In this podcast, Series 5, Chapter 6, Dr. Jeffrey Barsuk interviews Dr. Robert Jury. Dr. Jury is a Clinical Assistant Professor and Core Teaching Faculty member in the Counseling at Northwestern program at The Family Institute at Northwestern University. He earned his PhD in Counselor Education and Supervision from Northern Illinois University, graduate degrees in Counseling from Northwestern University, and in Narrative Therapy and Community Work from the University of Melbourne. Dr. Jury is also an ordained rabbi, commissioned cantor, Licensed Clinical Professional Counselor, and Board Certified Chaplain. He currently serves as President-elect of Neshama: Association of Jewish Chaplains and previously chaired its Certification Commission, helping oversee professional certification standards and the development of board-certified chaplains. Listen in while Drs. Barsuk and Jury discuss the application of simulation-based mastery learning to communication skills training for chaplains.
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
Speaker 1Welcome back to our podcast called Simulation and Healthcare Education. I'm Jeffrey H. Barsak, Executive Medical Director of Northwestern Simulation. Today, I have the honor of interviewing Dr. Robert T. Jury.
Speaker 2Dr.
Speaker 1Jury is a clinical assistant professor and core teaching faculty member in the Counseling at Northwestern program at the Family Institute at Northwestern University. He earned his Ph.D. in counselor education and supervision from Northern Illinois University and holds graduate degrees in counseling from Northwestern University and in narrative therapy and community work from the University of Melbourne.
Speaker 2Dr.
Speaker 1Jury is also an ordained rabbi in Commission Cantor, a licensed clinical professional counselor, and a board-certified chaplain. He is president-elect of Neshama, Associate of Jewish Chaplains and previously served as Chair of its Certification Commission, where he helped oversee professional certification standards and formation for board-certified chaplains. His work spans counselor education, clinical practice, healthcare chaplaincy, addiction and recovery, spirituality,
Speaker 1and professional formation. His current scholarship focuses on simulation-based mastery learning, deliberate practice, and the use of artificial intelligence in professional skill development. Rob, it's an honor to interview you today.
Speaker 2What an honor to be here, and thank you so much. A longtime fan of the podcast and really excited to be in conversation with you today. So I always start my podcasts, as you probably know, with a little bit of background. It's a bit unusual that I'm talking to an ordained rabbi and a counselor because we don't think of simulation in this setting as much as we do in the medical professional clinical training. So tell me a little bit about your background and how you got involved in simulation. That's great. My work in simulation really grew
Speaker 2out of a simple question. How do we help counselors, chaplains, and other helping professionals learn practice difficult interpersonal skills and not just learn about them. I first became involved in simulation-based mastery learning by attending your outstanding course at Northwestern Simulation to learn a little more about how do we address the curricular needs in thinking through simulation. There's a lot out there about elements of simulation, and really, you
Speaker 2all do the best at how do we address this curricular work. From there, I began applying simulation in a number of settings. I developed and led simulation lab experiences for Neshama Association of Jewish Chaplains. We say NAJC to not have to say all those words all the time. We held them at conferences for chaplains preparing for board certification. These simulations really focused on our professional practice skills, an important component of the high-stakes chaplaincy certification process. The goal
Speaker 2was to give candidates an opportunity to practice these competencies and do so in realistic scenarios, receive feedback, and become more comfortable in demonstrating them before entering the formal certification process. At the same time, I began incorporating simulation and deliberate practice into counselor education classrooms and into my research, particularly around spirituality, religion, identity, addiction, and other difficult or emotionally
Speaker 2charged clinical conversations. As the work was developing, I wanted more of that formal training. And so I also completed a postgraduate certificate in simulation-based clinical education at the University of Edinburgh. And that training helped me think more broadly and systematically about simulation design, the debriefing assessment, and how simulation can be integrated into professional education. in a rigorous way. Since then, the work's continued to expand. I've used simulation
Speaker 2and deliberate practice across counselor education, chaplaincy training, and a number of research studies focused on professional responses to anti-Semitism, to spiritual and religious competencies, and to development of professional practice skills. More recently, this has turned to the use of AI-supported simulation. And I'm studying the use of conversational AI avatars for skill development with clinical mental health counselors, chaplains, addiction counselors, even peer recovery support specialists as the use
Speaker 2of a broader AI-supported simulation in education. I'm really excited about that work.
Speaker 1Wow, that's impressive. Again, I never would have thought that this would be done in this realm. We've certainly attacked this with some difficult conversations. and communication skills among clinicians, but to teach chaplaincy and conversations around religion and spirituality, anti-Semitism even, that is just something I never imagined would happen. So what was your educational problem that led you to simulation and eventually mastery learning?
Speaker 2Great, that educational problem starts with the need to perform skills on a consistent basis in complex communication skills. And one of the things that comes in clinical education for chaplains or even clergy education, we find that we talk about the skills and we hope that simply through living them out in practice that we'll develop them. It turns out then learning about the skills and being able to
Speaker 2perform them consistently in a professional clinical way, two different needs. And so we turn to the behavioral rehearsal that simulation allows and the opportunity to engage in difficult, complex conversations without requiring the people at the center of the work to be the teachers for us to practice in addressing antisemitism or in complex conversations involving race or gender or other identities. We need a way to practice those.
Speaker 1So was your needs assessment based on observation, or were there manuscripts discussing the variability in skills among individuals that were doing this?
Speaker 2There's a little bit of both. Some of it was a needs assessment when I was chairing the certification commission for NAJC, and we were looking at the number of folks coming up for a high-stakes certification exam to become a board-certified chaplain. Many of the listeners in medical settings may be familiar with chaplains in the hospitals. Those chaplains undergo their own certification exam called board certification chaplain, BCC. And we watched the pass rate plummet as we looked
Speaker 2at more behavior-based competencies. And in that moment, we were able to identify that traditional chaplaincy training didn't provide enough additional skills practice before the high-stake exam. So we began to use that to address post-basic training and pre-certification exam.
Speaker 1That is amazing that you see the same thing in chaplaincy that we see in medicine. that when we test practicing physicians or nurses or whoever it is in the clinic, we find that they have a variability of skill and now you're seeing it in chaplains. So somehow you came to us and you took our mastery learning course. I'm curious how that came about. I mean, certainly it's a competency, but how did it turn from a competency-based assessment to like true mastery and where I would say now we
Speaker 1would define as someone that's well-prepared to do a task individually without supervision and safely for the patient. So for communication skills in a way that is pretty high level.
Speaker 2I love the focus on mastery-based learning, that we can continue to learn until we've mastered the skill. we started to look for not only the use of simulation, but also curricular design that didn't stop people as you either have the skill or you don't. In high-stake exams, and particularly in communication work, the ability to say you have the skill or not misses the fact that most people need to reach a competency and maintain that competency. We
Speaker 2didn't want to screen out otherwise limited qualified candidates by saying, well, now you just don't have it. So it was really the mastery-based learning that brought me to you all, is it was a radically different way to approach the cutoff scores and the way of designs for education.
Speaker 1So in our communication training skills assessment, we had the advantage of using the spikes format, and there were published literature on methods that can be used. What the heck did you use? as your rubric or your assessment, and how did you define mastery?
Speaker 2So I love this idea, and the spikes format has been a really interesting place for me to look at it, both from watching physicians... while attending the course and then looking at it from the area of counselor education, where our communication requirements involve many things that don't exist in the SPIKES format, including nonverbal skills and minimal encouragers, other attending skills that aren't present in that format. So fascinating place. For me, we use deliberate practice and Tony Romaniere's work
Speaker 2in deliberate practice in order to establish what was going to be the skill requirement. So in deliberate practice, according to the school of Tony Romaniere and all of his folks working on this, deliberate practice identifies both a skill gap, so they would call it a skill deficit, and then operationalizing skill criteria to meet that deficit. It takes communication competencies and breaks them down into micro skills. And so we took our competencies that were required across all the
Speaker 2certifying bodies, and we broke them down into these micro skills to make up the subcomponents. For me, I think that's what's complex about communication skills in a simulation. is that often we need a series of micro skills coming together in different ways. And those different ways might not follow the same order of operations that we might follow when performing surgery. And we need the ability to both have the skills and to be able to improvise within
Speaker 2the skills at a high level of mastery and communication skills.
Speaker 1So it sounds like to break down those micro skills that you may have used the science of expertise. That's Carl Wieman's work originally in STEM and in physics teaching. So we don't have to get into the nitty gritty, but was it observation of other people doing this effectively or people you considered experts that helped you figure out what are those micro skills within the competencies?
Speaker 2Great. In the science of expertise, we did a combination of both, both some observation and then two bring some folks together in Delphi rounds in order to refine what we had observed. And so we took some panel of experts of both chaplaincy educators as well as other board-certified chaplains and asked them to take a look at the literature that was developing out of the observation and now to use some Delphi rounds to refine the micro-skills for that.
Speaker 2And then we've begun a process that hasn't completed on a national level, though it's underway, of looking at minimal passing scores. Where's the cutoff that we would need to say you did enough of these skills, that you've performed the skill well?
Speaker 1So you're evaluating learning curves, basically, it seems.
Speaker 2Definitely evaluating learning curves, and then also re-examining Are these the skills that are actually required in order to perform the competency that's named? It turns out when you put a bunch of theologians together to come up with behavioral skills competencies, they're known for their wordiness. And perhaps we haven't quite narrowed in on the exact words that are capturing the skills well. So that's also an active component in the field right now.
Speaker 1Tell me what it looks like now. So can you walk us through, if I am a chaplain and I'm going through your program, what does it look like for me?
Speaker 2Great. We're going to start by choosing one of the certification skills, one of these professional practice skills. And we're going to take that skill and we're going to break it down into skill components for that. What are the skill components in order to perform this piece that will be on your exam? We'll then give you an opportunity to practice it, and we'll give, coming out of the communication work of Tony Romaniere, we'll give you a stimulus or trigger to start with, and then we'll have you
Speaker 2give your first improvised attempt. And so in this version, the practice that's happening is frequently read by a role player. This role player has a structured role play. They can only read the script given to them. you'll then, as the clinician, improvise your response. We'll then pause and ask you, what did you do well? It turns out many high-performing folks in clinical spaces have difficulty naming what they did well and are experts at naming what didn't go well.
Speaker 2In order to build the competencies in communication skills, you actually need to know what you did well in order to repeat it again. And so folks start by identifying that. Then we'd do it again. It's rapid cycle deliberate practice. The same stimulus, and the person improvises a second time. Then we use consent to help establish psychological safety, and we ask in real time, are you ready for feedback? When the person has consented to the feedback, then the role player now offers something they thought the person did
Speaker 2well based on the skill criteria, and something they'd like them to try differently based on the skill criteria. We then Do it all again, rapid cycle. You're going to do multiple attempts at this. After three or four times, we take a look to see, is this a good practice? Sometimes in rapid cycle deliberate practice, we might be doing something that's too easy or too difficult. And we want to keep the challenge as a good practice. So we use the International Deliberate Practice Society's measurement to be able
Speaker 2to see, is this a good practice? practice. And if it is, we keep going typically for 20 or 30 minutes at a time until we have practiced the skill and can do it on command.
Speaker 1Are these role players standardized patients or are they other participants in the course or who are they?
Speaker 2We don't start with standardized patients as the cost of that is pretty high. Instead, you can start the actual practice to address the skill deficit with anybody. Because it's a standardized role play, you don't need a standardized patient for it. The person's going to read the stimulus for themselves, which will stay the same each time. Then we have the stimuli that are what we call beginner, intermediate, or advanced. And the difference among the stimuli tends to have to
Speaker 2do with the emotional content of what's being said. So the skill is still the same, and the difficulty isn't necessarily about your ability to find the words. It's rather, can you perform that as we increase the emotional intensity?
Speaker 1Is there a pre-test, a post-test?
Speaker 2We often do both a pre- and a post-test to be able to do a level set How did this come across the first time based on the skill criteria? Then a series of rapid cycle deliberate practice, ideally for 30 minutes. And then a post-test. How would we evaluate you based on the certification rubric, given what you were able to do? There's a secondary challenge in certification and chaplaincy, which is also a reflection in the ability to explain why you did what you did. So think of it
Speaker 2as a mini Schwartz round. happening after the practice as well. And so chaplains also have a chance to practice their reflection to be able to explain why they did what they did.
Speaker 1Now, I don't know a lot about chaplaincy certification exams, but is there a skills component to it or is it all written?
Speaker 2Also a skills component. There's a written component first and then what we call an interview where you meet in front of a panel of who will ask you about the skills and sometimes even ask you to demonstrate certain skills in a live session of a panel.
Speaker 1So they might actually even role play during the panel.
Speaker 2Absolutely. That's been known to happen when the demonstration of the skill is otherwise not apparent, then we'll be able to use a role play to be able to check it out.
Speaker 1So the emotional aspects and showing empathy seem to be the hardest things to grade because different observers may feel it differently. How do you account for that and how do you achieve inter-rater reliability on your assessments? Great.
Speaker 2The emotional aspects, I think, are both hard to grade and also where the practice actually is. It's one of my thoughts of the limitations of a SPIKES framework is you're not actually evaluating that when someone is going silent versus yelling back at you when breaking the bad news. And there's a significant differentiator between someone's kind of taking it in and shutting down, and we might think you're delivering it well because there's not a wild pushback and
Speaker 2you followed the framework, and someone who is processing that and reacting with strong emotion. So one of the things that we'll look at is how well did you respond in the emotional context? That involves the ability to do emotional regulation for the clinician. And that ability to perform a skill while remaining emotionally regulated is part of what had brought us into the work with anti-Semitism and the work addressing racism in these skills, because the person who's
Speaker 2responding also needs to attend to their own emotional regulation. And that's really what we're assessing. What was your emotional regulation? And then we can get feedback What was the impact of the response on another person? You can get that feedback in rapid cycle deliberate practice while talking with your partner. What was that like? And then we can do some of that internal recognition of the emotional regulation, even using AI simulated patients for that.
Speaker 1This is amazing. So we're going to get into your AI-supported conversational avatars in a second. I'm a little bit jealous of what you can do because, as you know, in health care, we are limited by the fact that every institution wants to do it their way or for themselves. And I don't have a national standard that I can have all the hospitals or all the med schools or all the nursing schools do it this way. And you're in a national organization. So tell me, what is your plan then? You said you're working on in the process
Speaker 1of getting this a part of the program nationally. Can you talk to that a little bit and how you had implementation struggles throughout that? Because I'm sure you did.
Speaker 2I think one of the first challenges was the move in chaplaincy to outcomes-based education and competency-based education. Because chaplaincy grows out of a clergy tradition, and those traditions predate outcomes-based learning, there was some resistance even there. We've been for about a decade in competency-based, outcomes-based for our certification standards.
Speaker 2That's brought us to a new place of being able to examine that more directly and to say, are we actually examining a demonstration of competency? Or are we having a theoretical discussion? And as we have, as a field, addressed this, we've started to come up with this rubric version where we can better assess demonstration of competency. I'm not saying that's not without some resistance in the field.
Speaker 2That changes is difficult. As we have better operationalized our certification rubrics, which is an ongoing process among our certification partners. We've also then looked at the gaps that have appeared in certifying bodies. So what had prompted me to look at simulation was our pass rate dropped significantly when we shifted to a competency-based demonstration. And the goal wasn't to have fewer certified chaplains. The goal was to attune chaplaincy
Speaker 2to that move that had taken place already in the medical fields. And As we're in that direction, looking at what's a minimum passing score for this and better operationalizing the micro skills to demonstrate has become really crucial. There's a number of ongoing studies, some of which I'm conducting, some of which other folks are actively conducting, in order to address that gap so we can continue to do this as an evidence-based practice as opposed to more wishful thinking.
Speaker 1So it sounds like there's a lot of stakeholders involved. I always talk about the importance of stakeholder engagement. We're going through this process currently. We're trying to change the way we train gastroenterology fellows and endoscopy skills, because right now it's just based on a number that you're required to complete of each type while you're a fellow. And we know that those numbers can be variable, not only because there's supervised practice and sometimes the The attending physician
Speaker 1will take the scope from the fellow, but also we know that numbers don't equal competency. So we're in an attempt now to see if we can make a standard for competency assessment as a requirement to graduate your endoscopy fellowship. We have the similar limitation that there's resistance to change, and there's a lot of organizations that want a stake in the game that you have to get to agree. We did a multi-center study of endoscopy training, and we're
Speaker 1having problems getting it published because the reviewers don't feel that simulation and competency-based assessment is the way to go, and they're very resistant. And the constant argument is that, well, simulation isn't the same as working on a patient. But what they don't understand is you handle the endoscope and the simulator the same way you do in a patient, regardless of the simulator. So either way, there's a lot of resistance to change. And you were more diplomatic when
Speaker 1you talked about it, but it sounds like you had to deal with other organizations. And I do remember having conversations with you during the Mastery Learning course a few years back about those resistance to change.
Speaker 2Very much. I think some of the resistance to change comes from a place of not fully understanding what simulation makes possible. And we'll hear similar arguments that simulation isn't the same as having actual lived experience in the moment. And I think that argument misses what it is to develop skill competency and misses what it is to be able to evaluate the benefit of behavioral rehearsal on the impact of performing the skill. We have a similar piece
Speaker 2where our certification requires a certain number of years post-graduation of just working as a chaplain, as if you will increase your skills simply by performing the role. And I think we know that isn't correlated with skill development. And what we really want to do is help those who are resistant understand that skills can be learned, skills can be improved, and that simulation-based Mastery-based learning, I think, addresses
Speaker 2some of that worry. Our stakeholders were pushing back, saying, well, you're going to exclude people who would otherwise be qualified because of simulation. And mastery-based learning flips that on its head, and it says everyone can become qualified. It's how long it will take for them to become qualified. And I've found our conversations using mastery-based learning have been a way to reduce the resistance. We're not looking to exclude people. We're looking to have rigor and qualifications and the performance
Speaker 2of skills.
Speaker 1And that's scary because they're willing to accept the status quo. You're going to limit the people that can become certified, but by the way, they're not qualified, but we'll certify them anyhow. So yes, and then mastery prevents that because all it does is you take more time to get certified than you would otherwise. Yeah.
Speaker 2Well, AI is hot.
Speaker 1And there's a lot of research being done in AI, and you're doing that in chaplaincy. And you're using AI-supported simulation and conversational avatars. So what can AI do and not do to replace clinical and relational skills? And what can it do that simulation can't do?
Speaker 2Great. So I think of AI as a simulation environment. And one of the things that it's able to do is provoke an emotional response in someone who's practicing consistently. As opposed to the emotional weight of a simulated patient or the cost of a simulated patient to provoke that emotional response, we can have the AI avatar speak and provoke that response in you. For me, that was the
Speaker 2moment where I got behind the usage of AI in our practice. is when I started to have those embodied and feeling experiences in the conversation with the AI avatar, similar to what I would have had with a human being. That can allow us to change the emotional intensity of your practice. It can allow us to repeatedly find the right words until we've found the communication practice without having a human being need to experience that. I think that's
Speaker 2one of, when we go back to that, Spike's framework, finding the way and allowing the clinician to find the words so it sounds like them and also follows the protocol sometimes gets rushed. AI doesn't care how many times we want to practice. It'll just keep repeating. So I found that to be really helpful.
Speaker 1What have you learned from the AI-based simulated clients with counselors, chaplains, addiction counselors, whoever specialists that you need to train?
Speaker 2Yeah, so one of the things I'm really interested in is that difference between using AI as an information tool and using it as the practice environment. And I'm really much more interested in it as the practice environment for the people in training. The AI avatar can give a learner repeated opportunities to encourage a challenging interaction. to experiment with different language responses. It creates a possibility for practice at a scale that's difficult to achieve with faculty members
Speaker 2or peers or even standardized patients. I don't think it's going to replace human supervision, human feedback giving, real clinical relationships. It's another place where practice can occur, a practice partner, not a replacement for people. It does some things pretty well, It can... increase access to repetition. If I want the learner to practice a difficult conversation 10 times, historically that can require a great deal of faculty time, which is
Speaker 2why we don't practice it 10 times, or for a standardized patient or for peer coordination. the AI simulation can make those repetitions more accessible. But there's also the things it doesn't reproduce fully. You can't get genuine human vulnerability from the AI. The complexity of a therapeutic relationship or embodied presence isn't the same. And the ethical responsibilities that come with caring for another human being. So I'm really
Speaker 2interested in where AI belongs in the process and not replacing all of the human parts. in professional formation.
Speaker 1The embodied experience, I think, is so important, whether it involves physical touch or just being present in the moment for, or at least in real life, certainly you cannot get to that with AI. We at Northwestern use AI more for practice, like you're using it, rather than assessment, because we find that assessment is quite variable. The literature is a little bit all over the place now with how reliable AI is at assessing OSCEs, for instance, in clinical education.
Speaker 1So where do you see simulation-based mastery learning and AI fitting into the future of education for counselors and even healthcare professionals?
Speaker 2Yeah, I think these professions all rely heavily on interpersonal performance. I find the practice using the AI is particularly compelling in the context of telemental health. So the AI, when it's placed in a frame that looks like a telemental health frame, is reasonably good at reproducing those feelings because we're not in person together. It's a pretty good facsimile for telehealth. We can ask people to enter emotionally complex situations,
Speaker 2listen carefully, regulate themselves, respond ethically. even make decisions in real time. And historically, much of that learning happened through coursework or supervision as opposed to experience with actual clients or patients. Simulation gives us another layer between the classroom knowledge and the real world of practice. And it allows someone to make mistakes and receive feedback and try again before another person has to bear the consequences of the
Speaker 2learner still developing the skills. We're now using AI-simulated avatars at the Family Institute at Northwestern in our counselor education program. Our methods courses, which are the skills courses for counseling, clinical mental health counselors, all incorporate AI avatars in the practice. It used to be that we didn't have simulation, and your clinical mental health counselors were practicing potentially practicing on
Speaker 2you as the client or the patient. Now we can both assess before human interactions, and that's, I think, one of the places in the future is not a summative assessment using AI simulation, but rather a formative assessment. Are you developed enough in your interactions using counseling skills to that we can now let you begin your practical and you can start to see human patients.
Speaker 1So Robert, it was great interviewing you. As you know, I've been trying to get you on the podcast for a long time because what you do is so unique using the mastery learning model that I have been so excited about it ever since you told us about it in the mastery learning course. And this kind of gives me a chance also to catch up with you and figure out all of the stuff that you have completed since doing the mastery learning course. Are there any publications yet we can look at? We have publications under review,
Speaker 1and I'm hoping in the next couple of months to have multiple publications coming out. We've had a couple accepted, and we're waiting for them, and a couple more in review. So stay tuned for things to be reading in early 27th.
Speaker 2Awesome. Awesome.
Speaker 1To our listeners, thank you for listening today. Please tune in to future episodes as we continue to talk with experts in the simulation-based healthcare education field. Bye for now.
In this podcast, Series 5, Chapter 1, Dr. Barsuk interviews Jim Farruggio. Jim is a professional actor and standardized patient in Chicago. His acting career involves numerous appearances in the theatre and on...
In this podcast, Series 4, Chapter 9, Dr. Barsuk interviews Dr. Matthew Charnetski. Matt is the Director of Simulation-Based Education and Research at Dartmouth-Hitchcock Medical Center in Lebanon, NH. Matt is also a...
In this podcast, Series 4, Chapter 8, Dr. Barsuk interviews Molly Schoblocher. Molly has an MBA in Emergency Management and Public Safety Communications, and a certificate in Healthcare Emergency Manager (Professional)...
In this podcast, Series 4, Chapter 7, Dr. Barsuk interviews Dr. Jayfus Doswell. Dr. Doswell has a PhD in Information Technology focused on artificial intelligence to improve human learning and performance. He started...
In this podcast, Series 4, Chapter 6, Dr. Barsuk interviews Dr. Martin Pusic, Associate Professor of Pediatrics and Emergency Medicine at Harvard Medical School and Senior Associate Faculty for Boston Children’s...