MEdIC Series: The Case of the Failure to Fail

2017-06-16T08:38:59+00:00

failure to fail

Welcome to season 4, episode 8 of the ALiEM Medical Education in Cases (MEdIC) series! Our team (Drs. Tamara McColl, Teresa Chan, John Eicken, Sarah Luckett-Gatopoulos, Eve Purdy, Alkarim Velji and Brent Thoma) is pleased to welcome you to our online community of practice where we discuss the practice of academic medicine!

This month, we present a case of an attending emergency physician who struggles with the ingrained “failure to fail” culture amongst his colleagues when faced with a resident who has significantly underperformed throughout his emergency medicine rotation.

MEdIC: The Case of the Failure to Fail

By Dr. Tamara McColl

Mark walked into the emergency department boardroom for the monthly departmental meeting. He sat down next to his friend Aaron and quickly scanned the meeting agenda. His eyes honed in on item number 3, a discussion of the recent learners who had rotated through the department.

“Hey Aaron, did you work with Trevor this month?”

“Yeah…” Aaron groaned, rolling his eyes.

“I’m definitely going to bring up his performance this month. Far too many red flags. He’s not performing at his level of training at all. Frankly, it’s dangerous to let him work with patients unsupervised!”

Trevor was a first-year surgical resident with whom Mark had worked several shifts earlier in the month. Within a few hours of their first shift, he noticed Trevor’s poor attention to detail, dismissive attitude towards concerning historical features, and very narrow differentials surrounding his cases. Amongst his many clinical missteps he had misdiagnosed a septic joint, planned to discharge a patient with unstable angina, and was overconfident with a central line and inadvertently cannulated the carotid artery.

Mark provided Trevor with honest feedback throughout the rotation and tried to help him progress, but didn’t notice much improvement with subsequent shifts. A big part of the issue was his attitude. He seemed resistant to constructive feedback and was defensive whenever Mark attempted to debrief various mishandled cases. Mark mentioned his concerns about Trevor to several colleagues throughout the month and it seemed like everyone was on the same page regarding his performance.

The meeting progressed quickly and before he knew it, they had reached the topic of resident progression. Dr. Singh went through each resident individually and allowed the group to comment and voice their concerns. When he arrived at Trevor’s name, Mark was shocked.

“I noticed some discrepancy in Trevor’s evaluations. Most of his scores are “meets expectations” and “exceeds expectations” aside from yours, Mark. I see you were the only staff who had some reservations about his performance?” said Dr. Singh, glancing over his folder at Mark.

“Yes, I don’t believe he’s met the objectives of our rotation. I had outlined some specific examples of cases we had together as well as a few critical incidents identified on shift. He’s overconfident and frankly, I believe his practice is unsafe,” Mark replied. “It’s interesting that I’m the only one who raised concerns since a few of us had discussed his performance and it seemed like we all had similar reservations.”

“Well he’s certainly not an all-star like our own residents, but for an off-service resident, he’s fine. We’ve definitely passed residents that were far worse than he is!” joked Dr. Davis, a senior physician in the group.

“Do you know what kind of a process it is to fail a learner? Not worth the hassle! He’ll be someone else’s problem next week.” added Dr. Collins. “Plus, do you really want him on shift again for another month?!”

The room broke out in whispers, chuckles, and smiles. Mark looked over at Aaron, hoping for some support but was met with a dispassionate shrug. The conversation in the room moved onto the next resident and then onto other departmental business. The group had overlooked Trevor’s deficiencies and ultimately stamped a “pass” on his final evaluation.

Discussion Questions

  1. Why do you think the physicians generally scored Trevor’s performance as “meets” or “exceeds” expectations rather than providing feedback consistent with their earlier remarks to Mark?
  2. As a clinician teacher who feels strongly about providing honest feedback and remediating struggling learners, how should Mark approach this situation? Is it worth speaking up again?
  3. Why do we have a general “failure to fail” culture in medicine in which we seem to pass learners who would likely significantly benefit from additional time and remediation? Do you think implementation of competency-based evaluation will change this culture as evaluation becomes more concrete and task based?
  4. What interventions could be implemented that would help make the process of reporting poor learner performance easier for staff physicians?

Weekly Wrap Up

As always, we will post the expert responses and a curated commentary derived from the community responses 2 weeks after the case is published.

This month, our 2 expert teams are:

  • Dr. Nancy Dudek & Dr. Jessica Trier
  • Dr. Karen Hauer & Dr. Vanessa Thompson

On June 16,  2017 we will post the curated commentary and expert responses to this case! After that date, you may continue to comment below, but your commentary will no longer be integrated into the curated commentary. That said, we’d love to hear from you, so please comment below!

All characters in this case are fictitious. Any resemblance to real persons, living or dead, is purely coincidental. Also, as always, we will generate a curated community commentary based on your participation below and on Twitter. We will try to attribute names, but if you choose to comment anonymously, you will be referred to as your pseudonym in our writing.

MEdIC Series: The Concept
Tamara McColl, MD FRCPC MEd(c)

Tamara McColl, MD FRCPC MEd(c)

Associate Editor, ALiEM MEdIC Series
Emergency Physician, St. Boniface Hospital, WRHA
Academic Lead, Educational Scholarship
Department of Emergency Medicine
University of Manitoba
  • Nadim Lalani

    Grossly inflated performance ratings are Not a phenomenon exclusive to medical education. Even university profs overestimate their performance – gulp.

    1. Physicians as poor raters:
    Almost always the missing competency is a soft skill (attitude) which are hard to articulate.

    This is compounded by a former lack of awareness and faculty development which undermines self-efficacy of faculty, but this is changing. (e.g around rater biases and providing “negative feedback” and having “crucial conversations” )

    We do little to no direct observation – which means little to no direct objective evidence to support or refute claims.

    People don’t want to own the problem. It takes courage to get in someone’s face and most are doves not hawks.

    50% of us are disengaged and 30% for sure burnt out. You’re asking a lot of people who don’t have the capacity to go the extra mile.

    2. Speaking up
    For sure! “What you permit you promote”. This goes for every single time in medicine you encounter something you don’t like – SPEAK UP! Why?

    1) Poor soft skills HARM patients

    2) The ddx of a struggling learner is wellness! Wellness! Wellness! (Then attitude, soft skills, then knowledge, then learning disability and program issues). Almost all who are identified are correctly course-corrected and thrive – so it’s an opportunity not an obstacle. Even those that are written off as “insight-less” can be coached around – trust me!

    3. General “failure to fail” culture in medicine and CBME:
    I really think we need to invent another word for “remediation”. That aside:

    We lack skills, are biased and don’t take time to set up the shift or observe directly. Systemic pressures compound this problem.

    CBME should in theory fix this as the onus will be on the learner to ensure competencies are checked off, but this still requires an extraordinary investment in faculty (and remuneration) which UGME PGME have been generally unable to provide to date.

    4. What interventions could be implemented that would help make the process of reporting poor learner performance easier for staff physicians?

    Teach around it! Make people aware! Identify the poor raters and give THEM objective feedback! LOL (“Nadim we have noted that you have a tendency to fall into “central tendency bias”)

    Culturally – stop the tolerance. Daily rotation evals with more comments than checkboxes. McMAP – is a win here.

    Create a culture of “objective feedback” – let’s lose the term “negative feedback” – we are dealing with generally high achievers and so their work matters to them. They need objective measures of what they need to work on.

    Maybe have a rule that any feedback – even it comes AFTER a rotation, can trigger a review. This may encourage people to speak up?

    What I do?
    Set up the shift expectations at the beginning:
    I always tell learners that I am going to give them feedback on their ABCS – NO! not airway breathing and circulation!
    Attitude
    Behaviours
    Clinical Skills
    Soft Skills
    Guess what? When I tell them that I feel that they mailed it in (at the end of the shift) there is no blow-back, there is no surprise. Only a “okay – how can I get better” – for me this leads to conversations around creating meaning and purpose which I love (but for YOU – you can consult it out! – suggest they talk to a mentor or coach! Document this)
    Have a pull model for feedback so that faculty can just consult out the problem – even if it’s … “I don’t know how to articulate it, but there was something off with this learners work day before yesterday … just thought I’d let you know” … It’s never too late … this provides and opportunity to dig deeper. Most programs have a dedicated rotation director who can then dig into this.
    Dedicated rotation directors should really be a team of “scrutinizers” who can drop in and scrutinize.

    We all need to own this – it’s called accountability. – I’d really be interested in hearing Eve’s take on this In terms of tribe dynamic and norms.

    There’s also some really good stuff on the CANADIEM site on this ☺

    cheers

    NL

    • 1. Why do you think the physicians generally scored Trevor’s performance as “meets” or “exceeds” expectations rather than providing feedback consistent with their earlier remarks to Mark?

      I agree with Nadim, faculty are chickens. It is hard to tell someone they are inadequate. Though passing an incompetent physician has higher stakes than telling them in one instance that they need work, it sure doesn’t seem like that at the time.

      We tried an experiment of telling faculty to “act like coaches.” Your batting coach has no qualms telling you your stance is too narrow or your grip sucks. And you accept that feedback as instructions for improvement. This DID NOT work with faculty. They had trouble providing advice to improve to the learner’s face and harder time writing it down.

      I’d also add that the cognitive load of most evaluation forms are too much for the already mentally taxing end-of-shift activities. The path of least resistance is to just circle that last column. We should design forms to be easy and intuitive and complete them in a space and place that’s calmer.

      —–

      2. As a clinician teacher who feels strongly about providing honest feedback and remediating struggling learners, how should Mark approach this situation? Is it worth speaking up again?

      The inertia of the group will be difficult to overcome. Everyone just wants to get out of that meeting. Go talk to Dr. Singh later when he or she is able to listen.

      —–

      3. Why do we have a general “failure to fail” culture in medicine in which we seem to pass learners who would likely significantly benefit from additional time and remediation? Do you think implementation of competency-based evaluation will change this culture as evaluation becomes more concrete and task based?

      In medical school and even more so in residency, we don’t have the flexibility of “letting time be the variable and ability the constant,” at least not in our current model. If you have 60% of expected Ddx skills by the end of the rotation, you still move on. You have to, someone has to cover the unit. However, Ddx is still taught in the unit. We’d have to have forward feed, which is probably assumed in CBME but not in our tea-steeping model.

      —–

      4. What interventions could be implemented that would help make the process of reporting poor learner performance easier for staff physicians?

      Ideally? All workplace based assessments are formative. Faculty and learners knew this and so were not afraid to give or receive constructive criticism. High stakes decisions were made on some later summative assessment divorced from faculty coaching. I have no idea how to make that happen without big bags of cash.

    • I think it’s possible to create a culture of feedback though….

    • Eve Purdy

      Hey Nadim,

      Sorry for being so late to chime in on this one. Culture is a learned set of values, behaviours and actions. Culture is key to this discussion. .

      Rotating through a variety of different specialties in two years as a largely off service resident has provided me with a cross-specialty comparison on cultures around feedback and failure. I can confirm that there is a near universal fear of failure amongst learners and near universal failure to appropriately address short comings in a way that is truly constructive amongst our teachers.

      A distaste for failure, given the life-and-death consequences of our work, is understandable – but it can paralyze progress by creating an apprehensive mindset that transcends clinical and non-clinical domains.

      So, how do we create a culture of true feedback? I think frequent, low-stakes assessments are important but agree that the people doing those assessments cannot be stretched so thin that it seems that it is just another thing on their plate. CBME is putting more onus on the learner but it is still going to be very taxing for our faculty…

      In my thinking around attitudes that would create a positive feedback culture I find myself coming back to admissions. Our admissions process for medical school and residency actively select against people who are comfortable with failure. We need to come up with innovative ways to find people who thrive and rise to the challenge of improving with feedback.

      Eve

  • Krishan Yadav

    There’s a few things I find highly interesting about a ‘failure to fail’ culture. It is very rare for a medical student or resident to be held back in their training or to be told they cannot continue altogether. Yet it is more common to hear about a poor learner who is really underperforming. Speaking to colleagues in other specialties, there’s also a feeling that if you fail someone on a rotation (for example), you are then ‘stuck’ with this person for another block of time. Wrong attitude I know, but an honest reflection of how some people feel.

    I agree with Nadim that most are doves and not hawks. I think part of that is the teacher is also evaluated by the learner. Part of the issue is many evaluators are hesitant to give poor scores because they (incorrectly or correctly) feel this will result in a poor feedback evaluation for them. Numbers on evaluations are really unhelpful. I find most learners tend to focus on the number rather than the comments written.

    Overall, I think there’s an underlying culture that if you’re in medicine, you must be pretty smart and failing is not part of the equation. Hopefully CBME can fix these (and more) issues with medical education.

    • Tamara McColl

      Thanks for sharing your insights Krishan!

  • Swapnil Hiremath

    This is a perfectly pitched post, and I am sure many people are nodding along at this time. Failing is difficult. Sometimes one is in Mark’s shoes not for an R1 but an R4 or an R5, and one has to wonder, how did they reach so far? Definitely a failure to fail.

    From colleagues who have been brave enough to fail, the institutional culture, I am told, doesn’t help. In our field (medicine) it is crucial to fail. I would suggest, it should be done more often.

    Perhaps what would help, would be to then proactively offer remedial opportunities to shape up. These exist, but make the evaluators aware that they are not being cruel and unusual. Perhaps this should be the standard – that everyone gets certain things wrong and they need remedial rotations. So there will be less shame attached to ‘failing’. Everyone fails.

    • Tamara McColl

      Interesting you mention institutional culture. Is it from a University or more local departmental level where you feel the barriers exist?

  • Dr. J.

    In Emergency Medicine we have so many high performing residents rotate through our departments that we have become accustomed to learners who perform well and who improve with relatively little effort from the preceptor. As is the case in this story the learner in difficulty is often framed as a burden; “…added Dr. Collins. “Plus, do you really want him on shift again for another month?!” Our culture, the culture that believes a resident who is struggling is a burden, is a performance culture and an excellence culture, but it is also sometimes a culture in denial of the humanity of our work and the humanity of being a medical learner. “He’s certainly not a star like our own residents…” says one of the senior physicians in the vignette, but flip that around and consider what it is like to be a ‘star resident’ in such a program, what is the message? I think it is ‘Be a star or become a burden.’ This creates a culture where people do not strive be be their best self, to chase excellence rather they seek to not be not excellent, or to not become a burden. Failure is how we grow, as doctors and as people. When we are scared of failure and when the consequences of failure are high we choose the safest paths, paths that can lead to stagnation.

    Imagine what it is like to be Trevor. Trevor is intelligent, (he got into medical school) he is likely a hard worker (he matched to a surgical residency) and quite possibly for the first time in his life he is struggling. He almost certainly knows he is struggling (pure insight problems are rare) but he is given variable feedback and even those who tell him he is struggling don’t seem to be able to tell him why he is struggling. He can sense that people are unhappy with him. People have given up on him, they just want to pass him along, and he knows it.

    As a medical educator, and as a residency program director the learner in difficulty is (in my opinion) the most intellectually interesting part of my job. A person with a track record of success is struggling, that’s interesting!

    Why is the learner in difficulty such a challenging issue in medicine? I don’t claim to be an expert but I will share some observations, some opinions and an approach to the problem that I have found useful.

    1) Preceptors often identify the learner in difficulty through type 1 thinking processes. This is the situation where a preceptor says ‘I don’t know why but they’re just not as good as the other residents.’ For the most part these assessments are accurate but they miss the subtle difficulties and the well hidden difficulties that learners face. Heuristic identification of the learner in difficulty by a seasoned preceptor has decent specificity but lacks sensitivity.

    2) Preceptors become quickly frustrated with the learner in difficulty and struggle to identify the underlying problem. As emergency physicians we like to make quick decisions. We live in a world where type 1 thinking is rewarded, but type one thinking fails in this circumstance. It fails simply because almost none of us will gain enough experience with learners in difficulty to develop accurate or reliable heuristics that we can use to assess the situation. This manifests as evaluations that say “below average clinical performance” or “has a lot of knowledge gaps”. It is important to recognize that these are the phenotypes, the outward expression on the learner in difficulty but only rarely the genotype (or etiology) of the problem itself.

    3) Preceptors put too much pressure on themselves. As a program director I have no expectation that a preceptor will figure out the ‘why’ of a learner in difficulty, that’s what I get paid the big program director bucks to do. What is most valuable to me from a preceptor is a narrative of observable behaviours and themes that occurred during the shift. I don’t really care for rating scales, if a preceptor is unsure about a resident I would always prefer they leave the scale blank and provide me with a written record of their observations instead.

    4) It is best to use a familiar process in assessing the ‘why’ of a learner in difficulty. I prefer to think of it as the differential diagnosis of the learner in difficulty and use the CANMEDS framework to start to unpack the issue. This is not the only possible approach, but it is an approach that seems familiar and a good way to get started. As Dr. Lalani states in the comments the issue is often about wellness and almost always involves wellness. You may find that there are skills issues, insight issues, cognitive processing issues, residency mis-match, etc.. You may need help to figure it out from experts. All that being said, if you approach the situation from the point of view of curiosity and in a deliberate and organized way you stand a much better chance of getting to the ‘why’ of the situation.

    5) It is important to approach the situation with curiosity and an open mind, active listening is important. It is less important to get to the answer quickly. Start the conversation with the resident early, this isn’t a court case and you don’t need all of the evidence to proceed. Here is a good first conversation; “I’m wondering if you are okay?” Just say that and then listen. If the resident is defensive resist the urge to counter with the evidence, again this is not a trial, and make a statement of genuine concern. Get the wellness issues on the table early, you will never get at any other issues while wellness remains unstated or unaddressed.

    6) Have a try at untangling the identity problem for the learner. I am well aware that the process of medical education results in a very entangled idea of ourselves as doctor-people and that our behaviours and performance as doctors is how we value ourselves as persons. Reminding the resident that they have integral value as a person, completely separate from being a doctor can be important.

    I would like to meet a resident like Trevor. As a medical educator I think that there is a lot to offer him, and an certainly the opportunity to have a meaningful impact on his education and his career. A resident like Trevor deserves better than being passed along, of being thought of as a burden. He deserves the time and space to work through this important challenge in his career and his life.

    The mark of a great educator is not how many stars pass through your program. The star residents are the stars with or without you. A great educator helps learners get near to their own personal best. It’s messy, it’s uncertain and it requires slow and deliberate thinking, but it is one of the most important and rewarding processes in medical education.
    Aaron Johnston

  • Kaif Pardhan

    This case certainly resonated with me.

    I am definitely in agreement with Nadim and Eve’s points regarding physicians as raters, resource requirements to do feedback well and the population we are admitting to medical school being many of the links in this chain. Unfortunately, CBME (or the Royal College branded CBD) will not be a panacea for this as the issues around culture and feedback will ultimately remain given the same set of faculty, resource constraints and admissions policies we faced before the new curriculum is implemented.

    The point that I’d like to bring to the discussion centres around teaching faculty, specifically how we select, train and reward these individuals. There is an assumption made that as medical students and residents progress through their training, they become excellent (or at least competent) teachers. Some universities in Canada have well developed “Residents as Teachers” programs and the Professional Association of Residents of Ontario (PARO) has recently developed and is rolling out a teaching to teach program. However, these programs are far from universal and, with the emphasis through training on the medical expert competencies, particularly as residents are preparing for the exam, there is an unstated devaluation of enhancing and maintaining this competency.

    Similarly, once in independent practice, the emphasis from the universities where we train our medical students and residents is on researchers and research productivity rather than clinical teaching excellence. The comment made by Dr. Collins in the case above is only too true – failing a trainee is quite an onerous process and takes a professional and emotional toll on the faculty member as well as the trainee. Part of the culture change that everyone has commented on is providing training and support for both new in practice physicians as well as more experienced faculty who are responsible for frontline clinical teaching. Part of being an excellent clinical teacher is knowing when a trainee is not meeting expectations and being able to have a frank conversation with them and make the ultimate decision to give a failing assessment. This excellence has to be supported both by the teaching faculty and by the institution in order for the culture change to start taking effect.

  • Dr. Aaron Johnston

    In Emergency Medicine we have so many high performing residents rotate through our departments that we have become accustomed to learners who perform well and who improve with relatively little effort from the preceptor. As is the case in this story the learner in difficulty is often framed as a burden; “…added Dr. Collins. “Plus, do you really want him on shift again for another month?!” Our culture, the culture that believes a resident who is struggling is a burden, is a performance culture and an excellence culture, but it is also sometimes a culture in denial of the humanity of our work and the humanity of being a medical learner. “He’s certainly not a star like our own residents…” says one of the senior physicians in the vignette, but flip that around and consider what it is like to be a ‘star resident’ in such a program, what is the message? I think it is ‘Be a star or become a burden.’ This creates a culture where people do not strive be be their best self, to chase excellence rather they seek to not be not excellent, or to not become a burden. Failure is how we grow, as doctors and as people. When we are scared of failure and when the consequences of failure are high we choose the safest paths, paths that can lead to stagnation.
    Imagine what it is like to be Trevor. Trevor is intelligent, (he got into medical school) he is likely a hard worker (he matched to a surgical residency) and quite possibly for the first time in his life he is struggling. He almost certainly knows he is struggling (pure insight problems are rare) but he is given variable feedback and even those who tell him he is struggling don’t seem to be able to tell him why he is struggling. He can sense that people are unhappy with him. People have given up on him, they just want to pass him along, and he knows it.
    As a medical educator, and as a residency program director the learner in difficulty is (in my opinion) the most intellectually interesting part of my job. A person with a track record of success is struggling, that’s interesting!
    Why is the learner in difficulty such a challenging issue in medicine? I don’t claim to be an expert but I will share some observations, some opinions and an approach to the problem that I have found useful.
    1) Preceptors often identify the learner in difficulty through type 1 thinking processes. This is the situation where a preceptor says ‘I don’t know why but they’re just not as good as the other residents.’ For the most part these assessments are accurate but they miss the subtle difficulties and the well hidden difficulties that learners face. Heuristic identification of the learner in difficulty by a seasoned preceptor has decent specificity but lacks sensitivity.
    2) Preceptors become quickly frustrated with the learner in difficulty and struggle to identify the underlying problem. As emergency physicians we like to make quick decisions. We live in a world where type 1 thinking is rewarded, but type one thinking fails in this circumstance. It fails simply because almost none of us will gain enough experience with learners in difficulty to develop accurate or reliable heuristics that we can use to assess the situation. This manifests as evaluations that say “below average clinical performance” or “has a lot of knowledge gaps”. It is important to recognize that these are the phenotypes, the outward expression on the learner in difficulty but only rarely the genotype (or etiology) of the problem itself.
    3) Preceptors put too much pressure on themselves. As a program director I have no expectation that a preceptor will figure out the ‘why’ of a learner in difficulty, that’s what I get paid the big program director bucks to do. What is most valuable to me from a preceptor is a narrative of observable behaviours and themes that occurred during the shift. I don’t really care for rating scales, if a preceptor is unsure about a resident I would always prefer they leave the scale blank and provide me with a written record of their observations instead.
    4) It is best to use a familiar process in assessing the ‘why’ of a learner in difficulty. I prefer to think of it as the differential diagnosis of the learner in difficulty and use the CANMEDS framework to start to unpack the issue. This is not the only possible approach, but it is an approach that seems familiar and a good way to get started. As Dr. Lalani states in the comments the issue is often about wellness and almost always involves wellness. You may find that there are skills issues, insight issues, cognitive processing issues, residency mis-match, etc.. You may need help to figure it out from experts. All that being said, if you approach the situation from the point of view of curiosity and in a deliberate and organized way you stand a much better chance of getting to the ‘why’ of the situation.
    5) It is important to approach the situation with curiosity and an open mind, active listening is important. It is less important to get to the answer quickly. Start the conversation with the resident early, this isn’t a court case and you don’t need all of the evidence to proceed. Here is a good first conversation; “I’m wondering if you are okay?” Just say that and then listen. If the resident is defensive resist the urge to counter with the evidence, again this is not a trial, and make a statement of genuine concern. Get the wellness issues on the table early, you will never get at any other issues while wellness remains unstated or unaddressed.
    6) Have a try at untangling the identity problem for the learner. I am well aware that the process of medical education results in a very entangled idea of ourselves as doctor-people and that our behaviours and performance as doctors is how we value ourselves as persons. Reminding the resident that they have integral value as a person, completely separate from being a doctor can be important.
    I would like to meet a resident like Trevor. As a medical educator I think that there is a lot to offer him, and certainly the opportunity to have a meaningful impact on his education and his career. A resident like Trevor deserves better than being passed along, of being thought of as a burden. He deserves the time and space to work through this important challenge in his career and his life.
    The mark of a great educator is not how many stars pass through your program. The star residents are the stars with or without you. A great educator helps learners get near to their own personal best. It’s messy, it’s uncertain and it requires slow and deliberate thinking, but it is one of the most important and rewarding processes in medical education.
    Aaron Johnston