MEdIC Series | The Case of the Awkward Assessors

2017-01-20T12:30:40+00:00

awkward assessorWelcome to season 3, episode 4 of the ALiEM Medical Education in Cases (MEdIC) series! Our team (Brent Thoma, Sarah Luckett-Gatopoulos, Tamara McColl, Eve Purdy, and Teresa Chan) is pleased to welcome you to our online community of practice where we discuss difficult medical education cases each month. As usual, the community discussion will be reviewed using qualitative research methods to produce a curated summary that will be combined with two expert responses to create a functional teaching resource.

This month’s case features a couple of colleagues who are trying how to evaluate a medical student appropriate. How can we give critical feedback in a busy emergency department? What is our obligation as physicians in terms of reporting negative feedback to our institutions? Should learners who are uninterested in emergency medicine be held to a different standard? Please read the case and join in the discussion below!

MEdIC Series: The Concept

MEdIC: The Case of the Awkward Assessors

by Dr. Eddie Garcia

“Hey, Tom, crazy day, no?”

Tom looked up from the computer screen.  A fourth-year medical student, Patrick, had a grin plastered on his face. He repeated himself, “Crazy, huh?”

“Yeah, crazy day, definitely.” Tom turned back to computer to catch up on charting. “So you learn anything today?”

“Of course! I saw a traction pin being placed and looked at so many x-rays!” Patrick couldn’t hide his enthusiasm. “So, how did I do today?”

Tom hesitated for a second. He didn’t know exactly how to respond to Patrick. After doing an I&D together at the start of the shift, Tom hadn’t seen Patrick for most of the day. Maybe they had just missed each other in the chaos, but Tom didn’t think so. Was it possible that Patrick hadn’t been in the department that day?

Tom considered probing further, but he was tired. He gave the easy response, “You’re doing a good job. You did great with that I&D. Keep it up.”

“Fantastic! Listen, I need an assessment form filled out by an attending. Do you think you could tell Dr. Pam that I did a good job? I only have one more shift and I really need to get one before I’m done. Thanks.”

*******

Tom stretched and signed his last note. He was about to head out when Dr. Pam sat down at the computer next to him. Tom remembered Patrick’s request.

“Hey, Dr. Pam, you know that MS4, Patrick? He told me he asked you for an assessment form. I thought I’d pass along that he did a pretty good job on an I&D in the morning.”

“Oh, did he?” replied Dr. Pam. “That’s great. What other patients did you see together?”

Tom sensed a funny tone in Dr. Pam’s voice. He weighed his words carefully, “Well, none that I can remember, but it was pretty busy.”

“That’s because he spent most of the day parading around with various surgical services,” said Dr. Pam matter-of-factly. “I’ve worked with him before and that seems to be his usual. Yesterday, he spent half the day tailing ortho because his first patient had a fracture.”

“But, hey, I get it,” continued Dr. Pam. “Emergency medicine isn’t for everyone. Let him go where his interests lead him. As for his eval, I’ll probably just leave it blank. I’d rather do that than give him a real assessment. After all, he seems nice enough.”

“Makes sense, although he did say he really needed . . .” Tom trailed off.

Patrick stood silently in the doorway, assessment form in hand.

Dr. Pam was cool and collected. “Hey there, Patrick. Why don’t you sit down with us? We were just talking about you. Let’s discuss that assessment form.”

If you were Dr. Pam, how would you handle this situation? (More questions for discussion follow)


Discussion Questions

  1. With close quarters and constant traffic, the emergency department is a high-risk zone for eavesdropped conversations. However, as physicians, we are supposed to be experts at keeping conversations private! Considering the often large number of collaborative assessments and verbal feedback sessions required in the academic setting, how can we keep private conversations private?
  2. Tom gives the “easy response” to Patrick when asked for feedback. If feedback were part of the oral boards, what would be the critical fails?
  3. Patrick is not interested in emergency medicine. How do you approach assessing students going into emergency medicine versus those going into other fields?*

*Author’s note: The question refers to assessing students in general and not students specifically in Patrick’s situation.


Weekly Wrap Up

As always, we will post the expert responses and a curated commentary derived from the community responses one week after the case was published. This time the two experts are:

  • Dr. Inna Leybell is an Assistant EM Residency Program Director at NYU / Bellevue Medical Centers.
  • Dr. Karen Hauer is Associate Dean for Assessment and Professor of Medicine at the UCSF School of Medicine.

On January 29, 2015 we posted the Expert Responses and Curated Community Commentary for the Case of the Awkward Assessors.  After that date, you may continue to comment below, but your commentary will no longer be integrated into the curated commentary.  That said, we’d love to hear from you, so please comment below!

All characters in this case are fictitious. Any resemblance to real persons, living or dead, is purely coincidental. Also, as always, we will generate a curated community commentary based on your participation below and on Twitter. We will try to attribute names, but if you choose to comment anonymously, you will be referred to as your pseudonym in our writing.


Brent Thoma, MD MA
ALiEM Associate Editor
Emergency Medicine Research Director at the University of Saskatchewan
Editor/Author at CanadiEM.org
Brent Thoma, MD MA
  • Matthew Siedsma

    To me the main issue is one of professionalism. In any residency (or fellowship) you are going to be mandated to participate in off-service rotations that are meant to improve your knowledge base and provide you with experiences, that while tangential to your ultimate career path, have been deemed useful and or necessary for a trainee in that specialty. I would sit Patrick down and give him the opportunity to be honest about how he’s spent his time in the ED during his scheduled shifts. The onus is on Tom to ask him what other patients he saw today and who he saw them with. During an 8-12 hour ED shift it’s unreasonable to see only one patient together and then expect an evaluation. A way to combat this is to make that expectation clear up front by whomever is responsible for orienting new students to their clinical EM rotations.

    If Patrick is honest about his whereabouts and activities during his ED shifts, he should be thanked for his honesty and then ask him how he can make it right. Hopefully he will realize he needs to repeat at least 2 or 3 shifts where he demonstrates the appropriate professionalism. If he claims he was in the ED all day then he needs to provide some level of proof, find the resident or attending who he worked with and have them vouch for the patients he saw and worked up. If he can’t do that then explain your concerns and suspicions and allow him to respond. Explain that this is an issue of professionalism and as a future surgical intern he will be asked to rotate through a number of surgical specialty services that he may have no interest in as a future career path. Ask him if he thinks it would be appropriate to abandon his duties as a surgical intern because he wasn’t interested in the specialty. If he doesn’t make the connection on his own, make it quite clear for him.

    There are baseline expectations that should be met by each and every medical student regardless of their interest in your specialty. Professionalism and how it applies to any rotation should be a no-brainer. You can go from there in terms of expecting more from students who are actively pursuing a career in your specialty. Make those expectations clear from the outset and you’ll run into much fewer issues.

    In terms of having conversations in private in the ED, this is one of those situations where you break your rule once, you get burned, and then all subsequent conversations of a sensitive manner happen behind a closed door in an office or room elsewhere. Or over the phone when you’re both out of the department.

    • Hi Matt!
      How would you find out more about the situation that lead up to Patrick’s apparent absenteeism?

      • Matthew Siedsma

        Hi Teresa

        I think there are two viewpoints that are important. One is Patrick’s and why he would feel his absenteeism is allowable. The other is the viewpoint of Dr Pam and the other faculty who have worked with Patrick. From the vignette it seems that Tom had not worked with Patrick very much prior to this shift. If I was Dr Pam or Tom I would address the problem directly. The problem is that (as others have mentioned) we’re a little behind the eight ball here. There are clearly others aware of this behavior and have done nothing to address it. The longer it goes on unchecked the more it becomes equal parts the problem of the faculty and the student.

        If Patrick isn’t standing right there in front of you then the most reasonable thing to do is to poll the faculty that have worked with Patrick and find out their opinions of Patrick and his work during his rotation. I would do that by asking via email “We’re trying to put together a composite evaluation of an MS4, Patrick. Anyone that’s worked with him, please drop me a line so we can talk further.” I’d then ask them broadly how he performed during their shifts, initially without leading questions. If their answers indicate that they spent significant time together and he present in the department, then perhaps it was a limited problem. However if their answers indicate that he wasn’t around very much, I would ask follow-up questions to get specifics of how many patients they saw together . I would eventually ask if they had suspicions that he was spending significant time outside the department and if they had directly observed such behavior (as Pam seems to indicate).

        If Patrick is standing in front of you then I think you have no choice but to confront him with your observations. Dr Pam claims he spent “half the day with ortho” and Tom only saw him with one patient. State those observations and ask if he has an explanation. Who else did he work with during those shifts, how many patients did he see? If he’s honest, then again you try to find out why he thought this was ok. That’s the best case scenario. However, if he claims this isn’t the truth then he is tasked with finding the other faculty he was working with who can vouch for his presence in the department. I would follow that up with the implication that if he hasn’t been in the department and it is discovered that he is lying (we all have colleagues in other departments who can be asked as to his whereabouts during his ED shifts) then we will be forced to fail him with comments discussing his lack of professionalism.

  • Elisha Targonsky

    I agree with Matthew. Attending MD has to take onus on making sure that the learner is actually performing the duties for which he/she is responsible. This should be done during the shift. That aside, I think that recognizing Patrick’s main interests (surgery) is important to do at the start of the discussion. I would probably carry out the conversation with something like this:
    “Patrick, I see you have a lot of interest in the surgical patients and I think it’s great that you are so keen. You did great with the I&D. I am concerned though that you are getting distracted with these cases and not fully carrying out the tasks you should be doing during your EM rotation. I know EM isn’t what you had in mind for a career, but like all medical students it’s important for your learning and achieving your core competencies. It is also important for me or any other attending physician to adequately evaluate you. Do you understand what I mean?”
    Then let Patrick reflect and recognize the situation and respond.

    As far as privacy goes, this is the discussion you take to a quiet area without patients/learners/nurses. Should be done on the spot though.

    My two cents.

    • Thanks for your comments Elisha. Your blurb is a really good one!
      What are you thoughts on attendings gathering feedback from other learners (especially more senior ones) before filling out an assessment form?

  • Pik Mukherji

    “I only have one more shift left…”
    Unfortunately, this train has left the station. The time for formative feedback was long ago if the rotation is nearly done. So we’ve kind of failed Patrick here. He’s portrayed here as an enthusiastic puppy dog and budding surgeon. Not as an eye-rolling disinterested party who KNOWS he’s falling down on the job. If I was Tom:

    Hey, Patrick, I passed along your I+D report to Dr. Pam. She asked about other cases we had for the day, and of course I had nothing else to say…

    The ensuing pause should give us an idea of where Patrick’s head is at. If he wants an eval. it will be a bad one. If he wants to be an orthopedist, he can have a personal letter that talks about his interest in the field. But the time to set expectations, improve Patrick’s behavior, and expose him to EM is long gone. If there ARE professionalism issues, that this vignette doesn’t necessarily make clear, then he should get an appropriate eval. and possibly make up the rotation.

    • What would you do if you were Dr. Pam?
      What about as the clerkship director receiving a note from Dr. Pam about this incident?

      • Pik Mukherji

        In the awkward scenario above, where Patrick walks in on the discussion?

        I guess if I’m Dr. Pam I apologize to Patrick for not giving him this feedback earlier when he had a chance at salvaging the rotation. I’d probe as to whether HE thinks he deserves a good evaluation for his work in the ED. As much as I am trying to not ascribe bad intent here, most 4th yr students would have a very good idea that they hadn’t lived up to their responsibility as a learner or as a medical professional.

        • I like it! Converting this very awkward moment into a teachable one! Guided reflection is very important to coach in our learners, and I agree with you that this is clearly an opportunity.

          Now to challenge you (and all other readers out there)… What if Patrick displayed NO INSIGHT? This is not unheard of in medicine, as generally speaking much research has suggested a norming phenomenon that occurs in self assessment within medical trainees. People who are great doubt themselves, and those who are underperforming believe that they are on par with peers?

          P.S. for those interested in the self-assessment literature, here are a few papers that will lead you down that rabbit hole. 😀 Drs. Kevin Eva & Glenn Regehr (out of UBC’s CHES unit) have written about this extensively:

          1. Eva K, Regehr G. “I’ll Never Play Professional Football” and Other Fallacies of Self-Assessment. J Contin Educ Health Prof. 2008;28(1):14–9.

          2. Eva KW, Regehr G. Self-assessment in the health professions:
          a reformulation and research agenda. Acad Med. 2005;80(10 Suppl):S46–54.

  • jeff riddell

    Seems like some of this could have been avoided if Tom had given clear expectations at the beginning of the shift. Maybe not. I agree with Pik that the ship may have sailed not just for this shift, but for the whole rotation. Though there can probably be some salvaging by Pam at this point.

    Q1: I usually label the conversation and take it out of the work area. I ask the resident or student if they want to go “do feedback” and we go around the corner to a private spot. And when it is labeled as “feedback time”, it isn’t as awkward to walk off to a private spot, as if someone was in trouble. Doesn’t always work, but it is the best idea that I’ve been able to steal from my senior colleagues.

    Q3: I have different expectations of an EM-bound MS4 who has already done one EM rotation than I do of a psychiatry-bound medical student fulfilling a requirement. Though much of the core objectives are the same, I think their individual objectives are different and we should recognize that. I’m content if, by the end of a shift, the psychiatry-bound student can list the deadly causes of chest pain that we should rule out. I want the EM-bound student to understand the PERC rule. It is different expectation and so I assess them differently. Not sure if I’m doing the right thing.

    • Ahhh… I does not seem like your undergraduate assessments are using criterion standards (comparing students to a threshold set by a criteria – like names deadly CP DDx), but rather normative standards (i.e. comparing students to each other), right Jeff?

      In the incoming age of Competency Based Medical Education (CBME), defining your expectations and objectives will be very key for determining standards.

      Who’s responsibility is it at your shop to set these? The medical school? Your clerkship director? What do they do to message out to the faculty? Do they have workshops? Do they make explicit rubrics on forms?

  • Kaif Pardhan

    This is probably one of the biggest challenges we face in assessment. It is particularly acute, but certainly not limited to, emergency medicine given that a preceptor may work with a trainee for a single shift and may not notice important patterns of behaviour or evidence of the struggling trainee. Failing to provide an accurate assessment of a trainee, no matter what level, is doing them a disservice and may be doing patients a disservice down the line – particularly since we may be able to link them with helpful resources or remediation if they are, in fact, identified.

    This case speaks to a few issues:

    1. Clinical preceptors may not be supported in giving honest assessments. When faced with an assessment card that has a five point Likert scale (Unsatisfactory -> Consistently Exceeds Expectations), it is much easier to simply give a “Meets Expectations” and send the struggling or unprofessional trainee on their way. This is particularly true given that there is pressure to continue seeing patients, finish your documentation, get the trainee out on time and/or get yourself out relatively on time.

    2. In the absence of predetermined and broadly agreed upon criteria, it is up to the individual preceptor to decide and, more importantly, defend when they believe someone has NOT met expectations. If a preceptor chooses to give an honest evaluation that is below meets expectations, the onus is on them to provide concrete and objective examples that they have observed. A similar standard is not in place for meets/exceeds expectations and it becomes much easier to say “great job, keep up the good work”, rather than document all the ways they haven’t.

    3. There may not be a mechanism to discuss trainees amongst preceptors who supervise them. This leads to the situation faced by Tom and Dr. Pam, where they have to discuss how a trainee is doing in a sub-optimal setting. This normalizes “water-cooler” conversations about trainees, which will not add value to their assessment or allow for a frank discussion.

    To speak to the questions:

    – We need to create a space where preceptors have the opportunity to discuss their experiences with the trainees they supervise – both positive and negative. This may be during the physician group staff meeting or a separate dedicated time. This allows for a frank discussion that celebrates those who excel and identifies those who are struggling and what intervention would most benefit them.

    – If feedback were part of our certification exams, it would imply that it was part of our training – which it may or may not be depending on your training site. Feedback and assessment also imply that expectations have been set and that trainees are being measured against them. In this case, perhaps Patrick had been told on his first shift that he could follow consultant services as they managed referred patients – particularly given that he wasn’t interested in EM. He may have felt, rightly or wrongly, that he could then do this on all subsequent shifts. Is it now fair to penalize him, on his penultimate shift, when no preceptor had given him any indication that what he was doing was wrong?

    – In the Canadian context, 120 Canadian graduates ranked emergency medicine as their top choice for residency. This is out of approximately 3000 Canadian graduates applying to the Match. The vast majority of trainees that come through the ED are not interested in a career in EM so, while we should certainly ensure that those who are interested have an exceptional experience, we should also ensure that we aren’t shortchanging those who have legitimate learning outcomes they need to achieve and to be assessed on them while they are on our service.

    • Brent Thoma

      Stellar answer, Kaif!

      Feedback is hard for so many reasons, especially when we haven’t had good training for it. I learned a ton about debriefing / feedback during my sim fellowship but still find this case tough for many of the reasons that you outlined. As you alluded to, I think it’s particularly important to figure out where Patrick is coming from with his behaviour. The school of ‘debriefing with good judgement’ would go there by objectively stating the behaviour (following other services around instead of seeing ED patients), noting that this is considered inappropriate while on the ED block, and then offering him a chance to provide his perspective.

      Thanks for commenting!

  • Alvin C

    Great case!! However, I think that the vignette (probably intentionally) does leave some important questions unanswered:

    1. The assumption here is that Dr. Pam has correctly identified that Patrick had spent the majority of his shift with other specialties, outside of the emergency department where he is currently placed for his rotation. Is this actually the case or did he work with other emergency physicians during his shift and was assigned Dr. Pam specifically for his assessment? In my school, we are assigned to a certain preceptor for the shift and evaluation but there have been occasions where the preceptor has too many learners and gives the student the opportunity to join another staff with his/her patients.

    2. As mentioned below, there is definitely question as to whether or not Patrick is insightful to the fact that his assessors (Tom and Dr. Pam) felt that he was quite inappropriately absent during the shift, specifically without the permission of his supervisor. Was it the case that Patrick independently chose to work with other specialties, without permission, during his allotted time in the emergency or did a preceptor previously mention something along the lines of him having completed the core objectives and should tailor his education and exposure to the specialty that he was interested in/matched to (for the remainder of the shifts)?

    3. It seems that Dr. Pam has previously worked with Patrick and has established this impression of the student but there is no indication as to whether or not any feedback was given regarding this issue at hand. Has Dr. Pam already mentioned this as a problem or did Dr. Pam previously endorse it or inadvertently support this behaviour?

    I think as a junior learner (medical student), we are generally explicitly provided with the goals of each rotation and our responsibilities. At the same time, I personally have experienced a number of specialties where staff/residents encouraged me (usually in the last week or two of a rotation where I have already met the rotation objectives) to take on cases/consults that are more aligned with my interests.

    The biggest problem in this situation seems to be with communication. While I agree that students should have a level of understanding and professionalism to recognize their responsibilities on each rotation, I would hope that in situations where a learner is confused/mistaken about these expectations, that a supervisor/preceptor would help communicate and point out the gap. I personally have appreciated preceptors that take 2-5 minutes in the middle of a shift to point out the strengths and weaknesses I have been demonstrating so far in the shift, so that I have an opportunity to adjust my practice with the feedback. In fact, I would say that this may be especially important in emergency medicine where a student may work with an individual preceptor only once or twice during their entire rotation and the expectations may be constantly changing.

    In short, there are many questions unanswered in this case that force us to draw conclusions from various assumptions. This problem may have been mitigated if open communication between the supervisor and student was improved. If the student has significant lack in insight, I think it does pose an issue, but it would be unfortunate if the student was allowed to continue this behaviour without any constructive feedback. Personally, if I was the student in this situation, I would be open and honest about where I had been for the entire shift, the reason for my misunderstanding of the expectations, and finally, ask how my behaviour could be changed to better match the expectations for future shifts.

    • Brent Thoma

      Thanks for providing the student perspective, Alvin! I totally agree that it is important to give the student the feedback and also allow them some space to explain themselves. It’s amazing how often I’m surprised by why learners are doing what they are.

  • Daren

    Hi all, good case and good comments. Everyone should be given the benefit of the doubt, and deserve a fair process. We should assume the best in people. However, I will play devil’s advocate for sake of discussion. If the train has left the station, then is Patrick playing chicken with it? If Tom calls Pat on the problem, Pat may be able to say he had no idea – he was just helping out with the tough cases and trying to make his shift applicable to him. He was being self-directed. He lost track of Dr. Pam and Tom in the chaos, and after all, that first ER patient, was HIS patient and he wanted to follow him. (On the back of the daily eval, there should be a log of patients seen which can help Dr. Pam and Tom in the future.) Most people in Tom’s position would back down and at worst write “meets expectations” – and move on (it’s a busy day in the ED!). If Tom does the unlikely thing and writes a really bad daily eval, would the comment even make it to the final eval that goes on his transcript? That final is written by the clerkship director (or similar student advisor) who has to sort through a bunch of evals and might see Tom’s as an outlier (but maybe there were other bad shifts just no other bad evals). Is there anyone who matters to Pat reading the final eval, as his residency matching process was done 6 months ago? If the clerkship director writes a bad eval – it is unlikely that he or she would actually fail Pat on this issue after a “glowing” medical school career and matching to an excellent surgical specialty. Was it just a misunderstanding? Even if the clerkship director asks Pat to repeat some shifts at this point, it actually would mean delay of graduation because there is no further time to repeat shifts. Has Pat done anything to make him miss convocation with his family, classmates, and friends? Maybe he deserves that if he has been intentionally unprofessional, but Pat knows it’s highly unlikely it will go that far, and everything can be ascribed to a simple misunderstanding. So he smiles and hands over the daily eval, and knows Tom won’t call his bluff. If he does, Pat knows the cards he will lay down. I have had this situation come up in a student with multiple offences and I sat in the room with all the Deans and the student to discuss this issue of last minute unprofessionalism. The solution was to write a letter to his residency program director in his accepted residency program, so to watch for similar issues of professionalism. We allowed him to graduate with his class.

  • Daren

    Hi all, good case and good comments. Everyone should be given the benefit of the doubt, and deserve a fair process. We should assume the best in people. However, I will play devil’s advocate for sake of discussion. If the train has left the station, then is Patrick playing chicken with it? If Tom calls Pat on the problem, Pat may be able to say he had no idea – he was just helping out with the tough cases and trying to make his shift applicable to him. He was being self-directed. He lost track of Dr. Pam and Tom in the chaos, and after all, that first ER patient, was HIS patient and he wanted to follow him. (On the back of the daily eval, there should be a log of patients seen which can help Dr. Pam and Tom in the future.) Most people in Tom’s position would back down and at worst write “meets expectations” – and move on (it’s a busy day in the ED!). If Tom does the unlikely thing and writes a really bad daily eval, would the comment even make it to the final eval that goes on his transcript? That final is written by the clerkship director (or similar student advisor) who has to sort through a bunch of evals and might see Tom’s as an outlier (but maybe there were other bad shifts just no other bad evals). Is there anyone who matters to Pat reading the final eval, as his residency matching process was done 6 months ago? If the clerkship director writes a bad eval – it is unlikely that he or she would actually fail Pat on this issue after a “glowing” medical school career and matching to an excellent surgical specialty. Was it just a misunderstanding? Even if the clerkship director asks Pat to repeat some shifts at this point, it actually would mean delay of graduation because there is no further time to repeat shifts. Has Pat done anything to make him miss convocation with his family, classmates, and friends? Maybe he deserves that if he has been intentionally unprofessional, but Pat knows it’s highly unlikely it will go that far, and everything can be ascribed to a simple misunderstanding. So he smiles and hands over the daily eval, and knows Tom won’t call his bluff. If he does, Pat knows the cards he will lay down. I have had this situation come up in a student with multiple offences and I sat in the room with all the Deans and the student to discuss this issue of last minute unprofessionalism. The solution was to write a letter to his residency program director in his accepted residency program, so to watch for similar issues of professionalism. We allowed him to graduate with his class.

    • Brent Thoma

      Daren,
      This is a great outline of the many issues with evaluation. I think, as you noted, that many of them seem to stem from many of us being unwilling to provide negative feedback for the many reasons outlined in these comments. Any ideas for solutions to this?
      Thanks for your comment!

      • Daren

        All our evals are exceptionally “right-shifted”, likely more so than any other field. When you give a student a mark of 8/10 on history and physical, she might look angry or upset. When you write a letter of recommendation and write “this student is well above average and would make a good addition to your residency program”; you have doomed him to an unmatched position and ruined his hopes and dreams forever.

        It would require a fundamental change, but to answer your question with one of many possible solutions, we would need to track each faculty’s evaluations and give them personal feedback as to the distribution of their ratings of their students over time. If a faculty member’s last ten students all received “top 5% of students in my career” – it is valuable feedback and accountability to the program and to the faculty member. Comparison of an individual’s eval metrics to all the evals in the department are important – since it is statistically possible to get a run of exceptional students once in a while.

        It would require buy-in from the entire undergraduate program and postgraduate program, as well as clear communication to all external program directors (perhaps by way of a letter from the Dean with every CaRMS application). This culture of having most of a department’s evals fall close to the mean, would bring well-deserved value to the exceptional student’s performance, while creating an atmosphere which allows discussion of areas for improvement. Feedback means something. A good eval is earned and not expected.

        • Brent Thoma

          Great answer, Daren. Thanks!

          Also, I think you might be interested in this blog post and video that I published on ALiEM last year. It describes the implementation of some of the things that you are discussing within the context of a PGME anesthesia program. They z-scored their faculty eval’s to standardize the influence across both ‘hawk’ and ‘dove’ evaluators (the latter of which are much more prominent!):

          https://www.aliem.com/assessment-in-medical-education/

          The data collection and analysis that are required are complex and multiple small formative/summative points of evaluation are required, but I suspect that this might be where we’re headed with CBME.

          Really appreciate your comments!

  • Pik Mukherji

    Agree with Darren’s comments and the logistics of the eval. As far as Dr. Pam actually giving this feedback and doing the fact-finding: We teach the SFED model for Feedback in our teaching course-

    Self-Assess: (see where your learner is) How’d it do? How do you think you did? What could have gone better?

    Feedback: (Give objective data points) So you had one case with Tom today, an I+D, and one with me yesterday. Typically rotating 4th yrs will see 4-10 patients over the course of a shift….PAUSE

    -this space for response and bidirectional conversation to get Patrick’s side of the story-

    Encourage: I think we can improve on these things. You’ve done well in challenging rotations before, so I’m sure you’ll rise to the occasion on your next shift.

    Direction: (Coaching and specifics) Let’s see who you’re working with next shift. I’ll go over the plan for you to see at least 4 pts. primarily and present them to the attending. We’ll have a much better idea of where you are then.

    OR possibly in this case-
    Encourage: It looks like you’re set in a Surgery career and weren’t trying to game the system. We’ll write a letter rather than our SLOE that can be honest without noting your performance and EM skills.

    OR even, if Patrick is less than forthright, right to Direction: We’ll see how your last shift goes and write you a letter of eval rather than our standard SLOE, but also a note to the Dean. You’ll understand that if this wasn’t an isolated incident and you typically engage in absenteeism, then it’s something your school should know about.

  • Nadim Lalani

    Q1: every ED I’ve worked in has a docs room that comes with a door. I give feedback (good or bad) in private. One should try and always be a pro. One should always have a purpose to these conversations….preface venting by saying that you are venting and that the conversation should stay private. An attending should be careful about potentially placing a resident in the awkward position of listening to them vent about another learner.

    Q2: Tom shouldn’t be afraid to be honest. He only saw one interaction – the I & D. That should be his basis for his evaluation period. It’s important to keep it objective. It is also important that learners have objectives | milestones that they are DIRECTLY observed achieving. Learners today more than ever need to take ownership and direction for their learning.

    Q3: I always “diagnose the learner” at the beginning of the shift. I get a sense of their learning needs, but let’s not forget that learners are responsible for ALL the CanMeds competencies – not just ‘medical expert’ (e.g. collaborator | manager etc) and so it doesn’t matter if the learner rotating through is an EM gunner or not – they have much to learn from me about being a physician in 2016. Lastly I always tell learners that they are going to get evaluated on their “ABC’s – Attitude, Behaviour and Clinical skills” – so there shouldn’t be any surprises when I give negative feedback regarding attitude.
    back to the case – Maybe Patrick’s enthusiasm needs more direction. Maybe previous attendings let him do whatever [it wouldn’t be the first time in history faculty give mixed messages). I feel Dr. Pam let herself down venting about Patrick and got caught. So now she needs to eat a bit of crow, apologise and invite Patrick in to sit down and have a courageous conversation. Get his side, explain hers and then try and come to a solution. thanks
    NL