John Dalrymple (center) looks on while Monica Sircar, MD, HMS research fellow in medicine at Mass General, demonstrates an exam skill on a standardized patient to HMS student Gabriella Herrera in the Clinical Skills Center. Image: Gretchen Ertl for HMS
For Jessica Stuart and Kirstin Woody Scott, third-year Harvard Medical School students, one of the great advantages of attending HMS is the wealth of clinical learning opportunities afforded by the School’s renowned affiliated teaching hospitals.
But when HMS students get together to compare notes on their clinical experiences, they do find differences.
“I think there are certainly differences, and I think there are certainly strengths at some of the hospitals that can be applied to some of the others,” said Stuart, who spent her clinical year at Brigham and Women’s Hospital.
To Stuart’s point, HMS would like to ensure that, no matter where students are assigned for their clinical year, there will be comparable learning objectives—ensuring, for example, that all students are equally well prepared for their objective structured clinical examinations (OSCEs).
Both Stuart and Scott agree that the educational programs at HMS teaching hospitals can be strengthened if administrators have the information they need to make decisions about adopting one another’s best practices.
Scott, who did her clinical year at Massachusetts General Hospital and came to HMS with a PhD in health policy from Harvard, noted that there is an abundance of riches in terms of learning environments at HMS, but “without data it’s hard to know where to improve and what’s worth prioritizing.”
New Data Warehouse
Carolyn Wood, director of the School’s new Office of Educational Quality Improvement (OEQI), was brought on board this summer from Harvard's John F. Kennedy School of Government, where she spent 15 years helping to build a teaching and learning center.
One of her tasks at HMS will be to facilitate the flow of information derived from data stored in a new data warehouse now in development, information that can help faculty better evaluate how students are doing in core competencies because they will be able to analyze attributes, performance, outcomes and other indicators which will all be available in one place.
“This is the first time the School is investing in a group approach for gathering data from across the student life cycle and working across the phases of our curriculum to track how our students are doing against learning goals,” said Wood.
HMS has always gathered a broad spectrum of student data, but it has been spread across several different systems. The new data warehouse, being developed by Wood, the Program in Medical Education and the HMS educational IT group, will consolidate these information streams, providing a much more comprehensive picture of student performance. The plan is to have the first phase of the database online by spring of 2018.
“It will enable us to develop dashboards at the press of a button to track how students are doing,” said Wood.
Faculty Development
Having such a data solution will also make it easier to get information to the School’s far-flung faculty, providing them with a sophisticated tool that can help facilitate inquiry, assessment and reflection. Faculty can use the data to refine their curricula and hone their teaching skills.
“[This database] will help [faculty] understand what’s working and what’s not working,” Wood said, adding that it will also create an environment conducive to faculty development.
“We started the OEQI to empower our faculty to manage their courses and clerkships with an evidence-based education philosophy,” said Dean for Medical Education Ed Hundert, the HMS Daniel D. Federman, M.D. Professor in Residence of Global Health and Social Medicine and Medical Education.
“When outcomes data suggest that students in a particular section of a course, or site of a clerkship, are not learning some core competency as well as they should, we can send people to that faculty group to tailor a faculty development program to empower them and enable them to teach those particular skills and competencies more effectively. And so, it closes the loop between the teaching and the learning in an evidence-based way,” Hundert said.
This data consolidation approach is a new trend across medical schools, he said, adding that faculty are enthusiastic about soon getting comprehensive quarterly outcomes reports.
“Faculty are always wanting to know how our students are doing, in terms of learning and acquisition of knowledge and skill,” said John Dalrymple, Director of Assessment, Evaluation and Quality Improvement and the Dr. Mark and Karen Lawrence Director of Professionalism and Humanism in Medicine.
Dalrymple is also an associate professor of obstetrics, gynecology and reproductive biology at Beth Israel Deaconess Medical Center as well as faculty lead on the new initiative, which the School hopes will foster an environment of continuous quality improvement in teaching and learning.
“Assessment and evaluation can be challenging for many reasons, but everyone recognizes it is essential to be able to see if you’re achieving what you hope to achieve,” Dalrymple said. “I think the way this is going to benefit the students most is that they are going to be able to see how much they’ve learned and how much they’ve grown in a tangible way.”
Hundert noted that students are now always included as members of the curricular committees that evaluate the new data, making the entire endeavor a true partnership between faculty, students and staff in service to improving the School’s MD program.
“Student representatives will be part of this discussion of looking at the outcomes data and giving feedback on what kinds of faculty development programs would best help improve the quality of their education,” he said.
Bird's Eye View
The new data can also provide a bird’s eye view of the most effective teaching methods across hospital sites and provide benchmarks.
“Comparability of learning experiences is essential, especially for a program like ours,” Dalrymple said, explaining that it’s important to ensure that all HMS students are getting the same high-quality educational experience while also helping faculty know that they’re performing on par with colleagues at other sites.
“We’ll be able to look at student performance data and feedback data across sites,” said Wood. “If there’s a promising model in one of the hospitals … this helps us ID that pocket of excellence and perhaps we can scale it out and replicate it.”
This approach will also encourage innovation within the curriculum, Hundert said, so that when one hospital site discovers an effective way to teach a specific idea or technique and wants to demonstrate how it works, faculty will now have the tools to do so and will be encouraged to publish outcomes and disseminate broadly what they have learned.
“Every continuous quality improvement pilot can become research on medical education,” he said.
“We can share with the rest of the world all of the innovative things that are happening in every learning studio and classroom and at every department and every hospital and every part of the curriculum,” Hundert added.
The beauty of continuous quality improvement is that there is room to customize instruction at individual sites, said Wood. “You can’t use a cookie-cutter model, but at the same time the students need to come out with a comparable set of skills and competencies,” she said. “You want to ensure as much consistency as you can, but at the same time you can’t be too prescriptive. We’re not producing widgets.”
Stuart acknowledges that it will be a “tricky balance,” but she believes it’s an endeavor worth pursuing. Hospitals each have their own way of running programs that might present a barrier, but they will now be able to see how their own data compares to those from other teaching sites, which will be a “motivating factor,” she said.
“I think it’s an amazing idea and it’s going to provide so many opportunities for improvement of the clerkship experience without having to reinvent the wheel,” Stuart said.
For Wood, the new data warehouse is just the beginning of a more comprehensive approach to medical education, but it’s one both she and Hundert believe will have tremendous benefits for both faculty and students for generations to come.
“Excellence in teaching is not something that is ‘one and done.’ It’s a long-term process,” said Wood.
