Michael Barbella, Managing Editor03.24.23
Electronic health records may improve access to patient information, but charting increases the clerical burden on physicians and can contribute to burnout. A new study presented earlier this month at the 2023 American Academy of Orthopaedic Surgeons (AAOS) Annual Meeting analyzed the use of documentation modalities including artificial intelligence (AI)-based virtual scribe services to determine the overall quality and time it takes to capture a patient encounter. While AI proved to be a promising tool, some verification and correction is necessary for accuracy.
"In our practice, we created a task force to better understand and correct physician burnout to study what we know to be the top reason for burnout—patient documentation," said Michael Rivlin, M.D., FAAOS, orthopedic surgeon, Rothman Institute and associate professor, Thomas Jefferson University, both in Philadelphia. "We wanted to look at ways to maximize the physician's workload at the maximum level of their license and remove burdens that can lead to burnout by finding methods to outsource certain tasks, such as documentation, as this can be time consuming and redundant."
Since the quality of AI-generated documentation had not been analyzed, the team compared the quality and time spent documenting common orthopaedic encounters in hand surgery using the following modalities during a patient visit:
"Our physicians who were not involved in the documentation acted out these vignettes and each scenario contained an element of distraction to determine if the AI would be thrown off by various nuances that might occur during a clinical visit—such as a parent and a minor sharing their thoughts, or a patient interjecting a story about a friend's experience with hand surgery in the middle of providing an update on their own surgery," Rivlin explained.
In total, 118 clinical encounters were documented including 30 AI scribe, 30 VRM, 28 transcription service and 30 medical scribe notes. Clinical notes were deemed as acceptable or unacceptable and assigned a letter grade (A, B, C or F) using an eight-point scoring system. Additionally, an attorney reviewed all notes for medical legal risk.
Overall, all modalities performed well with similar documentation outputs between each. Specific findings include:
With more than 39,000 members, AAOS is the world's largest medical association of musculoskeletal specialists. It provides education to help orthopedic surgeons and allied health professionals best treat patients in their daily practices. The AAOS is the source for information on bone and joint conditions, treatments and related musculoskeletal healthcare issues; and it leads the healthcare discussion on advancing quality.
"In our practice, we created a task force to better understand and correct physician burnout to study what we know to be the top reason for burnout—patient documentation," said Michael Rivlin, M.D., FAAOS, orthopedic surgeon, Rothman Institute and associate professor, Thomas Jefferson University, both in Philadelphia. "We wanted to look at ways to maximize the physician's workload at the maximum level of their license and remove burdens that can lead to burnout by finding methods to outsource certain tasks, such as documentation, as this can be time consuming and redundant."
Since the quality of AI-generated documentation had not been analyzed, the team compared the quality and time spent documenting common orthopaedic encounters in hand surgery using the following modalities during a patient visit:
- AI-based virtual scribe service – An AI program runs on a tablet, and everything said in the room is extracted by the machine.
- Medical scribe – A human being who is either physically in the office visit or participates virtually and transcribes the patient encounter.
- Transcription service – The physician uses a dictaphone to record an audio file about the patient visit and sends it to a third-party company that transcribes the dictation.
- Voice recognition mobile (VRM) application – A program available on an electronic medical record platform that types the words based on voice recognition.
"Our physicians who were not involved in the documentation acted out these vignettes and each scenario contained an element of distraction to determine if the AI would be thrown off by various nuances that might occur during a clinical visit—such as a parent and a minor sharing their thoughts, or a patient interjecting a story about a friend's experience with hand surgery in the middle of providing an update on their own surgery," Rivlin explained.
In total, 118 clinical encounters were documented including 30 AI scribe, 30 VRM, 28 transcription service and 30 medical scribe notes. Clinical notes were deemed as acceptable or unacceptable and assigned a letter grade (A, B, C or F) using an eight-point scoring system. Additionally, an attorney reviewed all notes for medical legal risk.
Overall, all modalities performed well with similar documentation outputs between each. Specific findings include:
- The AI scribe scored significantly lower than the other modalities for one specific question: "Is the plan correct?" – whereas AI captured most of the verbal and implied elements of medical documentation; however, plan formulation was at times deficient compared to its human counterpart and manual edit of the plan section was required.
- Documenting clinical encounters through transcription services and voice recognition mobile applications requires substantial time compared to auto-populated AI-based notes. The average time per a note for VRM and the transcription service was 3.48 minutes and 3.22 minutes, respectively.
- AI-based scribe services rely on verbalized narrative throughout the entire encounter for accurate documentation, but some verification and correction are needed unlike a human scribe.
With more than 39,000 members, AAOS is the world's largest medical association of musculoskeletal specialists. It provides education to help orthopedic surgeons and allied health professionals best treat patients in their daily practices. The AAOS is the source for information on bone and joint conditions, treatments and related musculoskeletal healthcare issues; and it leads the healthcare discussion on advancing quality.