X-ray vision, context-sensitive guidance, coordinator, training assistant and more: augmented reality (AR) has hit the ORWhile still in its infancy AR does grow rapidly and has already shown enormous potential.
University Professor Dr Rüdiger von Eisenhart-Rothe, Chair of Orthopaedics and Sports Orthopaedics at the Technical University Munich, explains the advantages of different AR applicationsInterface between the engineers and the surgeons, it is Professor von Eisenhart-Rothe’s task to provide feedback from the physician’s perspective whenever new technology is about to be implemented in his OR so the engineers can tailor their solutions to the actual needs and requirements of the OR staff.
Virtual reality (VR) has become an important element of the training for joint replacement surgeryUsing head-mounted displays OR processes are simulated in an entirely computer-generated environmentAugmented reality in contrast merges reality and virtual reality: virtual content is shown in the surgeon’s field of view to supplement the visible information on the real-world OR tableThus the physician’s eyes can return to the patient rather than having to focus on the screen.
AR is useful for training purposes since a further dimension is added“In VR there is usually no haptic feedbackThe user doesn’t get the physical sensation for example when a saw or a biopsy needle hits a harder object,” Professor von Eisenhart-Rothe reportsIn AR different devices can provide haptic feedback on the interaction with the patient thus, he says, “the additional reaction significantly improves the learning effect.”
Interface between engineer and surgeons
Any AR solution requires a lot of fine-tuning of the hardware and the software before it can be used in the ORThe goggles, for example, need to be lightweight so they don’t become a nuisance during long interventionsImportant questions need to be answered in extensive tests such as the kind of data that is important and the location of the data on the surgeon’s display“Often enough the engineers don’t know what the surgeons really need in the OR – and the surgeons don’t know what’s technically feasibleTherefore it is crucial to have an interface between these two worlds,” Professor von Eisenhart-Rothe describes his task.
In orthopaedic surgery implants of different manufacturers are used and fixed with special toolsHere AR can help: detailed instructions can be shown in the head-mounted display and even a product specialist can be brought in who sees the current situation via a camera and can virtually show the steps or devices neededBut the entire OR team benefits as Professor von Eisenhart-Rothe points out: “The OR nurse will also receive this information and can prepare all necessary instruments,”
X-ray vision for smaller incisions
Superimposing image data from CT or MRI is still being tested and not yet used in the real-world ORA major obstacle is the fact that current hardware does not offer the required computing power“Obviously, it depends which kind of data needs to be displayed in the surgeon’s field of view,” says the orthopaedistShowing vital signs or the tilt of an instrument is not a problemIf however a live CT scan has to be displayed that shows the exact site of a fracture – that’s an entirely different ballgameImmense computing power is required since complex data has to be displayed in real-time with millimetre precision on the patient body“This would give the surgeon x-ray visionSuch functionality is still in the experimental stage since precise matching is a major challenge,” he underlinesIn addition to high-performance computers, specific markers are requiredThey are placed on the patient body to tell the camera where certain parts of the anatomy are located when the patient is moved or when the surgeon changes her viewing angle.
Von Eisenhart-Rothe is sure that the effort will be worthwhile particularly for tumour surgery: “Patients with abdominal tumours often require large incisions for the surgeon to get a good overview of complex anatomyIf the exact site of the pelvis and the tumour were shown in the head-mounted display, much smaller incision could be made.” Combined with intelligent navigation systems ‘collateral damage’ during surgery could be further reduced.
Through the eyes of the surgeon
Another very promising idea according to Professor von Eisenhart-Rothe is “remote sitting-in” which is already being applied in select cases: a surgeon performs an intervention in her field of expertiseThe data goggles allow less experienced colleagues to join“Particularly now – during the pandemic – this has many advantages”, the expert saysBut even in non-Corona times this technology could make long and expensive travel obsoleteMoreover: “In minimally invasive” interventions the incisions are only a few centimetresEven colleagues right at the OR table scarcely see anythingWhen the steps can be observed from the point of view of the surgeon that is much more instructive,” von Eisenhart-Rothe explainsAnd vice versa – an experienced surgeon can be called upon to follow the surgery and support the colleague when needed.
University Professor Dr Rüdiger von Eisenhart-Rothe is Chair for Orthopaedics at the Medical Faculty of the Technical University of Munich and Director of the Clinic and Polyclinic for Orthopedics and Sports Orthopedics at the Rechts der Isar Hospital of the University of Technology MunichHe is Vice President of the German Knee Society, Founding President of the German Hip Society and Member of the Board of the German Society for Joint Replacement (Deutsche Gesellschaft für Endoprothetik)Von Eisenhart-Rothe publishes the journal “Der Orthopäde” and has (co-)authored more than 300 scientific books and papers.