If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Address correspondence to Daniel P. Berthold, M.D., Department of Orthopaedic Sports Medicine, Technical University of Munich, Ismaninger Str. 22, 81675 Munich, Germany.
To conduct a systematic review to determine the efficacy of head-mounted display (HMD) virtual reality (VR) in orthopaedic surgical training.
Methods
A thorough search was conducted on PubMed for articles published between January 2000 and August 2020. Studies were included if they (1) concerned orthopaedic surgery, (2) dealt with an HMD VR device, (3) the technology was being used for training purposes, and (4) was a randomized control trial (RCT).
Results
Eight articles met the inclusion criteria. Analysis of the 8 RCTs reveals 6 of the 8 demonstrating HMD VR to be a superior training method to traditional-based training modules. However, in the remaining 2 articles, authors found no significant difference between the VR group and controls, but showed at least equivalent ability to train novice surgeons.
Conclusions
RCTs show promising evidence that HMD VR is an efficacious tool in surgical training for orthopaedic procedures, with most randomized clinical trials showing improvement in novice students/surgeons compared with controls.
Clinical Relevance
As VR technology advances, so must the research directed at determining the efficacy of such technologies at educating our novice surgeons. RCTs are already demonstrating the role HMD VR can play in the education of novice orthopaedic surgeons.
The paradigm for surgical education and training has remained relatively unchanged for more than a century, with the surgical residency training program proposed by Dr. William Halsted leading the way. Halsted’s traditional method of “See One, Do One, Teach One” has more recently come under scrutiny, as there are growing concerns for patient safety and recent reductions in resident hours.
In 2003 and 2011, there were movements to add restrictions on the number of hours per week, maximum shift hours, and mandated minimum time off between shifts imposed by the Accreditation Council for Graduate medical Education.
showed that patient outcomes have not been effected by more flexible resident hours, the reality of it is that resident hour restrictions limit their time in the operating room and diminishes the feasibility of the Halstedian model of surgical education. These restrictions drive the need for surgical simulation, as residents are spending less time in the operating room. Surgical simulation is now more important than ever.
Classical surgical simulation has consisted of cadaveric models; however, technology has continued to improve and virtual reality (VR) is becoming more prevalent within orthopaedic surgical education. VR consists of creating a completely simulated experience outside of the real world. This may include the use of computer screens and monitors, as well as head-mounted displays (HMDs). This training technique has been implemented within orthopaedic surgical education/training in a variety of ways. This technology has been embraced, to some extent, in various fields within orthopaedics. To date, there is limited literature on the vast applications of the newer HMD VR systems within orthopaedic surgical education as a whole. As the HMD VR systems are the next step in virtual reality, even outside of the scope of surgical training, the idea of VR is very exciting to many in the field, but questions still remain regarding its applications and limitations.
The purpose of this study was to conduct a systematic review to determine the efficacy of HMD VR in orthopaedic surgical training. The authors hypothesized that there would be relatively limited randomized control trials (RCTs) available, however, within this small sample size, HMD VR would prove to be a useful tool in orthopaedic education.
Methods
Study Enrollment
The study was performed at the University of Connecticut Health Center/UConn Musculoskeletal Institute
Ethical Approval
Ethical approval was obtained via Human Research Determination Form to the institutional review board of the University of Connecticut and it was documented that no institutional review board approval was required.
Identification of Studies
A comprehensive search was conducted on an electronic database (PubMed) for articles published between January 1, 2000, and August 11, 2020, according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines by 2 independent reviewers (K.Q., D.P.B.) The specific search term used during this search was generated using the following key words: orthopedics, virtual reality, head-mounted display, surgical training, and surgical education. Using these key words, the following search term was used:
“(Orthopedics) AND (Virtual Reality OR Head-Mounted Device) AND (Surgical Training OR Surgical Education)”
This search term was intended to be general, to capture all relevant literature pertaining to HMD VR within orthopaedic surgical training/education. Studies found in this search were included if they (1) concerned orthopaedic surgery, (2) dealt with a HMD virtual reality device, (3) the technology was being used for training purposes, and (4) was specifically an RCT. The restrictions on article type were that the full-text article must be available in English, and the article must be a RCT. Discrepancies were resolved through consensus.
Results
Search Results
The search strategy resulted in 299 articles. No duplicates were identified in this search. Of the 299 articles found in the initial search, 265 were excluded when article type (“RCT”) filter was applied, leaving 34 articles for screening (Fig 1). During the title/abstract screening process, an additional 11 articles were excluded, as they were found not to deal with orthopaedic training/education (Fig 1). This left 23 articles to be screened via the full text. During the full-text review of the remaining 23 articles, 14 were excluded due to non-HMD VR devices used in these studies, and 1 was not available for access (Fig 1). Ultimately, this left 8 articles to be included in this review.
Fig 1Flowchart displaying exclusion and inclusion criteria (PRISMA flow diagram). Ultimately, 8 articles were included in this review. (HMD, head-mounted display; VR, virtual reality.)
All articles found were published between 2019 and 2022, pointing to the innovative nature of the HMD VR technology. While only 8 articles fit the inclusion criteria for this study, there remained a range of procedural variety within the final articles. The orthopaedic cases that were assessed across the 8 articles include total hip arthroplasty (THA), reverse shoulder arthroplasty, pedicle screw placement, tibial intramedullary nail placement, glenoid exposure, and unicompartmental knee arthroplasty (UKA). All studies dealt with a VR group and control group partaking in traditional training methods. All studies also used HMD VR technology, although there was some variation in the software used depending on the particular study.
The general development of these technologies follows a fairly standardized sequence. Once the concept or device is developed, face/content validity must be established with expert insight and analysis. Next, construct validity must be established, showing that skills on the simulator corresponds to real world skill (i.e., novice performs worse than expert). Finally, one must show transfer validity and equal or improved efficacy as a training tool from traditional methods. This final step is accomplished through randomized control studies with a pre- and postsimulation assessment. This final step is would be the primary focus of the vast majority of the studies discussed here; however, one study did focus gather data focusing in on face, content, and construct validity.
conducted an RCT consisting of 24 novice, orthopaedic surgical trainees. The focus of this study was on a HMD with handheld controllers, specifically HTC Vive System (Taipei, Taiwan) running THA VR simulation v1.1 (Pixelmolkerei, Chur, Switzerland). This study consisted of 2 groups, a VR-trained group and a control group being trained with conventional preparatory materials. Both groups were given a 6-week curriculum using their respective training technique. Postcurriculum assessment was done using a cadaveric THA, with the primary outcome being the subjects’ procedure-based assessment (PBA). The PBA is a compulsory and widely used objective structured assessment tool used in the United Kingdom, and subjects were given a score based on their performance on postcurriculum procedure.
The study also had secondary outcomes, including completion of task-specific checklist, degree of error in acetabular component orientation, and procedure duration. This group found VR curriculum to be overall superior to traditional preparatory materials. The VR group had a mean PBA of 3b (ranging from 3a-4a), as compared with the control group mean PBA of 2a (ranging from 1b-2b). According to the PBA global summary score, 3b is described as “procedure performed competently without guidance or intervention but lacked fluency,” whereas a PBA score of 2a is described as “guidance required for most/all of the procedure (or part performed).” This difference among the groups was statistically significant, with a P < .001. Secondary outcomes were in favor of VR as well. Task-specific checklist assessment resulted in VR subjects completing an average of 23 key steps compared with 12 in the control (P < .001). The acetabular component implant was placed with 12 degrees greater accuracy in the VR group when compared with the control. Finally, operative time was faster in VR group (42-minute average) relative to control (51-minute average) with a P < .03.
conducted a similar study in 2019 looking at the THA procedure and HMD VR as a possible training tool. This study consisted of 14 postgraduate year 1 orthopaedic residents. This group also used HMD with handheld controllers, but used the Oculus Rift CV1 (Menlo Park, CA) hardware and VR-THA Simulation, ORamaVR (Heraklion, Crete, Greece) software. This study consisted of 2 groups, VR trained serving as an experimental group and a control group. All subjects completed a pre-VR assessment on a cadaveric model, which was graded using a “novel checklist” to establish a baseline score, as well as a procedural knowledge test. The subjects were then randomized to a VR group plus standard study materials or a standard study materials-only group. The VR group then completed 2 virtual THA procedures in a 2-week period. All subjects were then reassessed on cadaveric model and procedural knowledge two weeks after the first procedure. The primary outcomes in this study were the results of the cadaveric model assessment and the procedural knowledge test results, comparing the pre- and post-VR (and study time) 2 weeks apart. The results of this study demonstrated the VR group improved their cadaveric score by 18 points, which was statistically significant at a P = .48; however, this required an adjustment based on grader strictness. The VR cohort demonstrated greater improvement in procedural steps, technical performance, visuospatial skills, efficiency, and flow, however, only the improvement in technical performance was statistically significant (P = .009). The VR group also showed greater improvement on the procedural knowledge quiz; however, this was not statistically significant. Overall, this group concluded that the HDM VR simulation is worth developing as a tool for resident education.
Moving into 2020, there were 5 studies published that were included in this review. The first by Lohre et al.
was conducted in Vancouver, Canada, with a sample of 19 senior orthopaedic surgical residents. This group focused on reverse shoulder arthroplasty, using the PrecisionOS platform version 3.0 (Vancouver, Canada). Two cohorts were randomized—a VR group and a control group that was given instructional videos as preparatory material. Both groups were assessed using a cadaveric model and given a spoken knowledge test after the assigned intervention period process. The Objective Structured Assessment of Technical Skills (OSATS) score was used as the primary outcome for participants. Secondary outcomes included error rate and procedure duration. This study also dove deeper into training validity measures, such as the Global Rating Scale, transfer of training, transfer effectiveness ratio, and cost-effectiveness ratio. The mean cumulative OSATS score of the VR group (15.9) was greater than that of the control group (9.4) with statistical significance (P < .001). Furthermore, this group demonstrated a decrease in error rate among the VR group as compared with the control group (15% vs 65%, respectively, P < .001). The VR group also completed the procedure, on average, faster than those in the control group; however, no statistical significance was shown. This group concluded that training complex procedural skills and critical steps in a HDM VR simulation was superior to technical video training.
also published in 2020, focusing on pedicle screw placement. This group used the Immersive Virtual Reality Surgical Simulator (IVRSS), which consisted of an HMD with specialized nailing equipment, including simulated force feedback, as controllers. Commercial software, UG NX8.0 (Siemens, Munich, Germany), also was used. This group differed from all other studies in this review, as they used attending spinal surgeons (all <1 year as attending) as their study subjects. The sample size was 24, with 12 surgeons randomized to an IVRSS group and 12 randomized to a control group using a conventional model of observing a spinal model first, followed by teaching videos of spinal surgery. Both groups underwent baseline assessment of nailing skills, followed by designated intervention period, and finally were reassessed on nailing skills. Primary outcomes included success rate and accuracy rate of pedicle screw placement on post-intervention assessment. VR-trained surgeons had an 82.9% success rate, as compared with the control group success rate of 74.2% (P < .05). Accuracy rate followed a similar trend, with the VR-trained group showing a rate of 69.6% and control showing a rate of 55.4% (P < .05). Furthermore, the VR group improved from their baseline score, with a baseline success rate average of 69.2% improving to 82.9% after IVRSS training. Note that this improvement was seen specifically within a group of surgeons with a certain level of experience. This study concluded the data suggest that IVRSS is an effective tool in improving the skills of young surgeons, with a certain clinical value.
conducted a study with first- and second-year medical students as participants. The total sample size in this study was 25 individuals, and the focus was on tibial intramedullary nail placement. This group used an HMD with handheld controllers and OssoVR (Palo Alto, CA) software. Unlike other studies in this review, this group chose to have 3 separate cohorts—a VR group (8), a VR plus technique guide group (9), and a technique guide only (8, control) group. The groups involved in VR simulation were allowed 3 separate sessions using the tool, and the groups involved in the technique guide were allowed to prepare as they desired. After 10 to 14 days, all individuals were assessed on their ability to insert a tibial intramedullary nail on a compact bone model, which was without surrounding soft tissue. The primary outcome in this study was completion of the compact bone model task. Secondary outcomes also were considered, including proportion of incorrect steps, number of hints requested, and the mean time to completion of the procedure. The results of this study found that the VR groups were superior to the technique guide only group. There was no statistically significant difference between the 2 VR groups (VR with/without technique guide). The VR only group saw 6 of 8 participants complete the task, whereas 7 of the 8 in the VR plus technique guide completed the task. This was compared to the technique guide only group where only 2 of 8 were able to complete the task (VR groups to technique guide only, P = .01). There was statistical significance in the mean number of errors made by VR groups and control, with VR only averaging 3.2, VR plus technique guide averaging 3.1, and the technique guide only averaging 5.7 (P = .02). Finally, there was statistical significance in the mean completion time, with the VR plus technique guide requiring an average of 18 minutes, VR only requiring 19 minutes, and the control requiring 24 minutes (P = .03). While the VR groups on average requested fewer hints during the procedure, there was no statistical significance to support this difference. This group concluded that virtual reality, specifically the HMD used, has potential to be used in surgical residency as a tool to catalyze learning.
conducted another study, similar to their previously described work, in 2020. This second study consisted of 19 orthopaedic residents (resident group) and 7 consultant shoulder arthroplasty surgeons (expert group). The addition of the expert group in this study allows for increased ability to demonstrate face and content validity, as well as the new opportunity to demonstrate construct validity, while still maintaining a structured RCT to address transfer validity and efficacy. Similar to their previous study, this group was using PrecisionOS Technology (Vancouver, Canada) software, specifically Glenoid Exposure Module, version 1.4. Both residents and expert surgeons were randomized into either a VR group (12) or a control group (11), with the control group gaining access to a comprehensive technical journal outlining steps for achieving glenoid exposure in shoulder arthroplasty, rather than training on VR simulation. After the given intervention period, both groups completed glenoid exposure on a cadaveric model, completed an 8-question knowledge test, and subjective Likert-scale questionnaire about their experience. Keeping with their previous study design, primary outcomes included OSATS score and time to completion on cadaveric model, and knowledge score. Secondary outcomes included face, content, construct, and transfer validity. The results of this study found no statistical difference in composite OSATS score between the VR and control groups. Statistical significance was seen specifically in the OSATS score regarding instrument handling, with the VR group scoring on average 3.25 and control scoring 3.0 (P = .03). There was also no statistical significance in the difference between knowledge scores of the 2 groups. However, the completion time was significantly shorter in the VR group relative to the control; on average, the groups spent 14 and 21 minutes, respectively (P = .04). Face, content, construct, and transfer validity of the used HMD simulation was established through this study. Authors concluded HMD VR simulation was superior to traditional model in time to completion and instrument handling, and was equivalent in teaching nontechnical skills, all with improved efficiency.
The final study published in 2020 came from Blumstein et al.
This group recruited 20 first- and second-year medical students to serve as the study subjects. This study focused on tibial intramedullary nail placement, similar to a previously described study. However, this group used an Oculus Rift VR headset with Oculus Touch motion controllers, running OssoVR software. Students were randomized into 2 groups—a VR group (10) and a technique guide or control group (10). Both groups were given a maximum of 20 minutes with their respective interventions, after which they were assessed on a SawBones (Vashon, WA) tibial model. Students were assessed on the SawBones model by blinded experts using the Global Assessment 5-Point Rating Scale and a Procedure-Specific Checklist, both of which served as the primary outcomes of this study. Duration of the assessment procedure served as a secondary outcome. This study found that the VR group completed 38% more steps correctly on the procedure-specific checklist (VR: 63% correct steps, compared with control: 25% correct steps, P = .003). The VR group performed superiorly in all categories of the Global Assessment 5-Point Rating Scale, with statistical significance. The average aggregate score of the VR group was 17.5, compared with the control group’s score of 7.5 (P = .0004). Finally, the VR group completed the task on average 147 seconds faster than the control group, averaging 615 seconds and 762 seconds, respectively (P = .002). These results allowed the group to conclude that HMD VR simulation may serve as a useful tool in resident education.
The final study found in this review was published in 2022 by Zaid et al.
This group’s primary focus was on UKA training on a HMD VR simulation. Orthopaedic residents (20) and fourth-year medical students (2) were recruited as study subjects. This study used a VR headset with handheld controllers, operating OssoVR. Subjects were randomized to either a VR intervention (11) or control intervention (11). The control group had access to a technical guide and video demonstrations of UKA. Both groups were given 45 minutes of training with their assigned intervention, followed by assessment on SawBones model. SawBones assessment was carried out by blinded fellowship-trained arthroplasty surgeons, using OSATS scoring as the evaluation metric. OSATS scores served as the primary outcome. Secondary outcomes included a postassessment survey collecting subjective data about user experience in their respective group, and time to completion of SawBones model. This group found no statistical significance between the VR and control group with respect to OSATS score and time to completion. However, 77% of participants reported VR as a useful tool for resident education, and 86.4% reported a likeliness to use VR for case preparation if available. The postassessment survey showed improvement in participant confidence in performing an independent UKA in both groups. This group concluded that HMD VR simulation was equivalent with regards to SawBones surgical competency, as compared with traditional training methods, with the vast majority of participants finding the VR simulation to be a useful tool.
Discussion
This review yielded 8 well-designed RCTs. Although the sample size is small, there is evidence to suggest that HMD VR is an effective tool in orthopaedic training. VR is growing field in the world of technology. At the forefront of VR technology is the emergence of HMD hardware, with innovative software to accompany it. Various forms of virtual simulation has been shown to be efficacious tools in surgical training.
Training and transfer effect of FluoroSim, an augmented reality fluoroscopic simulator for dynamic hip screw guidewire insertion: A single-blinded randomized controlled trial.
Traditionally, these simulators have a virtual component that involves a monitor or some form of external display that does away with the tested and true cadaveric models. What these tested virtual devices lack is the immersive, “life-like” experience that the HMD devices can offer. The surgical field as a whole has always adapted and grown with advancing technology, and the emergence of HMD simulation should not be the exception. While HMD is still in its infancy as a technology, with improved software being developed constantly, there is evidence to point to its use in orthopaedic surgical education. These 8 RCTs are some of the first to objectively look at HMD simulation and compare it with more traditional models of surgical training or preparation—and the results are very promising.
Early studies are showing that there is equivalent efficacy in HMD VR technology, with a majority of them pointing to superior efficacy. Trainees with access to these immersive environments are able to complete procedure after procedure without limitation based on supplies or the need to preserve cadaveric models. That is not to say that HMD VR simulation is without its cost burden to any institution that were to use this technology. Having said that, cost analysis is an important area that requires more analysis and out of the scope of this review. However, when considering the idea of HMD, it is clear that the technology could allow for smaller institutions, without an established bioskills lab, access to highly effective and beneficial simulation even in limited space and time. At its core, HMD is without a doubt, an efficient and helpful tool in helping residents and students learn the procedural components of a surgery. By placing the learner in a virtual operating room, with the same tools they would come across in the real world, there is an unique opportunity to become accustomed to the individual steps including the surgical instruments required for a given step.
While this review finds promise in early studies, there is need for further study into the efficacy of HMD VR simulation in orthopaedic surgical training. Of course, continued investigation into how HMD stands up to the traditional cadaveric model, which has been held as the gold standard of simulation for many decades, is needed. Furthermore, additional comparison is required to challenge the older forms of VR simulation (monitor, computer-based, etc.); to go beyond showing equivalence or superiority to cadaveric models, and establish HMD superiority in the VR-specific field.
Limitations
There are a few limitations to this systematic review. First, PubMed was the primary database used, while there is generally consistency across platforms, other databases may have more literature meeting the inclusion criteria of this study. Second, this study focused specifically on RCTs. While this is generally the final step in determining efficacy of a training tool, there is further discussion to be had regarding the face, content, and construct validity of these tools, which was not discussed in this review (other than Lohre et al.
). Generally, if RCTs are being done on a particular device, these other validities have already been established.
Conclusions
The interest in VR is growing within the realm of surgical education and training. Many VR devices have been proven as effective training tools in surgical training; however, as VR technology grows, the surgical training paradigm must grow with it. This review illuminates the next phase of VR platforms in orthopaedic training—HMD. RCTs show promising evidence that HMD VR is an efficacious tool in surgical training for orthopaedic procedures, with most randomized clinical trials showing improvement in novice students/surgeons compared with controls.
Training and transfer effect of FluoroSim, an augmented reality fluoroscopic simulator for dynamic hip screw guidewire insertion: A single-blinded randomized controlled trial.
The authors report the following potential conflicts of interest or sources of funding: A.D.M. is a consultant for OrthoFix and Arthrex and receives research grants from Arthrex. M.P.C. receives personal fees from AANA. S.S. is a consultant for Arthrex. The University of Connecticut Health Center/UConn Musculoskeletal Institute has received direct funding and material support from Arthrex. The company and the society had no influence on study design, data collection, or interpretation of the results or the final manuscript. Full ICMJE author disclosure forms are available for this article online, as supplementary material.
00086-4&id=gr1.jpg){kind=link}