Validation of collaborative cyberspace virtual reality oculometry enhanced with near real-time spatial audio
Subjects
Three members from our study group simultaneously explored (Fig. 3) the utility of the proposed VR technology from geographically distant places (Lucerne, Switzerland, Barcelona, Spain, and London, UK).
Hardware
All graders used HMDs, including two motion controllers (HP Reverb G2 VR Headsets Rev. 2, Hewlett-Packard Company, Palo Alto, United States). The VR system was tethered to gaming laptops (two Asus G14 GA401V, Microsoft Windows 10 home × 64, AMD Ryzen 9 4900HS with Radeon Graphics, 3.0 GHz, 8 cores, NVIDIA GeForce RTX 2060 with Max-Q design, 16 GB RAM, and one ASUS Zephyrus GX501GI, Windows 10 Pro × 64, Intel ® core i7-8750H 2.2 GHz 6 cores, 3 NVIDIA GeForce GTX 1080 Max-Q design, 2 GB RAM). The computers were connected to the internet using gigabit Ethernet and were communicating through a relay server. The relay server was running on a virtual host with six virtualized CPUs, each corresponding to an Intel Core CPU (Broadwell, IBRS) running at 2.2 GHz and 8 GB RAM.
Software
The medical visualisation software Specto used in this project was created with the cross-platform Unity Engine (Unity Technologies, San Francisco, USA) and was extended with additional features for this work. The software supports loading any volumetric medical data from standard file formats, such as DICOM or Nearly Raw Raster Data (NRRD). The loaded data is then rendered via a custom ray-marching-based volume renderer, with no manual processing required. Multiple performance optimizations are used, such as empty space skipping45,46 and early ray termination, to achieve the necessary refresh rate of at least 90 Hz and thereby avoid motion sickness. During a collaborative session, several users can connect to a host and receive the loaded volumetric data via an encrypted transmission.
A connection to other users can be established directly via an Internet Protocol (IP) address or indirectly over a relay server. The relay server software provides an end-to-end encrypted room system and makes it possible for the VR host and VR clients to communicate with each other even through firewalls, which are common in a hospital network environment. The relay server is application independent and only forwards the underlying Transmission Control Protocol (TCP) communication. The VR client is free to initiate a separate Transport Layer Security (TLS) cryptographic protocol and negotiate with the VR host to choose the best encryption algorithm supported by both machines. In this architecture the VR applications can communicate through the relay server without running the risk of a compromised relay server getting access to sensitive medical data.
Cyberspace virtual reality environment
The VR software was installed on each computer, which enabled the graders to take part in this study. After all graders had immersed themselves in cyberspace with their HMDs, they joined the VR arena after login into a password-secured environment. The arena consisted of a darkened room with four walls, a ceiling, and a floor. In the middle of the room, one volume-rendered OCT cube (4.5 mm × 3 mm × 1.9 mm) was made available for digital manipulation, which consisted of stacked 261 spectral-domain OCT cross-sectional images from a healthy macula from a right eye (Spectralis, Heidelberg Engineering, Heidelberg, Germany). The OCT scan angle was 30° with a scan pattern of 15° × 10°, interslice distance of 12 µm. Scan-enhanced depth imaging was activated, and each B-scan was averaged for 20 scans using the automatic averaging and tracking feature. This resulted in an image quality of 29 dB.
Consensus and diameter measurements
Before grading, each grader received consensus documentation (Supplementary File Fig. S1). The graders then executed the tasks in a 5-h sequence, which included two 10-min pauses. The session started with an instruction by the most experienced user (PMM), who explained the VR room, the operation of the handles, and the safety measures to the others using seamless spatial-audio utility.
This was followed by consensus diameter measurements on different B-scans: A grader specified a diameter to be measured, which could be located within the retina but could also extend to the choroid. These two landmarks were initially depicted on a B-scan with the yellow marker tips of the handles, whereby the display was performed alternately by different graders. Preferably, the outer borders of vessels were used as the starting point or end point, respectively, as they displayed the highest contrast. The measurement line was visible only to the individual grader. The length of the measurement was also hidden on the measuring monitor. After the first line was measured, the remaining graders were engaged in measuring the identical line between the proposed landmarks. Finally, the values were unlocked on the measurement board with a click from the specific grader, and the values were discussed with each other. By masking the lines, the end points, and the measured values, maximum blinding of the measurements was ensured. The obtained values were locked from any changes and transferred to the host computer for storage as a Microsoft Excel file (Microsoft Corporation, Redmond, United States) for each single diameter with three measurements per identical diameter by clicking on an export button. All three measurements were then cleared to start a new run. This was repeated for each of the 10 consensus measurements. Afterwards, the same procedure was performed for the real-life-time diameter measurements until enough values were collected. All measured values were stored on the host computer and merged into one final file (Supplementary File S1) for further data analysis.
Simulator Sickness Questionnaire and software evaluation
After exposure, the three participants were asked to complete a Simulator Sickness Questionnaire (SSQ)47 to assess nausea, disorientation, and oculomotor disturbances. For this purpose, a custom written script was applied, and the results were evaluated. Additionally, a software usability analysis was conducted to investigate the potential benefits and limits of the proposed application.
Statistical analysis
The summary statistics mean, median, min, max, standard deviation, and coefficient of variation (CoV) were calculated for the recorded length measurements with the library pandas v1.5.2 in Python v3.10.9. For each of the 109 objects, the consensus length was defined as the mean of the three length measurements of graders g1, g2, and g3. The mean absolute differences between all pairs of graders and the consensus were calculated with the library NumPy v1.23.548 yielding a distance matrix.
This distance matrix was used as input for metric multidimensional scaling (MDS). MDS converts distances among data points into positions of those data points in a two-dimensional coordinate system while preserving the distances as best as possible. MDS was performed using library scikit-learn v1.2.049. The MDS plot was centered at the consensus.
All plots were all drawn with the library matplotlib v3.6.250 using a red-green colourblindness-friendly colour-map51.
The influence of the independent variables “grader” and “line colour” on the dependent variable “measured length” was investigated with an analysis of variance (ANOVA). Since the size of the 109 measured objects varied from 0.23 mm to 3.97 mm, “object ID” was included as an additional independent variable. The effect of “object ID” is not of primary interest, but it was included as a covariate to adjust for when investigating the effect of “grader” and “line color.” Based on a quantile–quantile plot, four of the 327 data points were removed from the analysis because they violated the assumption of normally distributed residuals. A Type II sum of squares estimation was used. ANOVA was performed with the package car v3.0.2 using R Statistical Software v3.6.152.
Ethics declarations
Written informed consent was obtained from all graders. In addition, all study participants provided informed consent to publish the information/image(s) in an online open-access publication. All methods were carried out in accordance with relevant guidelines and regulations and all experimental protocols were approved by the local ethics committee (Ethikkommission Nordwest- und Zentralschweiz, ID EKN 2016-01948).
