Publications
All Publications, sorted by years
2022 2021 2020 2019 2018 2017 2015 [All]
Selected Publications
Journal Articles (8)
Workflow Augmentation of Video Data for Event Recognition with Time-Sensitive Neural Networks.
In eprint, 2021

Abstract:
Supervised training of neural networks requires large, diverse and well annotated data sets. In the medical field, this is often difficult to achieve due to constraints in time, expert knowledge and prevalence of an event. Artificial data augmentation can help to prevent overfitting and improve the detection of rare events as well as overall performance. However, most augmentation techniques use purely spatial transformations, which are not sufficient for video data with temporal correlations. In this paper, we present a novel methodology for workflow augmentation and demonstrate its benefit for event recognition in cataract surgery. The proposed approach increases the frequency of event alternation by creating artificial videos. The original video is split into event segments and a workflow graph is extracted from the original annotations. Finally, the segments are assembled into new videos based on the workflow graph. Compared to the original videos, the frequency of event alternation in the augmented cataract surgery videos increased by 26%. Further, a 3% higher classification accuracy and a 7.8% higher precision was achieved compared to a state-of-the-art approach. Our approach is particularly helpful to increase the occurrence of rare but important events and can be applied to a large variety of use cases.
Simulation-Based Estimation of the Number of Cameras Required for 3D Reconstruction in a Narrow-Baseline Multi-Camera Setup.
In Journal of Imaging, vol. 7(5) , pp. 87, 2021

Abstract:
Graphical visualization systems are a common clinical tool for displaying digital images and three-dimensional volumetric data. These systems provide a broad spectrum of information to support physicians in their clinical routine. For example, the field of radiology enjoys unrestricted options for interaction with the data, since information is pre-recorded and available entirely in digital form. However, some fields, such as microsurgery, do not benefit from this yet. Microscopes, endoscopes, and laparoscopes show the surgical site as it is. To allow free data manipulation and information fusion, 3D digitization of surgical sites is required. We aimed to find the number of cameras needed to add this functionality to surgical microscopes. For this, we performed in silico simulations of the 3D reconstruction of representative models of microsurgical sites with different numbers of cameras in narrow-baseline setups. Our results show that eight independent camera views are preferable, while at least four are necessary for a digital surgical site. In most cases, eight cameras allow the reconstruction of over 99% of the visible part. With four cameras, still over 95% can be achieved. This answers one of the key questions for the development of a prototype microscope. In future, such a system can provide functionality which is unattainable today
Mesh structure-independent modeling of patient-specific atrial fiber orientation.
In Current Directions in Biomedical Engineering, vol. 1(1) , pp. 409-412, 2015

Abstract:
The fiber orientation in the atria has a significant contribution to the electrophysiologic behavior of the heart and to the genesis of arrhythmia. Atrial fiber orientation has a direct effect on excitation propagation, activation patterns and the P-wave. We present a rule-based algorithm that works robustly on different volumetric meshes composed of either isotropic hexahedra or arbitrary tetrahedra as well as on 3-dimensional triangular surface meshes in patient-specific geometric models. This method fosters the understanding of general pro-arrhythmic mechanisms and enhances patient-specific modeling approaches.
The Impact of Standard Ablation Strategies for Atrial Fibrillation on Cardiovascular Performance in a Four-chamber Heart Model.
In arXiv, 2022

Abstract:
Atrial fibrillation is one of the most frequent cardiac arrhythmias in the industrialized world and ablation therapy is the method of choice for many patients. However, ablation scars alter the electrophysiological activation and the mechanical behavior of the affected atria. Different ablation strategies with the aim to terminate atrial fibrillation and prevent its recurrence exist but their impact on the hemodynamic performance of the heart has not been investigated thoroughly. In this work, we present a simulation study analyzing five commonly used ablation scar patterns and their combinations in the left atrium regarding their impact on the pumping function of the heart using an electromechanical whole-heart model. We analyzed how the altered atrial activation and increased stiffness due to the ablation scar affect atrial as well as ventricular contraction and relaxation. We found that systolic and diastolic function of the left atrium is impaired by ablation scars and that the reduction of atrial stroke volume of up to 11.43% depends linearly on the amount of inactivated tissue. Consequently, the end-diastolic volume of the left ventricle, and thus stroke volume, was reduced by up to 1.4% and 1.8%, respectively. During ventricular systole, left atrial pressure was increased by up to 20% due to changes in the atrial activation sequence and the stiffening of scar tissue. This study provides biomechanical evidence that atrial ablation has acute effects not only on atrial contraction but also on ventricular pumping function. Our results have the potential to help tailoring ablation strategies towards minimal global hemodynamic impairment.
A statistical shape model for radiation-free assessment and classification of craniosynostosis.
In arXiv, 2022

Abstract:
The assessment of craniofacial deformities requires patient data which is sparsely available. Statistical shape models provide realistic and synthetic data enabling comparisons of existing methods on a common dataset. We build the first publicly available statistical 3D head model of craniosynostosis patients and the first model focusing on infants younger than 1.5 years. For correspondence establishment, we test and evaluate four template morphing approaches. We further present an original, shape-model- based classification approach for craniosynostosis on photogrammetric surface scans. To the best of our knowledge, our study uses the largest dataset of craniosynostosis patients in a classification study for craniosynostosis and statistical shape modeling to date. We demonstrate that our shape model performs similar to other statistical shape models of the human head. Craniosynostosis-specific pathologies are represented in the first eigenmodes of the model. Regarding the automatic classification of craniosynostis, our classification approach yields an accuracy of 97.3 %, comparable to other state-of-the-art methods using both computed tomography scans and stereophotogrammetry. Our publicly available, craniosynostosis-specific statistical shape model enables the assessment of craniosynostosis on realistic and synthetic data. We further present a state-of-the-art shape-model- based classification approach for a radiation-free diagnosis of craniosynostosis.
3D-Guided Face Manipulation of 2D Images for the Prediction of Post-Operative Outcome After Cranio-Maxillofacial Surgery.
In IEEE Transactions on Image Processing, vol. 30, pp. 7349-7363, 2021

Abstract:
Cranio-maxillofacial surgery often alters the aesthetics of the face which can be a heavy burden for patients to decide whether or not to undergo surgery. Today, physicians can predict the post-operative face using surgery planning tools to support the patient’s decision-making. While these planning tools allow a simulation of the post-operative face, the facial texture must usually be captured by another 3D texture scan and subsequently mapped on the simulated face. This approach often results in face predictions that do not appear realistic or lively looking and are therefore ill-suited to guide the patient’s decision-making. Instead, we propose a method using a generative adversarial network to modify a facial image according to a 3D soft-tissue estimation of the post-operative face. To circumvent the lack of available data pairs between pre- and post-operative measurements we propose a semi-supervised training strategy using cycle losses that only requires paired open-source data of images and 3D surfaces of the face’s shape. After training on “in-the-wild” images we show that our model can realistically manipulate local regions of a face in a 2D image based on a modified 3D shape. We then test our model on four clinical examples where we predict the post-operative face according to a 3D soft-tissue prediction of surgery outcome, which was simulated by a surgery planning tool. As a result, we aim to demonstrate the potential of our approach to predict realistic post-operative images of faces without the need of paired clinical data, physical models, or 3D texture scans.
Laplace-Beltrami Refined Shape Regression Applied to Neck Reconstruction for Craniosynostosis Patients.
In Current Directions in Biomedical Engineering, vol. 7(2) , pp. 191-194, 2021

Abstract:
This contribution is part of a project concerning the creation of an artificial dataset comprising 3D head scans of craniosynostosis patients for a deep-learning-based classification. To conform to real data, both head and neck are required in the 3D scans. However, during patient recording, the neck is often covered by medical staff. Simply pasting an arbitrary neck leaves large gaps in the 3D mesh. We therefore use a publicly available statistical shape model (SSM) for neck reconstruction. However, most SSMs of the head are constructed using healthy subjects, so the full head reconstruction loses the craniosynostosis-specific head shape. We propose a method to recover the neck while keeping the pathological head shape intact. We propose a Laplace- Beltrami-based refinement step to deform the posterior mean shape of the full head model towards the pathological head. The artificial neck is created using the publicly available Liverpool-Y ork-Model. W e apply our method to construct artificial necks for head scans of 50 scaphocephaly patients. Our method reduces mean vertex correspondence error by approximately 1.3 mm compared to the ordinary posterior mean shape, preserves the pathological head shape, and creates a continuous transition between neck and head. The presented method showed good results for reconstructing a plausible neck to craniosynostosis patients. Easily generalized it might also be applicable to other pathological shapes.
Electrocardiographic Imaging Using a Spatio-Temporal Basis of Body Surface Potentials—Application to Atrial Ectopic Activity.
In Frontiers in Physiology, vol. 9:1126, 2018

Abstract:
Electrocardiographic imaging (ECGI) strongly relies on a priori assumptions and additional information to overcome ill-posedness. The major challenge of obtaining good reconstructions consists in finding ways to add information that effectively restricts the solution space without violating properties of the sought solution. In this work, we attempt to address this problem by constructing a spatio-temporal basis of body surface potentials (BSP) from simulations of many focal excitations. Measured BSPs are projected onto this basis and reconstructions are expressed as linear combinations of corresponding transmembrane voltage (TMV) basis vectors. The novel method was applied to simulations of 100 atrial ectopic foci with three different conduction velocities. Three signal-to-noise ratios (SNR) and bases of six different temporal lengths were considered. Reconstruction quality was evaluated using the spatial correlation coefficient of TMVs as well as estimated local activation times (LAT). The focus localization error was assessed by computing the geodesic distance between true and reconstructed foci. Compared with an optimally parameterized Tikhonov-Greensite method, the BSP basis reconstruction increased the mean TMV correlation by up to 22, 24, and 32% for an SNR of 40, 20, and 0 dB, respectively. Mean LAT correlation could be improved by up to 5, 7, and 19% for the three SNRs. For 0 dB, the average localization error could be halved from 15.8 to 7.9 mm. For the largest basis length, the localization error was always below 34 mm. In conclusion, the new method improved reconstructions of atrial ectopic activity especially for low SNRs. Localization of ectopic foci turned out to be more robust and more accurate. Preliminary experiments indicate that the basis generalizes to some extent from the training data and may even be applied for reconstruction of non-ectopic activity.
Conference Contributions (4)
Development of a real-time virtual reality environment for visualization of fully digital microscope datasets.
In Proceedings of SPIE, vol. 10868, pp. 108681F1-9, 2019

Abstract:
Current surgical microscope systems have excellent optical properties but still involve some limitations. A future fully digital surgical microscope may overcome some major limitations of typical optomechanical systems, like ergonomic restrictions or limited number of observers. Furthermore, it can leverage and provide the full potential of digital reality. To achieve this, the frontend, the reconstruction of the digital twin of the surgical scenery, as well as the backend, the 3-D visualization interface for the surgeon, need to work in real-time. To investigate the visualization chain, we developed a virtual reality environment allowing pretesting this new form of 3-D data presentation. In this study, we wanted to answer the following question: How must the visualization pipeline look like to achieve a real-time update of the 3-D digital reality scenery. With our current approach, we were able to obtain visualizations with a frame rate of 120 frames per second and a 3-D data update rate of approximately 90 datasets per second. In a further step, a first prototype of a real-time mixed-reality head mounted visualization system could be manufactured based on the knowledge gained during the virtual reality pretesting.
MATLAB Simulation Environment for Estimating the Minimal Number and Positions of Cameras for 3D Surface Reconstruction in a Fully-Digital Surgical Microscope.
In Current Directions in Biomedical Engineering, vol. 4(1) , pp. 517-520, 2018

Abstract:
Contemporary surgical microscope systems have excellent optical properties but some desirable features re- main unavailable. The number of co-observers is currently re- stricted, by spatial and optical limitations, to only two. More- over, ergonomics poses are a problem: Current microscope systems impede free movement and sometimes demand that surgeons take uncomfortable postures over long periods of time. To rectify these issues, some companies developed surgi- cal microscope systems based on a streaming approach. These systems remove some of the limitations. Multi-observer po- sitions, for example, are not independent from each other, for example. In order to overcome the aforementioned limitations, we are currently developing an approach for the next genera- tion of surgical microscope: Namely the fully digital surgi- cal microscope, where the current observation system is re- placed with a camera array, allowing real-time 3D reconstruc- tion of surgical scenes and, consequently, the rendering of al- most unlimited views for multiple observers. These digital mi- croscopes could make the perspective through the microscope unnecessary allowing the surgeon to move freely and work in more comfortable postures. The requirements on the camera array in such a system have to be determined. For this purpose, we propose of estimation the minimal number of cameras and their positions needed for the 3D reconstruction of microsurgi- cal scenes. The method of estimation is based on the require- ments for the 3D reconstruction. Within the MATLAB simu- lation environment, we have developed a 3D model of a mi- crosurgical scene, used for the determination of the number of required cameras. In a next step a small, compact and cost- ef cient s ystem w ith f ew o pto-mechanical c omponents could be manufactured.
Estimation of the interpolation error of a three-step rotation algorithm using recorded images with rotated test pattern as ground truth.
In Current Directions in Biomedical Engineering, vol. 3(2) , pp. 555-558, 2017

Abstract:
Nowadays, the surgical microscope is the goldstandard for microsurgical procedures. Additional functionalities such as surgical navigation, data injection or imageoverlay are providing additional valuable information to the surgeon. For substituting the conventional optical system by a fully-digital multi-camera setup the three dimensional (3D) reconstruction of the scenery in the field of view is required. However, for in camera-based systems, an exact alignment of the cameras is a challenging task. Therefore, a final adjustment through a digital image rotation becomes necessary. Even though the digital rotation is a commonly used procedure, it leads to unavoidable errors because of the discretized grid of the image. Previous research reported in literature has demonstrated that the method of digitally rotating the images combined with the Fourier interpolation delivers the results of best quality. Nevertheless, the performance evaluation of this algorithm was carried out rotating an image in multiple threestep rotations to a total of 90 or 180 degrees and comparing it to the original image rotated in one step. This is a valid approach because a rotation of 90 or 180 degrees does not produce rotation artifacts. In this research project, we verify the performance of the three-step rotation algorithm using recorded images for which the test pattern was rotated as ground truth. A series of photographs with a rotation angle of 3 to 45 degrees was created. The advantage of this setup is that the result of the digital rotation can be directly compared to the recorded image. In addition, with the knowledge obtained about the interpolation error, we can improve pixel matching in the further triangulation used for 3D reconstruction. By doing so, the estimation of the interpolation error helps to reduce the triangulation error.
Design of an experimental four-camera setup for enhanced 3D surface reconstruction in microsurgery.
In Current Directions in Biomedical Engineering, vol. 3(2) , pp. 539542, 2017

Abstract:
Future fully digital surgical visualization systems enable a wide range of new options. Caused by optomechanical limitations a main disadvantage of todays surgical microscopes is their incapability of providing arbitrary perspectives to more than two observers. In a fully digital microscopic system, multiple arbitrary views can be generated from a 3D reconstruction. Modern surgical microscopes allow replacing the eyepieces by cameras in order to record stereoscopic videos. A reconstruction from these videos can only contain the amount of detail the recording camera system gathers from the scene. Therefore, covered surfaces can result in a faulty reconstruction for deviating stereoscopic perspectives. By adding cameras recording the object from different angles, additional information of the scene is acquired, allowing to improve the reconstruction. Our approach is to use a fixed four-camera setup as a front-end system to capture enhanced 3D topography of a pseudo-surgical scene. This experimental setup would provide images for the reconstruction algorithms and generation of multiple observing stereo perspectives. The concept of the designed setup is based on the common main objective (CMO) principle of current surgical microscopes. These systems are well established and optically mature. Furthermore, the CMO principle allows a more compact design and a lowered effort in calibration than cameras with separate optics. Behind the CMO four pupils separate the four channels which are recorded by one camera each. The designed system captures an area of approximately 28mm × 28mm with four cameras. Thus, allowing to process images of 6 different stereo perspectives. In order to verify the setup, it is modelled in silico. It can be used in further studies to test algorithms for 3D reconstruction from up to four perspectives and provide information about the impact of additionally recorded perspectives on the enhancement of a reconstruction.