This PDF file contains the front matter associated with SPIE Proceedings Volume 6509, including the Title Page, Copyright information, Table of Contents, Introduction, and the Conference Committee listing.

Modern medical imaging provides a variety of techniques for the acquisition of multi-modality data. A typical example is the combination of functional and anatomical data from functional Magnetic Resonance Imaging (fMRI) and anatomical MRI measurements. Usually, the data resulting from each of these two methods is transformed to 3D scalar-field representations to facilitate visualization. A common method for the visualization of anatomical/functional multi-modalities combines semi-transparent isosurfaces (SSD, surface shaded display) with other scalar visualization techniques like direct volume rendering (DVR). However, partial occlusion and visual clutter that typically result from the overlay of these traditional 3D scalar-field visualization techniques make it difficult for the user to perceive and recognize visual structures. This paper addresses these perceptual issues by a new visualization approach for anatomical/functional multi-modalities. The idea is to reduce the occlusion effects of an isosurface by replacing its surface representation by a sparser line representation. Those lines are chosen along the principal curvature directions of the isosurface and rendered by a flow visualization method called line integral convolution (LIC). Applying the LIC algorithm results in fine line structures that improve the perception of the isosurface's shape in a way that it is possible to render it with small opacity values. An interactive visualization is achieved by executing the algorithm completely on the graphics processing unit (GPU) of modern graphics hardware. Furthermore, several illumination techniques and image compositing strategies are discussed for emphasizing the isosurface structure. We demonstrate our method for the example of fMRI/MRI measurements, visualizing the spatial relationship between brain activation and brain tissue.

Modern 3D visualization environments for medical image data provide high interactivity and flexibility but depend on the expert knowledge and the experience of the user with respect to the software application. The definition of the visualization parameters is a manual time-consuming process and as a result, inter-patient or inter-study comparisons are extremely difficult. To overcome these drawbacks in case of the analysis and diagnosis of pathologies, standardization of 3D visualization is an important issue. For this purpose automatically generated digital video sequences can be used to convey the most important information contained in the data. In this paper, we present an improvement of our existing web-based service which is now able to calculate the video sequences in much shorter time exploiting the power of a GPU-cluster. The system requires to transfer a medical volume dataset from an arbitrary computer connected via Internet and sends back a number of video files automatically generated with direct volume rendering. To achieve an optimal load balancing of the available resources, the tasks of automatic adjustment of transfer functions, volume rendering, and video encoding are divided into small sub-requests, which are distributed to the different cluster nodes in order to be performed in parallel. An additional preview mode, which renders a number of dedicated frames, provides a direct feedback and quick overview. For the evaluation, we were focusing on the analysis of intracranial aneurysms and were able to show that the system can be successfully applied. Further on, the system was developed in a way that allows easy integration of other analysis tasks.

The Medical Image Processing Group (MIPG) at the University of Pennsylvania has been developing and distributing medical image analysis and visualization software systems for a long period of time. Our most recent system, 3DVIEWNIX, was first released in 1993. Since that time, a number of significant advancements have taken place with regard to computer platforms and operating systems, networking capability, the rise of parallel processing standards, and the development of open source toolkits. The development of CAVASS by our group is the next generation of 3DVIEWNIX. CAVASS will be freely available, open source, and is integrated with toolkits such as ITK and VTK. CAVASS runs on Windows, Unix, and Linux but shares a single code base. Rather than requiring expensive multiprocessor systems, it seamlessly provides for parallel processing via inexpensive COWs (Cluster of Workstations) for more time consuming algorithms. Most importantly, CAVASS is directed at the visualization, processing, and analysis of medical imagery, so support for 3D and higher dimensional medical image data and the efficient implementation of algorithms is given paramount importance. This paper focuses on aspects of visualization. All of the most of the popular modes of visualization including various 2D slice modes, reslicing, MIP, surface rendering, volume rendering, and animation are incorporated into CAVASS.

In this paper we present four visualization modes based on piecewise and global registration. These visualization modes are evaluated by two interventional radiologists. We are using three routinely acquired datasets from three patients who underwent RF Liver Ablation procedure. Piecewise 2D-3D registration is applied to define a subvolume in the preinterventional dataset that compensates for tissue deformation. The liver in the CT-Fluoroscopy (CT-Fluoro) slice is respectively divided into four and six rectangles. Every single rectangle is independently registered and the intersection of the resulting planes, define a subvolume. This subvolume is minimally traversed in an animation sequence. The visualizations are evaluated qualitatively with regard to the information displayed. Two alternative visualization approaches are also evaluated. The first alternative approach is to display axial slices around the global registered slice, since the interventional radiologists are used to evaluate axial slices. The second alternative approach is to define two envelope planes of the previously defined subvolume and display this envelope volume. The evaluation results show that the visualization of the minimum volume comprised in the registered four and six planes is preferred over the axial visualization or the envelope planes since it shows the lesion from different angles and follows the breathing movement. The interventional radiologists also appreciated the facilitated assessment of the neighborhood of the lesion.

As the imaging modalities used in medicine transition to increasingly three-dimensional data the question of how best to interact with and analyze this data becomes ever more pressing. Immersive virtual reality systems seem to hold promise in tackling this, but how individuals learn and interact in these environments is not fully understood. Here we will attempt to show some methods in which user interaction in a virtual reality environment can be visualized and how this can allow us to gain greater insight into the process of interaction/learning in these systems. Also explored is the possibility of using this method to improve understanding and management of ergonomic issues within an interface.

Movement disorders affect over 5,000,000 people in the United States. Contemporary treatment of these diseases involves high-frequency stimulation through deep brain stimulation (DBS). This form of therapy is offered to patients who have begun to see failure with standard medical therapy and also to patients for which medical therapy is poorly effective. A DBS procedure involves the surgical placement, with millimetric accuracy, of an electrode in the proximity of functional areas referred to as targets. Following the surgical procedure, the implant, which is a multi-contact electrode is programmed to alleviate symptoms while minimizing side effects. Surgical placement of the electrode is difficult because targets of interest are poorly visible in current imaging modalities. Consequently, the process of implantation of a DBS electrode is an iterative procedure. An approximate target position is determined pre-operatively from the position of adjacent structures that are visible in MR images. With the patient awake, this position is then adjusted intra-operatively, which is a lengthy process. The post-surgical programming of the stimulator is an equally challenging and time consuming task, with parameter setting combinations exceeding 4000. This paper reports on the status of the Vanderbilt University DBS Project, which involves the development and clinical evaluation of a system designed to facilitate the entire process from the time of planning to the time of programming.

The medialization laryngoplasty is a surgical procedure to improve the voice function of the patient with vocal fold paresis and paralysis. An image guided system for the medialization laryngoplasty will help the surgeons to accurately place the implant and thus reduce the failure rates of the surgery. One of the fundamental challenges in image guided system is to accurately register the preoperative radiological data to the intraoperative anatomical structure of the patient. In this paper, we present a combined surface and fiducial based registration method to register the preoperative 3D CT data to the intraoperative surface of larynx. To accurately model the exposed surface area, a structured light based stereo vision technique is used for the surface reconstruction. We combined the gray code pattern and multi-line shifting to generate the intraoperative surface of the larynx. To register the point clouds from the intraoperative stage to the preoperative 3D CT data, a shape priori based ICP method is proposed to quickly register the two surfaces. The proposed approach is capable of tracking the fiducial markers and reconstructing the surface of larynx with no damage to the anatomical structure. We used off-the-shelf digital cameras, LCD projector and rapid 3D prototyper to develop our experimental system. The final RMS error in the registration is less than 1mm.

We have developed a novel 3D visualization and guidance system for handheld optical imaging devices. In this paper, the system is applied to measurements of breast/cancerous tissue optical properties using a handheld diffuse optical spectroscopy (DOS) instrument. The combined guidance system/DOS instrument becomes particularly useful for monitoring neoadjuvant chemotherapy in breast cancer patients and for longitudinal studies where measurement reproducibility is critical. The system uses relatively inexpensive hardware components and comprises a 6 degrees-of-freedom (DOF) magnetic tracking device including a DC field generator, three sensors, and a PCI card running on a PC workstation. A custom-built virtual environment combined with a well-defined workflow provide the means for image-guided measurements, improved longitudinal studies of breast optical properties, 3D reconstruction of optical properties within the anatomical map, and serial data registration. The DOS instrument characterizes tissue function such as water, lipid and total hemoglobin concentration. The patient lies on her back at a 45-degrees angle. Each spectral measurement requires consistent contact with the skin, and lasts about 5-10 seconds. Therefore a limited number of positions may be studied. In a reference measurement session, the physician acquires surface points on the breast. A Delaunay-based triangulation algorithm is used to build the virtual breast surface from the acquired points. 3D locations of all DOS measurements are recorded. All subsequently acquired surfaces are automatically registered to the reference surface, thus allowing measurement reproducibility through image guidance using the reference measurements.

Fiducial tracking is a widely used method in image guided procedures such as image guided radiosurgery and radiotherapy. Our group has developed a new fiducial identification algorithm, concurrent Viterbi with association (CVA) algorithm, based on a modified Hidden Markov Model (HMM), and reported our initial results previously. In this paper, we present an extensive performance evaluation of this novel algorithm using phantom testing and clinical images acquired during patient treatment. For a common three-fiducial case, the algorithm execution time is less than two seconds. Testing with a collection of images from more than 35 patient treatments, with a total of more than 10000 image pairs, we find that the success rate of the new algorithm is better than 99%. In the tracking test using a phantom, the phantom is moved to a variety of positions with translations up to 8 mm and rotations up to 4 degree. The new algorithm correctly tracks the phantom motion, with an average translation error of less than 0.5 mm and rotation error less than 0.5 degrees. These results demonstrate that the new algorithm is very efficient, robust, easy to use, and capable of tracking fiducials in a large region of interest (ROI) at a very high success rate with high accuracy.

For quantitative C-arm fluoroscopy, we had earlier proposed a unified mathematical framework to tackle the issues of pose estimation, correspondence and reconstruction, without the use of external trackers. The method used randomly distributed unknown points in the imaging volume, either naturally present or induced by placing beads on the patient. These points were then inputted to an algorithm that computed the 3D reconstruction. The algorithm had an 8° region of convergence, which in general could be considered sufficient for most applications. Here, we extend the earlier algorithm to make it more robust and clinically acceptable. We propose the use of a circle/ellipse, naturally found in many images. We show that the projection of elliptic curves constrain 5 out of the 6 degrees of freedom of the C-arm pose. To completely recover the true C-arm pose, we use constraints in the form of point correspondences between the images. We provide an algorithm to easily obtain a virtual correspondence across all the images and show that two correspondences can recover the true pose 95% of the time when the seeds employed are separated by a distance of 40 mm. or greater. Phantom experiments across three images indicate a pose estimation accuracy of 1.7° using an ellipse and two sufficiently separated point correspondences. Average execution time in this case is 130 seconds. The method appears to be suffciently accurate for clinical applications and does not require any significant modification of clinical protocol.

A novel omnidirectional endoscope which covers a field-of-view of ±135° away from the optical axis and 360° panoramically (3π steradians) can significantly improve the visual reality for in-vivo minimally invasive surgery and diagnostics. The inventive integration of a wide angle objective lens and catadioptric optics provides an omnidirectional viewing angle without severe optical distortion. Optical fibers/LEDs are used for illumination of the entire field-of-view. The omnidirectional viewing capability of this endoscope enables the user to visualize and relate positions in the entire operating field eliminating the need for registration when using multiple scopes. It also prevents repetitive insertions of conventional endoscopes with different direction of view and reduces the risk of misguidance due to the limited field-of-view of conventional endoscopes.

Minimally invasive laparoscopic surgeries are known to lead to improved outcomes, less scarring, and significantly faster patient recovery as compared with conventional open invasive surgeries. Laparoscopes, used to visualize internal anatomy and guide laparoscopic surgeries, however, remain limited in visualization capability. Not only do they provide a relatively flat representation of the three-dimensional (3D) anatomy, they show only the exposed surfaces. A surgeon is thus unable to see inside a structure, which limits the precision of current-generation minimally invasive surgeries and is often a source of complications. To see inside a structure before dissecting it has been a long-standing need in minimally invasive laparoscopic surgeries, a need that laparoscopy is fundamentally limited in meeting. In this work we propose to use continuous computed tomography (CT) of the surgical field as a supplementary imaging tool to guide laparoscopic surgeries. The recent emergence of 64-slice CT and its continuing evolution make it an ideal candidate for four-dimensional (3D space + time) intraoperative imaging. We also propose a novel, elastic image registration-based technique to keep the net radiation dose within acceptable levels. We have successfully created 3D renderings from multislice CT corresponding to anatomy visible within the field of view of the laparoscope in a swine. These renderings show the underlying vasculature along with their latest intraoperative orientation. With additional developments, our research has the potential to help improve the precision of laparoscopic surgeries further, reduce complications, and expand the scope of minimally invasive surgeries.

The development of image-guided interventions requires validation studies to evaluate new protocols. So far, these validation studies have been limited to animal models and to software and physical phantoms that simulate respiratory motion but cannot accommodate needle punctures in a realistic manner. We have built a computer-controlled pump that drives an anthropomorphic respiratory phantom for simulating natural breathing patterns. This pump consists of a power supply, a motion controller with servo amplifier, linear actuator, and custom fabricated pump assembly. By generating several sample waveforms, we were able to simulate typical breathing patterns. Using this pump, we were able to produce chest wall movements similar to typical chest wall movements observed in humans. This system has potential applications for evaluating new respiratory compensation algorithms and may facilitate improved testing of image-guided protocols under realistic interventional conditions.

The lumbar puncture is performed by inserting a needle into the spinal chord of the patient to inject medicaments or to extract liquor. The training of this procedure is usually done on the patient guided by experienced supervisors. A virtual reality lumbar puncture simulator has been developed in order to minimize the training costs and the patient's risk. We use a haptic device with six degrees of freedom (6DOF) to feedback forces that resist needle insertion and rotation. An improved haptic volume rendering approach is used to calculate the forces. This approach makes use of label data of relevant structures like skin, bone, muscles or fat and original CT data that contributes information about image structures that can not be segmented. A real-time 3D visualization with optional stereo view shows the punctured region. 2D visualizations of orthogonal slices enable a detailed impression of the anatomical context. The input data consisting of CT and label data and surface models of relevant structures is defined in an XML file together with haptic rendering and visualization parameters. In a first evaluation the visible human male data has been used to generate a virtual training body. Several users with different medical experience tested the lumbar puncture trainer. The simulator gives a good haptic and visual impression of the needle insertion and the haptic volume rendering technique enables the feeling of unsegmented structures. Especially, the restriction of transversal needle movement together with rotation constraints enabled by the 6DOF device facilitate a realistic puncture simulation.

Technological development of devices used in image-guided surgery and therapy has progressed due to the potential advantages such technology can bring to procedure efficacy and safety. This paper describes the technical design of a real-time catheter-motion sensor for use in investigating applied motion in catheter-based interventions and for use as an input device for a remote catheter navigation system. The device is comprised of three stages: the first two are passive stages used to measure axial and radial motion of the catheter and the third stage is an active brake used to impede motion. As a catheter is moved through the device, axial and radial measurement is achieved by mechanically coupling two optical encoders to the catheter. An electronic 24-bit counter and micro-controller are used to record incremental motion of the catheter. A computer loaded with a custom Python driver initializes and controls the micro-controller through a RS-232 port. The use of real clinical catheters with this device will allow the device to be used as an input to an image-guided remote catheter navigation system. User feedback may be achieved by linking a sensor in the slave device of a remote catheter navigation system, with the feedback system of the device.

The first and perhaps most important phase of a surgical procedure is the insertion of an intravenous (IV) catheter. Currently, this is performed manually by trained personnel. In some visions of future operating rooms, however, this process is to be replaced by an automated system. We previously presented work for localizing near-surface veins via near-infrared (NIR) imaging in combination with structured light ranging for surface mapping and robotic guidance. In this paper, we describe experiments to determine the best NIR wavelengths to optimize vein contrast for physiological differences such as skin tone and/or the presence of hair on the arm or wrist surface. For illumination, we employ an array of NIR LEDs comprising six different wavelength centers from 740nm to 910nm. We capture imagery of each subject under every possible combination of illuminants and determine the optimal combination of wavelengths for a given subject to maximize vein contrast using linear discriminant analysis.

Optical tracking systems pioneered the use of position sensors in surgical navigation. The requirement to maintain a clear line-of-sight between the emitters and detectors, however, renders them unsuitable for tracking flexible invasive instruments. On the other hand, advances in electromagnetic tracking systems permit a key-enabling role in imageguided procedures. First-generation magnetic systems present a significant challenge for tracker designers to improve both performance and acceptance. Troublesome magnetic problems include inaccuracies due to the presence of metallic distorters in the tracking volume and to dynamic motion of the tracked object. A new magnetic tracker (3D Guidance^TM), recently developed at Ascension Technology, seeks to address these problems. Employing third-generation pulsed-DC magnetic tracking technology and new signal processing techniques, the new tracker overcomes the distorting effects of non-magnetic conductive metals (300-series stainless steel, titanium and aluminum) and composite tables experienced by AC trackers. Ascension has developed a break-through flat transmitter that negates ferrous metal distortion emanating from procedural tables. The tracker development has also significantly advanced the state of the art in sensor miniaturization. The 3D Guidance^TM features the world's smallest electromagnetic tracking sensors, opening the door to new applications for minimally invasive procedures. Finally, dynamic accuracy has been significantly improved with the implementation of Kalman based algorithms. Test results are reported.

Integrated solutions for navigation systems with CT, MR or US systems become more and more popular for medical products. Such solutions improve the medical workflow, reduce hardware, space and costs requirements. The purpose of our project was to develop a new electromagnetic navigation system for interventional radiology which is integrated into C-arm CT systems. The application is focused on minimally invasive percutaneous interventions performed under local anaesthesia. Together with a vacuum-based patient immobilization device and newly developed navigation tools (needles, panels) we developed a safe and fully automatic navigation system. The radiologist can directly start with navigated interventions after loading images without any prior user interaction. The complete system is adapted to the requirements of the radiologist and to the clinical workflow. For evaluation of the navigation system we performed different phantom studies and achieved an average accuracy of better than 2.0 mm.

This work presents an integrated system for multimodality image guidance of minimally invasive medical procedures. This software and hardware system offers real-time integration and registration of multiple image streams with localization data from navigation systems. All system components communicate over a local area Ethernet network, enabling rapid and flexible deployment configurations. As a representative configuration, we use X-ray fluoroscopy (XF) and ultrasound (US) imaging. The XF imaging system serves as the world coordinate system, with gantry geometry derived from the imaging system, and patient table position tracked with a custom-built measurement device using linear encoders. An electromagnetic (EM) tracking system is registered to the XF space using a custom imaging phantom that is also tracked by the EM system. The RMS fiducial registration error for the EM to X-ray registration was 2.19 mm, and the RMS target registration error measured with an EM-tracked catheter was 8.81 mm. The US image stream is subsequently registered to the XF coordinate system using EM tracking of the probe, following a calibration of the US image within the EM coordinate system. We present qualitative results of the system in operation, demonstrating the integration of live ultrasound imaging spatially registered to X-ray fluoroscopy with catheter localization using electromagnetic tracking.

The purpose of this study was to quantify the effects of a computed tomography (CT) scanner environment on the positional accuracy of an AC electromagnetic tracking system, the second generation NDI Aurora. A three-axis positioning robot was used to move an electromagnetically tracked needle above the CT table throughout a 30cm by 30cm axial plane sampled in 2.5cm steps. The corresponding position data was captured from the Aurora and was registered to the positioning system data using a rigid body transformation minimizing the least squares L2-norm. Data was sampled at varying distances from the CT gantry (three feet, two feet, and one foot) and with the CT table in a nominal position and lowered by 10cm. A coordinate system was defined with the x axis normal to the CT table and the origin at the center of the CT table, and the z axis spanning the table in the lateral direction with the origin at the center of the CT table. In this coordinate system, the positional relationships of each sampled point, the CT table, and the Aurora field generator are clearly defined. This allows error maps to be displayed in accurate spatial relationship to the CT scanner as well as to a representative patient anatomy. By quantifying the distortions in relation to the position of CT scanner components and the Aurora field generator, the optimal working field of view and recommended guidelines for operation can be determined such that targeting inside human anatomy can be done with reasonable expectations of desired performance.

This paper investigates the utilization of the ultra-tiny electromagnetic tracker (UEMT) in a bronchoscope navigation system. In a bronchoscope navigation system, it is important to track the tip of a bronchoscope or catheter in real time. An ultra-tiny electromagnetic tracker (UEMT), which can be inserted into the working channel of a bronchoscope, allows us to track the tip of a bronchoscope or a catheter in real time. However, the accuracy of such UEMTs can be easily a.ected by ferromagnetic materials existing around the systems. This research tries to utilize a method for obtaining a function that compensates the outputs of a UEMT in a bronchoscope navigation system using a method proposed by Sato et al. This method uses a special jig combining a UEMT and an optical tracker (OT). Prior to bronchoscope navigation, we sweep this jig around an examination table and record outputs of both the UEMT and the OT. By using the outputs of the OT as reference data, we calculate a higher-order polynomial that compensates the UEMT outputs. We applied this method to the bronchoscope navigation system and performed bronchoscope navigation inside a bronchial phantom on the examination table. The experimental results showed that this method can reduce the position sensing error from 53.2 mm to 3.5 mm on a conventional examination table. Also, by using compensated outputs, it was possible to produce virtual bronchoscopic images synchronized with real bronchoscopic images.

Clinical research has been rapidly evolving towards the development of less invasive surgical procedures. We recently embarked on a project to improve intracardiac beating heart interventions. Our novel approach employs new surgical technologies and support from image-guidance via pre-operative and intra-operative imaging (i.e. two-dimensional echocardiography) to substitute for direct vision. Our goal was to develop a versatile system that allowed for safe cardiac port access, and provide sufficient image-guidance with the aid of a virtual reality environment to substitute for the absence of direct vision, while delivering quality therapy to the target. Specific targets included the repair and replacement of heart valves and the repair of septal defects. The ultimate objective was to duplicate the success rate of conventional open-heart surgery, but to do so via a small incision, and to evaluate the efficacy of the procedure as it is performed. This paper describes the software and hardware components, along with the methodology for performing mitral valve replacement as one example of this approach, using ultrasound and virtual tool models to position and fasten the valve in place.

We have constructed a fourth generation anthropomorphic phantom which, in addition to the realistic description of the human anatomy, includes a coronary artery disease model. A watertight version of the NURBS-based Cardiac-Torso (NCAT) phantom was generated by converting the individual NURBS surfaces of each organ into closed, manifold and non-self-intersecting tessellated surfaces. The resulting 330 surfaces of the phantom organs and tissues are now comprised of ~5×10⁶ triangles whose size depends on the individual organ surface normals. A database of the elemental composition of each organ was generated, and material properties such as density and scattering cross-sections were defined using PENELOPE. A 300 μm resolution model of a heart with 55 coronary vessel segments was constructed by fitting smooth triangular meshes to a high resolution cardiac CT scan we have segmented, and was consequently registered inside the torso model. A coronary artery disease model that uses hemodynamic properties such as blood viscosity and resistivity was used to randomly place plaque within the artery tree. To generate x-ray images of the aforementioned phantom, our group has developed an efficient Monte Carlo radiation transport code based on the subroutine package PENELOPE, which employs an octree spatial data-structure that stores and traverses the phantom triangles. X-ray angiography images were generated under realistic imaging conditions (90 kVp, 10° Wanode spectra with 3 mm Al filtration, ~5×10¹¹ x-ray source photons, and 10% per volume iodine contrast in the coronaries). The images will be used in an optimization algorithm to select the optimal technique parameters for a variety of imaging tasks.

A method is proposed that allows for a fully automated computation of a series of high-resolution volumetric reconstructions of a patient's coronary arteries based on a single rotational acquisition. During the 7.2 second acquisition the coronary arteries are injected with contrast material while the imaging system rotates around the patient to obtain a series of X-ray projection images over an angular range of 180 degrees. Based on the simultaneously recorded ECG-signal the projection images corresponding to the same cardiac cycle can be utilized to reconstruct three-dimensional (3D) high-spatial-resolution angiograms of the coronary arteries in multiple (3D+t) cardiac phases within the cardiac cycle. The proposed acquisition protocol has been applied to 22 patients and the tomograpic reconstructions depicted the main arteries as well as the main bifurcations in multiple cardiac phases in all enrolled patients. For the first time, this feasibility study shows that a three-dimensional description of the coronary arteries can be obtained intraprocedurally in a conventional interventional suite by means of tomographic reconstruction from projection images without any user interaction.

This study presents a multi-modality image registration method that evaluates left atrial scarring after radiofrequency (RF) ablation for pulmonary vein (PV) isolation. Our group has recently developed a delayed enhancement magnetic resonance imaging (DE-MRI) method with the potential to visualize and monitor non-invasively post-ablation scarring in the left atrium and the PV ostia. We wished to compare the 3D configuration of scarring in the DE-MRI image and the ablation points recorded by electroanatomical mapping (EAM) system, hypothesizing that scarring detected by DE-MRI overlaps with ablation points recorded by the EAM system used in the procedure. Methods and Results: Three data sets, DE-MRI images and pulmonary vein MR angiography (PV-MRA) images, and EAM data (CARTO-XP, Biosense-Webster, Inc., Diamond Bar, CA) from a patient who underwent PV ablation, were used for the multi-modal image registration. Contrast-enhanced MR imaging was performed 38 days after the ablation procedure. PV-MRA and DE-MRI were fused by intensity-based rigid registration. Scar tissue was extracted from the DE-MRI images using multiple threshold values. EAM data was further fused with segmented PV-MRA by the iterative closest point algorithm (ICP). After registration, the distance from PV-MRA to the scar was 2.6 ± 2.1 mm, and from ablation points to the surface of the scar was 2.5 ± 2.3 mm. The fused image demonstrates the 3D relationship between the PV ostia, the scar and the EAM recording of ablation points. Conclusion: Multimodal data fusion indicated that the scar tissue lesion after PV isolation showed good overlap with the ablation points.

Image-guided therapy for electrophysiology applications requires integration of pre-procedural volumetric imaging data with intra-procedural electroanatomical mapping (EAM) information. Existing methods for fusion of EAM and imaging data are based on fiducial landmark identification or point-to-surface distance minimization algorithms, both of which require detailed EAM mapping. This mapping procedure requires specific selection of points on the endocardial surface and this point acquisition process is skill-dependent, time-consuming and labor-intensive. The mapping catheter tip must first be navigated to a landmark on the endocardium, tip contact must be verified, and finally the tip location must be explicitly annotated within the EAM data record. This process of individual landmark identification and annotation must be repeated carefully >50 times to define endocardial and other vascular surfaces with sufficient detail for iterated-closest-point (ICP)-based registration. To achieve this, 30-45 minutes of mapping just for the registration procedure can be necessary before the interventional component of the patient study begins. Any acquired EAM point location that is not in contact with the chamber surface can adversely impact the quality of registration. Significantly faster point acquisition can be achieved by recording catheter tip locations automatically and continuously without requiring explicit navigation to and annotation of fiducial landmarks. We present a novel registration framework in which EAM locations are rapidly acquired and recorded in a continuous, untriggered fashion while the electrophysiologist manipulates the catheter tip within the heart. Results from simulation indicate that mean registration errors are on the order of 3-4mm, comparable in magnitude to conventional registration procedures which take significantly longer to perform. Qualitative assessment in clinical data also reflects good agreement with physician expectations.

Interventional cardiac MRI has been undergoing rapid development because of the availability of MRI compatible interventional catheters, and the increased performance of the MRI systems. Intravascular techniques do not require an open access scanner, and hence higher imaging performance during procedures can be achieved. Now, with the availability of a short, relatively open cylindrical bore scanner high imaging performance is also available to guide direct surgical procedures.

Catheter-based ablation in the left atrium and pulmonary veins (LAPV) for treatment of atrial fibrillation in cardiac electrophysiology (EP) are complex and require knowledge of heart chamber anatomy. Electroanatomical mapping (EAM) is typically used to define cardiac structures by combining electromagnetic spatial catheter localization with surface models which interpolate the anatomy between EAM point locations in 3D. Recently, the incorporation of pre-operative volumetric CT or MR data sets has allowed for more detailed maps of LAPV anatomy to be used intra-operatively. Preoperative data sets are however a rough guide since they can be acquired several days to weeks prior to EP intervention. Due to positional and physiological changes, the intra-operative cardiac anatomy can be different from that depicted in the pre-operative data. We present an application of contrast-enhanced rotational X-ray imaging for CT-like reconstruction of 3D LAPV anatomy during the intervention itself. Depending on the heart size a single or two selective contrastenhanced rotational acquisitions are performed and CT-like volumes are reconstructed with 3D filtered back projection. In case of dual injection, the two volumes depicting the left and right portions of the LAPV are registered and fused. The data sets are visualized and segmented intra-procedurally to provide anatomical data and surface models for intervention guidance. Our results from animal and human experiments indicate that the anatomical information from intra-operative CT-like reconstructions compares favorably with preacquired imaging data and can be of sufficient quality for intra-operative guidance.

Simulation systems are becoming increasingly essential in medical education. Hereby, capturing the physical behaviour of the real world requires a sophisticated modelling of instruments within the virtual environment. Most models currently used are not capable of user interactive simulations due to the computation of the complex underlying analytical equations. Alternatives are often based on simplifying mass-spring systems, being able to deliver high update rates that come at the cost of less realistic motion. In addition, most techniques are limited to narrow and tubular vessel structures or restrict shape alterations to two degrees of freedom, not allowing instrument deformations like torsion. In contrast, our approach combines high update rates with highly realistic motion and can in addition be used with respect to arbitrary structures like vessels or cavities (e.g. atrium, ventricle) without limiting the degrees of freedom. Based on energy minimization, bending energies and vessel structures are considered as linear elastic elements; energies are evaluated at regularly spaced points on the instrument, while the distance of the points is fixed, i.e. we simulate an articulated structure of joints with fixed connections between them. Arbitrary tissue structures are modeled through adaptive distance fields and are connected by nodes via an undirected graph system. The instrument points are linked to nodes by a system of rules. Energy minimization uses a Quasi Newton method without preconditioning and, hereby, gradients are estimated using a combination of analytical and numerical terms. Results show a high quality in motion simulation when compared to a phantom model. The approach is also robust and fast. Simulating an instrument with 100 joints runs at 100 Hz on a 3 GHz PC.

Static X-ray computed tomography (CT) volumes are often used as anatomic roadmaps during catheter-based cardiac interventions performed under X-ray fluoroscopy guidance. These CT volumes provide a high-resolution depiction of soft-tissue structures, but at only a single point within the cardiac and respiratory cycles. Augmenting these static CT roadmaps with segmented myocardial borders extracted from live ultrasound (US) provides intra-operative access to real-time dynamic information about the cardiac anatomy. In this work, using a customized segmentation method based on a 3D active mesh, endocardial borders of the left ventricle were extracted from US image streams (4D data sets) at a frame rate of approximately 5 frames per second. The coordinate systems for CT and US modalities were registered using rigid body registration based on manually selected landmarks, and the segmented endocardial surfaces were overlaid onto the CT volume. The root-mean squared fiducial registration error was 3.80 mm. The accuracy of the segmentation was quantitatively evaluated in phantom and human volunteer studies via comparison with manual tracings on 9 randomly selected frames using a finite-element model (the US image resolutions of the phantom and volunteer data were 1.3 x 1.1 x 1.3 mm and 0.70 x 0.82 x 0.77 mm, respectively). This comparison yielded 3.70±2.5 mm (approximately 3 pixels) root-mean squared error (RMSE) in a phantom study and 2.58±1.58 mm (approximately 3 pixels) RMSE in a clinical study. The combination of static anatomical roadmap volumes and dynamic intra-operative anatomic information will enable better guidance and feedback for image-guided minimally invasive cardiac interventions.

Minimally invasive techniques for use inside the beating heart, such as mitral valve replacement and septal defect repair, are the focus of this work. Traditional techniques for these procedures require an open chest approach and a cardiopulmonary bypass machine. New techniques using port access and a combined surgical guidance tool that includes an overlaid two-dimensional ultrasound image in a virtual reality environment are being developed. To test this technique, a cardiac phantom was developed to simulate the anatomy. The phantom consists of an acrylic box filled with a 7% glycerol solution with ultrasound properties similar to human tissue. Plate inserts mounted in the box simulate the physical anatomy. An accuracy assessment was completed to evaluate the performance of the system. Using the cardiac phantom, a 2mm diameter glass toroid was attached to a vertical plate as the target location. An elastic material was placed between the target and plate to simulate the target lying on a soft tissue structure. The target was measured using an independent measurement system and was represented as a sphere in the virtual reality system. The goal was to test the ability of a user to probe the target using three guidance methods: (i) 2D ultrasound only, (ii) virtual reality only and (iii) ultrasound enhanced virtual reality. Three users attempted the task three times each for each method. An independent measurement system was used to validate the measurement. The ultrasound imaging alone was poor in locating the target (5.42 mm RMS) while the other methods proved to be significantly better (1.02 mm RMS and 1.47 mm RMS respectively). The ultrasound enhancement is expected to be more useful in a dynamic environment where the system registration may be disturbed.

Three-dimensional (3D) ultrasound is ideally suited to monitor internal organ motion since it offers real-time volumetric imaging without exposing the patient to radiation. We extend a two dimensional (2D) region-tracking algorithm, which was originally used in computer vision, to monitor internal organ motion in 3D. A volume of interest is first selected in an ultrasound volume as a reference. The sum of squared differences is used as the similarity measure to register the reference to each successive volume frame. A transformation model is used to describe the motion and geometric deformation of the reference. The Gauss-Newton method is used to solve the optimization problem. In order to improve the algorithm's efficiency, the Jacobian matrix is decomposed as a product of a time-varying matrix and a constant matrix. The constant matrix is pre-computed to reduce the load of online computation. The algorithm was tested on targets under respiratory motion and cardiac motion. The experimental results show that the transformation model of the algorithm can approximate the geometric distortion of the reference template. With a properly selected reference with rich texture information, the algorithm is sufficiently accurate and robust to follow target motion, and fast enough to be used in real time.

Sonography has good topographic accuracy for superficial lymph node assessment in patients with head and neck cancers. It is therefore an ideal non-invasive tool for precise inter-fraction volumetric analysis of enlarged cervical nodes. In addition, when registered with computed tomography (CT) images, ultrasound information may improve target volume delineation and facilitate image-guided adaptive radiation therapy. A feasibility study was developed to evaluate the use of a prototype ultrasound system capable of three dimensional visualization and multi-modality image fusion for cervical node geometry. A ceiling-mounted optical tracking camera recorded the position and orientation of a transducer in order to synchronize the transducer's position with respect to the room's coordinate system. Tracking systems were installed in both the CT-simulator and radiation therapy treatment rooms. Serial images were collected at the time of treatment planning and at subsequent treatment fractions. Volume reconstruction was performed by generating surfaces around contours. The quality of the spatial reconstruction and semi-automatic segmentation was highly dependent on the system's ability to track the transducer throughout each scan procedure. The ultrasound information provided enhanced soft tissue contrast and facilitated node delineation. Manual segmentation was the preferred method to contour structures due to their sonographic topography.

This paper presents a novel graphical user interface developed for a navigation system for ultrasound-guided computer-assisted shoulder arthroscopic surgery. The envisioned purpose of the interface is to assist the surgeon in determining the position and orientation of the arthroscopic camera and other surgical tools within the anatomy of the patient. The user interface features real time position tracking of the arthroscopic instruments with an optical tracking system, and visualization of their graphical representations relative to a three-dimensional shoulder surface model of the patient, created from computed tomography images. In addition, the developed graphical interface facilitates fast and user-friendly intra-operative calibration of the arthroscope and the arthroscopic burr, capture and segmentation of ultrasound images, and intra-operative registration. A pilot study simulating the computer-aided shoulder arthroscopic procedure on a shoulder phantom demonstrated the speed, efficiency and ease-of-use of the system.

Biopsy of the prostate using ultrasound guidance is the clinical gold standard for diagnosis of prostate adenocarinoma. However, because early stage tumors are rarely visible under US, the procedure carries high false-negative rates and often patients require multiple biopsies before cancer is detected. To improve cancer detection, it is imperative that throughout the biopsy procedure, physicians know where they are within the prostate and where they have sampled during prior biopsies. The current biopsy procedure is limited to using only 2D ultrasound images to find and record target biopsy core sample sites. This information leaves ambiguity as the physician tries to interpret the 2D information and apply it to their 3D workspace. We have developed a 3D ultrasound-guided prostate biopsy system that provides 3D intra-biopsy information to physicians for needle guidance and biopsy location recording. The system is designed to conform to the workflow of the current prostate biopsy procedure, making it easier for clinical integration. In this paper, we describe the system design and validate its accuracy by performing an in vitro biopsy procedure on US/CT multi-modal patient-specific prostate phantoms. A clinical sextant biopsy was performed by a urologist on the phantoms and the 3D models of the prostates were generated with volume errors less than 4% and mean boundary errors of less than 1 mm. Using the 3D biopsy system, needles were guided to within 1.36 ± 0.83 mm of 3D targets and the position of the biopsy sites were accurately localized to 1.06 ± 0.89 mm for the two prostates.

We introduce a novel navigation system to support minimally invasive prostate surgery. The system utilizes transrectal ultrasonography (TRUS) and needle-shaped navigation aids to visualize hidden structures via Augmented Reality. During the intervention, the navigation aids are segmented once from a 3D TRUS dataset and subsequently tracked by the endoscope camera. Camera Pose Estimation methods directly determine position and orientation of the camera in relation to the navigation aids. Accordingly, our system does not require any external tracking device for registration of endoscope camera and ultrasonography probe. In addition to a preoperative planning step in which the navigation targets are defined, the procedure consists of two main steps which are carried out during the intervention: First, the preoperatively prepared planning data is registered with an intraoperatively acquired 3D TRUS dataset and the segmented navigation aids. Second, the navigation aids are continuously tracked by the endoscope camera. The camera's pose can thereby be derived and relevant medical structures can be superimposed on the video image. This paper focuses on the latter step. We have implemented several promising real-time algorithms and incorporated them into the Open Source Toolkit MITK (www.mitk.org). Furthermore, we have evaluated them for minimally invasive surgery (MIS) navigation scenarios. For this purpose, a virtual evaluation environment has been developed, which allows for the simulation of navigation targets and navigation aids, including their measurement errors. Besides evaluating the accuracy of the computed pose, we have analyzed the impact of an inaccurate pose and the resulting displacement of navigation targets in Augmented Reality.

A system for fusion of realtime transrectal ultrasound (TRUS) with pre-acquired 3D images of the prostate was designed and demonstrated in phantoms and volunteer patients. Biopsy guides for endocavity ultrasound transducers were equipped with customized 6 degree-of-freedom (DoF) electromagnetic (EM) tracking sensors, compatible with the Aurora EM tracking system (Northern Digital Inc, NDI, Waterloo, ON, Canada). The biopsy guides were attached to an ultrasound probe and calibrated to map tracking coordinates with ultrasound image coordinates. Six cylindrical gold seeds were placed in a prostate phantom to serve as fiducial markers. The fiducials were first identified manually in 3T magnetic resonance (MR) images collected with an endorectal coil. The phantom was then imaged with tracked realtime TRUS and the fiducial markers were identified in the live image using custom software. Rigid registrations between MR and ultrasound image space were computed and evaluated using subsets of the fiducial markers. Twelve patients were scanned with 3T MRI and TRUS for biopsy and seed placement. In ten patients, volumetric ultrasound images were reconstructed from 2D sweeps of the prostate and were manually registered with the MR. The rigid registrations were used to display live TRUS images fused with spatially corresponding realtime multiplanar reconstructions (MPRs) of the MR image volume. Registration accuracy was evaluated by segmenting the prostate in the MR and volumetric ultrasound and computing distance measures between the two segmentations. In the phantom experiments, registration accuracies of 2.2 to 2.3 mm were achieved. In the patient studies, the average root mean square distance between the MR and TRUS segmentations was 3.1 mm, the average Hausdorff distance was 9.8 mm. Deformation of the prostate during MR and TRUS scan was identified as the primary source of error. Realtime MR/TRUS image fusion is feasible and is a promising approach to improved target visualization during TRUS-guided biopsy or therapy procedures.

The prostate is known to move between daily fractions during the course of radiation therapy using external beams. This movement causes problem with 3D conformal or intensity-modulated radiation therapy, in which tight margins are used for treatment planning. To minimize the adverse effect of this motion on dose delivery, daily localization of the prostate with respect to the planning CT is necessary. Current ultrasound-based localization systems require manual alignment of ultrasound images with the planning CT. The resulting localization is subjective and has high interobserver variability. To reduce the alignment uncertainty and increase the setup efficiency, we proposed an automatic prostate alignment method using a volume subdivision-based elastic image registration algorithm. The algorithm uses normalized mutual information as the measure of image similarity between the daily 3D ultrasound images and the planning CT. The prostate contours on the CT are mapped to the ultrasound space by applying the transformation fields from image registration. The displacement of the center-of-mass of the mapped contours is calculated for automatic patient setup. For validation purposes, six experts independently and manually aligned the archived CT and 3D ultrasound images using the SonArray system and reported their readings as shifts along the three principal axes. The mean shift and standard deviation of the readings along each axis were calculated. We regarded the automatic alignment as being acceptable if the difference between the mean shift and the automatic shift is within two times the standard deviation. Three out of five patients were successfully aligned with two failures.

Prostate brachytherapy involves permanent implantation of radioactive sources into the prostate gland. Since fluoroscopy and transrectal ultrasound (TRUS) imaging modalities currently complement each other by providing good visualization of seeds and soft tissue, respectively, the registration of these two imaging modalities could lead to the intraoperative dosimetry analysis of brachytherapy procedures, thus improving patient outcome and reducing costs. Although it is desirable to register TRUS and fluoroscopy images by using the implanted seeds as fiducial markers, an operator, based on our experience, can locate only a small fraction of implanted seeds in axial TRUS images. Therefore, to perform TRUS-fluoroscopy registration in a clinical setting, there is a need for (1) a new method that can reliably perform registration at low seed detection rates and (2) a new imaging technique to enhance the seed visibility. We previously developed iterative optimal assignment (IOA), which can perform registration at seed detection rates below 20%, to address the former. In this paper, we present a new TRUS acquisition method where we acquire images of the prostate by rotating the longitudinal transducer of a biplanar probe in the clockwise/counter-clockwise direction. We acquired post-implant fluoroscopy and TRUS images from 35 patients who underwent a seed implant procedure. The results show that the combined use of IOA and rotational images makes TRUS-fluoroscopy registration possible and practical, thus our goal of intraoperative dosimetry can be realized.

In recent years, an increasing number of liver tumor indications were treated by minimally invasive laparoscopic resection. Besides the restricted view, a major issue in laparoscopic liver resection is the enhanced visualization of (hidden) vessels, which supply the tumorous liver segment and thus need to be divided prior to the resection. To navigate the surgeon to these vessels, pre-operative abdominal imaging data can hardly be used due to intraoperative organ deformations mainly caused by appliance of carbon dioxide pneumoperitoneum and respiratory motion. While regular respiratory motion can be gated and synchronized intra-operatively, motion caused by pneumoperitoneum is individual for every patient and difficult to estimate. Therefore, we propose to use an optically tracked mobile C-arm providing cone-beam CT imaging capability intraoperatively. The C-arm is able to visualize soft tissue by means of its new flat panel detector and is calibrated offline to relate its current position and orientation to the coordinate system of a reconstructed volume. Also the laparoscope is optically tracked and calibrated offline, so both laparoscope and C-arm are registered in the same tracking coordinate system. Intra-operatively, after patient positioning, port placement, and carbon dioxide insufflation, the liver vessels are contrasted and scanned during patient exhalation. Immediately, a three-dimensional volume is reconstructed. Without any further need for patient registration, the volume can be directly augmented on the live laparoscope video, visualizing the contrasted vessels. This augmentation provides the surgeon with advanced visual aid for the localization of veins, arteries, and bile ducts to be divided or sealed.

In this paper, we evaluate the target position estimation accuracy of a novel soft tissue navigation system with a custom-designed respiratory liver motion simulator. The system uses a real-time deformation model to estimate the position of the target (e.g. a tumor) during a minimally invasive intervention from the location of a set of optically tracked needle-shaped navigation aids which are placed in the vicinity of the target. A respiratory liver motion simulator was developed to evaluate the performance of the system in-vitro. It allows the mounting of an explanted liver which can be moved along the longitudinal axis of a corpus model to simulate breathing motion. In order to assess the accuracy of our system we utilized an optically trackable tool as target and estimated its position continuously from the current position of the navigation aids. Four different transformation types were compared as base for the real-time deformation model: Rigid transformations, thinplate splines, volume splines, and elastic body splines. The respective root-mean-square target position estimation errors are 2.15 mm, 1.60 mm, 1.88 mm, and 1.92 mm averaged over a set of experiments obtained from a total of six navigation aid configurations in two pig livers. The error is reduced by 76.3%, 82.4%, 79.3%, and 78.8%, respectively, compared to the case when no deformation model is applied, i.e., a constant organ position is assumed throughout the breathing cycle.

Similar to the well documented brain shift experienced during neurosurgical procedures, intra-operative soft tissue deformation in open hepatic resections is the primary source of error in current image-guided liver surgery (IGLS) systems. The use of bio-mechanical models has shown promise in providing the link between the deformed, intra-operative patient anatomy and the pre-operative image data. More specifically, the current protocol for deformation compensation in IGLS involves the determination of displacements via registration of intra-operatively acquired sparse data and subsequent use of the displacements to drive solution of a linear elastic model via the finite element method (FEM). However, direct solution of the model during the surgical procedure has several logistical limitations including computational time and the ability to accurately prescribe boundary conditions and material properties. Recently, approaches utilizing an atlas of pre-operatively computed model solutions based on a priori information concerning the surgical loading conditions have been proposed as a more realistic avenue for intra-operative deformation compensation. Similar to previous work, we propose the use of a simple linear inverse model to match the intra-operatively acquired data to the pre-operatively computed atlas. Additionally, an iterative approach is implemented whereby point correspondence is updated during the matching process, being that the correspondence between intra-operative data and the pre-operatively computed atlas is not explicitly known in liver applications. Preliminary validation experiments of the proposed algorithm were performed using both simulation and phantom data. The proposed method provided comparable results in the phantom experiments with those obtained using the traditional incremental FEM approach.

Radiofrequency ablation (RFA) is emerging as the primary mode of treatment of unresectable malignant liver tumors. With current intraoperative imaging modalities, quick, precise, and complete localization of lesions remains a challenge for liver RFA. Fusion of intraoperative CT and preoperative PET images, which relies on PET and CT registration, can produce a new image with complementary metabolic and anatomic data and thus greatly improve the targeting accuracy. Unlike neurological images, alignment of abdominal images by combined PET/CT scanner is prone to errors as a result of large nonrigid misalignment in abdominal images. Our use of a normalized mutual information-based 3D nonrigid registration technique has proven powerful for whole-body PET and CT registration. We demonstrate here that this technique is capable of acceptable abdominal PET and CT registration as well. In five clinical cases, both qualitative and quantitative validation showed that the registration is robust and accurate. Quantitative accuracy was evaluated by comparison between the result from the algorithm and clinical experts. The accuracy of registration is much less than the allowable margin in liver RFA. Study findings show the technique's potential to enable the augmentation of intraoperative CT with preoperative PET to reduce procedure time, avoid repeating procedures, provide clinicians with complementary functional/anatomic maps, avoid omitting dispersed small lesions, and improve the accuracy of tumor targeting in liver RFA.

Image guided radiofrequency (RF) ablation has taken a significant part in the clinical routine as a minimally invasive method for the treatment of focal liver malignancies. Medical imaging is used in all parts of the clinical workflow of an RF ablation, incorporating treatment planning, interventional targeting and result assessment. This paper describes a software application, which has been designed to support the RF ablation workflow under consideration of the requirements of clinical routine, such as easy user interaction and a high degree of robust and fast automatic procedures, in order to keep the physician from spending too much time at the computer. The application therefore provides a collection of specialized image processing and visualization methods for treatment planning and result assessment. The algorithms are adapted to CT as well as to MR imaging. The planning support contains semi-automatic methods for the segmentation of liver tumors and the surrounding vascular system as well as an interactive virtual positioning of RF applicators and a concluding numerical estimation of the achievable heat distribution. The assessment of the ablation result is supported by the segmentation of the coagulative necrosis and an interactive registration of pre- and post-interventional image data for the comparison of tumor and necrosis segmentation masks. An automatic quantification of surface distances is performed to verify the embedding of the tumor area into the thermal lesion area. The visualization methods support representations in the commonly used orthogonal 2D view as well as in 3D scenes.

In image-guided surgery, discrete fiducials are used to determine a spatial registration between the location of surgical tools in the operating theater and the location of targeted subsurface lesions and critical anatomic features depicted in preoperative tomographic image data. However, the lack of readily localized anatomic landmarks has greatly hindered the use of image-guided surgery in minimally invasive abdominal procedures. To address these needs, we have previously described a laser-based system for localization of internal surface anatomy using conventional laparoscopes. During a procedure, this system generates a digitized, three-dimensional representation of visible anatomic surfaces in the abdominal cavity. This paper presents the results of an experiment utilizing an ex-vivo bovine liver to assess subsurface targeting accuracy achieved using our system. During the experiment, several radiopaque targets were inserted into the liver parenchyma. The location of each target was recorded using an optically-tracked insertion probe. The liver surface was digitized using our system, and registered with the liver surface extracted from post-procedure CT images. This surface-based registration was then used to transform the position of the inserted targets into the CT image volume. The target registration error (TRE) achieved using our surface-based registration (given a suitable registration algorithm initialization) was 2.4 mm ± 1.0 mm. A comparable TRE (2.6 mm ± 1.7 mm) was obtained using a registration based on traditional fiducial markers placed on the surface of the same liver. These results indicate the potential of fiducial-free, surface-to-surface registration for image-guided lesion targeting in minimally invasive abdominal surgery.

Deep brain stimulation (DBS) surgery is a treatment for patients suffering from Parkinson's disease and other movement disorders. The success of the procedure depends on the implantation accuracy of the DBS electrode array. Surgical planning and navigation are done based on the pre-operative patient scans, assuming that brain tissues do not move from the time of the pre-operative image acquisition to the time of the surgery. We performed brain shift analysis on nine patients that underwent DBS surgery using a 3D non-rigid registration algorithm. The registration algorithm automatically aligns the pre-operative and the post-operative 3D MRI scans and provides the shift vectors over the entire brain. The images were first aligned rigidly and then non-rigidly registered with an algorithm based on thin plate splines and maximization of the normalized mutual information. Brain shift of up to 8 mm was recorded in the nine subjects, which is significant given that the size of the targets in the DBS surgery is a few millimeters.

We are developing and evaluating a system that will facilitate the placement of deep brain stimulators (DBS) used to treat movement disorders including Parkinson's disease and essential tremor. Although our system does not rely on the common reference system used for functional neurosurgical procedures, which is based on the anterior and posterior commissure points (AC and PC), automatic and accurate localization of these points is necessary to communicate the positions of our targets. In this paper, we present an automated method for AC and PC selection that uses non-rigidly deformable atlases. To evaluate the accuracy of our multi-atlas based method, we compare it against the manual selection of the AC and PC points by 43 neurosurgeons (38 attendings and 5 residents) and show that its accuracy is submillimetric compared to the median of their selections. We also analyze the effect of AC-PC localization inaccuracy on the localization of common DBS targets.

In this paper, preliminary results from an image-to-physical space registration platform are presented. The current platform employs traditional and novel methods of registration which use a variety of data sources to include: traditional synthetic skin-fiducial point-based registration, surface registration based on facial contours, brain feature point-based registration, brain vessel-to-vessel registration, and a more comprehensive cortical surface registration method that utilizes both geometric and intensity information from both the image volume and physical patient. The intraoperative face and cortical surfaces were digitized using a laser range scanner (LRS) capable of producing highly resolved textured point clouds. In two in vivo cases, a series of registrations were performed using these techniques and compared within the context of a true target error. One of the advantages of using a textured point cloud data stream is that true targets among the physical cortical surface and the preoperative image volume can be identified and used to assess image-to-physical registration methods. The results suggest that iterative closest point (ICP) method for intraoperative face surface registration is equivalent to point-based registration (PBR) method of skin fiducial markers. With regard to the initial image and physical space registration, for patient 1, mean target registration error (TRE) were 3.1±0.4 mm and 3.6 ±0.9 mm for face ICP and skin fiducial PBR, respectively. For patient 2, the mean TRE were 5.7 ±1.3 mm, and 6.6 ±0.9 mm for face ICP and skin fiducial PBR, respectively. With regard to intraoperative cortical surface registration, SurfaceMI outperformed feature based PBR and vessel ICP with 1.7±1.8 mm for patient 1. For patient 2, the best result was achieved by using vessel ICP with 1.9±0.5 mm.

We introduce novel data structures and algorithms for clustering white matter fiber tracts to improve accuracy and robustness of existing techniques. Our novel fiber grid combined with a new randomized soft-division algorithm allows for defining the fiber similarity more precisely and efficiently than a feature space. A fine-tuning of several parameters to a particular fiber set - as it is often required if using a feature space - becomes obsolete. The idea is to utilize a 3D grid where each fiber point is assigned to cells with a certain weight. From this grid, an affinity matrix representing the fiber similarity can be calculated very efficiently in time O(n) in the average case, where n denotes the number of fibers. This is superior to feature space methods which need O(n²) time. Our novel eigenvalue regression is capable of determining a reasonable number of clusters as it accounts for inter-cluster connectivity. It performs a linear regression of the eigenvalues of the affinity matrix to find the point of maximum curvature in a list of descending order. This allows for identifying inner clusters within coarse structures, which automatically and drastically reduces the a-priori knowledge required for achieving plausible clustering results. Our extended multiple eigenvector clustering exhibits a drastically improved robustness compared to the well-known elongated clustering, which also includes an automatic detection of the number of clusters. We present several examples of artificial and real fiber sets clustered by our approach to support the clinical suitability and robustness of the proposed techniques.

In this paper we describe a novel approach to using morphological datasets (such as CT or MR) in the minimally invasive image guidance of intra-arterial and intra-venous endovascular devices in neuroangiography interventions. Minimally invasive X-ray angiography procedures rely on the navigation of endovascular devices, such as guide wires and catheters, through human vessels, using C-arm fluoroscopy. While the bone structure may be visible, and the injection of iodine contrast medium allows to guide endovascular devices through the vasculature, the soft-tissue structures remain invisible in the fluoroscopic images. We intend to present a method for the combined visualization of morphological data, a 3D rotational angiography (3DRA) reconstruction and the live fluoroscopy data stream in a single image. The combination of the fluoroscopic image with the 3DRA vessel tree offers the advantage that endovascular devices can be located with respect to the vasculature, without additional contrast injection, while the position of the C-arm geometry can be altered freely. The additional visualization of the morphological data, adds contextual information to the position of endovascular devices. This article addresses the clinical applications, the real-time aspects of the registration algorithms and fast fused visualization of the proposed method.

Recent studies have shown that more than 5 million bronchoscopy procedures are performed each year worldwide. The procedure usually involves biopsy of possible cancerous tissues from the lung. Standard bronchoscopes are too large to reach into the peripheral lung, where cancerous nodules are often found. The University of Washington has developed an ultrathin and flexible scanning fiber endoscope that is able to advance into the periphery of the human lungs without sacrificing image quality. To accompany the novel endoscope, we have developed a user interface that serves as a navigation guide for doctors when performing a bronchoscopy. The navigation system consists of a virtual surface mesh of the airways extracted from computed-tomography (CT) scan and an electromagnetic tracking system (EMTS). The complete system can be viewed as a global positioning system for the lung that provides pre-procedural planning functionalities, virtual bronchoscopy navigation, and real time tracking of the endoscope inside the lung. The real time virtual navigation is complemented by a particle filter algorithm to compensate for registration errors and outliers, and to prevent going through surfaces of the virtual lung model. The particle filter method tracks the endoscope tip based on real time tracking data and attaches the virtual endoscopic view to the skeleton that runs inside the virtual airway surface. Experiment results on a dried sheep lung show that the particle filter method converges and is able to accurately track the endoscope tip in real time when the endoscope is inserted both at slow and fast insertion speeds.

For exact orientation inside the tracheobronchial tree, clinicians are in urgent need of a navigation system for bronchoscopy. Such an image guided system has the ability to show the current position of a bronchoscope (instrument to inspect the inside of the lung) within the tracheobronchial tree. Thus orientation inside the complex tree structure is improved. Our approach of navigated bronchoscopy considers the problem of using a static image to navigate inside a constantly moving soft tissue. It offers a direct guidance to a preinterventionally defined target inside the bronchial tree to save intervention time spent on searching the right path and to minimize the duration of anesthesia. It is designed to adapt to the breathing cycle of the patient, so no further intervention to minimize the movement of the lung has to stress the patient. We present a newly developed navigation sensor with allows to display a virtual bronchoscopy in real time and we demonstrate an evaluation on the accuracy within a non moving ex vivo lung phantom.

This paper presents an easy and stable bronchoscope camera calibration technique for bronchoscope navigation system. A bronchoscope navigation system is strongly expected to be developed to make bronchoscopic examinations safer and more effective. In a bronchoscope navigation system, virtual bronchoscopic images are generated from a 3D CT image taken prior to an examination to register a patient's body and his/her CT image. It is absolutely indispensable to know correct intrinsic camera parameters such as focal length, aspect ratio, and the projection center of the camera for the generation of virtual bronchoscopic images. In the case of a bronchoscope, however, it is very complicated to obtain these camera parameters by calibration techniques applied to conventional cameras, since a bronchoscope camera has heavy barrel-type lens distortion. Also image resolution is quite low. Therefore, we propose an easy and stable bronchoscope camera calibration technique that does not require any special devices. In this method, a planar calibration pattern is captured at many different angles by moving the bronchoscope camera freely. Then we automatically detect feature points for camera calibration from captured images. Finally, intrinsic camera parameters are estimated from these extracted feature points by applying Zhang's calibration technique. We applied the proposed method to a conventional bronchoscope camera. The experimental results showed that reprojection error using estimated camera parameters was about 0.7 pixels. Also stable estimation was achieved by the proposed method.

In this paper, we present the design and implementation of a new rendering method based on high dynamic range (HDR) lighting and exposure control. This rendering method is applied to create video images for a 3D virtual bronchoscopy system. One of the main optical parameters of a bronchoscope's camera is the sensor exposure. The exposure adjustment is needed since the dynamic range of most digital video cameras is narrower than the high dynamic range of real scenes. The dynamic range of a camera is defined as the ratio of the brightest point of an image to the darkest point of the same image where details are present. In a video camera exposure is controlled by shutter speed and the lens aperture. To create the virtual bronchoscopic images, we first rendered a raw image in absolute units (luminance); then, we simulated exposure by mapping the computed values to the values appropriate for video-acquired images using a tone mapping operator. We generated several images with HDR and others with low dynamic range (LDR), and then compared their quality by applying them to a 2D/3D video-based tracking system. We conclude that images with HDR are closer to real bronchoscopy images than those with LDR, and thus, that HDR lighting can improve the accuracy of image-based tracking.

Physicians use endoscopic procedures to diagnose and treat a variety of medical conditions. For example, bronchoscopy is often performed to diagnose lung cancer. The current practice for planning endoscopic procedures requires the physician to manually scroll through the slices of a three-dimensional (3D) medical image. When doing this scrolling, the physician must perform 3D mental reconstruction of the endoscopic route to reach a specific diagnostic region of interest (ROI). Unfortunately, in the case of complex branching structures such as the airway tree, ROIs are often situated several generations away from the organ's origin. Existing image-analysis methods can help define possible endoscopic navigation paths, but they do not provide specific routes for reaching a given ROI. We have developed an automated method to find a specific route to reach an ROI. Given a 3D medical image, our method takes as inputs: (1) pre-defined ROIs; (2) a segmentation of the branching organ through which the endoscopic device will navigate; and (3) centerlines (paths) through the segmented organ. We use existing methods for branching-organ segmentation and centerline extraction. Our method then (1) identifies the closest paths (routes) to the ROI; and (2) if necessary, performs a directed search for the organ of interest, extending the existing paths to complete a route. Results from human 3D computed tomography chest images illustrate the efficacy of the method.

A 3D colon model is an essential component of a computer-aided diagnosis (CAD) system in colonoscopy to assist surgeons in visualization, and surgical planning and training. This research is thus aimed at developing the ability to construct a 3D colon model from endoscopic videos (or images). This paper summarizes our ongoing research in automated model building in colonoscopy. We have developed the mathematical formulations and algorithms for modeling static, localized 3D anatomic structures within a colon that can be rendered from multiple novel view points for close scrutiny and precise dimensioning. This ability is useful for the scenario when a surgeon notices some abnormal tissue growth and wants a close inspection and precise dimensioning. Our modeling system uses only video images and follows a well-established computer-vision paradigm for image-based modeling. We extract prominent features from images and establish their correspondences across multiple images by continuous tracking and discrete matching. We then use these feature correspondences to infer the camera's movement. The camera motion parameters allow us to rectify images into a standard stereo configuration and calculate pixel movements (disparity) in these images. The inferred disparity is then used to recover 3D surface depth. The inferred 3D depth, together with texture information recorded in images, allow us to construct a 3D model with both structure and appearance information that can be rendered from multiple novel view points.

The complete expansion of the stent during a percutaneous transluminal coronary angioplasty (PTCA) procedure is essential for treatment of a stenotic segment of a coronary artery. Inadequate expansion of the stent is a major predisposing factor to in-stent restenosis and acute thrombosis. Stents are positioned and deployed by fluoroscopic guidance. Although the current generation of stents are made of materials with some degree of radio-opacity to detect their location after deployment, proper stent expansion is hard to asses. In this work, we introduce a new method for the three-dimensional (3D) reconstruction of the coronary stents in-vivo utilizing two-dimensional projection images acquired during rotational angiography (RA). The acquisition protocol consist of a propeller rotation of the X-ray C-arm system of 180°, which ensures sufficient angular coverage for volume reconstruction. The angiographic projections were acquired at 30 frames per second resulting in 180 projections during a 7 second rotational run. The motion of the stent is estimated from the automatically tracked 2D coordinates of the markers on the balloon catheter. This information is used within a motion-compensated reconstruction algorithm. Therefore, projections from different cardiac phases and motion states can be used, resulting in improved signal-to-noise ratio of the stent. Results of 3D reconstructed coronary stents in vivo, with high spatial resolution are presented. The proposed method allows for a comprehensive and unique quantitative 3D assessment of stent expansion that rivals current X-ray and intravascular ultrasound techniques.

In this paper a method is introduced, to visualize bifurcated stent grafts in CT-Data. The aim is to improve therapy planning for minimal invasive treatment of abdominal aortic aneurysms (AAA). Due to precise measurement of the abdominal aortic aneurysm and exact simulation of the bifurcated stent graft, physicians are supported in choosing a suitable stent prior to an intervention. The presented method can be used to measure the dimensions of the abdominal aortic aneurysm as well as simulate a bifurcated stent graft. Both of these procedures are based on a preceding segmentation and skeletonization of the aortic, right and left iliac. Using these centerlines (aortic, right and left iliac) a bifurcated initial stent is constructed. Through the implementation of an ACM method the initial stent is fit iteratively to the vessel walls - due to the influence of external forces (distance- as well as balloonforce). Following the fitting process, the crucial values for choosing a bifurcated stent graft are measured, e.g. aortic diameter, right and left common iliac diameter, minimum diameter of distal neck. The selected stent is then simulated to the CT-Data - starting with the initial stent. It hereby becomes apparent if the dimensions of the bifurcated stent graft are exact, i.e. the fitting to the arteries was done properly and no ostium was covered.

Minimally invasive surgery is a highly complex medical discipline with various risks for surgeon and patient, but has also numerous advantages on patient-side. The surgeon has to adapt special operation-techniques and deal with difficulties like the complex hand-eye coordination, limited field of view and restricted mobility. To alleviate with these new problems, we propose to support the surgeon's spatial cognition by using augmented reality (AR) techniques to directly visualize virtual objects in the surgical site. In order to generate an intelligent support, it is necessary to have an intraoperative assistance system that recognizes the surgical skills during the intervention and provides context-aware assistance surgeon using AR techniques. With MEDIASSIST we bundle our research activities in the field of intraoperative intelligent support and visualization. Our experimental setup consists of a stereo endoscope, an optical tracking system and a head-mounted-display for 3D visualization. The framework will be used as platform for the development and evaluation of our research in the field of skill recognition and context-aware assistance generation. This includes methods for surgical skill analysis, skill classification, context interpretation as well as assistive visualization and interaction techniques. In this paper we present the objectives of MEDIASSIST and first results in the fields of skill analysis, visualization and multi-modal interaction. In detail we present a markerless instrument tracking for surgical skill analysis as well as visualization techniques and recognition of interaction gestures in an AR environment.

The surgical removal of brain tumors can lead to functional impairment. Therefore it is crucial to minimize the damage to important functional areas during surgery. These areas can be mapped before surgery by using functional MRI. However, functional impairment is not only caused by damage to these areas themselves. It is also caused by damage to the fiber bundles that connect these areas with the rest of the brain. Diffusion Tensor Images (DTI) can add information about these connecting fiber bundles. In this paper we present interactive visualization techniques that combine DTI, fMRI and structural MRI to assist the planning of brain tumor surgery. Using a fusion of these datasets, we can extract the fiber bundles that pass through an offset region around the tumor, as can be seen in Figure 1. These bundles can then be explored by filtering on distance to the tumor, or by selecting a specific functional area. This approach enables the surgeon to combine all this information in a highly interactive environment in order to explore the pre-operative situation.

Laparoscopic surgery without ventrotomy has been widely used in recent years for quick recovery and out of pain of patients. However, surgeons are required to accumulate various experiences for this surgery since the difficulty in perceiving the positions of tissues by the limited field of view (FOV) of laparoscopes and the operational difficulties of forceps. In this paper, we propose a new laparoscopic surgery supporting system using projected images. The image of the FOV of a laparoscope is projected directly onto the abdominal surface of a patient. The shape distortion of the projected images produced by the unevenness of the abdominal surface is corrected by grating projection. The distortion due to the viewing angle of the surgeon is also corrected by using an electromagnetic tracking sensor. It is shown that the proposed system is significant to laparoscopic surgery, particularly for forceps insertion, by experiments using a model of the abdomen made with a dry box.

Observer performance studies are time-consuming tasks, both for the participating observers and for the scientists collecting and analyzing the data. A possible way to optimize such studies is to perform the study in a completely digital environment. A software tool - ViewDEX (Viewer for Digital Evaluation of X-ray images) - has been developed in Java, enabling it to function on almost any computer. ViewDEX is a DICOM-compatible software tool that can be used to display medical images with simultaneous registration of the observer's response. ViewDEX is designed so that the user in a simple way can alter the types of questions and images presented to the observers, enabling ROC, MAFC and visual grading studies to be conducted in a fast and efficient way. The software can also be used for bench marking and for educational purposes. The results from each observer are saved in a log file, which can be exported for further analysis. The software is freely available for non-commercial purposes.

During the past decade, various volume visualization techniques have been developed for different purposes, and many of them, such as direct volume rendering, maximum intensity projection and non-photorealistic rendering, have been implemented on consumer graphics hardware for real time visualization. However, effective multi-volume visualization, a way to establish the visual connections between two or more types of data, has not been adequately addressed even though it has wide applications in medical imaging and numerical simulation based on 3D physical model. In this paper, we aim to develop an effective GPU-based system for multi-volume visualization which is able to reveal both the connections and distinctions among multiple volume data. To address the main challenge for multi-volume visualization on how to establish the visual correspondences while maintaining the distinctive information among multiple volumes, a multi-level distinction mechanism is developed including 2D transfer function, mixed rendering modes, and volume clipping. Taking advantage of the fast hardware-supported processing capabilities, the system is implemented based on the GPU programming. Several advanced volume rendering techniques based on segmented volume are also implemented. The resulting visualization is a highly interactive image fusion system with high quality image and three-level volume distinction. We demonstrate the effectiveness of our system with a case study in which the heat effect on brain tumor, represented as a temperature volume resulting from high intensity focused ultrasound beam exposure over time, is visualized in the context of a MRI head volume.

This paper presents an efficient graphics hardware-based method to segment and visualize level-set surfaces as interactive rates. Our method is composed of page manager, level-set solver, and volume renderer. The page manager which performs in CPU generates page table, inverse page table and available page stack as well as processes the activation and inactivation of pages. The level-set solver computes only voxels near the iso-surface. To run efficiently on GPUs, volume is decomposed into a set of small pages. Only those pages with non-zero derivatives are stored on GPU. These active pages are packed into a large 2D texture memory. The level-set partial differential equation (PDE) is computed directly on this packed format. The page manager is used to help managing the packing of the active data. The volume renderer performs volume rendering of the original data simultaneously with the evolving level set in GPU. Experimental results using two chest CT datasets show that our graphics hardware-based level-set method is much faster than software-based one.

This paper explores new methods to visualize and fuse multi-2D bioluminescence imaging (BLI) data with structural imaging modalities such as micro CT and MR. A geometric, back-projection-based 3D reconstruction for superficial lesions from multi-2D BLI data is presented, enabling a coarse estimate of the 3D source envelopes from the multi-2D BLI data. Also, an intuitive 3D landmark selection is developed to enable fast BLI / CT registration. Three modes of fused BLI / CT visualization were developed: slice visualization, carousel visualization and 3D surface visualization. The added value of the fused visualization is demonstrated in three small-animal experiments, where the sensitivity of BLI to detect cell clusters is combined with anatomical detail from micro-CT imaging.

The development of the concepts within 3DVIEWNIX and of the software system 3DVIEWNIX itself dates back to the 1970s. Since then, a series of software packages for Computer Assisted Visualization and Analysis (CAVA) of images came out from our group, 3DVIEWNIX released in 1993, being the most recent, and all were distributed with source code. CAVASS, an open source system, is the latest in this series, and represents the next major incarnation of 3DVIEWNIX. It incorporates four groups of operations: IMAGE PROCESSING (including ROI, interpolation, filtering, segmentation, registration, morphological, and algebraic operations), VISUALIZATION (including slice display, reslicing, MIP, surface rendering, and volume rendering), MANIPULATION (for modifying structures and surgery simulation), ANALYSIS (various ways of extracting quantitative information). CAVASS is designed to work on all platforms. Its key features are: (1) most major CAVA operations incorporated; (2) very efficient algorithms and their highly efficient implementations; (3) parallelized algorithms for computationally intensive operations; (4) parallel implementation via distributed computing on a cluster of PCs; (5) interface to other systems such as CAD/CAM software, ITK, and statistical packages; (6) easy to use GUI. In this paper, we focus on the image processing operations and compare the performance of CAVASS with that of ITK. Our conclusions based on assessing performance by utilizing a regular (6 MB), large (241 MB), and a super (873 MB) 3D image data set are as follows: CAVASS is considerably more efficient than ITK, especially in those operations which are computationally intensive. It can handle considerably larger data sets than ITK. It is easy and ready to use in applications since it provides an easy to use GUI. The users can easily build a cluster from ordinary inexpensive PCs and reap the full power of CAVASS inexpensively compared to expensive multiprocessing systems which are less efficient for CAVA operations.

After drawing and stacking contours of a structure, which is identified in the serially sectioned images, three-dimensional (3D) image can be made by surface reconstruction. Usually, software is composed for the surface reconstruction. In order to compose the software, medical doctors have to acquire the help of computer engineers. So in this research, surface reconstruction of stacked contours was tried by using commercial software. The purpose of this research is to enable medical doctors to perform surface reconstruction to make 3D images by themselves. The materials of this research were 996 anatomic images (1 mm intervals) of left lower limb, which were made by serial sectioning of a cadaver. On the Adobe Photoshop, contours of 114 anatomic structures were drawn, which were exported to Adobe Illustrator files. On the Maya, contours of each anatomic structure were stacked. On the Rhino, superoinferior lines were drawn along all stacked contours to fill quadrangular surfaces between contours. On the Maya, the contours were deleted. 3D images of 114 anatomic structures were assembled with their original locations preserved. With the surface reconstruction technique, developed in this research, medical doctors themselves could make 3D images of the serially sectioned images such as CTs and MRIs.

An exploration of techniques for developing intuitive, and efficient user interfaces for virtual reality systems. Work seeks to understand which paradigms from the better-understood world of 2D user interfaces remain viable within 3D environments. In order to establish this a new user interface was created that applied various understood principles of interface design. A user study was then performed where it was compared with an earlier interface for a series of medical visualization tasks.

Surgical navigation systems visualize the positions and orientations of surgical instruments and implants as graphical overlays onto a medical image of the operated anatomy on a computer monitor. The orthopaedic surgical navigation systems could be categorized according to the image modalities that are used for the visualization of surgical action. In the so-called CT-based systems or 'surgeon-defined anatomy' based systems, where a 3D volume or surface representation of the operated anatomy could be constructed from the preoperatively acquired tomographic data or through intraoperatively digitized anatomy landmarks, a photorealistic rendering of the surgical action has been identified to greatly improve usability of these navigation systems. However, this may not hold true when the virtual representation of surgical instruments and implants is superimposed onto 2D projection images in a fluoroscopy-based navigation system due to the so-called image occlusion problem. Image occlusion occurs when the field of view of the fluoroscopic image is occupied by the virtual representation of surgical implants or instruments. In these situations, the surgeon may miss part of the image details, even if transparency and/or wire-frame rendering is used. In this paper, we propose to use non-photorealistic rendering to overcome this difficulty. Laboratory testing results on foamed plastic bones during various computer-assisted fluoroscopybased surgical procedures including total hip arthroplasty and long bone fracture reduction and osteosynthesis are shown.

The inspection of a patient's data for diagnostics, therapy planning or therapy guidance involves an increasing number of 3D data sets, e.g. acquired by different imaging modalities, with different scanner settings or at different times. To enable viewing of the data in one consistent anatomical context fused interactive renderings of multiple 3D data sets are desirable. However, interactive fused rendering of typical medical data sets using standard computing hardware remains a challenge. In this paper we present a method to render multiple 3D data sets. By introducing local rendering functions, i.e. functions that are adapted to the complexity of the visible data contained in the different regions of a scene, we can ensure that the overall performance for fused rendering of multiple data sets depends on the actual amount of visible data. This is in contrast to other approaches where the performance depends mainly on the number of rendered data sets. We integrate the method into a streaming rendering architecture with brick-based data representations of the volume data. This enables efficient handling of data sets that do not fit into the graphics board memory and a good utilization of the texture caches. Furthermore, transfer and rendering of volume data that does not contribute to the final image can be avoided. We illustrate the benefits of our method by experiments with clinical data.

Volume ray casting algorithm is widely recognized for high quality volume visualization. However, when rendering very large volume data sets, the original ray casting algorithm will lead to very inefficient random accesses in disk and make it very slowly to render the whole volume data set. In order to solve this problem, an efficient out-of-core volume ray casting method with a new out-of-core framework for processing large volume data sets based on consumer PC hardware is proposed in this paper. The new framework gives a transparent and efficient access to the volume data set cached in disk, while the new volume ray casting method minimizes the data exchange between hard disk and physical memory and performs comparatively fast high quality volume rendering. The experimental results indicate that the new method and framework are effective and efficient for the visualization of very large medical data sets.

For the pre-operative definition of a surgical workspace for Navigated Control^® Functional Endoscopic Sinus Surgery (FESS), we developed a semi-automatic image processing system. Based on observations of surgeons using a manual system, we implemented a workflow-based engineering process that led us to the development of a system reducing time and workload spent during the workspace definition. The system uses a feature based on local curvature to align vertices of a polygonal outline along the bone structures defining the cavities of the inner nose. An anisotropic morphologic operator was developed solve problems arising from artifacts from noise and partial volume effects. We used time measurements and NASA's TLX questionnaire to evaluate our system.

Radiofrequency ablation is becoming an increasingly attractive option for minimally invasive treatment of liver tumors. In this procedure, the tumor and its margin are ablated using radiofrequency ablation probes that cover a region from 2cm to 7cm in diameter. For a large or irregularly shaped tumor, multiple ablations with overlapping probe placements are required. In this paper, we propose a treatment planning system to optimize these placements. A general optimization framework based on inverse planning methods is designed to generate the treatment plan. An objective function is defined to describe the coverage of the ablation volumes. Powell's method and simulated annealing algorithms are used to find the solution. Pre-computed mask volumes and an initial placement based on a Euclidean Distance Transform are used to speed up the computation, which can generally take a few seconds to several minutes. To ensure accurate placement of the ablation probe, we also propose a system architecture for integrating the treatment planning system with our previously developed image-guided surgery system, which uses an electromagnetic tracking device. We present some preliminary results from synthetic data to validate our treatment planning algorithm and system concept.

Percutaneous radiofrequency ablation is one of the most promising alternatives to open surgery for the treatment of liver cancer. This operation is a minimally invasive procedure that consists in inserting a needle in targeted tissues that are destroyed by heat. The success of such an operation mainly depends on the accuracy of the needle insertion, making it possible to destroy the whole tumor, while avoiding damages on other organs and minimizing risks of a local recurrence. We are developing a software that applies planning rules on patient-specific 3D reconstructions, in order to suggest relevant options for the choice of a path to the tumor, and that displays various information allowing to adjust the final choice. In this context we propose a method to compute automatically, quickly, and accurately, the possible insertion areas on the skin. Within these areas, an insertion of the probe targeting the tumor respects the numerous strong (boolean) constraints required for a radiofrequency ablation. Besides, these insertion zones define the research domain of the optimization process, taking into account soft constraints to refine the solutions. They are also displayed on the skin of the virtual patient to inform the physician about the different possibilities specific to each case, allowing him at the end of the automatic process, to modify interactively the proposed strategy, with a real-time update of the related information. We discuss in this paper about the importance of a precise delineation of these areas.

Due to very complex structure of nasal area that is covered by facial bones, a tracking of surgical instruments on the preoperative CT image is very important for obtaining an improved image guidance as well as preventing surgical accidents in the paranasal sinus surgery. In this contribution, we present our recently developed an efficient and compact navigation system for paranasal sinus surgery and its first clinical trial. In our system, we use an optical-based 3D range imaging device intra-operatively, in order to achieve registration and a tracking of instruments. Before the intervention, the range image of patient's face is acquired by a 3D range scanner and registered to corresponding surface extracted from the preoperative CT images. The surgical instrument fitted with spherical markers that also can be measured by range scanning device, is tracked during the procedure. The main advantages of our system are (a) markerless on the patient's body, (b) an easy semiautomatic registration, (c) frameless during surgery, thus, it is feasible to update a registration and to restart the tracking when a patient moves. In this paper, we describe a summary of used techniques in our approach including the benefits and limitations of the system, experimental results using a precise model based on a human paranasal structure and a first clinical trial in the surgical room.

In cochlear implant surgery an electrode array is permanently implanted to stimulate the auditory nerve and allow deaf people to hear. Current surgical techniques require wide excavation of the mastoid region of the temporal bone and one to three hours time to avoid damage to vital structures. Recently a far less invasive approach has been proposed-percutaneous cochlear access, in which a single hole is drilled from skull surface to the cochlea. The drill path is determined by attaching a fiducial system to the patient's skull and then choosing, on a pre-operative CT, an entry point and a target point. The drill is advanced to the target, the electrodes placed through the hole, and a stimulator implanted at the surface of the skull. The major challenge is the determination of a safe and effective drill path, which with high probability avoids specific vital structures-the facial nerve, the ossicles, and the external ear canal-and arrives at the basal turn of the cochlea. These four features lie within a few millimeters of each other, the drill is one millimeter in diameter, and errors in the determination of the target position are on the order of 0.5mm root-mean square. Thus, path selection is both difficult and critical to the success of the surgery. This paper presents a method for finding optimally safe and effective paths while accounting for target positioning error.

We evaluate two core modules of a novel soft tissue navigation system. The system estimates the position of a hidden target (e.g. a tumor) during a minimally invasive intervention from the location of a set of optically tracked needle-shaped navigation aids which are placed in the vicinity of the target. The initial position of the target relative to the navigation aids is obtained from a CT scan. The accuracy of the entire system depends on (a) the accuracy for locating a set of navigation aids in a CT image, (b) the accuracy for determining the positions of the navigation aids during the intervention by means of optical tracking, (c) the accuracy for tracking the applicator (e.g. the biopsy needle), and (d) the accuracy of the real-time deformation model which continuously computes the location of the initially determined target point from the current positions of the navigation aids. In this paper, we focus on the first two aspects. We introduce the navigation aids we constructed for our system and show that the needle tips can be tracked with submillimeter accuracy. Furthermore, we present and evaluate three methods for registering a set of navigation aid models with a given CT image. The fully-automatic algorithm outperforms both the manual method and the semi-automatic algorithm, yielding an average distance of 0.27 ± 0.08 mm between the estimated needle tip position and the reference position.

The design of mobile X-ray C-arm equipment with image tomography and surgical guidance capabilities involves the retrieval of repeatable gantry positioning in three-dimensional space. Geometry misrepresentations can cause degradation of the reconstruction results with the appearance of blurred edges, image artifacts, and even false structures. It may also amplify surgical instrument tracking errors leading to improper implant placement. In our prior publications we have proposed a C-arm 3D positioner calibration method comprising separate intrinsic and extrinsic geometry calibration steps. Following this approach, in the present paper, we extend the intrinsic geometry calibration of C-gantry beyond angular positions in the orbital plane into angular positions on a unit sphere of isocentric rotation. Our method makes deployment of markerless interventional tool guidance with use of high-resolution fluoro images and electromagnetic tracking feasible at any angular position of the tube-detector assembly. Variations of the intrinsic parameters associated with C-arm motion are measured off-line as functions of orbital and lateral angles. The proposed calibration procedure provides better accuracy, and prevents unnecessary workflow steps for surgical navigation applications. With a slight modification, the Misalignment phantom, a tool for intrinsic geometry calibration, is also utilized to obtain an accurate 'image-to-sensor' mapping. We show simulation results, image quality and navigation accuracy estimates, and feasibility data acquired with the prototype system. The experimental results show the potential of high-resolution CT imaging (voxel size below 0.5 mm) and confident navigation in an interventional surgery setting with a mobile C-arm.

Image guided surgery typically relies on preoperatively acquired image data. The major disadvantage is that changes that occur between image data acquisition and surgery, are not reflected by the image data. Furthermore, with the beginning of surgery, the image data is not valid anymore. The use of an intraoperative Computer Tomography (CT) suite is reported. The system consists of a single slice spiral CT scanner (Somatom Emotion, Siemens, Forchheim, Germany) and a operating room table with a radiolucent board (AWIGS, Maquet, Rastatt, Germany) to put the patient on. During CT scanning, the patient on the board is immobile, while the gantry of the CT scanner is moved on rails that are embedded in the floor of the operating room. Image data can be transferred immediately via local area network to a frameless stereotaxy system (VectorVision, Brainlab, Heimstetten, Germany). Furthermore, intraoperative image data acquisition in connection with the navigation system can be used for automated patient to image registration. Using the infrared camera of the navigation system, the position of the gantry can be measured during CT image data acquisition. With the patient being tracked simultaneously, registration of the image data can be performed fully automatically. The clinical use of intraoperative CT image data acquisition, the intraoperative workflow of the system, and the clinical applications are demonstrated.

A central purpose of image-guidance is to assist the interventionalist with feedback of geometric performance in the direction of therapy delivery. Tradeoffs exist between accuracy, precision and the constraints imposed by parameters used in the generation of images. A framework that uses geometric performance as feedback to control these parameters can balance such tradeoffs in order to maintain the requisite localization precision for a given clinical procedure. We refer to this principle as Active Image-Guidance (AIG). This framework requires estimates of the uncertainty in the estimated location of the object of interest. In this study, a simple fiducial marker detected under X-ray fluoroscopy is considered and it is shown that a relation exists between the applied imaging dose and the uncertainty in localization for a given observer. A robust estimator of the location of a fiducial in the thorax during respiration under X-ray fluoroscopy is demonstrated using a particle filter based approach that outputs estimates of the location and the associated spatial uncertainty. This approach gives an rmse of 1.3mm and the uncertainty estimates are found to be correlated with the error in the estimates. Furthermore, the particle filtering approach is employed to output location estimates and the associated uncertainty not only at instances of pulsed exposure but also between exposures. Such a system has applications in image-guided interventions (surgery, radiotherapy, interventional radiology) where there are latencies between the moment of imaging and the act of intervention.

In the X-ray coronary digital subtraction angiography, there are serious motion artifacts and noises, and backgrounds such as ribs, spine, cathers and etc, which are tube structures and like vessels. It's difficult to separate vessels from the background automatically if they are close each other. In this paper, an automatic extraction of coronary vessels from X-ray digital subtraction angiography is proposed. We used edge preserving smooth filter to reduce the noises in the images and keep the vessel edge firstly. Then affine and B-spline based FFD nonrigid registration is applied to the images. Compared with the segmentation method, the proposed method can remove background greatly and extract the coronary vessel very well.

Interventional cardiac electrophysiology (EP) procedures are typically performed under X-ray fluoroscopy for visualizing catheters and EP devices relative to other highly-attenuating structures such as the thoracic spine and ribs. These projections do not however contain information about soft-tissue anatomy and there is a recognized need for fusion of conventional fluoroscopy with pre-operatively acquired cardiac multislice computed tomography (MSCT) volumes. Rapid 2D-3D integration in this application would allow for real-time visualization of all catheters present within the thorax in relation to the cardiovascular anatomy visible in MSCT. We present a method for rapid fusion of 2D X-ray fluoroscopy with 3DMSCT that can facilitate EP mapping and interventional procedures by reducing the need for intra-operative contrast injections to visualize heart chambers and specialized systems to track catheters within the cardiovascular anatomy. We use hardware-accelerated ray-casting to compute digitally reconstructed radiographs (DRRs) from the MSCT volume and iteratively optimize the rigid-body pose of the volumetric data to maximize the similarity between the MSCT-derived DRR and the intra-operative X-ray projection data.

Endovascular interventional procedures are being used more frequently in cardiovascular surgery. Unfortunately, procedural failure, e.g., vessel dissection, may occur and is often related to improper guidewire and/or device selection. To support the surgeon's decision process and because of the importance of the guidewire in positioning devices, we propose a method to determine the guidewire path prior to insertion using a model of its elastic potential energy coupled with a representative graph construction. The 3D vessel centerline and sizes are determined for a specified vessel. Points in planes perpendicular to the vessel centerline are generated. For each pair of consecutive planes, a vector set is generated which joins all points in these planes. We construct a graph representing these vector sets as nodes. The nodes representing adjacent vector sets are joined by edges with weights calculated as a function of the angle between the corresponding vectors (nodes). The optimal path through this weighted directed graph is then determined using shortest path algorithms, such as topological sort based shortest path algorithm or Dijkstra's algorithm. Volumetric data of an internal carotid artery phantom (Ø 3.5mm) were acquired. Several independent guidewire (Ø 0.4mm) placements were performed, and the 3D paths were determined using rotational angiography. The average RMS distance between the actual and the average simulated guidewire path was 0.7mm; the computation time to determine the path was 3 seconds. The ability to predict the guidewire path inside vessels may facilitate calculation of vessel-branch access and force estimation on devices and the vessel wall.

In minimally invasive image-guided surgical interventions, different imaging modalities, such as magnetic resonance imaging (MRI), computed tomography (CT), and real-time three-dimensional (3D) ultrasound (US), can provide complementary, multi-spectral image information. Multimodality dynamic image registration is a well-established approach that permits real-time diagnostic information to be enhanced by placing lower-quality real-time images within a high quality anatomical context. For the guidance of cardiac procedures, it would be valuable to register dynamic MRI or CT with intraoperative US. However, in practice, either the high computational cost prohibits such real-time visualization of volumetric multimodal images in a real-world medical environment, or else the resulting image quality is not satisfactory for accurate guidance during the intervention. Modern graphics processing units (GPUs) provide the programmability, parallelism and increased computational precision to begin to address this problem. In this work, we first outline our research on dynamic 3D cardiac MR and US image acquisition, real-time dual-modality registration and US tracking. Then we describe image processing and optimization techniques for 4D (3D + time) cardiac image real-time rendering. We also present our multimodality 4D medical image visualization engine, which directly runs on a GPU in real-time by exploiting the advantages of the graphics hardware. In addition, techniques such as multiple transfer functions for different imaging modalities, dynamic texture binding, advanced texture sampling and multimodality image compositing are employed to facilitate the real-time display and manipulation of the registered dual-modality dynamic 3D MR and US cardiac datasets.

Intra-cardiac echocardiography (ICE) is commonly used to guide intra-cardiac procedures, such as the treatment of atrial fibrillation (AF). However, effective surgical navigation based on ICE images is not trivial, due to the low signal-to-noise ratio (SNR) and limited field of view of ultrasound (US) images. The interpretation of ICE can be significantly improved if correctly placed in the context of three-dimensional magnetic resonance (MR) or computed tomography (CT) images by simultaneously presenting the complementary anatomical information from the two modalities. The purpose of this research is to demonstrate the feasibility of multimodality image registration of 2D intra-cardiac US images with 3D computed tomography (CT) images. In our previous work, a two-step registration procedure has been proposed to register US images with MR images and was validated on a patient dataset. In this work, we extend the two-step method to intra-cardiac procedures and provide a detailed assessment of registration accuracy by determining the target registration errors (TRE) on a heart phantom, which had fiducial markers affixed to the surface to facilitate evaluation of registration accuracy. The resultant TRE on the heart phantom was 3.7 mm. This result is considered to be acceptable for guiding a probe in the heart during ablative therapy for atrial fibrillation. To our knowledge, there is no previous report describing multimodality registration of 2D intra-cardiac US to high-resolution 3D CT.

The use of intensity modulated radiation therapy promises to spare organs at risk by applying better dose distribution on the tumor. The specific challenge of this methods is the exact positioning of the patient and the localization of the exposured organ. With respect to the filling of rectum and bladder the prostate can move several millimeters up to centimeters. Therefore, the position of the prostate should be determinated and corrected daily before irradiation. We used a B-mode US machine (Ultramark 9, advanced Technology Laboratories, USA) which was calibrated using an optical tracking system (Polaris, NDI, Can). After correct positioning of the patient in the simulation room three anatomical markers (apex prostate, prostate lateral sinister/dexter) were identified and their positions calculated with respect to the coordinate system of the simulator. The same situation is given in the treatment room. Both, simulator and accelerator are registered by a simple point-to-point registration using a block with five drilled holes with known coordinates in the block coordinate system. The block is aligned by means of laser markers. When the patient is placed on the treatment table, the three anatomical landmarks are located on the US images and their positions are calculated with respect to the coordinate system of the treatment room. Applying a point-to-point registration results in a rotation matrix and a translation vector in the desired coordinate system which can be used for repositioning by translating and rotating the patient table. Additionally, a fiducial registration error (FRE) is calculated which gives a dimension of the accuracy the three points were identified. We found an fiducial registration error (FRE) of 2.4 mm +/- 1.2 mm for the point-to-point registration of the anatomical landmarks. The FRE for the point-to-point registration between the block and the optical tracking system was 0.5 mm +/- 0.2 mm. According to the US calibration we found an error of 0.8 mm +/- 0.2 mm.

To obtain 3D ultrasound image, traditionally, a 1D ultrasound transducer and an orientation system are used to get a series of 2D images and their positions. From these 2D images and position information, a 3D image is reconstructed. In this paper, the accuracy of the 3D image is determined by the accuracy of the position information, furthermore it affects the result of the measure from the 3D image. When in rotational scanning mode, the reconstructed 3D image is sensitive to the calibration parameters, including the orientation difference as well as the offset between the central line of each 2D US image and the rotational axis. To address this type of effects, we developed a 3D reconstruction technique considering the calibration uncertainty by building a transformation model with a 360 degrees scanning, and an accurate 3D reconstruction with calibration uncertainty is achieved. The experiments with both synthetic and scanning phantom data demonstrated the feasibility of our approach.

Ultrasound imaging has revolutionized the treatment of prostate cancer by producing increasingly accurate models of the prostate and influencing sophisticated targeting procedures for the insertion of radioactive seeds during brachytherapy. Three-dimensional (3D) ultrasound imaging, which allows 3D models of the prostate to be constructed from a series of two-dimensional images, helps to accurately target and implant seeds into the prostate. We have developed a compact robotic apparatus, as well as an effective method for guiding and controlling the insertion of transperineal needles into the prostate. This device has been designed to accurately guide a needle in 3D space so that the needle can be inserted into the prostate at an angle that does not interfere with the pubic arch. The physician can adjust manually or automatically the position of the apparatus in order to place several radioactive seeds into the prostate at designated target locations. Because many physicians are wary of conducting robotic surgical procedures, the apparatus has been developed so that the physician can position the needle for manual insertion and apply a method for manually releasing the needle without damaging the apparatus or endangering the patient.

The asymmetric vascular stent (AVS) is a new minimally invasive endovascular device, designed to reduce the potential for further growth and rupture of cerebral aneurysms by substantially modifying the aneurysmal inflow. The low porosity part of the AVS or patch must be deployed to either completely or partially cover the aneurysm orifice. In this study, we investigated the effect on aneurysm hemodynamics of partial coverage with an asymmetric stent using Computational Fluid Dynamics (CFD) analysis and visualization. The low porosity patch of an asymmetric stent was computationally created and deformed to fit into the vessel lumen. Such a patch was placed both in an idealized aneurysm model and in a patient-specific aneurysm model to cover only a portion of the aneurysm orifice either proximally or distally according to the flow direction. The CFD-generated hemodynamic image sequences in the untreated and stented aneurysm models were compared. The asymmetric stent effectively attenuated the aneurysmal flow when the primary inflow was blocked by the patch. Consequently, the Wall Shear Stress (WSS) was reduced, and flow stasis was substantially increased by stenting. For the idealized model, distal placement was better for reducing the inflow jet, whereas for the patient-specific model proximal placement was better. We can conclude that CFD visualizations may be essential to guide either the optimal positioning of a small low porosity region of the AVS or the acceptability of inaccurate placement of a larger AVS patch for partial aneurysm orifice coverage.

Brain shift poses a significant challenge to accurate image-guided neurosurgery. To this end, finite element (FE) brain models have been developed to estimate brain motion during these procedures. The significance of the brain-skull boundary conditions (BCs) for accurate predictions in these models has been explored in dynamic impact and inertial rotation injury computational simulations where the results have shown that the brain mechanical response is sensitive to the type of BCs applied. We extend the study of brain-skull BCs to quasi-static brain motion simulations which prevail in neurosurgery. Specifically, a frictionless brain-skull BC using a contact penalty method master-slave paradigm is incorporated into our existing deformation forward model (forced displacement method). The initial brain-skull gap (CSF thickness) is assumed to be 2mm for demonstration purposes. The brain surface nodes are assigned as either fixed (at bottom along the gravity direction), free (at brainstem), with prescribed displacement (at craniotomy) or as slave nodes potentially in contact with the skull (all the remaining). Each slave node is assigned a penalty parameter (β=5) such that when the node penetrates the rigid body skull inner-surface (master surface), a contact force is introduced proportionally to the penetration. Effectively, brain surface nodes are allowed to move towards or away from the cranium wall, but are ultimately restricted from penetrating the skull. We show that this scheme improves the model's ability to represent the brain-skull interface.

Mesh quality is an important factor for stable, repeatable numerical simulations. The Delaunay method is widely used for creation of 3D tetrahedral meshes. Two-dimensional triangulation via Delaunay exhibits the mathematical property of maximizing the minimum interior angle. This feature provides excellent quality meshes for a given node deployment. However, the 3D equivalent of this property, i.e. to maximize the minimum solid angle, is not assured with 3D Delaunay. The tetrahedron's interior solid angle is directly related to mesh quality, but it is independent of the Delaunay process. Consequently, sliver elements and poor quality meshes can be created via Delaunay tetrahedral formation. In this paper, we describe a method for maximizing the minimum solid angle of tetrahedral meshes by changing the locations of non-boundary nodes. The displacement of nodes uses a gradient-based approach. The process is iterative and terminates when the mesh quality exceeds a user specified quality or convergence criterion. The technique is robust. The relocation of vertices is local which avoids significant deformation of the mesh. The results show considerable improvements in mesh quality. Using a 3D human brain mesh (27,000+ elements), our algorithm reduced the number of ill-formed elements three fold. We are extending this approach to allow tangential motion along the boundary surfaces. Currently all boundary nodes are fixed which constrains some of the element qualities.

Biomechanical models of brain deformation are useful tools for estimating the shift that occurs during neurosurgical interventions. Incorporation of intra-operative data into the biomechanical model improves the accuracy of the registration between the patient and the image volume. The representer method to solve the adjoint equations (AEM) for data assimilation has been developed. In order to improve the computational efficiency and to process more intraoperative data, we modified the adjoint equation method by changing the way in which intraoperative data is applied. The current formulation is developed around a point-based data-model misfit. Surface based data-model misfit could be a more robust and computationally efficient technique. Our approach is to express the surface misfit as the volume between the measured surface and model predicted surface. An iterative method is used to solve the adjoint equations. The surface misfit criterion is tested in a cortical distension clinical case and compared to the results generated with the prior point-based methodology solved either iteratively or with the representer algorithm. The results show that solving the adjoint equations with an iterative method improves computational efficiency dramatically over the representer approach and that reformulating the minimization criterion in terms of a surface description is even more efficient. Applying intra-operative data in the form of a surface misfit is computationally very efficient and appears promising with respect to its accuracy in estimating brain deformation.

In this paper, we propose an adaptive seeding strategy for visualization of diffusion tensor magnetic resonance imaging (DT-MRI) data using streamtubes. DT-MRI is a medical imaging modality that captures unique water diffusion properties and fiber orientation information of the imaged tissues. Visualizing DT-MRI data using streamtubes has the advantage that not only the anisotropic nature of the diffusion is visualized but also the underlying anatomy of biological structures is revealed. This makes streamtubes significant for the analysis of fibrous tissues in medical images. In order to avoid rendering multiple similar streamtubes, an adaptive seeding strategy is employed which takes into account similarity of tensors in a given region. The goal is to automate the process of generating seed points such that regions with dissimilar tensors are assigned more seed points compared to regions with similar tensors. The algorithm is based on tensor dissimilarity metrics that take into account both diffusion magnitudes and directions to optimize the seeding positions and density of streamtubes in order to reduce the visual clutter. Two recent advances in tensor calculus and tensor dissimilarity metrics are utilized: the Log-Euclidean and the J-divergence. Results show that adaptive seeding not only helps to cull unnecessary streamtubes that would obscure visualization but also do so without having to compute the culled streamtubes, which makes the visualization process faster.

The automated segmentation of brain structures is an important step in many neuroimaging analyses. A variety of automated segmentation tools exist, however, most segmentation results are imperfect, and require manual editing of the resulting contours or surfaces. A new, integrated segmentation and visualization tool, the LONI Anatomist, was developed to provide an open architecture for applying automated segmentation algorithms and interactive tools to manually edit the automated segmentation results. Two automated segmentation algorithms were developed to skullstrip MR brain images and were integrated in the LONI Anatomist: a two-dimensional model-based level set (2D MLS) algorithm and a three-dimensional MLS algorithm. These MLS algorithms were based on the Level Set methods by incorporating two constraints into the level set framework to evolve the zero level set surface in 2D space and 3D space respectively. In the LONI Anatomist, the evolution of the level set was displayed in real time, and final results were corrected using easy-to-use interactive editing tools. Additional tools were provided to visualize the results, such as color overlays of 2D contours over the original gray-scale slices, 3D surface visualization, etc. The LONI Anatomist was implemented in Java using a portable imaging framework (the jViewbox) for medical image display and manipulation, using the Java Image I/O plug-ins for reading/writing DICOM, MINC, ANALYZE image files, and using the Java Advanced Imaging classes for image processing. The design of the system provides a framework for researchers to integrate more mathematical algorithms for converting the algorithms into practical use.

Often within the clinical environment of a neurosurgical brain tumor procedure, the surgeon is faced with the difficulty of orienting the patient's head to maximize the success of removing the pathology. Currently, these decisions are based on the experience of the surgeon. The primary objective of this paper is to demonstrate how a mathematical model can be used to evaluate the different patient positioning for tumor resection therapies. Specifically, therapies involving gravity-induced shift are used to demonstrate how a series of candidate approaches to the tumor can result in significantly different deformation behavior of brain tissue. To quantitatively assess the advantages and disadvantages of potential approaches, three different midline tumor locations were used to evaluate for the extent of tumor exposure and the magnitude of tensile stress at the brain-tumor interface, both of which are reliable indicators of the ease of resection. Preliminary results indicate that the lateral decubitus position is best suited for midline tumors.

PET (Positron Emission Tomography) scanning has become a dominant force in oncology care because of its ability to identify regions of abnormal function. The current generation of PET scanners is focused on whole-body imaging, and does not address aspects that might be required by surgeons or other practitioners interested in the function of particular body parts. We are therefore developing and testing a new class of hand-operated molecular imaging scanners designed for use with physical examinations and intraoperative visualization. These devices integrate several technological advances, including (1) nanotechnology-based quantum photodetectors for high performance at low light levels, (2) continuous position tracking of the detectors so that they form a larger 'virtual detector', and (3) novel reconstruction algorithms that do not depend on a circular or ring geometry. The first incarnations of this device will be in the form of a glove with finger-mounted detectors or in a "sash" of detectors that can be draped over the patient. Potential applications include image-guided biopsy, surgical resection of tumors, assessment of inflammatory conditions, and early cancer detection. Our first prototype is in development now along with a clinical protocol for pilot testing.

Low-dose X-ray imaging, diagnosis by image analysis and multi-modal medical imaging are example aspects that lead to more advanced image processing algorithms and the corresponding platforms on which they have to be executed. In this paper, we investigate the applicability of commercially available off-the-shelf components for a new computing platform. In the analysis, we will comply to some specific use cases. In cardiovascular minimal invasive surgery, physicians require low-latency imaging applications, as their actions must be directly visible on the screen. Typical image-processing algorithms in this domain are based on multi-resolution decomposition, noise reduction, image analysis and enhancement techniques. We have compared various solutions for possible processing architectures. The most interesting technology areas for constituting a new architecture are presented and we discuss the mapping of the use cases onto the various architectural proposals. Results show that a heterogeneous architecture gives the highest potential for current and upcoming image-processing applications. However, hardware and software solutions to support low-latency, high-bandwidth image streaming and an efficient concurrent distribution of functionality still need further development. This validates a clear direction for the future, which is based on modeling streaming computing architectures and special interconnect infrastructures.

In surgical incision, it is known that the surgeons control surgical knives by feeling of its reaction force, therefore it is necessary to render realistic haptic for development of the surgical incision training system. In our previous paper, we reported that the surgical incision training system used three translational degree-of-freedom (3-DOF) haptic rendering without consideration of rotational forces, distribution of hardness and viscosity of tissue. In this paper, we propose 6-DOF haptic rendering model for development of incision training system with three translational and rotational forces considered hardness and velocity. In this model, it is possible to render the cut, friction and clamping force acting on a surgical knife and those forces can be displayed on the 6-DOF haptic interface device in real time. It is shown the effectiveness of 6-DOF rendering model in comparison with 3-DOF by subjective rating experiment.

Minimally invasive procedures are increasingly attractive to patients and medical personnel because they can reduce operative trauma, recovery times, and overall costs. However, during these procedures, the physician has a very limited view of the interventional field and the exact position of surgical instruments. We present an image-guided platform for precision placement of surgical instruments based upon a small four degree-of-freedom robot (B-RobII; ARC Seibersdorf Research GmbH, Vienna, Austria). This platform includes a custom instrument guide with an integrated spiral fiducial pattern as the robot's end-effector, and it uses intra-operative computed tomography (CT) to register the robot to the patient directly before the intervention. The physician can then use a graphical user interface (GUI) to select a path for percutaneous access, and the robot will automatically align the instrument guide along this path. Potential anatomical targets include the liver, kidney, prostate, and spine. This paper describes the robotic platform, workflow, software, and algorithms used by the system. To demonstrate the algorithmic accuracy and suitability of the custom instrument guide, we also present results from experiments as well as estimates of the maximum error between target and instrument tip.

This paper proposes an assessment protocol that incorporates both hardware and analysis methods for evaluation of electromagnetic tracker accuracy in different clinical environments. The susceptibility of electromagnetic tracker measurement accuracy is both highly dependent on nearby ferromagnetic interference sources and non-isotropic. These inherent limitations combined with the various hardware components and assessment techniques used within different studies makes the direct comparison of measurement accuracy between studies difficult. This paper presents a multicenter study to evaluate electromagnetic devices in different clinical environments using a common hardware phantom and assessment techniques so that results are directly comparable. Measurement accuracy has been shown to be in the range of 0.79-6.67mm within a 180mm³ sub-volume of the Aurora measurement space in five different clinical environments.

C-arm fluoroscopy is modelled as a perspective projection, the parameters of which are estimated through a calibration procedure. It has been universally accepted that precise intra-procedural calibration is a prerequisite for accurate quantitative C-arm fluoroscopy guidance. Calibration, however, significantly adds to system complexity, which is a major impediment to clinical practice. We challenge the status quo by questioning the assumption that precise intra-procedural C-arm calibration is really necessary. Using our theoretical framework, we derive upper bounds on the effect of mis-calibration on various algorithms like C-arm tracking, 3D reconstruction and surgical guidance in virtual fluoroscopy - some of the most common techniques in intra-operative fluoroscopic guidance. To derive bounds as a function of mis-calibration, we model the error using an a.ne transform. This is fairly intuitive, since small amounts of mis-calibration result in predictably linear transformation of the reconstruction space. Experiments indicate the validity of this approximation even for 50 mm mis-calibrations.

Extraction of centerlines is useful to analyzing objects in medical images, such as lung, bronchia, blood vessels, and colon. Given the noise and other imaging artifacts that are present in medical images, it is crucial to use robust algorithms that are (1) noise tolerant, (2) computationally efficient, (3) accurate and (4) preferably, do not require an accurate segmentation and can directly operate on grayscale data. We propose a new centerline extraction method that employs a Gaussian type probability model to build a more robust distance field. The model is computed using an integration of the image gradient field, in order to estimate boundaries of interest. Probabilities assigned to boundary voxels are then used to compute a modified distance field. Standard distance field algorithms are then applied to extract the centerline. We illustrate the accuracy and robustness of our algorithm on a synthetically generated example volume and a radiologist supervised segmented head MRT angiography dataset with significant amounts of Gaussian noise, as well as on three publicly available medical volume datasets. Comparison to traditional distance field algorithms is also presented.

Constructing an accurate patient-specific 3D bone model from sparse point sets is a challenging task. A priori information is often required to handle this otherwise ill-posed problem. Previously we have proposed an optimal approach for anatomical shape reconstruction from sparse information, which uses a dense surface point distribution model (DS-PDM) as the a priori information and formulates the surface reconstruction problem as a sequential three-stage optimal estimation process including (1) affine registration; (2) statistical morphing; and (3) kernel-based deformation. Mathematically, it is formulated by applying least-squares method to estimate the unknown parameters of linear regression models (the first two stages) and nonlinear regression model (the last stage). However, it is well-known that the least-squares method is very sensitive to outliers. In this paper, we propose an important enhancement that enables to realize stable reconstruction and robustly reject outliers. This is achieved by consistently employing least trimmed squares approach in all three stages of the reconstruction to robustly estimate unknown parameters of each regression model. Results of testing the new approach on a simulated data are shown.

Tackling transluminal angioplasty planning, the aim of our work is to bring, in a patient specific way, solutions to clinical problems. This work focuses on realization of simple simulation scenarios taking into account macroscopic behaviors of stenosis. It means simulating geometrical and physical data from the inflation of a balloon while integrating data from tissues analysis and parameters from virtual tool-tissues interactions. In this context, three main behaviors has been identified: soft tissues crush completely under the effect of the balloon, calcified plaques, do not admit any deformation but could move in deformable structures, the blood vessel wall undergoes consequences from compression phenomenon and tries to find its original form. We investigated the use of Chain-Mail which is based on elements linked with the others thanks to geometric constraints. Compared with time consuming methods or low realism ones, Chain-Mail methods provide a good compromise between physical and geometrical approaches. In this study, constraints are defined from pixel density from angio-CT images. The 2D method, proposed in this paper, first initializes the balloon in the blood vessel lumen. Then the balloon inflates and the moving propagation, gives an approximate reaction of tissues. Finally, a minimal energy level is calculated to locally adjust element positions, throughout elastic relaxation stage. Preliminary experimental results obtained on 2D computed tomography (CT) images (100x100 pixels) show that the method is fast enough to handle a great number of linked-element. The simulation is able to verify real-time and realistic interactions, particularly for hard and soft plaques.

C-arm images suffer from pose dependant distortion, which needs to be corrected for intra-operative quantitative 3D surgical guidance. Several distortion correction techniques have been proposed in the literature, the current state of art using a dense grid pattern rigidly attached to the detector. These methods become cumbersome for intra-operative use, such as 3D reconstruction, since the grid pattern interferes with patient anatomy. The primary contribution of this paper is a framework to statistically analyze the distortion pattern which enables us to study alternate intra-operative distortion correction methods. In particular, we propose a new phantom that uses very few BBs, and yet accurately corrects for distortion. The high dimensional space of distortion pattern can be effectively characterized by principal component analysis (PCA). The analysis shows that only first three eigen modes are significant and capture about 99% of the variation. Phantom experiments indicate that distortion map can be recovered up to an average accuracy of less than 0.1 mm/pixel with these three modes. With this prior statistical knowledge, a subset of BBs can be sufficient to recover the distortion map accurately. Phantom experiments indicate that as few as 15 BBs can recover distortion with average error of 0.17 mm/pixel, accuracy sufficient for most clinical applications. These BBs can be arranged on the periphery of the C-arm detector, minimizing the interference with patient anatomy and hence allowing the grid to remain attached to the detector permanently. The proposed method is fast, economical, and C-arm independent, potentially boosting the clinical viability of applications such as quantitative 3D fluoroscopic reconstruction.

A novel and robust method for automatic scan planning of MRI examinations of knee joints is presented. Clinical knee examinations require acquisition of a 'scout' image, in which the operator manually specifies the scan volume orientations (off-centres, angulations, field-of-view) for the subsequent diagnostic scans. This planning task is time-consuming and requires skilled operators. The proposed automated planning system determines orientations for the diagnostic scan by using a set of anatomical landmarks derived by adapting active shape models of the femur, patella and tibia to the acquired scout images. The expert knowledge required to position scan geometries is learned from previous manually planned scans, allowing individual preferences to be taken into account. The system is able to automatically discriminate between left and right knees. This allows to use and merge training data from both left and right knees, and to automatically transform all learned scan geometries to the side for which a plan is required, providing a convenient integration of the automated scan planning system in the clinical routine. Assessment of the method on the basis of 88 images from 31 different individuals, exhibiting strong anatomical and positional variability demonstrates success, robustness and efficiency of all parts of the proposed approach, which thus has the potential to significantly improve the clinical workflow.

Image guidance in otologic surgery has been thwarted by the need for a non-invasive fiducial system with target registration error (TRE) at the inner ear below 1.5mm. We previously presented a fiducial frame for this purpose that attaches to the upper dentition via patient-specific bite blocks and demonstrated a TRE of 0.73mm (±0.23mm) on cadaveric skulls. In that study, TRE measurement depended upon placement of bone-implanted, intracranial target fiducials-clearly impossible to repeat clinically. Using cadaveric specimens, we recently presented a validation method based on an auditory implant system (BAHA System®; Cochlear Corp., Denver, CO). That system requires a skull-implanted titanium screw behind the ear upon which a bone-anchored hearing aid (BAHA) is mounted. In our validation, we replace the BAHA with a fiducial marker to permit measurement of TRE. That TRE is then used to estimate TRE at an internal point. While the method can be used to determine accuracy at any point within the head, we focus in this study on the inner ear, in particular the cochlea, and we apply the method to patients (N=5). Physical localizations were performed after varying elapsed times since bite-block fabrication, and TRE at the cochlea was estimated. We found TRE to be 0.97mm at the cochlea within one month and 2.5mm after seven months. Thus, while accuracy deteriorates considerably with delays of seven months or more, if this frame is used within one month of the fabrication of the bite-block, it achieves the goal and in fact exhibits submillimetric accuracy.

Ablation is a kind of successful treatment for cancer. The technique inserts a special needle into a tumor and produces heat from Radiofrequency at the needle tip to ablate the tumor. Open configure MR system can take MR images almost real time and now is applied in liver cancer treatments. During a surgery, surgeons select images in which liver tumors are seen clearly, and use them to guide the surgery. However, in some cases with severe chirrhosis, the tumors can't be visualized in the MR images. In such cases, the combination of preoperative CT images will be greatly helpful, if CT images can be registered to the position of MR images accurately. It is a difficult work since the shape of the liver in the MR image is different from that of CT images due to the influent of the surgery. In this paper, we use Bspline based FFD nonrigid image registration to attack the problem. The method includes four steps. Firstly the MRI inhomogeneity is corrected. Secondly, parametric active contour with the gradient vector flow is used to extract the liver as region of interest (ROI) because the method is robust and can obtain satisfied results. Thirdly, affine registration is use to match CT and MR images roughly. Finally, Bspline based FFD nonrigid registration is applied to obtain accuracy registration. Experiments show the proposed method is robust and accuracy.

Image to physical space registration is a very challenging problem in image guided surgical procedures for the liver, due to deformation and paucity of prominent surface anatomical landmarks. Iterative closest point (ICP) algorithm, the surface registration method used for registering the intraoperative laser range scanner (LRS) data with the preoperative CT data in image guided liver surgery, requires a good starting pose to reduce the number of iterations. Currently anatomical landmarks such as vessel bifurcations are used for an initial registration. This paper presents a computational approach to obtain the initial alignment that would reduce contact with probes for registration during surgical procedures. A priori user information about the anatomical orientation of the liver is incorporated and used to orient the point clouds for segmented CT data and LRS liver data. Four points are computationally selected on the anatomical anterior surface of CT point cloud data and corresponding points are localized on the LRS data using the orientation information. These four points are then used to find the rigid transformation using the singular value decomposition method. Nine datasets were tested using the computational approach and the results were evaluated using the anatomical landmarks method as the "gold standard". Seven of the nine datasets converged to the same solution using both the methods. The computational method, being an approximated approach, may increase the number of iterations to converge to the solution. However since the method does not require precise localization of anatomical landmarks, it could potentially reduce OR time.

The procedure currently used for cochlear implementation requires wide surgical exposure to identify anatomic landmarks. At our institution a minimally invasive technique is being developed that will permit to perform the procedure with a small burr hole. This technique does, however, require identifying pre-operatively a surgical path that reaches the cochlea without touching sensitive structures. This path can be found interactively by localizing a point in the facial recess and another point in the basal turn of the cochlea in the pre-operative CT images. Unfortunately, this is a difficult task because these structures are small and difficult to visualize. As an alternative, outlines of those two structures can be drawn first by a qualified surgeon in one image volume selected as an atlas. This atlas can then be registered to other image volumes to permit automatic localization. In this work a 12 parameter affine registration is performed first using mutual information as a similarity measure. After that a non rigid registration algorithm is applied to register the ear in the atlas to the patient's ear. The structures outlined in the atlas are deformed using the computed transformations and the resulting intensity centroids are used to draw the required safe path. The developed algorithm has been tested on eleven ears. In every instance, the path was deemed acceptable.

We are currently developing a PET/CT based navigation system for guidance of biopsies and radiofrequency ablation (RFA) of early stage hepatic tumors. For these procedures, combined PET/CT data can potentially improve current interventions. The diagnostic efficacy of biopsies can potentially be improved by accurately targeting the region within the tumor that exhibits the highest metabolic activity. For RFA procedures the system can potentially enable treatment of early stage tumors, targeting tumors before structural abnormalities are clearly visible on CT. In both cases target definition is based on the metabolic data (PET), and navigation is based on the spatial data (CT), making the system highly dependent upon accurate spatial alignment between these data sets. In our institute all clinical data sets include three image volumes: one CT, and two PET volumes, with and without CT-based attenuation correction. This paper studies the effect of the CT-based attenuation correction on the registration process. From comparing the pairs of registrations from five data sets we observe that the point motion magnitude difference between registrations is on the same scale as the point motion magnitude in each one of the registrations, and that visual inspection cannot identify this discrepancy. We conclude that using non-rigid registration to align the PET and CT data sets is too variable, and most likely does not provide sufficient accuracy for interventional procedures.

The target pose (position and orientation) of a spinal lesion can be determined using image registration of a pair of two-dimensional (2D) x-ray projection images and a pre-treatment three-dimensional (3D) CT image. This is useful for detecting, tracking and correcting for patient movement during image-guided spinal radiotherapy and radiosurgery. We recently developed a fiducial-less 2D-3D spine image registration that localizes spinal targets by directly tracking adjacent skeletal structures and thereby eliminates the need for implanted fiducials. Experience has shown this method to be robust under a wide range of clinical circumstances. However, image artifacts in digitally reconstructed radiographs (DRRs) that can be introduced by breathing during CT scanning or by other surrounding structures such as ribs have the negative effects on image registration performance. Therefore, we present an approach to eliminate the image artifacts in DRRs for a more robust registration. The spinal structures in the CT volume are approximately segmented in a semi-automatic way and saved as a volume of interest (VOI). The DRRs are then generated within the spine VOI for two orthogonal projections. During radiation treatment delivery, two X-ray images are acquired simultaneously in near real time. Then each X-ray image is registered with the DRR image to obtain 2D local displacements of skeletal structures. The 3D tumor position is calculated from the 2D displacements by 2D-to-3D back-projection and geometric transformation. Experiments on clinical data were conducted to evaluate the performance of the improved registration. The results showed that spine segmentation substantially improves image registration performance.

Recent advances in breast cancer imaging have generated new ways to characterize the disease. Many analysis techniques require a method for determining correspondence between a pendant breast surface before and after a deformation. In this paper, an automated point correspondence method that uses the surface Laplacian or the diffusion equation coupled to an isocontour matching and interpolation scheme are presented. This method is compared to a TPS interpolation of surface displacements tracked by fiducial markers. The correspondence methods are tested on two realistic finite element simulations of a breast deformation and on a breast phantom. The Laplace correspondence method resulted in a mean TRE ranging from 1.0 to 7.7 mm for deformations ranging from 13 to 33 mm, outperforming the diffusion method. The TPS method, in part because it utilizes fiducial information, performed better than the Laplace method, with mean TRE ranging from 0.3 to 1.9 mm for the same range of deformations. The Laplace and TPS methods have the potential to be used by analyses requiring point correspondence between deforming surfaces.

Slice-to-Volume registration is a special case of 2D/3D registration where a single slice obtained using a stationary scanner geometry is registered to a pre-interventional diagnostic volume scan. Examples include interventional magnetic resonance imaging (IMRI) or fluoroscopic computed tomography (CT). In a recent study in FluoroCT/ CT registration, we have shown that conventional cross correlation (CC), together with repeated use of conventional local optimization algorithms, provides an optimum measure for slice-to-volume registration for monoenergetic CT imaging data. If the required linear relationship between corresponding pixel pairs is offended (e. g. by using X-rays of different energy or by varying detector characteristics), CC becomes an unreliable measure of image similarity. A more general merit function like normalized mutual information (NMI) serves better in such a case but is stricken with local minima caused by sparse population of joint histograms. We present a novel merit function for 2D/3D registration named stochastic rank correlation (SRC), which is well-suited for intramodal dual-energy imaging. A first evaluation of SRC is given on a set of simulated and clinical FluoroCT/CT scan image data sets.

This paper describes a method for DRR generation as well as for volume gradients projection using hardware accelerated 2D texture mapping and accumulation buffering and demonstrates its application in 2D-3D registration of X-ray fluoroscopy to CT images. The robustness of the present registration scheme are guaranteed by taking advantage of a coarse-to-fine processing of the volume/image pyramids based on cubic B-splines. A human cadaveric spine specimen together with its ground truth was used to compare the present scheme with a purely software-based scheme in three aspects: accuracy, speed, and capture ranges. Our experiments revealed an equivalent accuracy and capture ranges but with much shorter registration time with the present scheme. More specifically, the results showed 0.8 mm average target registration error, 55 second average execution time per registration, and 10 mm and 10° capture ranges for the present scheme when tested on a 3.0 GHz Pentium 4 computer.