Tim Salcudean

Ultrasound elastography is a medical imaging technique that quantifies tissue mechanical properties such as elasticity. It involves applying a force to the tissue, and measuring the resulting deformations to calculate tissue elasticity by solving an inverse problem. 3D imaging provides more spatial information on tissue deformation compared to conventional 2D frames. However, 3D imaging has several bottlenecks that limit its practicality in clinical settings, including a long data acquisition time that can introduce artifacts from patient or sonographer motion and a low frame rate that limits the applied force’s frequency range by the Nyquist frequency. A wider frequency range can enable a more comprehensive tissue modeling and characterization. Furthermore, most ultrasound elastography techniques measure displacement only in one axial direction, but it is important to consider that any force applied to the tissue leads to 3D tissue deformation due to tissue incompressibility. Therefore, it is crucial to explore new approaches to overcome these limitations and fully utilize the potential of 3D imaging in clinical applications. Shear wave absolute vibro-elastography (S-WAVE) is an elastography technique where an external vibration source generates steady-state mechanical vibrations inside the tissue. This thesis presents two novel techniques for high-frame-rate S-WAVE volumetric data acquisition as well as the algorithms to calculate elasticity volumes. The standard quasi-real-time technique using a wobbler transducer involves sweeping the transducer mechanically and acquiring 2D images at each location. The images are then reconstructed into a 3D volume which takes less than 2 s to collect 100 data volumes. Alternatively, using a matrix array transducer, we introduce the first S-WAVE method with real-time volumetric data acquisition which takes 0.05 s to collect 100 data volumes. We propose a novel method for estimating axial, lateral, and elevational displacements which in turn enables the use of the curl of the displacements in the elasticity reconstruction to reduce artifacts. By employing high volume rates, we extend the range of S-WAVE excitation to 800 Hz. The proposed method is validated using homogeneous and heterogeneous phantoms and in ex vivo bovine liver studies.

View record

The most common treatment for organ confined prostate cancer is radical prostatectomy (RP), where cancerous prostate is incised out. Nowadays, mostly the da Vinci system is used to do robot assisted laparoscopic radical prostatectomy (RALRP), providing improved dexterity and significantly faster patient recovery times. However similar to open surgery, the RALRP has higher numbers of reported positive surgical margins. This is potentially due to a surgeon’s effort to remove the cancer and preserving the healthy tissue, when the cancerous and non-cancerous boundaries are indistinguishable in the endoscope. Therefore, the objective of this thesis is to clearly display these boundaries and tumors, using Augmented/Mixed Reality (AR/MR) technology, utilizing imaging data such as, magnetic resonance imaging and ultrasound (US). The successful intra-operative AR/MR depends on primarily two things. First, the surgically compatible calibration steps, to map the imaging data correctly in camera image. Second, the visualization of the co-located data to give reliable depth of subsurface structures, which is imperative to patient safety. Unlike existing methods, in our first work, we propose a method that performs the required hand-eye and camera calibrations without using external markers during surgery. To further streamline the process, in another work, we use an optimization scheme to combine both the calibrations in a single step. The method allows to register the robotic data to the camera within minutes. Additionally, we presented an evaluation of a full AR system that registers the phantom US to the camera image. Next, we address the well-known problem of occlusion while visualizing the overlayed imaging data. For this, our deep learning-based method segments surgical instruments in the human RALRPs videos without using any labelled data. In our other two works we explore the color and motion parallax as depth cues to provide a reliable depth judgement. The usefulness of these are validated through user studies showing significantly better depth perception when using our methods. In conclusion, this thesis presents multiple methods to make AR/MR guidance feasible for RALRP by addressing two pressing challenges in the field of surgical AR, i.e. surgically compatible calibrations and reliable visualization of the registered data.

View record

This thesis addresses the problem of tracking surgical tissue using camera data in robotically assisted minimally invasive surgery. Millions of robotically assisted surgeries are performed yearly. These surgeries are performed by a surgeon who uses a teleoperation console. This console gives the surgeon a 3D view of their environment as they perform surgery. If we would like to overlay preoperative scans onto the surgeon's field of view using augmented reality, we must understand where the tissue is and where it is moving. Additionally, to enable automation of tasks, or remembering measurements made at tissue locations, such as a biopsy, we require robust tracking.In this thesis, we first perform an in-depth review of the field. We then propose multiple interlocking pieces to enable tissue tracking. To enable tracking salient features, we design a real-time keypoint descriptor that is trained in an unsupervised manner, demonstrating improved performance over classical methods. Afterwards, we propose a novel method that uses these keypoints to parameterize motion in 2D space using graph neural networks. We demonstrate the efficiency of this graph interpolation method. We then incorporate a temporal model into this graph interpolation paradigm. Finally, we extend our graph interpolation algorithm into 3D.In developing our methodology, we realized that there is an acute need for more datasets for quantification. We develop a novel and complete dataset for tissue tracking and mapping and release our dataset for public use by researchers. We close with a summary of important future work.

View record

Prostate cancer (PCa) represents a substantial health concern for men, and this thesis focuses on improving its diagnosis and treatment. The first contribution introduces a registration framework that localizes transperineal template-guided mapping biopsy (TTMB) cores in volumetric ultrasound (US) using only standard TTMB information. With an average target registration error of 1.2 mm and ≈97 s of registration time, this framework provides fast and accurate TTMB core localization. The pathology of these registered cores can serve as ground truth for training US-based PCa classifiers.To improve diagnosis, three US systems are developed to perform Shear Wave Absolute Vibro-Elastography (S-WAVE), a technique for quantitative tissue stiffness assessment. The first system, compatible with TTMB, employs multi-frequency transperineal excitation for the first time and correlates with magnetic resonance elastography (MRE) imaging at 96%. Clinical studies revealed a positive correlation between its measurements and PCa detected from histopathology. A PCa classifier, trained using clinical S-WAVE data taken with this system, achieved an AUC of 0.87±0.12. The second system is the first hand-operated 3D S-WAVE system with a transducer-mounted exciter designed for targeting systematic biopsies. When compared to MRE, it achieves a cross-correlation of 99% and 94% on quality assurance phantoms and in vivo human livers, respectively. In the third system, both S-WAVE and strain elastography are implemented for the first time in a commercial microUS system thus enabling the possibility of multi-parametric microUS imaging.For localized PCa, low-dose-rate prostate brachytherapy (LDR-PB) is an effective treatment where small radioactive seeds are permanently implanted within the prostate. The arrangement of these seeds is pre-planned manually, which is an iterative and time-consuming process. To automate this, two new methods based on conditional generative adversarial networks are proposed. Comparable results to the high-quality manual plans were achieved, demonstrating 98.9% coverage of the clinical target volume (CTV) receiving the minimum prescribed dose. Planning times are significantly reduced, with the proposed methods running approximately 7 and 400 times faster.

View record

Magnetic resonance elastography (MRE) is a biomechanical imaging tool that reconstructs the tissue elasticity map by capturing tissue displacement with magnetic resonance imaging (MRI). MRE has the unique advantage of capturing all three directional displacement components with a moveable and deep field of view. In addition, MRE offers the functionality to incorporate other MRI contrast seamlessly to extend MRE for advanced biomechanical models. However, MRE is limited by its long scanning time, low signal-to-noise ratio, and low resolution. This thesis focuses on iterative model-based reconstruction techniques with structured sparsity for faster MRE acquisition and robust elastogram reconstruction. A new optimization technique for elastogram reconstruction is proposed using the bi-convexity of the elastodynamic model and the alternating direction method of multipliers (ADMM). The proposed optimization technique allows easy integration of structured sparsity and is robust to noise and initialization. Four elastography reconstruction methods were implemented in this thesis with different models and regularization prior: (a) scalar 2D shear wave model (2D-ERBA), (b) scalar 3D shear wave model (3D-ERBA), (c) vector 3D elastodynamic wave model (ERSA), and (d) multifrequency vector 3D elastodynamic wave model (MERSA). The former two elastography reconstruction methods used a sparsity prior to the shear modulus, while the latter used sparsity prior on both shear modulus and displacement. These methods were extensively validated for in silico, in vitro, and in vivo datasets and compared with state-of-the-art methods. Experiments showed that MERSA provides higher robustness to noise, higher robustness to wavelength-to-voxel ratio, and higher accuracy for both elasticity and viscosity. A comparison study of diagnostic performance in detecting liver disease of scalar 2D shear wave model, scalar 3D shear wave model, and vector 3D elastodynamic wave model with MERSA implementation concludes the reconstruction part of this thesis. A bi-convex optimization-based displacement regularized compressed sensing (DRCS) method is proposed for fast MRE acquisition. The proposed method uses separate sparsity prior on the magnitude, phase and displacement to recover the MR signal from a highly undersampled k-space. We compare the performance of DRCS with other compressed sensing methods for highly undersampled data from in silico, in vitro and in vivo datasets.

View record

Neural networks (NNs) have reached remarkable performance in computer vision. However, numerous parameters and complex structures make NNs opaque to humans. The failure to comprehend NNs may raise serious issues in real-world applications. My research aims to explore the NN interpretability in diverse visual tasks from post-hoc explanation and intrinsic interpretability perspectives.Convolutional neural networks (CNNs) have outperformed humans in image classification. However, the logic of network decisions remains a puzzle. As such, we propose concept-harmonized hierarchical inference, a post-hoc explanation framework, to explain the decision-making process of CNNs. Firstly, we interpret layered feature representations of NNs with hierarchical visual semantics. Then we explain the NN feature learning as a bottom-up decision logic from low to high semantic levels in which a deep-layer decision is decomposed as a sequence of shallow-layer sub-decisions.With the evolution of virtual reality, researchers are focusing increasingly on inverse rendering: reconstructing a 3D scene from multi-view 2D images. In this field, NNs achieved superior performance in novel view synthesis and 3D reconstruction. For both tasks, learning a 3D representation from input views is the key process where prior methods separately designed a CNN-based single-view feature extraction and a pooling-based multi-view fusion. This incoherent design damages their intrinsic interpretability and performance. Therefore, we aim to design coherent, interpretable NNs that can adequately exploit knowledge of relationships from data. For novel view synthesis, we propose a unified Transformer-based neural radiance field (TransNeRF) conditioned on source views to learn a generic 3D-scene representation. TransNeRF explores deep relationships between the target-rendering view and source views. TransNeRF also improves intrinsic interpretability by enhancing the shape and appearance consistency of a 3D scene. In experiments, TransNeRF outperforms prior neural rendering methods, and the interpretation results are consistent with human perception.We reformulate 3D reconstruction as a sequence-to-sequence prediction and propose an end-to-end Transformer-based framework (EVolT). EVolT jointly explores multi-level associations between input views and the output volume-based 3D representation within our encoder-decoder structure. EVolT achieves state-of-the-art accuracy in multi-view reconstruction with fewer parameters (70% fewer) than prior methods. Experimental results also suggest the strong scaling capability of EVolT.

View record

This thesis addresses the problem of assisting humans in robot-assisted surgery. Our research investigates this problem using the following two approaches: (i) designing interfaces that facilitate human control of the surgical robotics platform, and (ii) developing autonomous systems to perform repetitive parts of the surgical task, allowing humans to focus on the more demanding ones.Following the first approach, we explored how to use multiple types of data to facilitate the surgeon’s control of surgical robots, such as motion data of expert surgeons, video data from an additional camera, and eye gaze data. The main application area of this approach is surgical training and skill assessment. Our results show that combining hand-over-hand and trial and error training approaches, based on expert motion data, enables trainees to balance both the speed and accuracy of performing tasks better than using only one of these approaches alone. Furthermore, our results show that a two-view system can improve both training and skill assessment, with its application in training showing the most promise, compared with the traditional case of using only a single view. In addition, we discovered a gaze-based phenomenon called “Quiet Eye” in multiple minimally invasive surgery settings. We report how this phenomenon changes with surgeons’ experience level and/or successful task completion. This opens the door to leverage this phenomenon in surgical training and skill assessment.Following the second approach, in the context of autonomous robotic surgery, we worked on automating tasks such as moving the surgical camera and suturing. To automate the surgical camera, we proposed a rule-based method that uses both the position and 3D orientation information from structures in the surgical scene. We tested the effectiveness of using our autonomous camera method in video-based surgical skill assessment. To automate surgical tasks such as suturing, we leveraged the surgical robot’s capability to move multiple arms in parallel to devise autonomous execution models that go beyond the humans’ way of performing these tasks. Our simulation experiments show that our proposed parallel execution models can lead to at least 40% decrease in the tasks’ completion time, compared with the state-of-the art ones.

View record

The progression of chronic liver disease, including hepatitis B, C, and non-alcoholic steatotic hepatitis, is closely associated with advancing fibrosis which stiffens the liver tissue over time. Magnetic resonance elastography is a quantitative imaging method that measures the liver tissue shear modulus over a volume and provides the most accurate imaging-based fibrosis staging results when compared to biopsy. While ultrasound-based elastography methods have been developed for liver fibrosis staging, they are mostly confined to providing measurements for a 1D or a 2D region of interest and with a limited imaging depth. In this thesis, a novel 3D ultrasound liver shear wave absolute vibro-elastography (S-WAVE) imaging technique is developed. With the aid of state-of-the-art 3D ultrasound hardware, 3D S-WAVE aims to provide quantitative and volumetric hepatic stiffness measurements comparable to magnetic resonance elastography. The new 3D S-WAVE utilizes a high element count matrix array transducer. With a modified 3D color power angiography imaging sequence, large field-ofview multi-frequency shear wave imaging could be achieved with a significantly shorter scan time. Learning-based automatic image segmentation and cross-modality registration techniques have also been developed to streamline the image processing pipeline and assist the comparison study between the S-WAVE and magnetic resonance elastography methods. Real-time shear wave imaging capability has also been enabled to improve the efficiency and repeatability of the S-WAVE exam. Validation studies with liver tissue phantoms and in vivo subjects have been designed and implemented using magnetic resonance elastography as the ground truth. The results showed that 3D S-WAVE had a high consistency with magnetic resonance elastography and outperformed conventional ultrasound transient elastography. The collected 3D S-WAVE data were also utilized to assess the performance difference between 2D and 3D elasticity reconstruction techniques. The results indicated that the 3D method effectively overcame the overestimation and the spatial bias issues which were commonly introduced by the 2D imaging method.

View record

This thesis focuses on the automatic analysis of radiological and pathological images for cancer detection and classification. It addresses ultrasound imaging for prostate and breast cancer, and histopathology analysis for prostate cancer, where we propose several classification approaches based on novel features and deep learning. The goal of this thesis is to develop machine learning methods to assist clinicians in diagnosis, prognosis, and treatment planning for patients.To tackle the inherent data heterogeneity in prostate cancer research, we develop a novel framework based on the generative adversarial network to discard extraneous information. For benign vs. malignant classification, it achieves area-under-the-curve of 93.4%, sensitivity of 95.1%, and specificity of 87.7%, respectively, representing significant improvements of 5.0%, 3.9%, and 6.0% compared to using heterogeneous data.We propose novel methods that improve prostate cancer classification and risk stratification using multi-stain digital histopathology. For classification, we demonstrate that: (1) other stain types (Ki67, P63) improve classification performance upon H&E; (2) even without the presence of Ki67 and P63, by mimicking the stain types, H&E stain can better report the presence and severity of prostate cancer.For risk stratification, our proposed risk stratification pipeline, integrating clinicopathologic data and learned image features from multi-stain digital histopathology, outperforms the currently most common grading system, the Gleason grading system, in predicting clinical outcomes such as metastasis-free and overall survival. Using our risk models, 3.9% of low-risk patients are reclassified as high-risk and 21.3% of high-risk patients are reclassified as low-risk. These results demonstrate our risk stratification pipeline’s potential to guide the administration of adjuvant therapy after radical prostatectomy.For breast cancer, we propose a novel automatic pipeline for data processing, feature extraction, feature selection, and classification using ultrasound data. Our best results (95% confidence interval, area-under-the-curve = 95%±1.45%, sensitivity = 95%, and specificity = 93%) outperform the state-of-the-art results using shear wave absolute vibro-elastography. Moreover, our study proposes novel directions in the field of elasticity imaging for tissue classification.All the proposed methods have been tested on held-out sets and have demonstrated promising results, which would be useful in future cancer diagnosis, prognosis, and patient management.

View record

Despite the rapid advancement in medical imaging technologies, health care systems in Canada as well as in many other countries still cannot provide patients with sufficient medical imaging resources due to the ever-growing demands, lack of imaging devices, and fully trained medical practitioners. What makes the situation worse are persistent and prevalent work-related injury situations for medical practitioners, especially sonographers. To help these problems, the overall objective of this thesis is to improve the user interface design for ultrasound machines and contribute to automated medical imaging-based diagnosis with the help of the gaze tracking technology.Multi-themed study and research were conducted to achieve the objectives. On one hand, for the interface design improvement, we started by analyzing the characteristics of gaze signals through statistical modelling. Then we surveyed among sonographers to understand their daily usage of the ultrasound machines. Also, we analyzed different kinds of ultrasound machines to understand common design patterns and suboptimal control logic. To improve the suboptimal control logic and observed usage difficulties on ultrasound machines, we designed and implemented a gaze tracking contingent ultrasound machine. For the validation of our design ideas and machine effectiveness, comparative user studies were conducted. Additionally, we enhanced the user experience by solving the gaze tracker accuracy deterioration problem during normal usage of the devices by proposing a disruption-free auto-recalibration method. On the other hand, for the automated medical image diagnosis, with the integration of the gaze tracking dataset, we proposed deep learning methods that not only can provide a label that predicts which kind of abnormality is observed in the medical image, but also some reasoning that can guide humans to understand how the prediction is made. Multi-task learning methods and neural network attention mechanisms were used for enhanced automated diagnostic performance and better interpretability of the deep learning models.Our study has demonstrated the usefulness of integrating gaze tracking with human-computer interaction and image understanding in the medical field. Future work can be done to further processing and quantify gaze tracking data.

View record

Breast cancer affects millions of women worldwide each year, and is responsible for hundreds of thousands of deaths. Mammography, the standard screening method for these cancers, underperforms for women with dense breast tissue, who account for half of all women under the age of 50. Given that these women are also at higher risk of developing breast cancer, there is a significant need for better screening methods to serve this population. In this thesis, we develop a breast imaging platform which combines several modalities that have been clinically shown to aid in the detection and staging of breast malignancies. This operator-independent, completely automated scan, simultaneously acquires B-mode ultrasound, absolute elasticity, Doppler flow, and photoacoustic tomography of the entire breast in 20 minutes. We describe the hardware and software components which comprise each of these imaging subsystems, and conduct a preliminary study testing the combined system by imaging a phantom which we designed to incorporate inclusions which are uniquely visible in either elasticity or photoacoustic imaging. The photoacoustic tomography system constitutes the most significant contribution, and as such is the primary focus of the thesis. We have designed, built, and tested a custom, fiber-based tissue illuminator to accommodate the unique scanning geometry of the automated breast ultrasound scanner upon which we have based our system. We have also developed a novel data reconstruction scheme which can account for the spatial non-uniformity of this illumination. We tested this system in vitro, as well as a purpose-built wire phantom. Finally, we developed a data processing pipeline which uses generative adversarial networks to improve the signal-to-noise ratio of our raw photoacoustic data, and implemented a state-of-the-art regularized reconstruction scheme to remove imaging and reconstruction artifacts. We tested this method using several phantoms, including an anatomically realistic blood vessel phantom generated from real breast imaging data.

View record

In this thesis, we researched design developments for multi-axis force sensing at the surgeon and the patient consoles of the da Vinci® classic system. A systematic survey on the force sensing literature in Minimally Invasive Surgery (MIS) was conducted. It summarizes the design requirements, compares different technologies, and lists the pros and cons of different locations for sensor integration. While more than 100 articles were published on MIS force sensing, no prior work that addresses force sensing at the surgeon console, without limiting its dexterity, was found. We propose modifications in the wrist’s yaw link of the da Vinci’s Master Tool Manipulator (MTM) for integration of a commercial 6-axis force-torque sensor. The new design does not change the original manipulator’s kinematics and its dexterity. Two example applications of the MTM’s impedance control and joystick control of the Patient Side Manipulator (PSM) were presented to demonstrate the successful integration of the force sensor into the MTM.The mechanical design, electronics hardware, and firmware and software architectures of a novel 6-axis optical force sensor are discussed. The mechatronic design features simple integration, no overload, low-noise, wide dynamic range opto-electronics, and signal conditioning, coupled with co-located digital electronics based on a Field Programmable Gate Array (FPGA) that samples all sensing channels synchronously, enabling very low noise displacement sensing with a resolution of 1.62 nm, low measurement signal latency of 100 μs, high measurement bandwidth of 500 Hz, and high data transfer rates over 11.5 kHz for transmission of six-axis transducer data to a host computer. The transducer’s resolution is better than 0.0001% of the full-scale.The optical force sensor was used for measuring the forces applied to the distal end of a da Vinci® EndoWrist® instrument by mounting it onto its proximal shaft. A new cannula design comprising an inner tube and an outer tube was proposed. A mathematical model of the sensing principle was developed and used for model-based calibration. A data-driven calibration based on a shallow neural network architecture is discussed. The proposed force-sensing requires no modification of the instrument itself; therefore, it is adaptable to different instruments.

View record

During prostate surgeries, there are critical structures around the prostate that should be preserved. Therefore, an additional intra-operative prostate cancer (PCa) imaging method is needed to help the surgeon localize the cancer. Photoacoustic (PA) imaging as an emerging imaging modality shows great potential to detect cancerous tissue. In this thesis, we focus on intra-operative PA imaging of the prostate using the da Vinci robotic system.Towards this objective, we developed a PA reconstruction technique that works in the presence of the challenges of the linear transducers. These challenges include the directivity effect of the transducer and limited-view PA imaging that cause the rank deficiency of the reconstruction system. Therefore, a sparse representation of the PA absorber distribution using the Discrete Cosine Transform was proposed. This sparse representation helps improve the numerical conditioning of the system of equations and reduces the computation time of the approach.In addition, we evaluated the feasible scanning configurations for intra-operative PA tomography (PAT) of the prostate. There are two ultrasound transducers that can be used in prostate PAT: a transrectal ultrasound (TRUS) transducer located posterior to the prostate, and a pick-up ultrasound transducer carried by the da Vinci robotic system and located anterior to the prostate. We proposed a PAT acquisition system that includes a da Vinci system controlled by the da Vinci Research Kit. The configurations using the pick-up and the TRUS transducers to perform intra-operative prostate PAT were investigated.Finally, we developed intra-operative prostate PA imaging using the da Vinci robotic system and a pick-up ultrasound transducer. We proposed a new approach in which the da Vinci robot is programmed to acquire trajectories in a shared control configuration with virtual fixtures; the pick-up transducer is manually controlled but virtual fixtures keep it parallel to a single tomography axis, and keep its translation fixed to a single plane normal to this axis. The surgeon controls the transducer motion on the tissue along this virtual fixture. This thesis confirms that intra-operative da Vinci robot-assisted PA imaging with a pick-up transducer is feasible.

View record

The kidney is a vital organ within the human body and improvements in the ability to characterize the kidney tissue can create benefits for patients with kidney tumors and for kidney transplant recipients. Often, changes in tissue health or development of cancer are manifested in changes in tissue structure that affect tissue elastic properties. For example, the cancerous tissue of renal cell carcinoma is stiffer than healthy kidney tissue, and the development of fibrosis, which impairs kidney function, also causes the tissue to become stiffer over time. These changes can be imaged with ultrasound elastography, a technique for quantitatively assessing tissue elasticity. If proven effective, elastography tissue characterization can replace biopsy.The ultrasound elastography method used in this thesis, called Shear Wave Absolute Vibro-Elastography, or SWAVE, measures the wavelength of induced steady-state multi-frequency mechanical shear waves to calculate tissue elasticity. SWAVE can employ standard ultrasound transducers that image the kidney though the skin above the organ, or custom miniaturized transducers that are placed directly on the surface of the organ during surgery. The accuracy of SWAVE is vastly improved by the use of 3D ultrasound data. We propose and evaluate 3D SWAVE imaging based on the use of a tracked intra-operative ultrasound transducer designed for use with the da Vinci Robot. Different tracking methods are evaluated for future intra-operative use. Elasticity images of tissue phantoms are obtained using interpolated 3D tissue displacement data acquired with the da Vinci robot and the intra-operative transducer. The use of tracked ultrasound transducer opens the way for introducing registered preoperative imaging, including elastography, to improve surgical guidance. Different methods of characterizing kidney tissue using SWAVE imaging are examined. The elastic and viscous properties are estimated kidney tissue ex-vivo. The effect of arterial pressure on the measured kidney elasticity is characterized. It was found that increasing input pressure increases the measured elasticity. Finally, ultrasound and ultrasound elastography are applied to kidney transplant recipients in-vivo to assess the level of fibrosis development. A preliminary study indicates that it is possible to transmit shear waves into the transplanted kidney and measure the elastic properties of the kidney tissue.

View record

Changes in tissue elasticity are correlated with certain pathological changes, such as localized stiffening of malignant tumours or diffuse stiffening of liver fibrosis or placenta dysfunction. Elastography is a field of medical imaging that characterizes the mechanical properties of tissue, such as elasticity and viscosity. The elastography process involves deforming the tissue, measuring the tissue motion using an imaging technique such as ultrasound or magnetic resonance imaging (MRI), and solving the equations of motion. Ultrasound is well suited for elastography, however, it presents challenges such as anisotropic measurement accuracy and providing two dimensional (2D) measurements rather than three dimensional (3D). This thesis focuses on overcoming some of these limitations by improving upon methods of imaging absolute elasticity using 3D ultrasound. In this thesis, techniques are developed for 3D ultrasound acquired from transducers fitted with a motor to sweep the image plane, however many of the techniques can be applied to other forms of 3D acquisition such as matrix arrays. First, a flexible framework for 3D ultrasound elastography system is developed. The system allows for comparison and in depth analysis of errors in current state of the art 3D ultrasound shear wave absolute vibro-elastography (SWAVE). The SWAVE system is then used to measure the viscoelastic properties of placentas, which could be clinically valuable in diagnosing preeclampsia and fetal growth restriction. A novel 3D ultrasound calibration technique is developed which estimates the transducer motor parameters for accurate determination of location and orientation of every data sample, as well as for enabling position tracking of a 3D ultrasound transducer so multiple volumes can be combined. Another calibration technique using assumed motor parameters is developed, and an improvement to an existing N-wire method is presented. The SWAVE research system is extended to measure shear wave motion vectors with a new acquisition scheme to create synchronous volumes of ultrasound data. Regularization based on tissue incompressibility is used to reduce noise in the motion measurements. Lastly, multiple ultrasound volumes from different angles are combined for measurement of the full motion vector, and demonstrating accurate reconstructions of elasticity are feasible using the techniques developed in this thesis.

View record

Surgical removal of the prostate gland using the da Vinci surgical robot is the state of the art treatment option for organ confined prostate cancer. The da Vinci system provides excellent 3D visualization of the surgical site and improved dexterity, but it lacks haptic force feedback and subsurface tissue visualization. The overall objective of the work done in this thesis is to augment the existing visualization tools of the da Vinci with ones that can identify the prostate boundary, critical structures, and cancerous tissue so that prostate resection can be carried out with minimal damage to the adjacent criticalstructures, and therefore, with minimal complications. Towards this objective we designed and implemented a real-time image guidance system based on a robotic transrectal ultrasound (R-TRUS) platform that works in tandem with the da Vinci surgical system and tracks its surgical instruments. In addition to ultrasound as an intrinsic imaging modality, the system was first used to bring pre-operative magnetic resonance imaging (MRI) to the operating room by registering the pre-operative MRI to the intraoperative ultrasound and displaying the MRI image at the correct physical location based on the real-time ultrasound image. Second, a method of using the R-TRUS system for tissue palpation is proposed by expanding it to be used in conjunction with a real-time strain imaging technique. Third, another system based on the R-TRUS is described for detecting dominant prostate tumors, based on a combination of features extracted from a novelmulti-parametric quantitative ultrasound elastography technique. We tested our systems in an animal study followed by human patient studies involving n = 49 patients undergoing da Vinci prostatectomy. The clinical studies were conducted to evaluate the feasibility of using these systems in real human procedures, and also to improve and optimize our imaging systems using patient data.Finally, a novel force feedback control framework is presented as a solution to the lack of haptic feedback in the current clinically used surgicalrobots. The framework has been implemented on the da Vinci surgical system using the da Vinci Research Kit controllers and its performance hasbeen evaluated by conducting user studies.

View record

Noninvasive detection and localization of prostate cancer in medical imaging is an important, yet difficult task. Benefits range from diagnosis of cancer, to planning and guidance of its treatment. In order to characterize cancer and evaluate its localization in volumetric images, such as ultrasound or magnetic resonance imaging (MRI), their spatial correspondence with the "gold standard" provided by histopathology must be established.In this thesis, we propose a general framework for a multi-slice to volume registration that is applied to register a stack of sparse, unaligned two-dimensional histological slices to a three-dimensional volumetric imaging of the prostate. The approach uses particle filtering that allows deriving optimal pose parameters of the slices in a Bayesian approach.We then propose a novel registration method between in vivo and ex vivo MRI of the prostate to facilitate its registration to histopathology. The method incorporates elasticity information, acquired by magnetic resonance elastography (MRE), to generate a patient-specific biomechanical model of the prostate and periprostatic tissue.Next, we propose a registration method between preoperative MRI and intraoperative transrectal-ultrasound. The method can be incorporated with a robotic surgical system to augment the surgeon's visualization during robot-assisted prostatectomy. We also study the use of elasticity-based registration of ultrasound elastography and MRE.We then present an image processing approach for enhancing MRE data. The approach employs registration to compensate for motion of patients during the scan to improve the accuracy of the reconstructed elastogram. A super-resolution technique is employed to increase the resolution of the acquired images by utilizing unique properties of MRE.Finally, we develop a theory for optimization-based design of motion encoding in MRE that allows reducing scanning time and increasing signal-to-noise ratio of elasticity reconstruction. We formulate the displacement estimation of the mechanical wave as an experimental design problem, by which we quantify performance of sequences, and optimize multidirectional designs.The proposed methods have been evaluated in simulations and on a diverse set of clinical data. Results may pave the way for a broader clinical deployment of elastography and elastography-based image processing.

View record

This work presents new approaches to in-vivo and ex-vivo human prostate cancer imaging using magnetic resonance elastography (MRE) – a method to non-invasively image tissue elasticity using magnetic resonance imaging (MRI). From a clinical perspective, stiffness correlates with underlying tissue disease processes and has been traditionally probed with palpation. Thus, diagnosis based on mechanical properties may have great implications in terms of staging of prostate cancer, monitoring disease progression, treatment planning and post treatment follow up. In MRE, mechanical shear waves generted by an external transducer are imaged using an MRI scanner. From the acquired wave field it is possible to reconstruct mechanical properties such as the elastic modulus based on the wave equation. In this work two MR compatible trans-perineal transducers are developed for imaging of the human prostate on a 3T MR scanner. A new MRI pulse sequence is also developed to acquire the three dimensional wave fieldinduced by these transducers. The methods are validated in quality assurance phantoms and volunteer repeatability studies. The system is used for a patient study and the results are compared to the gold standard (whole-mount histopathology marked with Gleason score). Similarly, a transducer is developed for ex-vivo prostate studies on a 7T MR scanner. After validation, prostate specimens of patients are examined and the results are compared to the Gleason score. The overall conclusion are: (i) trans-perineal excitation is well tolerated by the subjects, (ii) the transducers do not interfere with the MR acquisition, (iii) the three dimensional wave field are successfully captured using the new pulse sequence, (iv) phantom validation studies prove that the methods are in fact repeatable and that thestiffness values match with the manufacturer’s specifications, (v) patient motion and the standing wave pattern degrade the repeatability of the reconstructed images, (vi) the prostate gland stands out in the stiffness and shear strain images, (vii) the central gland and in particular the transition zone are stiffer than the peripheral zone, (viii) cancer could indeed be detected with MRE with an area under the receiver-operator-curve of approximately 0.7, and finally (ix) the chemical fixation process degrades the stiffness contrast.

View record

Prostate cancer is the most prevalent type of cancer among men. Accurate delineation and appropriate visualization of the prostatic region can greatly affect treatment of prostate cancer and has the potential to reduce some of the treatment side-effects. The main goal of this research is to develop a prostate segmentation tool which is suitable to replace manual delineation. Manual segmentation, the current standard in procedures such as low dose rate prostate brachytherapy, is tedious, time consuming and observer dependent. We propose a 3D semi-automatic segmentation tool to overcome these limitations. To show the clinical value of this method we perform extensive dosimetric evaluation on in-vivo ultrasound images. This tool is currently being clinically used as part of the prostate brachytherapy treatment procedure at the BC Cancer Agency.Ultrasound is the common modality for imaging the prostate. Although safe and simple to use, it can not always allow the prostate to be reliably delineated. Vibro-elastography is a relatively new imaging method which is used to characterize mechanical properties of tissue. We investigate the suitability of vibro-elastography for visualizing the prostate. We compare in-vivo B-mode ultrasound and vibro-elastograpy images with the gold standard MRI in terms of contrast, edge visibility and the shape and size of the gland as seen in these images. Based on our results we develop a 3D automatic prostate segmentation tool in which, in addition to B-mode, information from vibro-elastography images is used in an iterative model-based segmentation approach.We conclude this work by studying the visibility of cancer itself in vibro-elastography images. Areas suspected for cancer are manually marked on the images and the results are compared to the marked cancer in registered pathology slices. Our preliminary results show that vibro-elastography has the potential to be used for detecting prostate cancer; however, we suggest a combined use of various modalities or image types to improve cancer detection.

View record

Low dose rate prostate brachytherapy is one of the most effective treatments for prostate cancer currently available. It involves the implantation of approximately one hundred small radioactive sources, or ‘seeds’, into the prostate gland. This is accomplished by depositing the seeds transperineally via 16-30 long needles. In British Columbia, over three thousand patients have been treated since 1998 using this technique, and fewer than 10% have suffered a recurrence to date. One of the principal challenges in low dose rate prostate brachytherapy is the limited reliability with which precise doses to the prostate and surrounding organs can be achieved. This is due both to the difficulty in accurately delivering the seeds to their planned positions, as well as movement of the seeds in the post-implant period during the resolution of procedure-induced edema. This uncertainty can lead to undetected deficits in the dose necessary to control the cancer. As patients with more advanced disease are being considered for brachytherapy, these deficits may have greater consequences on oncological outcomes. Treatment uncertainty also increases the risk of side effects, which have the potential to be severe.The overall aim of this thesis is to improve the scope and accuracy with which the dose distribution of stranded seeds can be measured after implant. This involved the development of an algorithm to uniquely identify seeds in post-implant CT data, along with a method to determine their orientation to improve dosimetric accuracy. An analysis of the displacement and migration patterns of seeds in the interval during the resolution of prostate edema was also undertaken. These results identified dosimetric deficiencies and modes of seed loss which have the potential to be rectified by the use of implants containing seeds of non-uniform strength (‘mixed-activity’ implants). Such treatments use fewer needles and may also reduce the incidence of treatment related urinary morbidity. Although the concept of mixed-activity implants is not novel, the algorithm developed to identify seeds after implant enables the post-implant assurance of their dosimetric quality in a clinically feasible way. This thesis concludes with a study investigating the dosimetric benefits of mixed-activity implants.

View record

Tissue deformation is common during many medical interventions. An accurate simulation of these procedures necessitates accounting for tissue displacements by modeling tissue deformation and medical tool interaction. In minimally-invasive procedures, due to lack of visibility, physicians rely on haptic feedback and medical imaging to assess the immediate anatomical configuration and relative medical tool position. These procedures are often difficult to learn and therefore extensive training becomes essential.Computerized training systems offer an alternative to cadavers and training on patients. To accurately model the tissue deformation, most such systems require a mesh representation of the anatomy. To replicate the medical imaging feedback offered during procedures, a realistic image simulation approach is also needed. In this thesis, a novel energy-based meshing technique taking medical images and producing desirable meshes for the finite element method is introduced. This method employs an image-based discretization energy combined with a geometry-based element quality energy. The former promotes each mesh element to cover similar intensity image regions, while the latter ensures the element suitability for finite element simulation. A method that can mimic realistic B-mode ultrasound images under deformation is also presented in this thesis. This method first maps the pixels of an image from a deformed mesh configuration back to the nominal configuration, and then interpolates them in a B-mode voxel volume reconstructed a priori.Needle insertion is involved in several medical procedures. These percutaneous procedures will benefit significantly from advances in simulating needle-tissue interaction, for which a 3D model is proposed in this thesis. Simulating needle flexibility is achieved fast and accurately using a novel approach employing torsional springs. The needle insertion simulation with haptic feedback is presented for a training scenario for prostate brachytherapy, where simulated ultrasound images coupled with deformation are also displayed. A scheme to generate patient models for this system is also devised using both the conventional meshing techniques in the literature and the proposed variational meshing method.

View record

Tissue motion estimation in ultrasound images plays a central role in many modern signal processing applications, including tissue characterization, strain and velocity imaging, and tissue viscoelasticity imaging. Therefore, the performance of tissue motion estimation is of significant importance. Also, its computational cost determines if it can be implemented in real-time so that it can be used clinically. This thesis presents several efficient methods for accurate estimation of tissue motion using digitized ultrasound echo signals.First, sample tracking algorithms are presented as a new class of motion estimators. These algorithms are based on the tracking of individual samples using a continuous representation of the reference echo signal. Simulations and experimental results on tissue mimicking phantoms show that sample tracking algorithms significantly outperform common algorithms in terms of accuracy, precision, sensitivity, and resolution. However, their performance degrades in the presence of noise.To improve the performance of motion estimation in multi-dimensions, pattern matching interpolation techniques are studied and new interpolation techniques are presented. Simulation and experimental results show that, with small computational overhead, the proposed interpolation techniques significantly improve the accuracy and the precision of motion estimation in both 2D and 3D. Employing these techniques, real-time 2D motion tracking software is developed. Furthermore, the performance of the proposed 2D estimators is compared with that of 2D tracking using angular compounding. The results show that the proposed interpolation methods bring the performance of pattern matching techniques close to that of 2D compound tracking. Finally, angular compounding is combined with custom pulse sequencing and delay cancellation techniques to develop a system that estimates the motion vectors at very high frame rates (> 500 Hz) in real-time. The application of the system in the study of the propagation of mechanical waves for tissue characterization is also presented.

View record

Prostate brachytherapy is a widely used treatment of localized prostate cancer. Intra-operative dose feedback would bring many benefits to patients and healthcare practitioners. Detection of brachytherapy seeds and segmentation of prostate boundaries play key roles in dosimetry for prostate brachytherapy. However, seed detection and prostate segmentation using conventional B-mode transrectal ultrasound still remains a challenge for prostate brachytherapy, mainly due to the small size of brachytherapy seeds in the relatively low-quality B-mode ultrasound images and due to the poor contrast between the prostate gland and surrounding tissues, speckle noise, shadowing and refraction artifacts.In this thesis, a new method called the reflected power imaging is presented to enhance the visibility and imaging of implanted seeds. It directly measures the reflected energy of ultrasound radio-frequency signals without logarithmic compression. Based on this method, we propose a new solution for brachytherapy seed detection in a 3D reflected power image computed from ultrasound radio-frequency signals, instead of conventional B-mode images. Then implanted seeds are segmented in 3D local search spaces that are determined by α priori knowledge, e.g. needle entry points and seed placements in a pre-operative dosimetry plan. Needle insertion tracks are also detected locally by using the Hough Transform. Experimental results show that the proposed solution works well for seed localization in a tissue-equivalent ultrasound prostate phantom implanted according to a realistic treatment plan with 136 seeds from 26 needles.As the prostate is a firm organ relative to surrounding tissues, elastography is a potential imaging modality for the guidance of prostate brachytherapy. A dynamic ultrasound elastography method named vibro-elastography can provide more complete dynamic tissue description in terms of transfer functions and coherence functions. In this thesis, we develop fast computational algorithms and programs to implement vibro-elastography imaging in real time. Phantom experiments demonstrate that the vibro-elastography techniques produce stable and operator-independent strain images with high contrast-to-noise ratio. Furthermore, the software for a 3D vibro-elastography imaging system has been designed, implemented and used in the data collection. Over 15 patients have been scanned at the British Columbia Cancer Agency, Vancouver Centre, and the results are encouraging.

View record

Low dose rate prostate brachytherapy has emerged as a treatment option forlocalized prostate cancer. During prostate brachytherapy, tiny radioactivecapsules - seeds - are implanted inside the tissue using long needles. The quality of the treatment depends on the accuracy of seed delivery to their desired positions. Prostate deformation and displacement during insertion and lack of sufficient visual feedback complicate accurate targeting and necessitates extensive training on the part of the physician. Needle insertion simulators can be useful for physician training. In addition, insertion of theneedle with optimized parameters can compensate for prostate deformation,can decrease the targeting errors and, subsequently, can increase thepost-treatment quality of life of the patients. Therefore, needle insertionsimulation and path planning have gained a lot of attention in the research community in the past decade. Moreover, several robots have been designed for brachytherapy; however, they are yet to be coupled with proper needle insertion path planning algorithms.In this thesis, steps toward a path planning algorithm for needles are taken. An optimization method is proposed that updates the initial insertion parameters of a rigid needle, iteratively, based on the simulated positions of the targets, in order to reduce the error between the needle and several targets in a 3D tissue model. The finite element method is used in the needle insertion simulator. The simulator requires a model for the needle-tissue interaction. Therefore, an experimental method has been developed to identify the force model and the tissue elasticity parameters usingmeasurements of insertion force, tissue displacement and needle position.Ultrasound imaging is used to measure tissue displacement. Ultrasound is a common imaging modality in the operating room and does not need beads or markers to track the tissue motion. Therefore, the experimental method can be used in patient studies. The needle-tissue modeling method and insertion parameter optimization were validated in experiments with tissue-mimicking phantoms.In order to facilitate the accommodation of needle flexibility for future applications, three flexible needle models have been compared. Based on these comparisons, it was determined that an efficient and accurate angular spring model is best suited for future studies.

View record

Commonly used medical imaging techniques can render many properties of the anatomy or function, but are still limited in their ability to remotely measure tissue mechanical properties such as elasticity and viscosity. A remote and objective palpation function would help physicians in locating possible tumors or malignancies. The branch of medical imaging that characterizes tissues mechanical properties in a non-invasive manner has enjoyed increasing interest in the past two decades. The basic principle is to apply an excitation, such as tissue compression, to a region of interest and measure the resulting tissue deformation. Tissue mechanical properties can then be inferred from the observed deformation at multiple locations in the region, and the properties can be displayed as an image. If the excitation is dynamic, the deformation is considered as a motion field that varies in time and location over the region of interest. Ultrasound is particularly well suited for measuring motion fields due to its ability to image in real-time, low cost, low risk and ease-of-accessibility. The focus of this thesis is the estimation of the viscoelastic parameters such as Young's modulus, viscosity and relaxation-time. For this purpose, a motion estimation method is proposed to measure axial tissue displacements from the peak of the ultrasound radio frequency signals. The displacements can be further processed to identify the mechanical properties. Two methods were developed: the first one is based on a one dimensional Voigt's model of soft tissue and the second one is based on a finite element model. In the first method, a single frequency or wide-band excitation is applied to the tissue and the local relaxation-time is recovered from the phase difference between the strains or displacements. In this method, the elasticity can also be reconstructed from the magnitudes of the spectra. In the second approach, a novel dynamic finite element model is proposed for the incompressible soft materials. An inverse problem of viscoelasticity is solved iteratively to reconstruct the viscosity and elasticity based on a two or three dimensional model. The theoretical aspect of compressional elastography and longitudinal wave propagation is investigated. It is shown to be feasible to apply dynamic or transient compressional excitation to recover the dynamic properties of soft tissue.

View record

A system for vessel characterization aimed at detecting deep vein thrombosis (DVT) in the lower limbs has been developed and evaluated using ultrasound image processing, location and force sensors measurements, blood flow information and a protocol based on the current clinical standard, compression ultrasound. The goal is to provide an objective and repeatable system to measure DVT in a rapid and standardized manner, as this has been suggested in the literature as an approach to improve overall detection of the disease.The system uses a spatial Kalman filter-based algorithm with an elliptical model in the measurement equation to detect vessel contours in transverse ultrasound images and estimate ellipse parameters, and temporal constant velocity Kalman filters for tracking vessel location in real-time. The vessel characterization also comprises building a 3-D vessel model and performing compression and blood flow assessments to calculate measures that indicate the possibility of DVT in a vessel. A user interface designed for assessing a vessel for DVT was also developed.The system and components were implemented and tested in simulations, laboratory settings, and clinical settings. Contour detection results are good, with mean and rms errors ranging from 1.47-3.64 and 3.69-9.67 pixels, respectively, in simulated and patient images, and parameter estimation errors of 5%. Experiments showed errors of 3-5 pixels for the tracking approaches. The measures for DVT were evaluated, independently and integrated in the system. The complete system was evaluated, with sensitivity of 67-100% and specificity of 50-89.5%. System learnability and memorability were evaluated in a separate user study, with good results.Contributions include a segmentation approach using a full parameter ellipse model in an extended Kalman filter, incorporating multiple measurements, an alternate sampling method for faster parameter convergence and application-specific initialization, and a tracking approach that includes a sub-sampled sum of absolutes similarity calculation and a method to detect vessel bifurcations using flow data. Further contributions include an integrated system for DVT detection that can combine ultrasound B-mode, colour flow and elastography images for vessel characterization, a system interface design focusing on usability that was evaluated with medical professionals, and system evaluations through multiple patient studies.

View record