G+PS regularly provides virtual sessions that focus on admission requirements and procedures and tips how to improve your application.
These videos contain some general advice from faculty across UBC on finding and reaching out to a potential thesis supervisor.
Dissertations completed in 2010 or later are listed below. Please note that there is a 6-12 month delay to add the latest dissertations.
Kidney cancers account for an estimated 140,000 global deaths annually. According to the Canadian Cancer Society, an estimated 6,600 Canadians were diagnosed with kidney cancer, and 1,900 Canadians died from it in 2017. Computed tomography (CT) imaging plays a vital role in kidney cancer detection, prognosis, and treatment response assessment. Automated CT-based cancer analysis is benefiting from unprecedented advancements in machine learning techniques and wide availability of high-performance computers.Typically, kidney cancer analysis requires a challenging pipeline of (a) kidney localization in the CT scan and general assessment of kidney functionality, (b) tumor detection within the kidney, and (c) cancer analysis.In this thesis, we developed deep learning techniques for automatic kidney localization, segmentation-free volume estimation, cancer detection, as well as CT features-based gene mutation detection, renal cell carcinoma (RCC) grading, and staging. Our convolutional neural network (CNN)-based kidney localization approach produces a kidney bounding box in CT, while our CNN-based direct kidney volume estimation approach skips the intermediate segmentation step that is often used for volume estimation at the cost of additional computational overhead. We also proposed a novel collage CNN technique to detect pathological kidneys, where we introduced a unique image augmentation procedure within a multiple instance learning framework. We further proposed a multiple instance decision aggregated CNN approach for automatic detection of gene mutations and a learnable image histogram-based deep neural network (ImHistNet) approach for RCC grading and staging. These approaches could be alternatives to renal biopsy-based whole-genome sequencing, RCC grading, and staging, respectively.Our automatic kidney localization approach reduced the mean kidney boundary localization error to 2.19 mm, which is 23% better than that of recent literature. We also achieved a mean total kidney volume estimation accuracy of 95.2%. Further, we showed a pathological vs. healthy kidney classification accuracy of 98% using our novel collage CNN approach. In our kidney cancer analysis works, our multiple-instance CNN demonstrated an approximately 94% accuracy in kidney-wise mutation detection. Also, our novel ImHistNet demonstrated 80% and 83% accuracies in RCC grading and staging, respectively.
View record
Developmental dysplasia of the hip (DDH) is the most common pediatric hip condition, representing a spectrum of hip abnormalities ranging from mild dysplasia to irreducible hip dislocation. Thirty-three years ago, the introduction of the Graf method revolutionized the use of ultrasound (US) and replaced radiography for DDH diagnoses. However, it has been shown that current US-based assessments suffer from large inter-rater and intra-rater variabilities which can lead to misdiagnosis and inappropriate treatment for DDH. In this thesis, we propose an automatic dysplasia metric estimator based on US and hypothesize that it significantly reduces the subjective variability inherent in the manual measurement of dysplasia metrics. To this end, we have developed an intensity invariant feature to accurately extract bone boundaries in US images, and have further developed an image processing pipeline to automatically discard US images which are inadequate for measuring dysplasia metrics, as defined by expert radiologists. If found adequate, our method automatically measures clinical dysplasia metrics from the US image. We validated our method on US images of 165 hips acquired through clinical examinations, and found that automatic extraction of dysplasia metrics improved the repeatability of diagnoses by 20%. We extended our automatic metric extraction method to three-dimensional (3D) US to increase robustness against operator dependent transducer placement and to better capture the 3D morphology of an infant hip. We present a new random forests-based method for segmenting the femoral head from a 3D US volume, and a method for automatically estimating a 3D femoral head coverage measurement from the segmented head. We propose an additional 3D hip morphology-derived dysplasia metric for identifying an unstable acetabulum. On 40 clinical hip examinations, we found our methods significantly improved the reproducibility of diagnosing femoral head coverage by 65% and acetabular abnormalities by 75% when compared to current standard methods.
View record
Network architectures and training strategies are crucial considerations in applying deep learning to neuroimaging data, but attaining optimal performance still remains challenging, because the images involved are high-dimensional and the pathological patterns to be modeled are often subtle. Additional challenges include limited annotations, heterogeneous modalities, and sparsity of certain image types. In this thesis, we have developed detailed methodologies to overcome these challenges for automatic feature extraction from multimodal neuroimaging data to perform image-level classification and segmentation, with applications to multiple sclerosis (MS).We developed our new methods in the context of four MS applications. The first was the development of an unsupervised deep network for MS lesion segmentation that was the first to use image features that were learned completely automatically, using unlabeled data. The deep-learned features were then refined with a supervised classifier, using a much smaller set of annotated images. We assessed the impact of unsupervised learning by observing the segmentation performance when the amount of unlabeled data was varied. Secondly, we developed an unsupervised learning method for modeling joint features from quantitative and anatomical MRIs to detect early MS pathology, which was novel in the use of deep learning to integrate high-dimensional myelin and structural images. Thirdly, we developed a supervised model that extracts brain lesion features that can predict conversion to MS in patients with early isolated symptoms. To efficiently train a convolutional neural network on sparse lesion masks and to reduce the risk of overfitting, we proposed utilizing the Euclidean distance transform for increasing information density, and a combination of downsampling, unsupervised pretraining and regularization during training. The fourth method models multimodal features between brain lesion and diffusion patterns to distinguish between MS and neuromyelitis optica, a neurological disorder similar to MS, to support differential diagnosis. We present a novel hierarchical multimodal fusion architecture that can improve joint learning of heterogeneous imaging modalities. Our results show that these models can discover subtle patterns of MS pathology and provide enhanced classification and prediction performance over the imaging biomarkers previously used in clinical studies, even with relatively small sample sizes. [An errata to this thesis/dissertation was made available on 2018-06-05.]
View record
Billions of people worldwide are affected by neurological disorders. Recent studies indicate that many neurological disorders can be described as dysconnectivity syndromes, and associated with changes in the brain networks prior to the development of clinical symptoms. This thesis presents contributions towards improving brain connectivity analysis based on graph theory representation of the human brain network. We propose novel multimodal techniques to analyze brain imaging data to better understand its structure, function and connectivity, i.e., brain connectomics. Our first contribution is towards improving parcellation, \ie brain network node definition, in terms of reproducibility, functional homogeneity, leftout data likelihood and overlaps with cytoarchitecture, by utilizing the neighbourhood information and multi-modality integration techniques. Specifically, we embed neighborhood connectivity information into the affinity matrix for parcellation to ameliorate the adverse effects of noise. We further integrate the connectivity information from both anatomical and functional modalities based on adaptive weighting for an improved parcellation. Our second contribution is to propose noise reduction techniques for brain network edge definition. We propose a matrix completion based technique to combat false negatives by recovering missing connections. We also present a local thresholding method which can address the regional bias issue when suppressing the false positives in connectivity estimates. Our third contribution is to improve the brain subnetwork extraction by using multi-pronged graphical metric guided methods. We propose a connection-fingerprint based modularity reinforcement model which reflects the putative modular structure of a brain graph. Inspired by the brain subnetwork's biological nature, we propose a provincial hub guided feedback optimization model for more reproducible subnetwork extraction. Our fourth contribution is to develop multimodal integration techniques to further improve brain subnetwork extraction. We propose a provincial hub guided subnetwork extraction model to fuse anatomical and functional data by propagating the modular structure information across different modalities. We further propose to fuse the task and rest functional data based on hypergraphs for non-overlapping and overlapping subnetwork extraction. Our results collectively indicate that combing multimodal information and applying graphical metric guided strategies outperform classical unimodal brain connectivity analysis methods. The resulting methods could provide important insights into cognitive and clinical neuroscience.
View record
This thesis proposes non-invasive automated scene analysis and augmentation techniques to improve navigation in image-guided therapy (IGT) applications. IGT refers to procedures in which physicians rely on medical images to plan, perform, and monitor an intervention. In IGT, the tomographic images acquired before the intervention may not directly correspond to what the physician sees via the intraoperative imaging. This is due to many factors such as: time-varying changes in the patient's anatomy (e.g., patient positioning or changes in pathology), risk of overexposure to ionizing radiation (restricted use of X-ray imaging), operational costs, and differences in imaging modalities. This inconsistency often results in a navigational problem that demands substantial additional effort from the physician to piece together a mental representation of complex correspondences between the preoperative images and the intraoperative scene. The first direction explored in this thesis, investigates the application of image-based motion analysis techniques for vessel segmentation. Specifically, we propose novel motion-based segmentation methods to enable safe, fast, and automatic localization of vascular structures from dynamic medical image sequences and demonstrated their efficacy in segmenting vasculature from surgical video and dynamic medical ultrasound sequences. The second direction investigates ways in which navigation uncertainties can be computed, propagated, and visualized in the context of IGT navigation systems that target deformable soft-tissues. Specifically, we present an uncertainty-encoded scene augmentation method for robot-assisted laparoscopic surgery, in which we propose visualization techniques for presenting probabilistic tumor margins. We further present a computationally efficient framework to estimate the uncertainty in deformable image registration and to subsequently propagate the effects of the computed uncertainties through to the visualizations, organ segmentations, and dosimetric evaluations performed in the context of fractionated image-guided brachytherapy. Our contributions constitute a step towards automated and real-time IGT navigation and may, in the near future, help to improve interventional outcomes for patients (improved targeting of pathologies) and increase surgical efficiency (less effort required by the physician).
View record
The oropharynx is involved in a number of complex neurological functions, such as chewing, swallowing, and speech. Disorders associated with these functions, if not treated properly, can dramatically reduce the quality of life for the sufferer. When tailored to individual patients, biomechanical models can augment the imaging data, to enable computer-assisted diagnosis and treatment planning. The present dissertation develops a framework for 3D, subject-specific biomechanical modeling and simulation of the oropharynx. Underlying data consists of magnetic resonance (MR) images, as well as audio signals, recorded while healthy speakers repeated specific phonetic utterances in time with a metronome. Based on this data, we perform simulations that demonstrate motor control commonalities and variations of the /s/ sound across speakers, in front and back vowel contexts. Results compare well with theories of speech motor control in predicting the primary muscles responsible for tongue protrusion/retraction, jaw advancement, and hyoid positioning, and in suggesting independent activation units along the genioglossus muscle. We augment the simulations with real-time acoustic synthesis to generate sound. Spectral analysis of resultant sounds vis-à-vis recorded audio signals reveals discrepancy in formant frequencies of the two. Experiments using 1D and 3D acoustical models demonstrate that such discrepancy arises from low resolution of MR images, generic parameter-tuning in acoustical models, and ambiguity in 1D vocal tract representation. Our models prove beneficial for vowel synthesis based on biomechanics derived from image data. Our modeling approach is designed for time-efficient creation of subject-specific models. We develop methods that streamline delineation of articulators from MR images and reduce expert interaction time significantly (≈ 5 mins per image volume for the tongue). Our approach also exploits muscular and joint information embedded in state-of-the-art generic models, while providing consistent mesh quality, and the affordances to adjust mesh resolution and muscle definitions.
View record
Deep learning methods have shown great success in many research areas such asobject recognition, speech recognition, and natural language understanding, dueto their ability to automatically learn a hierarchical set of features that istuned to a given domain and robust to large variability. This motivates the useof deep learning for neurological applications, because the large variability inbrain morphology and varying contrasts produced by different MRI scanners makesthe automatic analysis of brain images challenging.However, 3D brain images pose unique challenges due to their complex contentand high dimensionality relative to the typical number of images available,making optimization of deep networks and evaluation of extracted featuresdifficult. In order to facilitate the training on large 3D volumes, we havedeveloped a novel training method for deep networks that is optimizedfor speed and memory. Our method performs training of convolutional deep beliefnetworks and convolutional neural networks in the frequency domain, whichreplaces the time-consuming calculation of convolutions with element-wisemultiplications, while adding only a small number of Fourier transforms.We demonstrate the potential of deep learning for neurological image analysisusing two applications. One is the development of a fully automatic multiplesclerosis (MS) lesion segmentation method based on a new type of convolutionalneural network that consists of two interconnected pathways for featureextraction and lesion prediction. This allows for the automatic learning offeatures at different scales that are optimized for accuracy for any givencombination of image types and segmentation task. Our network also uses a novelobjective function that works well for segmenting underrepresented classes, suchas MS lesions. The other application is the development of a statistical modelof brain images that can automatically discover patterns of variability in brainmorphology and lesion distribution. We propose building such a model using adeep belief network, a layered network whose parameters can be learned fromtraining images. Our results show that this model can automatically discover theclassic patterns of MS pathology, as well as more subtle ones, and that theparameters computed have strong relationships to MS clinical scores.
View record
Mental and neurological diseases account for a major portion of the global disease burden. Neuroimaging has greatly contributed to the characterization and understanding of such disorders by enabling noninvasive investigation of the human brain. Among neuroimaging technologies, magnetic resonance imaging (MRI) stands out as a relatively widespread and safe imaging modality that can be sensitized to capture different aspects of the brain. Historically, MRI studies have investigated anatomy or function of the brain in isolation, which created an apparent dichotomy. In this thesis, we aim to bridge this divide using novel multimodal techniques. In particular, we present techniques to reconcile information regarding anatomical and functional connectivity (AC and FC) in the brain estimated from diffusion MRI (dMRI) and functional MRI (fMRI) data, respectively. Our first contribution is to show that the consistency between AC and FC is understated when standard analysis methods are used. We illustrate how the estimation of AC can be improved to increase the AC-FC consistency, which facilitates a more meaningful fusion of these two types of information. Specifically, we propose to improve AC estimation by the use of a dictionary based super-resolution approach to increase the spatial resolution in dMRI, reconstructing the white matter tracts using global tractography instead of conventional streamline tractography, and quantifying AC using fiber count as the metric. Our second contribution is to develop novel multimodal approaches for investigating functional segregation and integration in the human brain. We show that task fMRI data can be fused with dMRI and resting state fMRI data to mitigate the effects of noise and deconfound the effects of spontaneous fluctuations in brain activity on activation detection. Further, we show that sensitivity in unraveling the modular structure of the brain can be increased by fusing dMRI and fMRI data. Our results collectively suggest that combining dMRI and fMRI data outperforms classical unimodal analyses in understanding the brain's organization, bringing us one step closer to understanding the most complex organ in the human body.
View record
Neurological diseases constitute the leading disease burden worldwide. Existing symptom-based diagnostic methods are often insufficient to detect many of these diseases in their early stages. Recent advances in neuroimaging technologies have enabled non-invasive examination of the brain, which facilitates localization of disease-induced effects directly at the source. In particular, functional magnetic resonance imaging (fMRI) has become one of the dominant means for studying brain activity in healthy and diseased subjects. However, the low signal-to-noise ratio, the typical small sample size, and the large inter-subject variability present major challenges to fMRI analysis. Standard analysis approaches are largely univariate, which underutilize the available information in the data. In this thesis, we present novel strategies for activation detection, region of interest (ROI) characterization, functional connectivity analysis, and brain decoding that address many of the key challenges in fMRI research. Specifically, we propose: 1) new formulations for incorporating connectivity and group priors to better inform activation detection, 2) the use of invariant spatial features for capturing the often-neglected spatial information in ROI characterization, 3) an evolutionary group-wise approach for dealing with the high inter-subject variability in functional connectivity analysis, and 4) a generalized sparse regularization technique for handling ill-conditioned brain decoding problems. On both synthetic and real data, we showed that exploitation of prior information enables more sensitive activation detection, more refined ROI characterization, more robust functional connectivity analysis, and more accurate brain decoding over the current state-of-the-art. All of our results converged to the conclusion that integrating prior information is beneficial, and oftentimes, essential for tackling the challenges that fMRI research present.
View record
Three dimensional (3D) and four dimensional (4D) medical images are increasingly being used in many clinical and research applications. Due to their huge file size, 3D and 4D medical images pose heavy demands on storage and archiving resources. Lossless compression methods usually facilitate the access and reduce the storage burden of such data, while avoiding any loss of valuable clinical data. In this thesis, we propose novel methods for highly efficient storage and scalable access of 3D and 4D medical imaging data that outperform the state-of the-art. Specifically, we propose (1) a symmetry-based technique for scalable lossless compression of 3D medical images; (2) a 3D scalable medical image compression method with optimized volume of interest (VOI) coding; (3) a motion-compensation-based technique for lossless compression of 4D medical images; and (4) a lossless functional magnetic resonance imaging (fMRI) compression method based on motion compensation and customized entropy coding. The proposed symmetry-based technique for scalable lossless compression of 3D medical images employs wavelet transform technology and a prediction method to reduce the energy of the wavelet sub-bands based on a set of axes of symmetry. We achieve VOI coding by employing an optimization technique that maximizes reconstruction quality of a VOI at any bit-rate, while incorporating partial background information and allowing for gradual increase in peripheral quality around the VOI. The proposed lossless compression method for 4D medical imaging data employs motion compensation and estimation to exploit the spatial and temporal correlations of 4D medical images. Similarly, the proposed fMRI lossless compression method employs a motion compensation process that uses a 4D search, bi-directional prediction and variable-size block matching for motion estimation; and a new context-based adaptive binary arithmetic coder to compress the residual and motion vector data generated by the motion compensation process.We demonstrate that the proposed methods achieve a superior compression performance compared to the state-of-the-art, including JPEG2000 and 3D-JPEG2000.
View record
Analysis of brain tissues such as white matter (WM), gray matter (GM), cerebrospinal fluid (CSF), and pathological regions from magnetic resonance imaging (MRI) scans of normal adults and patients with neurodegenerative diseases such as multiple sclerosis (MS) allows for improved understanding of disease progression in vivo. As images are often confounded by acquisition noise and partial-volume effects, developing an automatic, robust, and efficient segmentation is essential to the accurate quantification of disease severity. Existing methods often require subjective parameter tuning, anatomical atlases, and training, which are impractical and undesirable.The contributions of this thesis are three-fold. First, a 3D deformable model was explored by integrating statistical and geometric information into a novel hybrid feature to provide robust regularization of the evolving contours. Second, to improve efficiency and noise resiliency, a 3D region-based hidden Markov model (rbHMM) was developed. The novelty of this model lies in subdividing an image into irregularly-shaped regions to reduce the problem dimensionality. A tree-structured estimation algorithm, based on Viterbi decoding, then enabled rotationally invariant estimation of the underlying discrete tissue labels given noisy observations. Third, estimation of partial volumes was incorporated in a 3D fuzzy rbHMM (frbHMM) for analyzing images suffering from acquisition-related resolution limitation by incorporating forward-backward estimations. These methods were successfully applied to the segmentation of WM, GM, CSF, and white matter lesions. Extensive qualitative and quantitative validations were performed on both synthetic 3D geometric shapes and simulated brain MRIs before applying to clinical scans of normal adults and MS patients. These experiments demonstrated 40% and 10% improvement in segmentation efficiency and accuracy, respectively, over state-of-the-art approaches under noise. When modeling partial-volume effects, an additional 30% reduction in segmentation errors was observed. Furthermore, the rotational invariance property introduced is especially valuable as segmentation should be invariant to subject positioning in the scanner to minimize analysis variability. Given such improvement in the quantification of tissue volumes, these methods could potentially be extended to the studies of other neurodegenerative diseases such as Alzheimer’s. Furthermore, the methods developed in this thesis are general and can potentially be adopted in other computer vision-related segmentation applications in the future.
View record
Current practice in orthopaedic surgery relies on intra-operative two dimensional (2D) fluoroscopy as the main imaging modality for localization and visualization of bone tissue, fractures, implants, and surgical tool positions. However, with such projection imaging, surgeons typically face considerable difficulties in accurately localizing bone fragments in three dimensional (3D) space and assessing the adequacy and accuracy of reduced fractures. Furthermore, fluoroscopy involves significant radiation exposure. Ultrasound (US) has recently emerged as a potential non-ionizing imaging alternative that promises safer operation while remaining relatively cheap and widely available. US image data, however, is typically characterized by high levels of speckle noise, reverberation, anisotropy and signal dropout which introduce significant difficulties in interpretation of captured data, automatic detection and segmentation of image features and accurate localization of imaged bone surfaces.In this thesis we propose a novel technique for automatic bone surface and surgical tool localization in US that employs local phase image information to derive symmetry-based features corresponding to tissue/bone or tissue/surgical tool interfaces through the use of 2D Log-Gabor filters. We extend the proposed method to 3D in order to take advantage of correlations between adjacent images. We validate the performance of the proposed approach quantitatively using realistic phantom and in-vitro experiments as well as qualitatively on in-vivo and ex-vivo data. Furthermore, we evaluate the ability of the proposed method in detecting gaps between fractured bone fragments. The current study is therefore the first to show that bone surfaces, surgical tools and fractures can be accurately localized using local phase features computed directly from 3D ultrasound image volumes. Log-Gabor filters have a strong dependence on the chosen filter parameters, the values of which significantly affect the outcome of the features being extracted. We present a novel method for contextual parameter selection that is autonomously adaptive to image content. Finally, we investigate the hypothesis that 3D US can be used to detect fractures reliably in the emergency room with three clinical studies. We believe that the results presented in this work will be invaluable for all future imaging studies with US in orthopaedics.
View record
Theses completed in 2010 or later are listed below. Please note that there is a 6-12 month delay to add the latest theses.
Developmental Dysplasia of the Hip (DDH) is a painful orthopaedic malformationdiagnosed at birth in 1-3% of all newborns. Left untreated, DDH canlead to significant morbidity including long term disability. Currently thecondition is clinically diagnosed using 2D ultrasound (US) imaging acquiredbetween 0-6 months of age. DDH metrics are manually extracted by highlytrained radiologists through manual measurements of relevant anatomy fromthe 2D US data, which remains a time consuming and highly error proneprocess. Recently, it was shown that combining 3D US imaging with deeplearning (DL)-based automated diagnostic tools may significantly improveaccuracy and reduce variability in measuring DDH metrics. However, robustnessof current techniques remains insufficient for reliable deploymentinto real life clinical workflows. In this thesis, we present a quantitativerobustness evaluation of state-of-the-art (SOTA) DL models in bone segmentationfor 3D US and demonstrate examples of failed or implausiblesegmentations with SOTA models under common data variations, e.g., smallchanges in image resolution or anatomical field of view (FOV) from thoseencountered in the training data. We propose a 3D extension of the Seg-Former architecture, a lightweight transformer-based model with hierarchicallystructured encoders producing multi-scale features, which we show toconcurrently improve accuracy and robustness. Specifically we show an increasein the 3% Dice score performance over the previous SOTA modelsfor 3D US segmentation. To allow researchers, collaborators, clinicians, anddoctors access to our DL models, we develop a prototype web-based applicationthat will allow users to upload three dimensional US data and visualizetheir data before eventually selecting from various DL models to run ontheir data. The DL models will run in the background segmenting out thehip anatomical structures and return the calculated DDH metrics as well asrelevant visualization of the segmentation and a 3D rendered mesh of thehip from the segmentation. We also investigate the use of learnable GaborFilter Banks as a preprocessing layer in DL models to mimic the humanvisual system.
View record
Skin cancer is the most common cancer worldwide, accounting for a third of all cancers. A major public health problem with shortages of dermatologists and high rates of misdiagnosis, Computer-aided diagnosis (CAD) offers much for easing the workload on clinicians and reducing human error. Deep neural networks (DNNs) have already shown promise in this domain and with impressive results in skin lesion segmentation (SLS) and skin lesion classification (SLC), both crucial steps in dermatology CAD systems. However, current approaches are still plagued with critical limitations that impede their full potential in real-life clinical applications. A main problem is their high susceptibility to data bias, which manifests as problematic unfairness in their decision-making. In dermatology, this is manifested with variable levels of model accuracy across skin tones (types). Another main problem relates to more recent DNNs that are based on vision transformer (VIT), in which, diverse, yet small-sized skin databases. Fare poorly due to the inherent nature of data-hungry models. In this thesis, we made contributions to both SLS and SLC. First, we proposed FairDisCo, a disentanglement DNN with contrastive learning, which adds a dedicated ‘fairness’ network branch that reduces sensitive attributes (skin-type information) from model representations, namely, and another contrastive branch to improve representation learning for better diagnosis accuracy. Second, we proposed MDViT, a multi-domain VIT with domain adapters to mitigate model data-hunger and to combat negative knowledge transfer (NKT) that decreases model performance on domains with inter-domain heterogeneity. MDViT also employs mutual knowledge distillation to enhance representation learning across domains.Third, we proposed AViT, an efficient framework that utilizes lightweight modules within the transformer layers to transfer a pre-trained VIT to the SLS task without a need for updating its pre-trained weights for data-hungry mitigation. AViT also employs a shallow convolutional neural network (CNN) to produce a prompt embedding with fine-grained information and to inject the CNNs’ inductive biases for better representation learning. All proposed models were thoroughly tested on publicly available databases and validated against state-of-the-art (SOTA) algorithms with comprehensive quantitative results demonstrating superior performance, both in terms of accuracy, fairness, and efficiency.
View record
Developmental Dysplasia of the Hip (DDH) - a condition characterized by hip joint instability, is one of the most common hip disorders in newborns. Clinical practice for diagnosis remains reliant on manual measurement of hip joint features from 2D Ultrasound (US) scans, a process plagued with high inter/intra operator and scan variability. Recently, 3D US was shown to be markedly more reliable with deeply-learned image features effectively used to localize and measure anatomical bone landmarks. However, standard Neural Network (NN) provide no means for assessing the reliability of computed results, a limitation that hampers deployment in clinical settings. In this thesis, we aim to improve the trustworthiness and reliability of deep-learning based DDH diagnostic system, addressing two components: uncertainty and calibration of NN. We propose interpretable uncertainty measures that allow for measuring hip joint segmentation reliability and quantifying scan adequacy in clinical DDH assessments from 3D US. Our approach measures variability of estimates generated from a Monte-Carlo (MC) dropout-based deep network optimized for hip joint localization. Results demonstrate US scans with lower dysplasia metric variability are strongly associated with those labelled as clinically adequate by a human expert. In segmentation tasks, quantifying levels of confidence can provide meaningfuladditional information to aid clinical decision making. We propose to quantifyconfidence in segmentation that incorporates voxel-wise uncertainty into the lossfunction used in the training regime. For ilium and acetabulum segmentation, wereport mean Dice score of 81% when trained with voxel-wise uncertainty loss vs.76% with cross-entropy loss. Recent works proposed Bayesian frameworks to quantify confidence in the segmentation process but the confidence measures tend to be miscalibrated. Wepropose a non-Bayesian-based system to calibrate the confidence values, in orderto reduce over-confident and under-confident predictions. We show deep ensembles optimized with compounded loss achieve low NLL of 11% and Brier score of 3% producing calibrated confidence estimates. Our findings suggest that the uncertainty quantification may improve clinical workflow acting as a quality control check on DL based analysis. This in turn may improve overall reliability of the DDH diagnostic process and the prospects of adoption in clinical settings.
View record
Developmental Dysplasia of the Hip is one of the most common congenital disorders. Misdiagnosis leads to financial consequences and reduced quality of life. The current standard diagnostic technique involves imaging the hip with ultrasound and extracting metrics such as the α angle. This has been shown to be unreliable due to human error in probe positioning, leading to misdiagnosis. 3D ultrasound, being more robust to errors in probe positioning, has been introduced as a more reliable alternative. In this thesis, we aim to further improve the image processing techniques of the 3D ultrasound-based system, addressing three components: segmentation, metrics extraction, and adequacy classification. Segmentation in 3D is prohibitively slow when performed manually and introduces human error. Previous work introduced automatic segmentation techniques, but our observations indicate lack of accuracy and robustness with these techniques. We propose to use deep Convolutional Neural Network (CNN)s for improving the segmentation accuracy and consequently the reproducibility and robustness of dysplasia measurement. We show that 3D-U-Net achieves higher agreement with human labels compared to the state-of-the-art. For pelvis bone surface segmentation, we report mean DSC of 85% with 3D-U-Net vs. 26% with CSPS. For femoral head segmentation, we report mean CED Error of 1.42mm with 3D-U-Net vs. 3.90mm with the Random Forest Classifier. We implement methods for extracting α₃D, FHC₃D, and OCR dysplasia metrics using the improved segmentation. On a clinical set of 42 hips, we report inter-exam, intra-sonographer intraclass correlation coefficients of 87%, 84%, and 74% for these three metrics, respectively, beating the state-of-the-art. Qualitative observations show improved robustness and reduced failure rates. Previous work had explored automatic adequacy classification of hip 3D ultrasound, to provide clinicians with rapid point-of-care feedback on the quality of the scan. We revisit the originally proposed adequacy criteria and show that these criteria can be improved. Further, we show that 3D CNNs can be used to automate this task. Our best model shows good agreement with human labels, achieving an AROC of 84%. Ultimately, we aim to incorporate these models into a fully automatic, accurate, reliable, and robust system for hip dysplasia diagnosis.
View record
Swallowing is a complex oropharyngeal process governed by intricate neuromuscular functions. Dysfunction in swallowing, clinically termed as dysphagia, can significantly reduce the quality of life. Modified barium swallow (MBS) studies are performed to produce vidoefluoroscopy (VF) for visualizing swallowing dynamics to diagnose dysphagia. To train the clinicians learning standardized dysphagia diagnosis, 2D animated videos coupled with VF are used. However, it is hypothesised that the physiologic components of the oral domain may benefit from extension of the training materials, such as inclusion of 3D models. We develop a 3D biomechanical swallowing model of the oropharyngeal complex to extend the clinical dysphagia diagnosis training materials. Our approach incorporates realistic geometries and accurate timing of swallowing events derived from training animations that have been clinically validated. We develop rigid body models for the bony structures and finite element models (FEM) for the deformable soft structures, and drive our coupled biomechanical model kinematically with accurate timing of swallowing events. We implement an airway-skin mesh using a geometric skinning technique that unifies geometric blending for rigid body model with embedded surface for FEMs to incorporate the deformation of upper airway during a swallowing motion. We use smoothed particle hydrodynamics (SPH) technique to simulate a fluid bolus in the airway-skin mesh where the model dynamics drive the bolus to emulate bolus transport during a swallowing motion. We validate this model in two phases. Firstly, we compare the simulated bolus movement with input data and match the swallowing kinematics identified in the standardized animations. Secondly, we extend existing training material for standardized dysphagia diagnosis with our 3D model. To test the usefulness of the extended training set using 3D visualizations, we conduct a pilot user study involving Speech Language Pathologists. The pilot data indicate that clinicians believe the additional 3D views are useful for identifying the salient features for differentiating between different swallowing impairments, such as direction, strength and timing of the tongue motion, and could be a useful addition to the current standardized MBSImP™© training system.
View record
Diffusion-weighted magnetic resonance imaging (dwMRI) provides unique capabilities for non-invasive imaging of neural fiber pathways in the brain. dwMRI is an increasingly popular imaging method and has promising diagnostic and surgical applications for Alzheimer's disease, brain tumors, and epilepsy, to name a few. However, one limitation of dwMRI (specifically, the more common diffusion tensor imaging scheme, DTI) is that it suffers from a relatively low resolution. This often leads to ambiguity in determining location and orientation of neural fibers, and therefore reduces the reliability of information gained from dwMRI. Several approaches have been suggested to address this issue. One approach is to have a finer sampling grid, as in diffusion spectrum imaging (DSI) and high-angular resolution imaging (HARDI). While this did result in a resolution improvement, it has the side effects of lowering the quality of image signal-to-noise ratio (SNR) or prolonging imaging time, which hinders its use in routine clinical practice. Subsequently, an alternative approach has been proposed based on super-resolution methods, where multiple low resolution images are fused into a higher resolution one. While this managed to improve resolution without reducing SNR, the multiple acquisitions required still resulted in a prolonged imaging time. In this thesis, we propose a processing pipeline that uses a super resolution approach based on dictionary learning for alleviating the dwMRI low resolution problem. Unlike the majority of existing dwMRI resolution enhancement approaches, our proposed framework does not require modifying the dwMRI acquisition. This makes it applicable to legacy data. Moreover, this approach does not require using a specific diffusion model. Motivated by how functional connectivity (FC) reflects the underlying structural connectivity (SC), we use the Human Connectome Project and Kirby multimodal dataset to quantitatively validate our results by investigating the consistency between SC and FC before and after super-resolving the data. Based on this scheme, we show that our method outperforms interpolation and the only existing single image super-resolution method for dMRI that is not dependent on a specific diffusion model. Qualitatively, we illustrate the improved resolution in diffusion images and illustrate the revealed details beyond what is achievable with the original data.
View record
Using ultrasound for tool navigation in an orthopaedic surgery requires bone localization in ultrasound, a procedure which remains challenging despite encouraging advances in its practice. Current methods, e.g., the local image phase-basedfeature analysis, have shown promising results but continue to rely on delicateparameter selection processes and to be prone to error at confounding soft tissueinterfaces of similar appearance to bone interfaces. In addition, the 3-dimensionalphase-features-based automatic bone segmentation method is found to be timeinefficient (at ~2 minutes).We have proposed a different approach to bone segmentation by combiningultrasound strain imaging and envelope power detection methods. After an estimation of the strain and envelope power maps, we subsequently fused thesemaps into a single combined map that corresponds robustly to the actual boneboundaries. This study has achieved a marked reduction in false positive boneresponses at the soft tissue interfaces. We also incorporated the depth-dependentcumulative power of the envelope into the elastographic data as well as incorporated an echo de-correlation measurement-based weight to fuse the strain andenvelope map. We also employed a data driven scheme to detect the presence ofany bone discontinuity in the scanned ultrasound image and introduced a multivariate non-parametric Gaussian mixture regression to be used over the maximumintensity points of the fused map. Finally, we developed a simple yet effectivemeans to perform 3-dimensional bone surface extractions using a surface growingapproach that is seeded from the 2-dimensional bone contours.We employed mean absolute error calculations between the actual and estimated bone boundaries to show the extent of the false positives created; ourmethods showed an average improvement in the mean absolute error of 20% onboth the 2- and 3-dimensional finite-element-models, and of 18% and 23%, respectively, on the 2- and 3-dimensional experimental phantom data, when compared with that of the local phase-based methods. Validation on the 2- and3-dimensional clinical in vivo data also demonstrates, respectively, an averageimprovement in the mean absolute fitting error of 55% and an 18 times improvement in the computation time.
View record
Diffusion tensor magnetic resonance imaging (DT-MRI) is a powerful non-invasive imaging modality whose processing, analysis and visualization has become a strong focus in medical imaging research. In this modality, the direction of the water diffusion is locally modeled by a Gaussian probability density function whose covariance matrix is a second order 3 X 3 symmetric positive definite matrix, also called the tensor here. The manifold-valued nature of the data as well as its high dimensionality makes the computational analysis of DT images complex. Very often, the data dimensionality is reduced to a single scalar derived from the tensors. Another common approach has been to ignore the restriction to the manifold of symmetric second-order tensors and, instead, treat the data as a multi-valued image. In this thesis, we try to address the above challenges posed by DT data using two different approaches. Our first contribution employs a geometric approach for representing DT data as low dimensional manifold embedded in higher dimensional space and then applying mathematical tools traditionally used in the study of Riemannian geometry for formulating first order and second order differential operators for DT images. Our second contribution is an algebraic one, where the key novel idea is to represent the DT data using the 8 dimensional hypercomplex algebra-biquaternions. This approach enables the processing of DT images in a holistic manner and facilitates the seamless introduction of traditional signal processing methodologies from biquaternion theory such as computing the Fourier transform, convolution, and edge detection for DT images. The preliminary results on synthetic and real DT data show great promise in both our approaches for DT image processing. In particular, we demonstrate greater detection ability of our features over scalar based approaches such as fractional anisotropy and show novel applications of our new biquaternion tools that have not been possible before for DT images.
View record
Pelvic fractures are serious injuries that are most commonly caused by motor vehicle accidents and affect people of all ages. Surgeries to realign the pelvis and fix the bone fragments with screws have inherent risks and rely on cumbersome intra-operative radioscopic imaging methods. Ultrasound (US) is emerging as a desirable imaging modality to replace fluoroscopy as an intra-operative tool for pelvic fracture surgery because it is safe, portable and inexpensive. Despite the many advantages of US, it suffers from speckle noise, a limited field of view and a low signal-to-noise ratio. Therefore, we must find a way to efficiently process and utilize ultrasound data so that it can be used to effectively visualize bone. In the past decade, there has been much research focused on fusing US with pre-operative Computed Tomography (CT) to be used in an intra-operative guidance system; however, current methods are either too slow or not robust enough to use in a clinical setting. We propose a method to automatically extract bone features in US and CT volumes and register them using a fast point-based method. We use local phase features to estimate the bone surfaces from B-mode US volumes. We simplify the bone surface using particle simulation, which we optimize using the hierarchical Barnes-Hut algorithm. To ensure the point cloud best represents the bone surface, we reinforce them with high curvature features. We then represent the point clouds using Gaussian Mixture Models (GMMs) and find the correspondence between them by minimizing a measure of distance between the GMMs. We have validated our proposed algorithm on a phantom pelvis and clinical data acquired from pelvic fracture patients. We demonstrate a registration runtime of 1.4 seconds and registration error of 0.769 mm.
View record
Image segmentation techniques are predominately based on parameter-laden optimization processes. The segmentation objective function traditionally involves parameters (i.e. weights) that need to be tuned in order to balance the underlying competing cost terms of image data fidelity and contour regularization. Conventionally, the associated parameters are set through tedious trial and error procedures and kept constant over the image. However, spatially varying structural characteristics, such as object curvature, combined with varying noise and imaging artifacts, significantly complicate the selection process of segmentation parameters.This thesis contributes to the field of image segmentation by proposing methods for spatially adapting the balance between regularization and data fidelity in energy minimization frameworks in an autonomous manner. We first proposed a method for determining the globally-optimum spatially adaptive regularization weight based on dynamic programming. We investigated this weight with a basic minimum-path segmentation framework. Our findings indicated that the globally-optimum weight is not necessarily the best weight as perceived by users, and resulted in poorer segmentation accuracy, particularly for high noise images. We then investigated a more intuitive approach that adapts the regularization weight based on the underlying local image characteristics to more properly address noisy and structurally important regions. This method, which we termed contextual (data-driven) weighting, involved the use of a robust structural cue to prevent excessive regularization of trusted (i.e. low noise) high curvature image regions and an edge evidence measure, where both measures are gated by a measure of image quality based on the concept of spectral flatness. We incorporated our proposed regularization weighting into four major segmentation frameworks that range from discrete to continuous methods: a minimum-path approach [9], Graph Cuts [14], Active Contours Without Edges [24], and a contextual Mumford-Shah based approach [38]. Our methods were validated on a variety of natural and medical image databases and compared against the globally-optimum weight approach and to two alternate approaches: the best possible (least-error) spatially-fixed regularization weight, and the most closely related data-driven spatially adaptive regularization method. In addition, we incorporated existing texture-based contextual cues to demonstrate the applicability of the data-driven weights.
View record
Faced with an increasingly aging and overweight population, our modern societies, particularly in the west, are set to witness a steep rise in various orthopaedic health problems in the coming decades, especially joint diseases such as arthritis. Better understanding of the way bones of the joints work is thus imperative for studying the nature and effects of these diseases and for finding cures. The data obtained from conventional sources such as skin markers and x-ray/fluoroscopy scans are generally useful but quite limited in terms of accuracy, quantification abilities and three-dimensional visualization potential. The continuous increase in the quality and versatility of various modern imaging modalities is enabling many new means for enhanced visualization and analysis of motion data of the joints. In this thesis we make use of ultrasound (US) and magnetic resonance (MR) imaging to facilitate robust, accurate and efficient analysis of the bones of joints in motion. We achieve this by obtaining motion data using 3D US with high temporal resolution which is then fused with a high spatial resolution, but static MRI volume of the same region (we mostly focus on the knee joint area). Our contributions include novel ways for improved segmentation and localization of the bones from image data. In particular, a highly effective method for improving bone segmentation in MRI volumes by enhancing the contrast at the bone-cartilage interface is proposed. Our contribution also focuses on robust and accurate registration of image data. To achieve this, a new method for stitching US bone volumes is proposed for generating larger fields of view. Further, we also present a novel method for US-MRI bone surface registration. The tools developed during the course of this thesis facilitate orthopaedic research efforts aiming to improving our understanding of the workings of the joints. The tools and methodologies proposed are versatile and expected to be applicable to other applications.
View record
If this is your researcher profile you can log in to the Faculty & Staff portal to update your details and provide recruitment preferences.
