S Sidney Fels

The sudden pivot to remote communication triggered by COVID-19 has intensi- fied the need for advanced teleconferencing technologies that closely mimic phys- ical presence. Achieving direct eye contact, understanding physical context, and enabling coordinated non-verbal cues are essential for enriching virtual interac- tions. These prerequisites have led to increased interest in 3D teleconferencing tools that offer an immersive persona projection, enhancing the overall commu- nication experience. Current 3D experiences primarily rely on Virtual Reality (VR) and Augmented Reality (AR) technologies, utilizing Head-Mounted Dis- plays (HMDs), semi-transparent 2D displays, and standalone 3D devices. How- ever, evaluating the Quality of User Experience (QoE) in these immersive environ- ments poses significant challenges, as traditional subjective feedback and objective video quality evaluation methods fall short in addressing VR’s interactive nature. Our study addresses this gap by developing new QoE metrics for 3D teleconfer- encing, leveraging Fish Tank Virtual Reality (FTVR) to simulate presence using stereo and motion parallax depth cues. We report the Just Noticeable Difference (JND) for resolution, latency, jitter, and framerate as 50dpi, 50ms, 0.06mm, 18FPS, respectively, providing tangible thresholds for evaluating the performance and user experience of 3D teleconferencing tools. Moreover, we acknowledge the lack of accessible platforms for comprehensive, end-to-end tests in 3D teleconferencing. While proprietary systems such as Google’s Starline and Microsoft’s Holoportation show promise, their limited accessibility and high costs hinder broader research en- deavors. To rectify this, we have developed an open-source end-to-end platform, enabling researchers to perform comprehensive tests and integrate enhancements into the 3D teleconferencing pipeline.

View record

The intricate and interdigitated musculature of the human tongue presentsa formidable challenge in quantifying functional traits for lingual behaviors, notably in speech articulation. To advance speech research and address treatment strategies, a harmonized approach blending quantitative and qualitative analyses of muscle functions within specific tongue movements is essential. A novel biomechanical ”atlas” model, incorporating morphological features from a diverse range of speakers, has the potential to predict and analyze muscle behavior across distinct subject-specific speech kinematics. This work centers on a Finite Element Model (FEM)constructed from multi-subject atlas MRI data, capturing the biomechanical intricacies of tongue movements during speech tasks such as "a souk" and "a geese." The tongue atlas model is used to inversely predict the muscle activations of eight native English speakers. To illuminate articulation differences within the same consonant sound production in distinct vowel contexts, we perform a Wilcoxon signed-ranked difference test on the estimated muscle activation patterns for each speaker. Our findings reveal that nearly all muscles engaged in producing the same /s/ sound exhibit different activation patterns in "a souk" vs. "a geese" across all speakers.Furthermore, we temporally align the muscle activation patterns for each speaker using a dynamic time warp (DTW) function and define a similarity index to measure resemblances in the employed motor strategies. Our results suggest that some speakers are more likely to employ similar motorstrategies when uttering words than others.

View record

In-class multitasking is a very common practice among university students as a strategy to tackle their many goals and commitments in life. However, research shows that in-class multitasking affects students' learning outcomes and academic performance negatively. Even though many students are aware of these potential drawbacks, in-class multitasking remains persistent in in-person and even more so in online classes. To learn more about students' motivations, perceptions, and challenges in in-class multitasking in online classes and to be able to propose solutions to their challenges, we conducted a small formative study (N=10) where we asked students to keep multitasking diaries in their online classes and followed up with semi-structured interviews. Through this study, we categorized students' challenges with in-class multitasking into 1. cognitive consequences which consist of content loss, context loss, and the overhead in time spent after the class to compensate for multitasking, and 2. metacognitive consequences which consist of the inhibition of time and attention management due to multitasking. Our insights from the preliminary study lead to the overarching requirement in design which is to support in-class multitasking by reducing its adverse effects on learning. Based on the cognitive consequences, our first design requirement is to enable immediate recovery of missed content in the class time frame. To satisfy this requirement, we propose a bichronous learning environment that enables accelerated guided viewing of past/missed content immediately. In an experimental evaluation (N=20) of these design elements, our prototype showed significant improvement in students' learning outcomes when they multitask, compared to a baseline system. Based on the metacognitive consequences, we define our second design requirement as providing assistance in metacognitive monitoring in in-class multitasking by enabling cognitive offloading. We propose topic, activity, and time-based offloading to help students manage their time and attention when multitasking under different circumstances. Our qualitative evaluation (N=9) of these design elements shows promise in their effectiveness in assisting students' self-regulation in in-class multitasking.

View record

To support effective pointing interactions with an intelligent virtual agent (IVA), the first question to answer is how accurately users can interpret the direction of IVAs' pointing. In this thesis, we designed an IVA and investigated its capability to point to the real world as accurately as a real person. We used a spherical Fish Tank Virtual Reality (FTVR) display as it provides effective 3D depth cues and is situated in real-world coordinates, allowing IVAs to point to the real world. We first conducted an experiment to determine the pointing cue, a fundamental design factor of our IVA. Specifically, we evaluated the effect of head and hand cues on users' perception of the IVA's pointing. The findings provide design guidelines for selecting pointing cues in virtual environments. Following the guideline, we further determined our IVA's other design factors, including appearance, subtleties on how it points, with rationales elaborated. Using our designed IVA, we conducted an experiment to investigate the difference between the IVA and natural human pointing, measured by users' accuracy of interpreting the pointing to a physical location. Results show that participants can interpret the IVA's pointing to a physical location more accurately than the real person's pointing. Specifically, the IVA outperformed the real person in the vertical dimension (5.2% less error) and yielded the same level of accuracy horizontally. Our IVA design mitigated the pointing ambiguity due to the eye-fingertip alignment commonly found in human pointing, which may account for the IVA's higher pointing accuracy. Thus, our findings provide design guidelines for visual representations of IVAs with pointing gestures.

View record

Handwritten note-taking with pen and paper is still the preferred medium to achieve information seeking and comprehension from diverse learning objects. But students, especially in video-based learning settings, exercise laborious practices to re-find the corresponding video context when reviewing notes. We propose the orchestration of students' handwritten notebook content as interoperable links to retrieve previously watched instructional videos. This work articulates the research objectives in two phases. In phase 1, we analyzed the characteristic features of notebook content of watched videos. And, in phase 2, we investigated student expectations and requirements of the proposed video retrieval system. Analysis of quality handwritten notebook samples and the related video materials in a lab study with ten engineering students revealed distinctive characteristic representations of note content such as text, formula, figures, and chiefly a hybrid of all the 3. A box plot interpretation of notes and the watched video content confirmed that at least 75% of the identified note samples demonstrated a verbatim overlap of 50% or more with the related video content, hinting at its potential use as a query artifact. Additionally, the video references to collected note samples exhibited referencing at three temporal levels: point, interval, and whole video. A 12-student lab study indicated higher satisfaction for video matches returned at the `interval' level and showcased students' existing workarounds for linking back to videos. Overall, students rated a positive Mean score for the system's usability to re-find note-specific video context. A medium-fidelity prototype was built, using off-the-shelf computer vision algorithms, to deduce technology requirements associated with the proposed approach. When tested on the 181 identified note samples, the prototype system matched 77.5% of the samples to corresponding watched videos. The proposed method worked exceptionally well to find suitable videos for textual notes --- yielding a 98% accuracy. The note content overlap with the video results further highlights the fragmented nature of the evaluated accuracy across all three temporal levels. Overall, the presented work ascertains the prospect of augmenting prevalent Personalized learning (PL) strategies, such as handwriting notes for future reference, to easily re-find and connect to the watched videos.

View record

The human speech motor control system takes advantage of the constraints in categorical speech perception space to reduce the index of difficulty of articulatory tasks. Taking this for inspiration, we introduce a perceptual mapping from speech-like complex multiple degree-of-freedom (DOF) movement of the hand to a controllable formant space, that allows us to leverage categorical perceptual constraints for reducing the difficulty level of hand motor control tasks. The perceptual network is modeled using long short term memory networks (LSTMs) aimed at optimizing a connectionist temporal classification (CTC) loss function. Our motor control mapping network consists of a graph convolutional neural network (GCNN) combined with LSTM encoder-decoder network, that is regularized with the help of the trained perception model. The mapping allows the user's hand to generate continuous kinematic trajectories at a reduced effort by altering the complexity of the task space. This is a human-in-the-loop system where the user plays the role of an expert assessor in evaluating the degree to which the network is capable of reducing the complexity of task space. Further, we quantitatively formulate the index of difficulty of the task space and the throughput of the user and demonstrate that our model is able to perform consistently well in generating trajectories with considerably reduced effort using mouse and data glove-based input devices.

View record

Learning to control is a complicated process, yet humans seamlessly control various complex movements. Motor theory suggests that humans start motor learning by learning to act in a feedforward manner. However, it is still unclear how humans learn feedforward control strategies. We hypothesize that this mechanism is governed by the criterion of success (reinforcement) or failure (penalty) of the task. Taking this for inspiration, we investigate how we can learn a feedforward controller utilizing reinforcement learning. Additionally, we investigate how the factors such as the difficulty of the task and noise present in the motor system are related to human motor control.Hence, a one-dimensional muscle-based biomechanical model is built to create a reaching task setup. The model contains an actuator controlled by the antagonistic and agonistic muscle pair and a goal or target to reach. Then, an end-to-end reinforcement-learning-based feedforward controller is learned to estimate control signals while taking the difficulty levels of a reaching task and noise levels into account. To design the learning-based controller, we adapted the model-free RL algorithm ``Soft Actor-Critic". As a result, during training, we observed that the SAC-based feedforward controller has learned to prepare co-activation to reach a target in the kinematic space using a minimum number of controller predictions. Moreover, we found that the controller has learned to estimate high-amplitude muscle activations as a way to adapt to the noise levels in the motor system. Finally, we conducted information analysis similar to Fitts' analysis to determine how the difficulty of the task and noise affected the controller. The effect of the task's difficulty and the noise in the system is determined by finding the relationship between the number of controller predictions, task difficulty, and the amount of noise. Our analysis demonstrates that the number of controller predictions increases exponentially with the increase in the difficulty of the task with the amount of noise kept constant. A linear relationship exists between the number of controller predictions and the amount of noise with ID kept constant. Additionally, we found that the effect of target width is more dominant than the distance, which confirms Welford's observation.

View record

Mandibular meshes segmented from computerized tomography (CT) imagescontain rich information of the dentition. Inconsistent dentition conditionsin healthy mandible data sets can impair data-driven premorbid shape predictionfor diseased mandibles. We developed a mesh segmentation methodthat includes a preprocessing algorithm using an off-the-shelf non-rigid registration,a surface mesh feature function, and an active contour model usinggeodesic distance. Constructive Solid Geometry (CSG) operations areemployed to separate the dentition area from the mandibular mesh. Aneasy-to-use interactive tool was implemented, allowing users to adjust thecontour position.We evaluated our method (preprocessing algorithm and user intervention)by comparing it with the traditional method of manual removal using3D Slicer. The results indicated that our method helped save the manualprocessing time by 40%, which largely improves the efficiency. From astatistical-shape-modeling-based shape completion test, we drew the conclusionthat edentulous mandibular data set could help make significantlybetter premorbid shape predictions (Z=-2.484,p=0.013) than data set withmixed dentition conditions.Besides tooth segmentation from 3D meshes, our research can assistvirtual planning for bone graft placement in the defective mandible andimplant placement in the bone graft. This work forms the underlying basisof a useful tool for coupling jaw reconstruction and restorative dentition forpatient treatment planning.

View record

The human voice is a complex but unique physiological process. It involves the neuromuscular control of articulators to form an intricate upper vocal tract geometry, which yields different speech sounds. The existing computational vocal tract models have significant limitations concerning acoustic precision and simulation performance. The high-dimensional vocal tract models can compute precise acoustic wave propagation at the expense of simulation run-time. This thesis aims to fill these lacunae through two major contributions.Firstly, we introduce a novel vocal tract that extends the existing two-dimensional (2D) vocal tract modelling approach while having three-dimensional behaviour. The proposed model (2.5D FDTD) employs the Finite-Difference Time-Domain numerical scheme to discretize and compute acoustic components on a staggered grid. The simulated acoustic outputs of our new model are shown to match with the 2D FDTD vocal tract model at a low spatiotemporal resolution for open static vocal tract shapes. Contrary to 2D FDTD, the model adds tube depth as an additional impedance parameter to the acoustic wave solver by lumping off-plane waves. This technique offers an excellent balance between computational cost and acoustic precision while promising better geometrical flexibility for vocal tract modelling.Secondly, we tested the model's basic capabilities through the acoustic simulation of cardinal English vowel sounds. For realistic modelling of vowel sounds, we built a vocal tract radiation model. We also couple the 2.5D vocal tract with a self-oscillatory lumped-element vocal fold model to illustrate a fully connected articulatory speech synthesizer. This study offers a speech synthesis tool that can generate static vowel sounds and set up a new pathway for lightweight vocal tract modelling and other computational acoustic research.

View record

Masticatory performance and the occlusal force are two of the main clinical metrics that are used to evaluate the masticatory function objectively. A comprehensive evaluation of masticatory function requires a correlative inspection of these two metrics. The complex multi-variant nature of the human mastication and the limitations of visualization and clinical measurement techniques, complicates the clinical investigation of masticatory function. A biomechanical model of oral food breakdown has the ability to bypass these difficulties. The currently available food breakdown models are either highly dependent on experimental data or are focused on food engineering applications. In this thesis, we attempted to solve these issues by building a two-dimensional fracture mechanics model to simulate the oral food breakdown. The different computational methods available to solve fracture mechanics problems have limits and strengths, which affects the accuracy of their solution. Extended Finite Element Method (XFEM) and Smoothed Particle Hydrodynamics (SPH) method use two very distinct approaches to solve fracture mechanics problems; comparing the effectiveness of these two methods can provide valuable insights into the computational possibilities. As the classical SPH formulation for solid mechanics suffers from numerical deficiencies, we first performed a set of modifications to build a corrected SPH model for solid and fracture mechanics. We solved fracture mechanics benchmark tests using XFEM and the modified version of SPH and investigated their strengths and weaknesses thoroughly. The SPH method eventually was selected to model the food breakdown procedure. We simulated the food breakdown following one chewing stroke using our two-dimensional SPH fracture model and measured the corresponding occlusal force and masticatory performance for a range of different food properties. The food breakdown model was able to simulate the experimental correlation between masticatory performance and the food properties. Although the simulated measurements for occlusal force were in accordance with the previous experimental studies, further detailed clinical investigation is required to validate the force pattern during chewing. The simplified biomechanical model of oral food comminution described in this work can be regarded as the first step toward building a patient-specific model to pre-assess the patient’s masticatory function following a maxillofacial reconstructive surgical plan.

View record

Currently, most evaluations of patient outcome following mandibular reconstructive surgery are defined by a combination of qualitative analyses consisting of patient-reported functional ability and masticatory performance. Metrics such as occlusal pressure and jaw kinematics provide quantitative assessments of masticatory function, facilitating a more comprehensive evaluation of patient outcomes. This thesis proposes a novel virtual mastication framework for evaluating occlusal force and area based on metrics of masticatory force and kinematics taken in a clinical setting. Statistical shape modeling was used to develop a mandible atlas based on the morphological averages which contribute to both universal model creation and prediction of missing anatomy. The simulation was able to predict clinically verified maximum occlusal forces and contact areas based on data inputs of intraoral dentition scans and jaw constraints provided through a controlled study of healthy volunteers. In assembly with this framework, a validation study of an occlusal force and contact area measurement system (Dental Prescale II) was performed to gain principal masticatory function information with measured accuracy. This work serves as a foundation for implementing virtual tools within the maxillofacial reconstructive surgery clinical workflow.

View record

Rewatching video segments is common in video-based learning, and video segments of interest need to be located first for this rewatching. However, learners are not well supported in the process of locating in-video information. To fill this gap, the presented work explores whether video annotations are effective in helping learners locate previously seen in-video information. A novel interface design consisting of two components for learning with videos is proposed and tested in the task of locating in-video information: an annotating mechanism based on an integration of text with video, and an annotation manager which enables the learner to see all annotations he/she has made on a video and provides quick access to video segments. A controlled lab experiment with 16 undergraduate students as subjects was carried out. Experiment results suggested that the use of video annotations significantly reduced time spent on searching for previously seen video segments by about 5 seconds (p
View record

Advancements in state-of-the-art 3D reconstruction algorithms have sped ahead of the development of interfaces or application programming interfaces (APIs) for developers, especially to those who are not experts in computer vision.In this thesis, we have designed a novel interface, specifically for 3D reconstruction techniques, which uses a description (covering the conditions of the problem) to allow a user to reconstruct the shape of an object without knowledge of 3D vision algorithms. The interface hides the details of algorithms by using a description of visual and geometric properties of the object. Our interface interprets the description and chooses from a set of algorithms those that satisfy the description. We show that this description can be interpreted to one appropriate algorithm, which can give a successful reconstruction result.We evaluate the interface through a proof of concept interpreter, which interprets the description and invokes one of three underlying algorithms for reconstruction. We demonstrate the link between the description set by the user and the result returned using synthetic and real-world datasets where each object has been imaged with the appropriate setup.

View record

Video continues to be used extensively as an instructional aid within modern educational contexts, such as in blended (flipped) courses, self-learning with MOOCs (Massive Open Online Courses), informal learning through online tutorials, and so on. One challenge is providing mechanisms for students to efficiently bookmark video content and quickly recall and review their video collection. We have run a background study to understand how students annotate video content, focusing especially on what words they would use most to bookmark video content. From this study, we proposed to leverage a quick tagging mechanism in an educational video interface comprised of a video filmstrip and transcript, both presented adjacent to a video player. The 'quick' tagging is defined as an easy and fast way to mark course video parts with predefined semantic tags. We use the metaphor of marking and highlighting textbook to achieve our quick tagging interaction. This mechanism was evaluated in a controlled study with handwritten notes. We found that participants using our quick tagging interface spent around 10% longer watching and learning from video on average than when taking notes on paper. Our participants also reported that tagging is a useful addition to instructional videos that helps them recall video content and finish learning tasks.

View record

The perceived duration of a time interval can seem shorter or longer relative to real time (i.e., solar time or clock time) depending on what fills that time interval. Research has suggested that increased immersion alters a users ability to reproduce a given duration whilst doing a simple task or playing a game in an Immersive Virtual Environment (IVE). Virtual Reality (VR) allows users to experience virtual environments similar to the real world. The contribution of this experimentalresearch is to explore the effects of undertaking a cognitive spatial task and immersion within a VR environment on a persons perception of time. A VR experience using a cognitive task (maze navigation) was compared with a non VR (control) experience of the same task to explore if the effects exist and if the effects are more significant in an IVE compared to a screen-based simple multimedia experience. Also, a VR experience of the environment without any task was compared to the same environment with the cognitive task to establish the effect of a spatial cognitive task on temporal perception. More specifically, this study measured how much temporal distortion is achievable utilizing cognitive tasks in a VR experience. In this thesis the use of cognitive tasks and VR are the independent variables and the perceived duration of the experiment (time) is the dependent variable. Obtained data suggest that being immersed in a VR experience results in 16.10% underestimation of time, while a non-VR experience results in 7.5% overestimation of time. Moreover, navigating mazes that involve a high cognitive load results in 6.45% underestimation of time. Finally, the combination of VR and high cognitive load (navigating the mazes without guiding lines in a VR experience) result in 22.18% underestimation of time. Finally, the implications of this research are discussed at the end of this thesis.

View record

The upper-airway complex is involved in a number of life-sustaining functions, such as swallowing, speech, breathing and chewing. Disorders associated with these functions can dramatically reduce the life quality of the suffers. Biomechanical modelling is a useful tool that can bridge the gap between the human knowledge and medical data.When tailored to individual patients, biomechanical models can augment the imaging data, to enable computer-assisted diagnosis and treatment planning. This thesis introduces a model-registration framework for creating subject-specific models of the upper-airway complex based on 3D medical images.Our framework adapts a state-of-art comprehensive biomechanical model of head and neck, which represents the generic upper-airway anatomy and function. By morphing this functional template to subject-specific data, we create upper-airway models for particular individuals. In order to preserve the functionality of the comprehensive model, we introduce a multi-structure registration technique, which can maintain the spatial relationship between the template components, and preserve the regularity of the underlying mesh structures. The functional information, such as the muscle attachment positions, joint positions and biomechanical properties, is updated to stay relevant to the subject-specific model geometry. We demonstrate the functionality of our subject-specific models in the biomechanical simulations.Two illustrative case studies are presented. First, we apply our modelling methods to simulating the normal swallowing motion of a particular subject based on the kinematics (of the airway boundary, jaw and hyoid) extracted from dynamic 3D CT images. The results suggest that our model tracks the oropharyngeal motion well, but has limited ability to reproduce the hyolaryngeal movements of normal swallowing. Second, we create two speaker-specific models based on 3D MR images, and perform personalized speech simulations of the utterance ageese. The models reproduce the speech motion of the tongue and jaw recorded in tagged and cine MRI data with sub-voxel tracking error, predict the muscular coordinating patterns of the speech motion.This study demonstrates the feasibility of using template-based subject-specific modelling methods to facilitate personalized analysis of upper-airway functions. The proposed model-registration framework provides a foundation for developing a systematic and advanced subject-specific modelling platform.

View record

Speech is unique to human beings as a means of communication and many efforts have been made towards understanding and characterizing speech. In particular, articulatory speech synthesis is a critical field of study as it works towards simulating the fundamental physical phenomena that underlines speech. Of the various components that constitute an articulatory speech synthesizer, vocal fold models play an important role as the source of the acoustic simulation. A balance between the simplicity and speed of lumped-element vocal fold models and the completeness and complexity of continuum-models is required to achieve time-efficient high-quality speech synthesis. In addition, most models of the vocal folds are seen in a vacuum without any coupling to the vocal tract model. This thesis aims to fill these lacunae in the field through two major contributions. We develop and implement a novel self-oscillating vocal-fold model, composed of an 1D unsteady fluid model loosely coupled with a 2D finite-element structural model. The flow model is capable of handling irregular geometries, different boundary conditions, closure of the glottis and unsteady flow states. A method for a fast decoupled solution of the flow equations that does not require the computation of the Jacobian matrix is provided. The simulation results are shown to agree with existing data in literature, and give realistic glottal pressure-velocity distributions, glottal width and glottal flow values. In addition, the model is more than order of magnitude faster than comparable 2D Navier-Stokes fluid solvers while better capturing transitional flow than simple Bernoulli-based flow models.Secondly, as an illustrative case study, we implement a complete articulatory speech synthesizer using our vocal fold model. This includes both lumped-element and continuum vocal fold models, a 2D finite-difference time-domain solver of the vocal tract, and a 1D tracheal model. A clear work flow is established to derive model components from experimental data or user-specified meshes, and run fully-coupled acoustic simulations. This leads to one of the few complete articulatory speech synthesizers in literature and a valuable tool for speech research to run time-efficient speech simulations, and thoroughly study the acoustic outcomes of model formulations.

View record

Lattice Boltzmann is a fixed grid particle based method originated from molecular dynamics which uses a kinetic-based approach to simulate fluid flows. The fixed grid nature and simplicity of lattice Boltzmann algorithm makes it an appealing approach for preliminary swallowing simulations. However, the issues of compressibility effect and boundary/initial condition implementation can be the source of instability and inaccuracy especially at high Reynolds simulations. The current work is an assessment of the lattice Boltzmann method with respect to high Reynolds number flow simulations, compressibility effect of the method, and the issue of boundary and initial condition implementation. Here we investigate the stability range of the lattice Boltzmann single relaxation and multi relaxation time models as well as the issue of consistent boundary/initial condition implementation. The superior stability of multi relaxation time (MRT) model is shown on the lid-driven cavity flow benchmark as a function of Reynolds number. The computational time required for the SRT model to simulate the li-driven cavity flow at Re=3200 is about 14 times higher than the MRT model and it’s shown that computational time is related to the third power of lattice resolution. It is suggested that single relaxation time model is inefficient for simulations with moderately high Reynolds number Re>1000 and the use of multi relaxation time model becomes necessary. Compressibility effect is the next topic of study where the incompressible lattice Boltzmann method is introduced. The compressibility error of the method surpasses the spatial discretization error and becomes the dominant source of error as the flow Reynolds number increases. It is shown on a 2D Womersley flow benchmark that the physical time step required for LBM is about 300 times larger than the physical time step of the finite volume implicit solver while generating results with the same order of accuracy at Re=2000. Due to the compressibility error inherent to the method, lattice Boltzmann is not recommended for preliminary swallowing simulations with high Reynolds number, since implicit time advancement methods can generate results with the same order of accuracy in noticeably less computational time.

View record

Smartphones today store large amounts of data that can be confidential, private or sensitive. To protect such data, all mobile OSs have a phone lock mechanism, a mechanism that requires user authentication in order to access applications or data on the phone, while also allowing to keep data-at-rest encrypted with encryption key dependent on the authentication secret. Recently Apple has introduced Touch ID feature that allows to use a fingerprint-based authentication to unlock an iPhone. The intuition behind such technology was that its usability would motivate users to use stronger passwords for locking their devices without sacrificing usability substantially. To this date, it is not clear, however, if users take an advantage of Touch ID technology and if they, indeed, employ stronger authentication secrets. It is the main objective and the contribution of this work to fill this knowledge gap. In order to answer this question we conducted three user studies (a) an in- person survey with 90 subjects, (b) an interview study with 21 participants, and (c) an online survey with 374 subjects. Overall we found that users do not take an advantage of Touch ID and use weak authentication secrets, mainly PIN-codes, similarly to those users who do not have Touch ID sensor on their devices. To our surprise, we found that more than 30% of subjects in each group did not know that they could use alphanumeric passwords instead of four digits PIN-codes. Others stated that they adopted PIN-codes due to better usability in comparison to passwords. Most of the subjects agreed that Touch ID, indeed, offers usability benefits such as convenience, speed and ease of use. Finally, we found that there is a disconnect between users desires for security that their passcodes have to offer and the reality. In particular, only 12% of participants correctly estimated the security PIN-codes provide while the rest had unjustified expectations.

View record

We propose the use of a personal video navigation history, which records a user's viewing behaviour, as a basis for casual video editing and sharing. Our novel interaction supports users' navigation of previously-viewed intervals to construct new videos via simple playlists. The intervals in the history can be individually previewed and searched, filtered to identify frequently-viewed sections, and added to a playlist from which they can be refined and re-ordered to create new videos. Interval selection and playlist creation using a history-based interaction is compared to a more conventional filmstrip-based technique. We performed several user studies to evaluate the usability and performance of this method and found significant results indicating improvement in video interval search and selection.

View record

Video annotation is a process of describing or elaborating on objects or events represented in video. Part of this process involves time consuming manual interactions to define spatio-temporal entities - such as a region of interest within the video.This dissertation proposes a pursuit method for video annotation to quickly define a particular type of spatio-temporal entity known as a point- based path. A pursuit method is particularly suited to annotation contexts when a precise bounding region is not needed, such as when annotators draw attention to objects in consumer video.We demonstrate the validity of the pursuit method with measurements of both accuracy and annotation time when annotators create point-based paths. Annotator tool designers can now chose a pursuit method for suitable annotation contexts.

View record

In the process of working with a real-time, gesture controlled speech and singing synthesizer used for musical performance, we have documented performer related issues and provided some suggestions that will serve to improve future work in the field from an engineering and technician's perspective. One particular, significant detrimental factor in the existing system is the sound quality caused by the limitations of the one-to-one kinematic mapping between the gesture input and output. In order to solve this a force activated bio-mechanical mapping layer was implemented to drive an articulatory synthesizer, and the results were and compared with the existing mapping system for the same task from both the performer and listener perspective. The results show that adding the complex, dynamic bio-mechanical mapping layer introduces more difficulty but allows a greater degree of expression to the performer that is consistent with existing work in the literature. However, to the novice listener, there is no significant difference in the intelligibility of the sound or the perceived quality. The results suggest that for browsing through a vowel space force and position input are comparable when considering output intelligibility alone but for expressivity a complex input may be more suitable.

View record

The ability to swallow is crucial in maintaining adequate nutrition. However, there is a high prevalence of dysphagia among the elderly and a high associated mortality rate. To study the various causes of the associated physiological changes, one must first understand the biomechanics of normal swallowing. However, functional studies of the anatomically complex head and neck region can prove to be difficult due to both technical and ethical reasons.To overcome the limitations of clinical studies, this thesis proposes the use of a 3D computer model for performing dynamic simulations. A state-of-the-art model of the hyolaryngeal complex was created for simulating swallowing-related motor tasks with a special focus on hyoid excursion since reduced hyoid motion is a major indicator of oropharyngeal dysphagia. The model was constructed using anatomical data for a male cadaver from the Visible Human Project and an open-source dynamic simulation platform, ArtiSynth.Hyoid motion data obtained from videofluoroscopy of subjects performing normal swallowing was applied to the model for inversely simulating the potential muscle activities of the extrinsic laryngeal muscles during hyoid excursion. Within a specific range, the model demonstrated the ability to reproduce realistic hyoid motion for swallowing. Selective usage of suprahyoid muscles was also examined and was found to be possible in achieving adequate hyoid excursion for successful swallows.Finally, this study investigated the relationship between muscle weakening and hyoid range of motion using the hyolaryngeal model. Loss of muscle strength is characteristic of the aging process. Simulation of the maximum hyoid displacement under various muscle conditions confirmed a nonlinear reduction in the hyoid motion range under a linear decline in muscle strength. With an assumed rate of muscle weakening, the proportion of hyoid range reduction was estimated for a person at various ages. The results suggest that severe muscle weakening might be required to reduce hyoid excursion sufficiently to impair swallowing to a significant degree.

View record

With the rapid rise of the popularity of online social networks (OSNs) in recent years, we have seen tremendous growth in the number of available OSNs. With newer OSNs attempting to draw users in by focussing on specific services or themes, it is becoming clearer that OSNs do not compete on the quality of their technology but rather the number of active users. This leads to vendor lock-in, which creates problems for users managing multiple OSNs or wanting to switch OSNs. Third party applications are often written to alleviate these problems but often find it difficult to deal with the differences between OSNs. These problems are made worse as we argue that a user will inevitably switch between many OSNs in his or her lifetime due to OSNs being incredibly fashionable things whose lifespan is dependent on social trends. Thus, these applications often only support a limited number of OSNs. This thesis examines how it is possible to help developers write apps that run against multiple OSNs. It describes the need for and presents a novel set of abstractions for apps to use to interface with OSNs. These abstractions are highly expressive, future proof, and removes the need for an app to know which OSNs it is running against. Two evaluations were done to determine the strength of these abstractions. The first evaluation analyzed the expressiveness of the abstractions while the latter analyzed the feasibility of the abstractions. The contributions of this thesis are a first step to better understanding how OSNs can be described at a high level.

View record

This thesis describes the evaluation of pCubee, a handheld outward-facing geometric display that supports high-quality visualization and tangible interaction with 3D content. Through reviewing existing literatures on 3D display technologies, we identified and examined important areas that have yet to be fully understood for outward-facing geometric displays. We investigated the performance of a dynamic visual calibration technique to compensate for tracking errors, and we demonstrated four novel interaction schemes afforded by tangible outward-facing geometric displays, including static content visualization, dynamic interaction with reactive virtual objects, scene navigation through display movements, and bimanual interaction. Two experiments were conducted to evaluate the impact of display seams and pCubee's potential in spatial reasoning tasks respectively. Two stimuli, a path-tracing visualization task and a 3D cube comparison task that was similar to a mental rotation task, were utilized in the experiments. In the first experiment, we discovered a significant effect on user performance in path-tracing that was dependent on the seam thickness. As seam size increased beyond a thickness threshold, subjects relied less on multiple screens and spent longer time to trace paths. In the second experiment, we found that subjects had significant preference for using the pCubee display compared to a desktop display setup when solving our cube comparison problem. Both time and accuracy using pCubee were as good as using a much larger, more familiar desktop display. This proved the utility of outward-facing geometric displays for spatial reasoning tasks. Our analysis and evaluation identified promising potential but current limitations of pCubee. The outcomes from our analysis can help to facilitate development and more systematic evaluations of similar displays in the future.

View record

In this thesis, we present the results of a user study that compares three different selection methods for moving targets in 1D and 2D space. The standard Chase-and-Click method involves pursuing an onscreen target with the mouse pointer and clicking on it once directly over it. The novel Click-to-Pause method involves first depressing the mouse button to pause all onscreen action, moving the cursor over the target and releasing the mouse button to select it. The Hybrid method combines the initial pursuit with the ability to pause the action by depressing the mouse button, affording an optimization of the point of interception. Our results show that the Click-to-Pause and Hybrid methods results in lower selection times than the Chase-and-Click method for small or fast targets, while the Click-to-Pause technique is the lowest overall for small-fast targets. We integrate the more practical Hybrid method into a multi-view video browser to enable the selection of hockey players in a pre-recorded hockey game. We demonstrate that the majority of correct player selections were performed while the video was paused and that our display method for extraneous information has no effect on selection task performance. We develop a kinematic model that is based on movement speed and direction in 1D as an adjustment to the effective width and distance of a target. Our studies show that target speed assists users when a target is approaching, up to a critical velocity where the direction is irrelevant and speed is entirely responsible for the index of difficulty. In addition, we suggest that existing linear and discrete models of human motor control are inadequate for modeling the selection of a moving target and recommend the minimum jerk law as a guide for measuring human motor acceleration. By combining our empirical results from moving target selection tasks in 1D with our theoretical model for motor control, we propose an extension to Fitts’ Law for moving targets in 2D polar space.

View record