Kaikki aineistot
Lisää
Abstract This thesis presents a mobile instrumentation middleware, AWARE, aimed at facilitating our understanding of human behavior. We demonstrate how to use AWARE to build context-aware applications, collect data, and study human behavior. Mobile phones are resource-constrained and several considerations need to be taken into account to create a research tool that ensures problem-free data collection. AWARE can mitigate researchers’ effort when building mobile data-logging tools and context-aware applications. By encapsulating implementation details of sensor data retrieval and exposing the sensed data as higher-level abstractions, researchers spend less time developing software and save more time for doing research and analyzing the collected data, both quantitative and qualitative. This thesis demonstrates AWARE’s use in a number of case studies. These vary in the research methods we have used: prototype-building; large-scale deployment; surveys; interviews; cognitive walkthroughs; heuristic evaluation; laboratory & field studies data logs; Day Reconstruction Method (DRM); and Experience Sampling Method (ESM). Together with these methods, we demonstrate how AWARE helps study human behavior in different research scenarios, such as: enabling human-smartphone awareness, understanding concerns on battery life, modeling the proximity of users to their smartphones, and capturing location sharing concerns. The thesis’ contributions are: the design, implementation and evaluation of a novel mobile instrumentation middleware to facilitate an understanding of human behavior.
Abstract High-quality data is a necessity for successful research and development endeavors. In this article, we review relevant literature for data quality (DQ) assessment methods in different domains and discuss the possibilities, challenges and constraints of applying them to mobile sensing. We identify DQ dimensions directly applicable to sensor data: believability (comparison with the correct operating bounds), completeness (missing values), free-of-error (erroneous values), consistency (over time), timeliness (delay), accuracy (deviation from true value) and precision (granularity of readings) are core aspects of high-quality sensor data. We also emphasize that sensor data must be representative of the originating type of sensor. We propose an altruistic approach to DQ assessment for sensor data that facilitates aggregating and sharing of domain knowledge through a community-contributed library of DQ assessment methods organized by sensor type.
Abstract Digital marketing is increasingly moving from desktop (e.g., browser) to mobile environments (e.g., within mobile applications). The means for delivering ads however, remains largely unchanged: banners and videos. In this work, we explore transforming ad delivery methods to the mobile environment while mitigating issues causing frustration and distractions to the users, evident in both web and mobile marketing. We demonstrate that softly enforcing interaction with the ad — with minimal usable screen space reduction — can improve user’s attitude towards mobile advertising. Brand recollection is also influenced via increased interactions with the ad delivery method.
Abstract This paper investigates the challenges of elderly care from the perspective of caregivers. More concretely, we identify caregivers’ workflows and pinpoint the main bottlenecks in a sheltered accommodation. As the general population is aging fast, the role of information and communication technology (ICT) has grown in importance for elderly care. This development has brought versatile ICT-related supportive systems to caregivers and laymen working with aging people. Our study first analyzed how professionals in elderly care perceived their workflow challenges. A new ICT system was developed and implemented to support their work. The results of our study inform the design of upcoming ICT systems for a sheltered accommodation that are in high demand today.
Abstract Parkinson’s disease (PD) is a second most common neurological disorder that affects up to 10 million people worldwide. It has an evolving nature and the symptoms may vary from patient to patient. Thus, to increase the effectiveness of PD treatment, it is necessary a personalized medication plan. Currently, PD patients undergo symptom observation on semiannual clinical visits. This work aims at the development of a new way of observation via smartphones, while at the same time offering the PD patient a tool to better understand his medication needs. Our mobile application leverages smartphone’s inbuilt sensors in order to keep track of subject’s medication adherence throughout the day, taking shape as a short-term accelerometer-based game played several times a day, and allows PD patients to record when they took medication. The combination of collected datasets can be used in further studies in order to estimate the changes in PD severity and medication effectiveness over time.
Abstract An empirical investigation of active/continuous authentication for smartphones is presented by exploiting users’ unique application usage data, i.e., distinct patterns of use, modeled by a Markovian process. Specifically, variations of hidden Markov models (HMMs) are evaluated for continuous user verification, and challenges due to the sparsity of session-wise data, an explosion of states, and handling unforeseen events in the test data are tackled. Unlike traditional approaches, the proposed formulation utilizes the complete app-usage information to achieve low latency. Through experimentation, empirical assessment of the impact of unforeseen events, i.e., unknown applications and unforeseen observations, on user verification is done via a modified edit-distance algorithm for sequence matching. It is found that for enhanced verification performance, unforeseen events should be considered. For validation, extensive experiments on two distinct datasets, namely, UMDAA-02 and Securacy, are performed. Using the marginally smoothed HMM a low equal error rate (EER) of 16.16% is reached for the Securacy dataset and the same method is found to be able to detect an intrusion within ~2.5 min of application use.
Abstract Background: Multimodal wearable technologies have brought forward wide possibilities in human activity recognition, and more specifically personalized monitoring of eating habits. The emerging challenge now is the selection of most discriminative information from high-dimensional data collected from multiple sources. The available fusion algorithms with their complex structure are poorly adopted to the computationally constrained environment which requires integrating information directly at the source. As a result, more simple low-level fusion methods are needed. Objective: In the absence of a data combining process, the cost of directly applying high-dimensional raw data to a deep classifier would be computationally expensive with regard to the response time, energy consumption, and memory requirement. Taking this into account, we aimed to develop a data fusion technique in a computationally efficient way to achieve a more comprehensive insight of human activity dynamics in a lower dimension. The major objective was considering statistical dependency of multisensory data and exploring intermodality correlation patterns for different activities. Methods: In this technique, the information in time (regardless of the number of sources) is transformed into a 2D space that facilitates classification of eating episodes from others. This is based on a hypothesis that data captured by various sensors are statistically associated with each other and the covariance matrix of all these signals has a unique distribution correlated with each activity which can be encoded on a contour representation. These representations are then used as input of a deep model to learn specific patterns associated with specific activity. Results: In order to show the generalizability of the proposed fusion algorithm, 2 different scenarios were taken into account. These scenarios were different in terms of temporal segment size, type of activity, wearable device, subjects, and deep learning architecture. The first scenario used a data set in which a single participant performed a limited number of activities while wearing the Empatica E4 wristband. In the second scenario, a data set related to the activities of daily living was used where 10 different participants wore inertial measurement units while performing a more complex set of activities. The precision metric obtained from leave-one-subject-out cross-validation for the second scenario reached 0.803. The impact of missing data on performance degradation was also evaluated. Conclusions: To conclude, the proposed fusion technique provides the possibility of embedding joint variability information over different modalities in just a single 2D representation which results in obtaining a more global view of different aspects of daily human activities at hand, and yet preserving the desired performance level in activity recognition.
Abstract We develop a Markov state transition model of smartphone screen use. We collected use traces from real-world users during a 3-month naturalistic deployment via an app-store. These traces were used to develop an analytical model which can be used to probabilistically model or predict, at runtime, how a user interacts with their mobile phone, and for how long. Unlike classification-driven machine learning approaches, our analytical model can be interrogated under unlimited conditions, making it suitable for a wide range of applications including more realistic automated testing and improving operating system management of resources.
Abstract Digital fabrication laboratories (FabLabs) influence how we think, ideate, do, make, and create. To enable the full capacity of materialization of the most creative ideas in the FabLab, a fundamental understanding of the processes in the FabLab is required. To accomplish this, we propose a framework for dynamically and ubiquitously capturing human-human (team) interactions, human-tool/machine interactions, and human-design-object interactions in the complex scenarios that occur in the paradigm of making in FabLabs. The framework elaborates three methods. The first method produces categories of creative spaces about activities and users in the FabLab. The second method identifies interactions between users and tools, and between users. Last, the third method identifies in-depth cognitive and thinking types of makers in the FabLab. The proposed framework can improve creative results and experiences of all stakeholders in the making process in the FabLab, and provide easy customization of the FabLab training for different audiences.
Abstract Smartphones have become an integral part of people’s everyday lives. Smartphones are used across all household locations, including in the bed at night. Smartphone screens and other displays emit blue light, and exposure to blue light can affect one’s sleep quality. Thus, smartphone use prior to bedtime could disrupt the quality of one’s sleep, but research lacks quantitative studies on how smartphone use can influence sleep. This study combines smartphone application use data from 75 participants with sleep data collected by a wearable ring. On average, the participants used their smartphones in bed for 322.8 s (5 min and 22.8 s), with an IQR of 43.7–456. Participants spent an average of 42% of their time in bed using their smartphones (IQR of 5.87–55.5%). Our findings indicate that smartphone use in bed has significant adverse effects on sleep latency, awake time, average heart rate, and HR variability. We also find that smartphone use does not decrease sleep quality when used outside of bed. Our results indicate that intense smartphone use alone does not negatively affect well-being. Since all smartphone users do not use their phones in the same way, extending the investigation to different smartphone use types might yield more information than general smartphone use. In conclusion, this paper presents the first investigation of the association between smartphone application use logs and detailed sleep metrics. Our work also validates previous research results and highlights emerging future work.
Abstract Research on mobile sensing for mental health monitoring has traditionally explored the correlation between smartphone and wearable data with self-reported mental health symptom severity assessments. The effectiveness of predictive techniques to monitor depression is limited, given the idiosyncratic nature of depression symptoms and the limited availability of objectively labelled depression sensor-driven behaviour. In this paper, we investigate the possibility of using unsupervised anomaly detection methods to monitor the fluctuations of mental health and its severity. Informed by literature, we created a mobile application that collects acknowledged data streams that can be indicative of depression. We recruited 11 participants for a 1-month field study. More specifically, we monitored participants’ mobility, overall smartphone interactions, and surrounding ambient noise. The participants provided three self-reports: Big five personality traits, sleep and depression. Our results suggest that digital markers, combined with anomaly detection methods are useful to flag changes in human behaviour over time; thus, enabling mobile just-in-time interventions for in-the-wild assistance.
Abstract Spiral drawing has been utilized for years as a clinical tool to observe tremors and other abnormal movements in the assessment of different movement disorders. Specifically, in Parkinson’s Disease (PD), patients’ motor functionalities are measured by various tests, and spiral drawing is one of the proven techniques for assessing the severity of PD motor symptoms. Traditionally, this test is performed on pen and paper, and visually assessed by a clinician. There have been successful efforts for digitizing this test on tablets. Here, we describe a smartphone-based digitized version of the spiral drawing test. Moreover, we introduce a square-shaped drawing to solve an identified challenge of a smaller screen estate: finger occlusion while drawing. Both approaches are evaluated with 8 Parkinson’s Disease patients and 6 age-matching control participants. Based on earlier studies and our data, we select suitable motion parameters for quantifying the task. Our results show an observable, statistically difference in performance between users with Parkinson’s Disease and the control group in drawing accuracy.
Abstract The leading cause of death during earthquakes is the collapse of urban infrastructures and the subsequent delay of emergency responders in identifying and reaching the affected sites. To overcome this challenge, we designed and evaluated a crowdsensing system that detects collapsed buildings using end-user smartphones as distributed sensors. We present our evaluation of smartphones’ accuracy in inferring a building collapse by detecting falls onto solid surfaces, and estimating the false positive rate. Further sensors can present more detailed information about each potential collapse event. We conduct simulations to identify strategies for dealing with false-positive data under scenarios of varying population density. Potential building collapses can be determined with 95% accuracy given 10 connected devices within a 125m radius, increasing to 99.99% for 50 devices. End-user devices can proactively offer valuable help to emergency responders during earthquakes, potentially saving lives.
Abstract We present a revisitation analysis of smartphone use to investigate the question: do smartphones induce usage habits? We analysed three months of application launch logs from 165 users in naturalistic settings. Our analysis reveals distinct clusters of applications and users which share similar revisitation patterns. However, we show that much of smartphone usage on a macro-level is very similar to web browsing on desktops, and thus argue that smartphone usage is driven by innate service needs rather than technology characteristics. On the other hand, on a micro-level we identify unique characteristics in smartphone usage, and we present a rudimentary model that accounts for 92% in the variability of our smartphone use.
Abstract The spread of smartphones allows us to freely capture video and diverse hardware sensors’ data (e.g., accel erometer, gyroscope). While recording such data is relatively simple, it is often challenging to share and restream this data to other people and applications. Such capability is very valuable for a range of applications such as a context-aware prototyping/developing platform, an integrated data recording and analysis tool, and a sensor-data based video editing system. To enable such complex operations, we propose Senbay, a platform for instant sensing, integrating, and restreaming multiple-sensor data streams. The platform embeds collected sensor data into a video frame using an animated two-dimensional barcode via real-time video processing. The video-embedded sensor data, dubbed Senbay Video, can be easily restreamed to other people and reused by data rich, context-aware applications.
Abstract Major depressive disorder is a complex and common mental health disorder that is heterogeneous and varies between individuals. Predictive measures have previously been used to predict depression in individuals. Given the complexity, heterogeneity of major depressive disorder in individuals, and the scarcity of labelled objective depressive behavioural data, predictive measures have shown limited applicability in detecting the early onset of depression. We present a developed system that collects similar smartphone sensor data like in previous predictive analysis studies. We discuss that anomaly detection and entropy analysis methods are best suited for developing new metrics for the early detection of the onset and progression of major depressive disorder.
Abstract Substantial ongoing research now uses smartphones as a research platform for various studies and interventions. With the aging population becoming a frequent focus of research, an increasing number of studies and projects attempt to develop technological interventions for the elderly population. The extent to which the elderly population (i.e., seniors) adopts and uses smartphones is not clear. Many studies acknowledge that today’s seniors are not particularly keen on using smartphones, but in the near future we can expect this trend to change.
Abstract In Parkinson’s disease (PD), patients’ motor functionalities are measured by various tests. Spiral drawing is one of the proven techniques for assessing the severity of PD motor symptoms. Commonly the test is performed with pen and paper, with the following visual observation by a clinician. This paper describes the implementation of the digitized version of the spiral drawing test for Android devices. Moreover, the application extends the spiral test and utilizes square-shape drawing accordingly. This artifact was tested in a trial with 8 PD patients and 6 age matching controls. The results have shown the observable difference in performance between PD and non-PD users in drawing accuracy and speed.
Abstract We discuss the design and initial evaluation of Gravity of Thought Waterfall, which is designed to be a surprising and creative prototype. Users can control the visual "gravity" effect of the waterfall with an EEG headset. The potentially surprising interactions with the prototype are evaluated through a set of questionnaires and a survey. Results show that the designed prototype is perceived to be surprising and creative; almost all participants were positively surprised when interacting with it.
Abstract As smartphones are increasingly an integral part of daily life, recent literature suggests a deeper relationship between personality traits and smartphone usage. However, this relationship depends on many complex factors such as geographic location, demographics, or cultural influence, just to name a few. These factors provide crucial knowledge for e.g. usage support, recommendations, marketing, general usage improvements. We use six months of application usage data from 739 Android smartphone user together with the IPIP 50-item Big Five personality traits questionnaire. As our main contribution, we show that even category-level aggregated application usage can predict Big Five traits at up to 86%–96% prediction fit in our sample. Our results show the effect of personality traits on application usage (mean error improvement on random guess 17.0%). We also identify which application usage data best describe the Big Five personality traits. Our work enables future personality-driven research, and shows that when studying personality, application categories can provide sufficient predictions in general traits.
Abstract Previous work suggests that Quantified-Self applications can retain long-term usage with motivational methods. These methods often require intermittent attention requests with manual data input. This may cause unnecessary burden to the user, leading to annoyance, frustration and possible application abandonment. We designed a novel method that uses on-screen alert dialogs to transform recurrent smartphone usage sessions into moments of data contributions and evaluate how accurately machine learning can reduce unintended interruptions. We collected sensor data from 48 participants during a 4-week long deployment and analysed how personal device usage can be considered in scheduling data inputs. We show that up to 81.7% of user interactions with the alert dialogs can be accurately predicted using user clusters, and up to 75.5% of unintended interruptions can be prevented and rescheduled. Our approach can be leveraged by applications that require self-reports on a frequent basis and may provide a better longitudinal QS experience.
Abstract We argue that improved data entry can motivate Quantified-Self (QS) users to better engage with QS applications. To improve data entry, we investigate the notion of transforming active smartphone usage into data logging contributions through alert dialogs. We evaluate this assertion in a 4-week long deployment with 48 participants. We collect 17,906 data entries, where 68.3% of the entries are reported using the alert dialogs. We demonstrate that QS applications can benefit from alert dialogs: to increase data precision, frequency, and reduce the probability of forgetfulness in data logging. We investigate the impact of usage session type (e.g., sessions with different goals or durations) and the assigned reminder delay on frequency of data contributions. We conclude with insights gathered from our investigation, and the implications they have on future designs.
Abstract Despite large investments in smartwatch development, the market growth remains smaller than forecasted. The purpose of smartwatch use remains unclear, indicated by the lack of large-scale adoption. Thus, we aim to better understand the early adoption and everyday smartwatch use. We investigate a diverse usage data of smartwatches logged over a period of up to 14 months from 79 individuals between December 2015 and March 2017, one of the largest wearable datasets collected. First, we identify both explorative and accepted behaviours that users exhibit and further investigate how the individual usage traits and features differ between the two categories. Our analysis offers an insightful perspective on how smartwatch use evolves organically. Our results improve our shared understanding of smartwatch use and users adapting their use of smartwatch over time to match the capabilities of the technology by validating numerous findings from previous literature.
Abstract This paper presents our findings on knowledge work environment usage behaviour through a combined automated mobile indoor positioning system and self-reports collected from the environment’s inhabitants. Contemporary work environments are increasingly flexible multi-occupant environments as opposed to cellular offices. Understanding persons’ task-related and situation-related environmental needs is critical to improve the design of future knowledge work environments. This study is conducted in a team office environment prior to and following an intervention in which the office layout was re-organized. The combined methodological approach described in this paper provides a new tool for architecture researchers aiming to understand the use of workspaces. Importantly, combining self-reports with context-aware location data collection provides researchers an efficient in situ tool to access participants experiences and decision-making process in choosing their workstation or workspace.
Abstract Background: Depression is a prevalent mental health challenge. Current depression assessment methods using self-reported and clinician-administered questionnaires have limitations. Instrumenting smartphones to passively and continuously collect moment-by-moment data sets to quantify human behaviors has the potential to augment current depression assessment methods for early diagnosis, scalable, and longitudinal monitoring of depression. Objective: The objective of this study was to investigate the feasibility of predicting depression with human behaviors quantified from smartphone data sets, and to identify behaviors that can influence depression. Methods: Smartphone data sets and self-reported 8-item Patient Health Questionnaire (PHQ-8) depression assessments were collected from 629 participants in an exploratory longitudinal study over an average of 22.1 days (SD 17.90; range 8–86). We quantified 22 regularity, entropy, and SD behavioral markers from the smartphone data. We explored the relationship between the behavioral features and depression using correlation and bivariate linear mixed models (LMMs). We leveraged 5 supervised machine learning (ML) algorithms with hyperparameter optimization, nested cross-validation, and imbalanced data handling to predict depression. Finally, with the permutation importance method, we identified influential behavioral markers in predicting depression. Results: Of the 629 participants from at least 56 countries, 69 (10.97%) were females, 546 (86.8%) were males, and 14 (2.2%) were nonbinary. Participants‘ age distribution is as follows: 73/629 (11.6%) were aged between 18 and 24, 204/629 (32.4%) were aged between 25 and 34, 156/629 (24.8%) were aged between 35 and 44, 166/629 (26.4%) were aged between 45 and 64, and 30/629 (4.8%) were aged 65 years and over. Of the 1374 PHQ-8 assessments, 1143 (83.19%) responses were nondepressed scores (PHQ-8 score <10), while 231 (16.81%) were depressed scores (PHQ-8 score ≥10), as identified based on PHQ-8 cut-off. A significant positive Pearson correlation was found between screen status–normalized entropy and depression (r=0.14, P<.001). LMM demonstrates an intraclass correlation of 0.7584 and a significant positive association between screen status–normalized entropy and depression (β=.48, P=.03). The best ML algorithms achieved the following metrics: precision, 85.55%–92.51%; recall, 92.19%–95.56%; F1, 88.73%–94.00%; area under the curve receiver operating characteristic, 94.69%–99.06%; Cohen κ, 86.61%-92.90%; and accuracy, 96.44%–98.14%. Including age group and gender as predictors improved the ML performances. Screen and internet connectivity features were the most influential in predicting depression. Conclusions: Our findings demonstrate that behavioral markers indicative of depression can be unobtrusively identified from smartphone sensors’ data. Traditional assessment of depression can be augmented with behavioral markers from smartphones for depression diagnosis and monitoring.
Abstract Mobile crowd sensing (MCS) is a relatively new paradigm for collecting real-time and location-dependent urban sensing data. Given its applications, it is crucial to optimize the MCS process with the objective of maximizing the sensing quality and minimizing the sensing cost. While earlier studies mainly tackle this issue by designing different combinatorial optimization algorithms, there is a new trend to further optimize MCS by integrating learning techniques to extract knowledge, such as participants’ behavioral patterns or sensing data correlation. In this paper, we perform an extensive literature review of learning-assisted optimization approaches in MCS. Specifically, from the perspective of the participant and the task, we organize the existing work into a conceptual framework, present different learning and optimization methods, and describe their evaluation. Furthermore, we discuss how different techniques can be combined to form a complete solution. In the end, we point out existing limitations, which can inform and guide future research directions.
Abstract The Experience Sampling Method is widely used to collect human labelled data in the wild. Using this methodology, study participants repeatedly answer a set of questions, constructing a rich overview of the studied phenomena. One of the methodological decisions faced by researchers is deciding on the question scheduling. The literature defines three distinct schedule types: randomised, interval-based, or event-based (in our case, smartphone unlock). However, little evidence exists regarding the side-effects of these schedules on response rate and recall accuracy, and how they may bias study findings. We evaluate the effect of these three contingency configurations in a 3-week within-subjects study (N=20). Participants answered various objective questions regarding their phone usage, while we simultaneously establish a ground-truth through smartphone instrumentation. We find that scheduling questions on phone unlock yields a higher response rate and accuracy. Our study provides empirical evidence for the effects of notification scheduling on participant responses, and informs researchers who conduct experience sampling studies on smartphones.
Abstract Mobile devices (smartphones, smartwatches, etc.) allow us to reach people anywhere, anytime. Collectively, these devices form a ubiquitous computer that offers valuable insights on the user. In addition to the benefits for researchers and developers, explored in previous UbiMI workshops, devices can also help individuals understand their own health, activities, and behaviour. The Ubiquitous Mobile Instrumentation (UbiMI) workshop focuses on using mobile devices as instruments to collect sensing data, to understand human-behaviour and routines, and to gather users’ context using sensor instrumentation.
Abstract While the population is aging the role of information and communication technology (ICT) has grown in elderly care. This development has brought versatile ICT-related supportive systems to professionals and laymen working with aging people. The current study analyzed how professionals in elderly care perceived their workflow challenges before new ICT is developed and implemented to support their work. The results of this study are set to inform the design of a novel ICT system for a sheltered care home.
Abstract We are increasingly in situations of divided attention, subject to interruptions, and having to deal with an abundance of information. Our cognitive load changes in these situations of divided attention, task interruption or multitasking; this is particularly true for older adults. To help mediate our finite attention resources in performing cognitive tasks, we have to be able to measure the real-time changes in the cognitive load of individuals. This paper investigates how to assess real-time cognitive load based on psycho-physiological measurements. We use two different cognitive tasks that test perceptual speed and visio-spatial cognitive processing capabilities, and build accurate models that differentiate an individual’s cognitive load (low and high) for both young and older adults. Our models perform well in assessing load every second with two different time windows: 10 seconds and 60 seconds, although less accurately for older participants. Our results show that it is possible to build a realtime assessment method for cognitive load. Based on these results, we discuss how to integrate such models into deployable systems that mediate attention effectively.
Abstract Researchers who analyse smartphone usage logs often make the assumption that users who lock and unlock their phone for brief periods of time (e.g., less than a minute) are continuing the same "session" of interaction. However, this assumption is not empirically validated, and in fact different studies apply different arbitrary thresholds in their analysis. To validate this assumption, we conducted a field study where we collected user-labelled activity data through ESM and sensor logging. Our results indicate that for the majority of instances where users return to their smartphone, i.e., unlock their device, they in fact begin a new session as opposed to continuing a previous one. Our findings suggest that the commonly used approach of ignoring brief standby periods is not reliable, but optimisation is possible. We therefore propose various metrics related to usage sessions and evaluate various machine learning approaches to classify gaps in usage.
Abstract Public controversies around the unethical use of personal data are increasing, spotlighting data ethics as an increasingly important field of study. MyData is a related emerging vision that emphasizes individuals’ control of their personal data. In this paper, we investigate people’s perceptions of various data management scenarios by measuring the perceived ethicality and level of felt concern concerning the scenarios. We deployed a set of 96 unique scenarios to an online crowdsourcing platform for assessment and invited a representative sample of the participants to a second-stage questionnaire about the MyData vision and its potential in the field of healthcare. Our results provide a timely investigation into how topical data-related practices affect the perceived ethicality and the felt concern. The questionnaire analysis reveals great potential in the MyData vision. Through the combined quantitative and qualitative results, we contribute to the field of data ethics.
Abstract We seek to quantify smartwatch use, and establish differences and similarities to smartphone use. Our analysis considers use traces from 307 users that include over 2.8 million notifications and 800,000 screen usage events, and we compare our findings to previous work that quantifies smartphone use. The results show that smartwatches are used more briefly and more frequently throughout the day, with half the sessions lasting less than 5 seconds. Interaction with notifications is similar across both types of devices, both in terms of response times and preferred application types. We also analyse the differences between our smartwatch dataset and a dataset aggregated from four previously conducted smartphone studies. The similarities and differences between smartwatch and smartphone use suggest effect on usage that go beyond differences in form factor.
Abstract A new generation of wearable devices now enable end-users to keep track of their sleep patterns. This paper reports on a longitudinal study of 82 participants who used a state-of-the-art sleep tracking ring for an average of 65 days. We conducted interviews and questionnaires to understand changes to their lifestyle, their perceptions of the tracked information and sleep, and the overall experience of using an unobtrusive sleep tracking device. Our results indicate that such a device is suitable for long-term sleep tracking and helpful in identifying detrimental lifestyle elements that hinder sleep quality. However, tracking one’s sleep can also introduce stress or physical discomfort, potentially leading to adverse outcomes. We discuss these findings in light of related work and highlight the near-term research directions that the rapid commoditisation of sleep tracking technology enables.
Abstract Mobile self-reports are a popular technique to collect participant labelled data in the wild. While literature has focused on increasing participant compliance to self-report questionnaires, relatively little work has assessed response accuracy. In this paper, we investigate how participant context can affect response accuracy and help identify strategies to improve the accuracy of mobile self-report data. In a 3-week study we collect over 2,500 questionnaires containing both verifiable and non-verifiable questions. We find that response accuracy is higher for questionnaires that arrive when the phone is not in ongoing or very recent use. Furthermore, our results show that long completion times are an indicator of a lower accuracy. Using contextual mechanisms readily available on smartphones, we are able to explain up to 13% of the variance in participant accuracy. We offer actionable recommendations to assist researchers in their future deployments of mobile self-report studies.
Abstract Mobile Crowd Sensing (MCS) is a method for collecting multiple sensor data from distributed mobile devices for understanding social and behavioral phenomena. The method requires collecting the sensor data 24/7, ideally inconspicuously to minimize bias. Although several MCS tools for collecting the sensor data from an off-the-shelf smartphone are proposed and evaluated under controlled conditions as a benchmark, the performance in a practical sensing study condition is scarce, especially on iOS. In this paper, we assess the data collection quality of AWARE iOS, installed on off-the-shelf iOS smartphones with 9 participants for a week. Our analysis shows that more than 97% of sensor data, provided by hardware sensors (i.e., accelerometer, location, and pedometer sensor), is successfully collected in real-world conditions, unless a user explicitly quits our data collection application.
Abstract Mobile crowdsourcing applications often run in dynamic environments. Due to limited time and budget, developers of mobile crowdsourcing applications usually cannot completely test their prototypes in real world situations. We describe a data integration technique for developers to validate their design in prototype testing. Our approach constructs the intended context by combining real-time, historical and simulated data. With correct context-aware design, mobile crowdsourcing applications presenting crowdsourcing questions in relevant context to users are likely to obtain high response quality.
Abstract Although mobile context instrumentation frameworks have simplified the development of mobile context-aware applications, it remains challenging to test such applications. In this paper, we present TestAWARE that enables developers to systematically test context-aware applications in laboratory settings. To achieve this, TestAWARE is able to download, replay and emulate contextual data on either physical devices or emulators. To support both white -box and black-box testing, TestAWARE has been implemented as a novel structure with a mobile client and code library. In blackbox testing scenarios, developers can manage data replay through the mobile client, without writing testing scripts or modifying the source code of the targeted application. In white-box testing scenarios, developers can manage data replay and test functional/non-functional properties of the targeted application by writing testing scripts using the code library. We evaluated TestAWARE by quantifying its maximal data replay speed, and by conducting a user study with 13 developers. We show that TestAWARE can overcome data synchronisation challenges, and found that PC-based emulators can replay data significantly faster than physical smartphones and tablets. The user study highlights the usefulness of TestAWARE in the systematic testing of mobile context-aware applications in laboratory settings.
Abstract Background: Physical and psychological symptoms are common during chemotherapy in cancer patients, and real-time monitoring of these symptoms can improve patient outcomes. Sensors embedded in mobile phones and wearable activity trackers could be potentially useful in monitoring symptoms passively, with minimal patient burden. Objective: The aim of this study was to explore whether passively sensed mobile phone and Fitbit data could be used to estimate daily symptom burden during chemotherapy. Methods: A total of 14 patients undergoing chemotherapy for gastrointestinal cancer participated in the 4-week study. Participants carried an Android phone and wore a Fitbit device for the duration of the study and also completed daily severity ratings of 12 common symptoms. Symptom severity ratings were summed to create a total symptom burden score for each day, and ratings were centered on individual patient means and categorized into low, average, and high symptom burden days. Day-level features were extracted from raw mobile phone sensor and Fitbit data and included features reflecting mobility and activity, sleep, phone usage (eg, duration of interaction with phone and apps), and communication (eg, number of incoming and outgoing calls and messages). We used a rotation random forests classifier with cross-validation and resampling with replacement to evaluate population and individual model performance and correlation-based feature subset selection to select nonredundant features with the best predictive ability. Results: Across 295 days of data with both symptom and sensor data, a number of mobile phone and Fitbit features were correlated with patient-reported symptom burden scores. We achieved an accuracy of 88.1% for our population model. The subset of features with the best accuracy included sedentary behavior as the most frequent activity, fewer minutes in light physical activity, less variable and average acceleration of the phone, and longer screen-on time and interactions with apps on the phone. Mobile phone features had better predictive ability than Fitbit features. Accuracy of individual models ranged from 78.1% to 100% (mean 88.4%), and subsets of relevant features varied across participants. Conclusions: Passive sensor data, including mobile phone accelerometer and usage and Fitbit-assessed activity and sleep, were related to daily symptom burden during chemotherapy. These findings highlight opportunities for long-term monitoring of cancer patients during chemotherapy with minimal patient burden as well as real-time adaptive interventions aimed at early management of worsening or severe symptoms.
Abstract Alcohol use in young adults is common, with high rates of morbidity and mortality largely due to periodic, heavy drinking episodes (HDEs). Behavioral interventions delivered through electronic communication modalities (e.g., text messaging) can reduce the frequency of HDEs in young adults, but effects are small. One way to amplify these effects is to deliver support materials proximal to drinking occasions, but this requires knowledge of when they will occur. Mobile phones have built-in sensors that can potentially be useful in monitoring behavioral patterns associated with the initiation of drinking occasions. The objective of our work is to explore the detection of daily-life behavioral markers using mobile phone sensors and their utility in identifying drinking occasions. We utilized data from 30 young adults aged 21–28 with past hazardous drinking and collected mobile phone sensor data and daily Experience Sampling Method (ESM) of drinking for 28 consecutive days. We built a machine learning-based model that is 96.6% accurate at identifying non-drinking, drinking and heavy drinking episodes. We highlight the most important features for detecting drinking episodes and identify the amount of historical data needed for accurate detection. Our results suggest that mobile phone sensors can be used for automated, continuous monitoring of at-risk populations to detect drinking episodes and support the delivery of timely interventions.
Abstract The latest smartphones have advanced sensors that allow us to recognize human and environmental contexts. They operate primarily on Android and iOS, and can be used as sensing platforms for research in various fields owing to their ubiquity in society. Mobile sensing frameworks help to manage these sensors easily. However, Android and iOS are constructed following different policies, requiring developers and researchers to consider framework differences during research planning, application development, and data collection phases to ensure sustainable data collection. In particular, iOS imposes strict regulations on background data collection and application distribution. In this study, we design, implement, and evaluate a mobile sensing framework for iOS, namely AWARE-iOS, which is an iOS version of the AWARE Framework. Our performance evaluations and case studies measured over a duration of 288 h on four types of devices, show the risks of continuous data collection in the background and explore optimal practical sensor settings for improved data collection. Based on these results, we develop guidelines for sustainable data collection on iOS.
Abstract Parkinson’s disease (PD) is the second most common neurodegenerative disorder, impacting an estimated seven to ten million people worldwide. Measuring the symptoms and progress of the disease, and medication effectiveness is currently performed using subjective measures and visual estimation. We developed and evaluated a mobile application, STOP for tracking hand’s motor symptoms, and a medication journal for recording medication intake. We followed 13 PD patients from two countries for a 1-month long real-world deployment. We found that PD patients are willing to use digital tools, such as STOP, to track their medication intake and symptoms, and are also willing to share such data with their caregivers and medical personnel to improve their own care.
Abstract Depression is a prevalent mental disorder. Current clinical and self-reported assessment methods of depression are laborious and incur recall bias. Their sporadic nature often misses severity fluctuations. Previous research highlights the potential of in-situ quantification of human behaviour using mobile sensors to augment traditional methods of depression management. In this paper, we study whether self-reported mood scores and passive smartphone and wearable sensor data could be used to classify people as depressed or non-depressed. In a longitudinal study, our participants provided daily mood (valence and arousal) scores and collected data using their smartphones and Oura Rings. We computed daily aggregations of mood, sleep, physical activity, phone usage, and GPS mobility from raw data to study the differences between the depressed and non-depressed groups and created population-level Machine Learning classification models of depression. We found statistically significant differences in GPS mobility, phone usage, sleep, physical activity and mood between depressed and non-depressed groups. An XGBoost model with daily aggregations of mood and sensor data as predictors classified participants with an accuracy of 81.43% and an Area Under the Curve of 82.31%. A Support Vector Machine using only sensor-based predictors had an accuracy of 77.06% and an Area Under the Curve of 74.25%. Our results suggest that digital biomarkers are promising in differentiating people with and without depression symptoms. This study contributes to the body of evidence supporting the role of unobtrusive mobile sensor data in understanding depression and its potential to augment depression diagnosis and monitoring.
Abstract Recognizing human emotions and responding appropriately has the potential to radically change the way we interact with technology. However, to train machines to sensibly detect and recognize human emotions, we need valid emotion ground truths. A fundamental challenge here is the momentary emotion elicitation and capture (MEEC) from individuals continuously and in real-time, without adversely affecting user experience. In this first edition of the one-day CHI 2020 workshop, we will (a) explore and define novel elicitation tasks (b) survey sensing and annotation techniques (c) create a taxonomy of when and where to apply an elicitation method.
Abstract We present CARE, a context-aware tool for nurses in nursing homes. The system utilises a sensors infrastructure to quantify the behaviour and wellbeing (e.g., activity, mood, social and nurse interactions) of elderly residents. The sensor data is offloaded, processed and analysed in the cloud, to generate daily and long-term summaries of residents’ health. These insights are then presented to nurses via an Android tablet application. We aim to create a tool that can assist nurses and increase their awareness to residents’ needs. We deployed CARE in a local nursing home for two months and evaluated the system through a post-hoc exploratory analysis and interviews with the nurses. The results indicate that CARE can reveal essential insights on the wellbeing of elderly residents and improve the care service. In the discussion, we reflect on our understanding and potential impact of future integrated technology in elderly care environments.
Abstract Parkinson’s disease (PD) is the second most common neurodegenerative disorder, impacting an estimated seven to ten million people worldwide. It is commonly accepted that improving medication adherence alleviates symptoms and maintains motor capabilities. Not following the medication regimen (e.g., skipping or over-medicating) may worsen side-effects, which mislead clinicians and patients. We developed and evaluated a mobile application, STOP, for screening the PD symptoms and medication intake. It contains a game for tracking the PD symptoms, and a medication journal for recording medical intake and adherence. We conducted a 1-month long real-world deployment with 13 PD patients from two countries. We found that the application medication adherence tracking provides non-bias information, and users are receptive to share such data with their care and medical personnel.
Abstract Elderly care is a pressing societal challenge: government’s financial burden is expected to exponentially increase in the next 20 years as the population is aging rapidly. Solutions to mitigate this challenge include the use of IoT and software solutions to minimise the effort of elderly care, in care centres and at home. To accomplish this, we set to quantify what are the most important elderly care metrics (i.e., what is important to support caregivers’ work) through field observations and interviews at a local care centre housing 14 old adults. We designed iteratively and evaluated the usefulness of a mobile application with 8 caregivers, to summarise and communicate the care metrics, juxtaposed with wellbeing data (e.g., social interaction, mobility and others), part of a larger elderly care support platform, CARE. The goal of the mobile application is to enable a better care service by raising awareness to daily needs and routines of the elderly and to provide quick access to their wellbeing information. Our findings advocate that our design could positively benefit the care personnel and assist them carrying out the daily duties at the care centre.
Abstract We present an assistive healthcare platform, CARE, which aims to provide daily support for elderly caregivers with context-aware, unobtrusive, and actionable information. This information is collected through a plethora of IoT sensors installed strategically at an elderly care centre and is accessed through an Android tablet application. The application’s goal is to empower nurses with a better understanding of elderly needs and ultimately, improve the care service. We investigate how IoT devices and sensors can enable a pervasive healthcare system, and discuss a wide-range of important parameters for integration of elderly care practices.
Abstract Background: Depression and anxiety are leading causes of disability worldwide but often remain undetected and untreated. Smartphone and wearable devices may offer a unique source of data to detect moment by moment changes in risk factors associated with mental disorders that overcome many of the limitations of traditional screening methods. Objective: The current study aimed to explore the extent to which data from smartphone and wearable devices could predict symptoms of depression and anxiety. Methods: A total of N = 60 adults (ages 24–68) who owned an Apple iPhone and Oura Ring were recruited online over a 2-week period. At the beginning of the study, participants installed the Delphi data acquisition app on their smartphone. The app continuously monitored participants’ location (using GPS) and smartphone usage behavior (total usage time and frequency of use). The Oura Ring provided measures related to activity (step count and metabolic equivalent for task), sleep (total sleep time, sleep onset latency, wake after sleep onset and time in bed) and heart rate variability (HRV). In addition, participants were prompted to report their daily mood (valence and arousal). Participants completed self-reported assessments of depression, anxiety and stress (DASS-21) at baseline, midpoint and the end of the study. Results: Multilevel models demonstrated a significant negative association between the variability of locations visited and symptoms of depression (beta = −0.21, p = 0.037) and significant positive associations between total sleep time and depression (beta = 0.24, p = 0.023), time in bed and depression (beta = 0.26, p = 0.020), wake after sleep onset and anxiety (beta = 0.23, p = 0.035) and HRV and anxiety (beta = 0.26, p = 0.035). A combined model of smartphone and wearable features and self-reported mood provided the strongest prediction of depression. Conclusion: The current findings demonstrate that wearable devices may provide valuable sources of data in predicting symptoms of depression and anxiety, most notably data related to common measures of sleep.