Leveraging smartphone potential for clinical trials: accessing inbuilt sensors and components to measure health outcomes

A smartphone is “a mobile phone that performs many of the functions of a computer, typically having a touchscreen interface, internet access, and an operating system capable of running downloaded apps” [1].  Global smartphone penetration continues to rise rapidly due to their increasing value and utility, their ease of use and their increasing affordability.  Due to the increasing miniaturisation of sensors and circuitry, smartphones have been able to leverage multiple inbuilt sensors to enhance user experience and provide other features.  For example, tri-axial accelerometers are useful in determining the spatial positioning of the smartphone and thus enabling the screen to re-orientate when the handset is rotated, and proximity sensors are used to switch off screen controls when the handset is held to the face during a phone call.

More recently, novel application of these inbuilt sensors has enabled new and inventive uses for the smartphone in the area of health and wellness.  In more recent operating system releases, AppleTM Inc., for example, have provided a health app with the iPhoneTM which, amongst other things, collects activity data from the smartphone accelerometer – measuring elements such as the number of steps, walking distance and flights climbed based on the tri-axial accelerometer measurements recorded whilst the phone is being carried in a pocket, bag or in the hand of the owner [2].  Because of the proliferation of sensors and components contained in modern smartphones, and the ability to develop apps that can access sensor readings by leveraging the open development capabilities of the main platforms, we have arrived at an exciting inflexion point in our ability to use the increasingly ubiquitous smartphone as a platform to measure a variety of novel health-related endpoints that may have value in clinical trials.

As with any way of capturing outcomes data in clinical trials, there needs to be evidence that the approach is providing valid and reliable data.  We reviewed published examples of concurrent validity studies comparing the results from smartphone sensor applications to other well-known or gold standard approaches with an aim to understanding the potential of developing solutions that leverage smartphone sensors to collect novel health outcomes.  We describe some of our observations below and in a poster presented at ISPOR, 2016

The Validity of Smartphone Sensors and Components to Measure Clinical Outcomes

ICON reviewed published validation studies performed that compare clinical outcomes assessments derived from smartphone sensor data to gold standard approaches. 

Figure 1.  Common sensors and components of modern smartphone

There are a growing number of scientific studies exploring the use of smartphone sensors to provide outcomes measures traditionally recorded by research grade devices, clinician subjective ratings or by other methodologies.

Milani et al. [4], for example, reported a literature review of smartphone apps that evaluate the range of motion and/or the angular measurement relating to various joint movements in patients with a range of conditions.  Examples reported included the use of the smartphone accelerometers and gyroscopes to measure knee range of motion [5]; cervical spine mobility [6], and the angle of trunk rotation during a forward bend test [7,8].  The extent of shoulder rotation using a smartphone gyroscope has been reported by El-Zayat et al. [9]; and researchers from the Republic of Korea similarly used a similar approach to measure the range of motion of the shoulder in subjects suffering from unilateral symptomatic shoulder.  They found that the smartphone gyroscope sensor results showed acceptable reliability and fairly high correlation with manual goniometer readings [10].

The iPhone magnometer has also been used to measure the cervical range of motion on the horizontal plane using an iPhone placed on the head of the subject during a clinical assessment by an investigator [6].  Another investigator app, Dr Goniometer, measures flexion of the elbow and knee using a virtual goniometer that is visible on the smartphone screen during camera use [11, 12].

In addition to range of motion, accelerometers are valuable in measuring gait and balance.  Kosse et al. [13] performed a validation study in which they compared the gait and posture measurement capabilities of the iPod TouchTM with that obtained simultaneously using a research grade accelerometer, the DynaPortTM Hybrid (McRoberts BV, The Hague, The Netherlands).  The study, conducted in 60 healthy volunteers aged 18 – 75 years who performed walking and stationary balance tests, concluded that the iPod Touch was able to produce valid and reliable measures of gait and postural control.  In a second study, Mellone et al. reported that the accelerometer in an Android smartphone (the HTC DesireTM) provided consistent results compared to the DynaPort Hybrid accelerometer when evaluating the Timed Up and Go (TUG) test [14].

Other researchers have used the accelerometer and gyroscope signals from an iPhone mounted on a specially designed glove enabling it to rest on the back of the hand to measure tremor in Parkinson’s Disease (PD) patients during a seated test in which patients kept their hands in their lap [15, 16].  This approach has also been applied by Roche Pharmaceuticals in their development of an Android app to assess tremor associated with PD using two tremor tests – a resting test where the smartphone is held with the hand resting in the lap while seated, and a postural tremor test where tremor is measured while the smartphone is held in the hand with the arm outstretched [17].  This app is currently being used in a phase 1 trial and also includes other short active tests including a phonation test, a balance test, a 20m walk test measuring aspects of gait, and a dexterity test where the user is instructed to tap two buttons alternately on the touchscreen.

In addition to dexterity tests, as described above, touchscreens also provide the ability to measure aspects of cognitive function.  Numerous examples of brain training apps exist in the health and wellness industry, and in clinical research a number of standard tests have been adapted for use on smartphones, where the touchscreen can be used to measure aspects of both the speed and the accuracy of task completion.  Apple Research Kit, for example, contains a number of examples of standard cognitive tests used in clinical research that have been adapted for use on the iPhone, including the spatial memory test, the paced auditory/visual serial addition test (PVSAT), and the simple reaction time test [18].  ICON has experience of using the PVSAT through Apple Research Kit in a study of 155 patients suffering form conditions causing chronic pain [19].  Within an interactive game environment, the Project:EVO app is used to measure interference processing, a key component of executive function.  This smartphone game is currently being tested in a variety of clinical studies and patient populations including ADHD, autism, depression, and traumatic brain injury.  Pfizer, for example, has recently funded a study to determine if the game can be used as a biomarker to enable the selection and longitudinal assessment of Alzheimer’s patients in clinical trials [20].

Shire is also funding investigations on the use of the game in ADHD clinical trials.
Voice acoustic measures have been shown to be sensitive indicators of disease severity and therapeutic response in many CNS disorders, including Parkinson's disease, depression, and schizophrenia [21]. Analysis of uninterrupted speech samples in depressed patients have shown good correlation with clinician ratings [22].  Aspects such as speaking rate and pitch variation have been shown to be well correlated with the total Hamilton Depression Rating Scale score, where it is observed that speech slows and becomes more monotonic with increasing severity of depression [23].  Similarly, voice acoustics endpoints have been shown to have value in assessment of voice and speech disorders in PD [24], which often present as an early indicator of the disease.  The availability of the smartphone microphone to record sound files facilitates easy collection of data for voice analysis from patients in the home or clinic settings.  For example, a phonation test has been used in the mPower app developed by Sage Bionetworks and the University of Rochester [25], in which users were instructed maintain sustained phonation at a comfortable pitch and loudness, and to keep this note constant for as long as possible by, for example, holding the sound “ahh”.  Analysis of this sound file in PD subjects might include, for example, calculation of summary measures such as the fundamental frequency (pitch), jitter (the extent of variation of the voice range) and shimmer (the extent of variation of expiratory flow).  Smartphone microphones have also been used to develop stethoscope apps [26].

The smartphone camera has the potential to be a valuable component for use in clinical trials.  The potential of taking and recording quality photographs pertinent to a medical condition means that participants in clinical trials may be able to provide much richer data about their conditions than previously possible form the home setting.  Examples might include psoriasis and other dermatological conditions, wound healing, injection site reactions and melanoma.  A recent addition to the Apple store, and built using Apple Research Kit, the Mole Mapper app developed by Oregon Health & Science University and Sage Bionetworks, uses photography to track changes in moles over time, providing a resource that can be shared with physicians [27].  Accessories that can be attached to the smartphone camera provide additional possibilities.  Cellscope Inc. (San Francisco, California) have developed an iPhone attachment that fits over the iPhone rear camera to create an otoscope with the added advantage that images and videos of the ear examination can be recorded.  The smartphone camera and light-emitting diode light has successfully shown to accurately measure heart rate when the index finger is pressed against the rear camera when compared to the Nonin 9560BT pulse oximeter (Nonin Medical Inc., Plymouth, Minnesota) [28].

Artificial intelligence approaches take the use of the smartphone camera to another level.  A new app built using Apple Research Kit uses the FaceTime HD camera in the iPhone along with innovative facial recognition algorithms to analyse a child’s emotional reaction whilst watching a short series of videos [29].  Developed by Duke University, Peking University and the University of Cape Town, this approach is being evaluated to determine whether video technology can detect emotional and developmental problems in children as young as 18 months, with an aim to providing a means to screen children in their homes for autism and other developmental challenges.

The potential of leveraging smartphone sensors to collect novel health outcomes measures and derive new study endpoints is huge.  New development s on the main platforms have facilitated this, such as Apple ResearchKit [30] and Android ResearchStack [31].  More work will need to be done to provide the evidence required to support the use of these health outcomes and endpoints collected in this way, in particular understanding the reliability and validity of data collected in this way.

[1] Oxford English Dictionary (2016).  http://www.oxforddictionaries.com/
[2] Apple Inc. (2015).  Health: An innovative new way to use your health and fitness information.  http://www.apple.com/uk/ios/health/
[3] Byrom B. et al. (2016).  A Review Evaluating the Validity of Smartphone Sensors and Components to Measure Clinical Outcomes in Clinical Research.  ISPOR 21st Annual International Meeting, May 2016.
[4] Milani, P., Coccetta, C.A., Rabini, A. et al. (2014). Mobile Smartphone Applications for Body Position Measurement in Rehabilitation: A Review of Goniometric Tools.  Physical Medicine and Rehabilitation; 6: 1038-1043.
[5] Jenny, J.Y. (2013). Measurement of the knee flexion angle with a smartphone application is precise and accurate. J Arthroplasty;28: 784-787.
[6] Tousignant-Laflamme, Y., Boutin, N., Dion, A.M., et al. (2013).  Reliability and criterion validity of two applications of the iPhone to measure cervical range of motion in healthy participants. J Neuroeng Rehabil.; 10:69-77.
[7] Izatt, M.T., Bateman, G.R., Adam, C.J. (2012).  Evaluation of the iPhone with an acrylic sleeve versus the Scoliometer for rib hump measurement in scoliosis. Scoliosis; 7:14-20.
[8] Franko, O.I., Bray, C., Newton, P.O. (2012).  Validation of a scoliometer smartphone app to assess scoliosis. J Pediatr Orthop; 32: e72-e75.
[9] El-Zayat, B.F., Efe, T., Heidrich, T., et al. (2011). Objective assessment of shoulder mobility with a new 3D gyroscope – a validation study.  BMC Musculoskelet Disord; 12: 168-176.
[10] Shin, S.H., Ro, D.H., Lee, O-S. et al. (2012). Within-day reliability of shoulder range of motion measurement with a smartphone. Manual Therapy; 17: 298–304.
[11] Ferriero, G., Sartorio, F., Foti, C., et al. (2011). Reliability of a new application for smartphones (DrGoniometer) for elbow angle measurement. Physical Medicine and Rehabilitation; 3: 1153-1154.
[12] Ferriero, G., Vercelli, S., Sartorio, F., et al. (2013).  Reliability of a smartphone-based goniometer for knee joint goniometry. Int J Rehabil Res; 36: 146-151.
[13] Kosse, N.M., Caljouw, S., Vervoort, D. et al. (2015).  Validity and Reliability of Gait and Postural Control Analysis Using the Tri-axial Accelerometer of the iPod Touch.  Annals of Biomedical Engineering; 43(8): 1935–1946.
[14] Mellone, S., Tacconi, C., Chiari, L. (2012). Validity of a Smartphone-based instrumented Timed Up and Go.  Gait & Posture; 36: 163–165.
[15] Kostikis, N., Hristu-Varsakelis, D., Arnaoutoglou, M., et al. (2011). “Towards remote evaluation of movement disorders via smartphones,” in Proc.  Engineering in Medicine and Biology Society (EMBC), 33rd Annual International Conference of the IEEE, pp. 5240–5243.
[16] Kostikis, N., Hristu-Varsakelis, D., Arnaoutoglou, M., et al. (2014). “Smartphone-based Evaluation of Parkinsonian Hand Tremor: Quantitative Measurements vs Clinical Assessment Scores,” in Proc.  Engineering in Medicine and Biology Society (EMBC), 36th Annual International Conference of the IEEE, pp. 906-909. 
[17] http://www.roche.com/media/store/roche_stories/roche-stories-2015-08-10.htm
[18] Apple Inc. (2016).  Research Kit Active Tasks.  http://researchkit.org/docs/docs/ActiveTasks/ActiveTasks.html
[19] Byrom, B. et al. (2017).  Applicability of Apple Research Kit to Deliver Cognitive Testing in Clinical Trials: Results of a Pilot Study.  ISPOR 22nd Annual International Meeting, May 2017.
[20] Akili Interactive Labs (2014). “Akili Interactive Labs announces partnership with Pfizer to test video game in people at risk of Alzheimer's disease,”  http://www.prnewswire.com/news-releases/akili-interactive-labs-announces-partnership-with-pfizer-to-test-video-game-in-people-at-risk-of-alzheimers-disease-239412291.html.
[21] Byrom, B. (2006).  Innovative ePRO: Tapping into the Potential.  Applied Clinical Trials; June: 64-72.
[22] Mundt, J.C., Snyder, P.J, Cannizzaro, M.S., et al. (2007). Voice acoustic measures of depression severity and treatment response collected via interactive voice response (IVR) technology.  J. Neurolingusitics; 20(1): 50-64.
[23] Cannizzaro, M., Harel, B., Reilly, N., et al. (2004).  Voice Acoustical Measurement of the Severity of Major Depression.  Brain and Cognition; 56: 30-35.
[24] Rusz, J.,  Čmejla, R., Růžičková, H., et al. (2011).  Acoustic Assessment of Voice and Speech Disorders in Parkinson’s Disease Through Quick Vocal Test.  Mov Disord.; 26(10): 1951-1952.
[25] Sage Bionetworks. (2015). Parkinson’s mPower Study App.  https://itunes.apple.com/WebObjects/MZStore.woa/wa/viewSoftware?id=972191200&mt=8&ls=1&v0=www-us-researchkit-itms-mpower
[26] Yoo, J.H. (2013). The meaning of information technology (IT) mobile devices to me, the infectious disease physician. Infect Chemother.; 45(2): 244–251.
[27] Sage Bionetworks. (2015).  Mole Mapper Melanoma Study App.  https://itunes.apple.com/us/app/mole-mapper-melanoma-study/id1048337814?mt=8
[28] Gregowski, M.J., Mueller, M., Vertegel, A. et al. (2012).  Development and validation of a smartphone heart rate acquisition application for health promotion and wellness telehealth applications.  Int J Telemed Appl. 2012:696324.
[29] Duke Medicine. (2015). Autism & Beyond.  https://itunes.apple.com/us/app/autism-beyond/id1025327516?mt=8
[30] Apple Inc. (2016).  https://www.apple.com/uk/researchkit/ 
[31] Android ResearchStack (2017).  http://researchstack.org/