???Evaluation of Machine Learning Algorithms for Classifying Deep Brain Stimulation Respective of ‘On’ and ‘Off’ StatusAbstract— Essential tremor is a prevalent neurodegenerative movement disorder. Deep brain stimulation represents a highly effective means of treatment, especially for scenarios for which medical intervention is no longer feasible. One of the major post-operative challenges is the determination of an optimal set of tuning parameters.  Optimizing the deep brain stimulation parameters can impart a time-intensive task to the clinician. The smartphone in the context of a wearable and wireless inertial sensor system offers the capability to objectively quantify the characteristics of the tremor. Machine learning in conjunction with a wearable and wireless inertial sensor system, such as a smartphone, can distinguish between disparate states, such as deep brain stimulation in ‘On’ and ‘Off’ status. Multiple machine learning classification techniques are available, such as the multilayer perceptron neural network, support vector machine, K-nearest neighbors, logistic regression, J-48 decision tree, and random forest.  The objective of this research is to evaluate these six machine learning classification algorithms for classification of deep brain stimulation regarding ‘On’ and ‘Off’ status for Essential tremor during a reach and grasp task.  The reach and grasp task is quantified through the smartphone as wearable and wireless inertial sensor system mounted to the dorsum of the hand and secured by latex glove.  Multiple feature set scenarios are considered, such as recordings from both the accelerometer and gyroscope, accelerometer, and gyroscope. These scenarios facilitate the determination of the most robust machine learning algorithms. The multilayer perceptron neural network, support vector machine, K-nearest neighbors, and logistic regression achieve the highest classification accuracy for three feature set scenarios: derived by recordings from both accelerometer and gyroscope, accelerometer, and gyroscope.I.     INTRODUCTIONEssential tremor is a predominant neurodegenerative movement disorder that is relatively more prevalent than Parkinson’s disease. Contrasting to Parkinson’s disease, Essential tremor also is represented through both resting tremor and kinetic tremor, which involves tremor during intentional movement [1, 2]. The tremor characteristics of these movement disorders has been successfully quantified through the application of wearable and wireless inertial systems [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]. In particular these wearable and wireless systems have been instrumental for the objective quantification of response to therapeutic intervention through the application of deep brain stimulation, such as ‘On’ and ‘Off’ status, through the distinction of machine learning classification [11, 12, 13].Deep brain stimulation offers a highly efficacious therapy intervention for the amelioration of movement disorder tremor symptoms [14]. The deep brain stimulation system is surgically implanted through stereotactic surgery by a skilled neurosurgeon [14, 15]. Another highly challenging task is the acquisition of the optimal parameter configuration [16, 17].The clinician is tasked with a substantial number of parameter configurations, which consist of a myriad of settings regarding electrode polarity, amplitude, pulse width, and simulation frequency.  Intuitively the convergence to an optimal configuration can present a time intensive and laborious endeavor [15, 16, 17].  Another matter of concern is in light of whether the temporal snapshot of a clinical meeting properly addresses the inherent variations observed during a neurodegenerative movement disorder.Two technologies offer considerable potential for advancing the efficacy of deep brain stimulation tuning optimization: wearable and wireless systems and machine learning. Wearable and wireless systems through their inertial sensor systems can quantify movement disorder tremor, which can be consolidated into a feature set for machine learning classification [11, 12, 13]. However, a challenge with the proper application of machine learning is the determination of the most appropriate machine learning algorithm [18, 19, 20]. There are many available machine learning algorithms for the scope of this research endeavor, and the following are under consideration: multilayer perceptron neural network, support vector machine, K-nearest neighbors, logistic regression, J-48 decision tree, and random forest. The objective of the research endeavor is to evaluate these six machine learning algorithms for the classification of deep brain stimulation using ‘On’ and ‘Off’ status for Essential tremor during a reach and grasp task that is measured by a smartphone as a wearable and wireless inertial sensor platform to establish a quantified feature set.II.   BackgroundA. Fundamental characteristics of Essential tremorEssential tremor and Parkinson’s disease are contrasted as disparate regarding the nature of their respective movement disorder [1, 21, 22].  Parkinson’s disease involves resting tremor, which can be measured with the patient sitting in a chair [8, 13]. However, Essential tremor is comprised of both resting and kinetic tremor, and kinetic tremor is observed while the subject conducts a voluntary movement task [1, 2]. For example, a reach and grasp task of a lightweight object on a table can satisfactorily demonstrate kinetic tremor, which can be recorded by the inertial sensor signal of a wearable and wireless system, such as a smartphone [11, 23].For approximately the last two millennia there have been attempts among the medical community to ameliorate the kinetic tremor morbidities evident for this type of movement disorder.  Further effort has emphasized the evolution of diagnostic acuity for Essential tremor [1, 24].  However, the neuroanatomical origins of Essential tremor have not been conclusively determined from a medical perspective. Contemporary positions attribute the origins of Essential tremor to the central nervous system, such as degeneration about the cerebellar pathways [1, 25].The tremor frequency regarding Essential tremor has been established as inversely proportional relative to age. The symptoms are more prevalently revealed by kinetic tremor compared to resting tremor. Kinetic tremor generally displays a frequency on the order of 4 Hz to 12 Hz [1].B. Conventional therapy for the treatment of Essential tremorMedical therapy through drug intervention is the traditional strategy for mitigating the influence of Essential tremor symptoms. Two widely applied drugs are propranolol and primidone.  However, with the neurological origins of the movement disorder not conclusively determined, the efficacy of a specific medical treatment can display variable degrees of success for a number of patients [1, 26].For some scenarios pharmacotherapy is determined to be intractable for treating Essential tremor, and irreversible neurosurgery becomes an inevitable contingency [1].  The thalamotomy is an irreversible neurosurgical procedure that disrupts pathways associated with the thalamus. The Gamma Knife applies radiosurgery that can achieve this alternate [27, 28].C. Deep brain stimulation for treatment of Essential tremorDeep brain stimulation offers considerable advantage for the treatment of Essential tremor. Intuitively this treatment does not have the permanency of ablative techniques. Deep brain stimulation is effectively a reversible procedure.  If the device is considered not effective for the individualized treatment of Essential tremor, then the deep brain stimulation system can be removed. Furthermore, the deep brain stimulation system is equipped with a considerable array of parameter configurations, which enables highly individualized therapeutic intervention [29].The target selection for the anatomical structure of the deep brain to be treated is dependent to the unique movement disorder scenario that is being treated. Regarding Essential tremor the ventral intermediate nucleus (VIM) is the prevalent target. The VIM receives pathways from the cerebellum and projects to the motor cortex [30, 31].The deep brain stimulation system is an integral device that consists of multiple components and subsystems. The component that interacts directly with the deep brain structure is the electrode lead.  The electrodes are prolonged through connecting wire that merges with the implantable pulse generator.  An internalized battery provides the energy source for the implantable pulse generator. The implantable pulse generator consists of the electronic circuitry that provides the stimulating electric signal, which stimulates the targeted aspect of the deep brain to treat Essential tremor [15]. The skilled implantation of a deep brain stimulation system by an expert neurosurgeon only represents a preliminary aspect of properly treating a person with Essential tremor through this novel form of therapy [17].There are multiple parameters that are adjusted to establish a configuration, and current deep brain stimulation systems applied four parameters:amplitudefrequencypulse widthelectrode polarity [15, 16]Acquiring the optimal tuning configuration, which is imperative for efficacious intervention, presents a time-consuming and challenging task. Given the considerable array of parameter configurations, a systematic approach would be beneficial in light of the limited medical resources available [17]. A quantified rather than subjective form of feedback to ascertain the efficacy of a deep brain stimulation system tuning configuration may considerably advance the optimization process.D. Wearable and wireless systems for quantifying movement disorderThe application of wearable and wireless systems to advance diagnostic acuity by the feedback acquired through the signal of an inertial sensor has been advocated as a means to objectively quantify movement disorder status [3, 4, 5, 6, 7].  During 2010 LeMoyne et al. applied a smartphone as a functional wireless accelerometer platform to quantify Parkinson’s disease tremor, which is another type of movement disorder [8].  One of the immediately discernible benefits for the application of wearable and wireless systems, such as the smartphone, is the observation that experimental and post-processing resources can be remotely situated anywhere in the world [4, 5, 6, 7].The smartphone is equipped with an inertial sensor package that consists of both an accelerometer and gyroscope. A software application enables the smartphone to function as a wireless inertial sensor platform.  The signal data is recorded and attached to an email for wireless transmission using the Internet.  In essence, this email resource is representative of a functional Cloud computing resource [4-7, 32].Subsequent evolutions of the role of wearable and wireless systems, such as the smartphone, pertained to the evaluation of deep brain stimulation efficacy for Essential tremor. The smartphone with its inertial sensor package quantified Essential tremor attributes for a reach and grasp task during the scenarios of deep brain stimulation ‘On’ and ‘Off’ modes. The signal data was consolidated into a feature set for machine learning classification, and considerable classification accuracy was attained differentiating between ‘On’ and ‘Off’ modes [11, 12].  A similar technique was recently applied for Parkinson’s disease hand tremor [13]. However, the appropriateness of a machine learning algorithm for classification is contextually unique to the experimental research conditions [33].Fig. 1. Mounting of the smartphone about the dorsum of the hand through a latex glove for reach and grasp of a lightweight object.Fig. 2. Deep brain stimulation system’s patient programmer.The current research objective is to evaluate the classification accuracy for an assortment of machine learning algorithms to distinguish between deep brain stimulation using ‘On’ and ‘Off’ modes for a subject with Essential tremor. Six machine learning algorithms have been selected for this research endeavor:Multilayer Perceptron Neural NetworkSupport Vector MachineK-Nearest NeighborsLogistic RegressionJ-48 Decision TreeRandom ForestThe feature set for machine learning classification is established by consolidating the inertial signal (accelerometer and gyroscope) obtained through a smartphone functioning as a wearable and wireless inertial sensor platform.III. Materials and MethodsThe application of multiple machine learning algorithms to establish classification accuracy for distinguishing between deep brain stimulation ‘On’ and ‘Off’ status was conducted from the perspective of engineering proof of concept. One subject with Essential tremor being treated with deep brain stimulation was selected. Informed consent was established through the Institutional Review Board provided by Allegheny General Hospital of Pittsburgh, Pennsylvania.  Tremor characteristics were acquired through the use of a smartphone.The smartphone utilized a software application that enabled the simultaneous recording of both the accelerometer signal and gyroscope signal. The inertial sensors for the smartphone recorded signal data at a rate of 100Hz for approximately 10 seconds. The data was attached to an email for wireless transmission using the Internet for remote post-processing. The smartphone was mounted to the dorsum of the hand and secured by a latex glove. Figure 1 shows a representative picture of the strategy for mounting the smartphone. Figure 2 displays the deep brain stimulation system’s patient programmer.The six machine learning algorithms were provided by Waikato Environment for Knowledge Analysis (WEKA) using ten-fold cross validation. Software automation through Matlab consolidated the inertial signal data acquired by the smartphone into a feature set using an Attribute-Relation File Format (ARFF). The feature set was composed of five attributes maximum, minimum, mean, standard deviation, and coefficient of variation of the roll, pitch, and yaw components of the gyroscope signal and the acceleration magnitude. These feature set attributes have been successfully applied for multiple machine learning classification scenarios pertaining to wearable and wireless inertial sensors [12, 13, 34, 35, 36, 37].The following experimental protocol was applied to acquire 10 trials while the deep brain stimulation system was set to ‘On’ mode and 10 trials in ‘Off’ mode for a subject with Essential tremor.Experimental Protocol:1. Mount the smartphone to the dorsum of the hand using a latex glove.2. Initiate the smartphone for the accelerometer and gyroscope recording countdown.3. Instruct the subject to reach and grasp a lightweight object (rolled flexible bandage) on a table after the audio tone indicates the signal recording has commenced.4. Transmit the recorded inertial signal data for post-processing using wireless Internet connectivity.IV.     Results and DiscussionThe deep brain stimulation system notably mitigated tremor for the Essential tremor subject using the ‘On’ mode by contrast to the ‘Off’ mode. The suppression of tremor is discernible with the deep brain stimulation system set to ‘On’ compared to ‘Off’. The inertial sensors of the smartphone provide objectively quantified definition of the nature of the tremor. Figure 3 demonstrates the acceleration magnitude for the subject while conducting the experiment using ‘On’ mode.  Tremor is notably amplified for figure 4, which represents the acceleration magnitude for the Essential tremor subject with deep brain stimulation set to ‘Off’ status.Fig. 3. Acceleration magnitude of Essential tremor subject conducting a reach and grasp task with deep brain stimulator set to ‘On’ status.Fig 4. Acceleration magnitude of Essential tremor subject conducting a reach and grasp task with deep brain stimulator set to ‘Off’ status.Four inertial sensor signals were applied to develop the feature set: acceleration magnitude and roll, pitch, and yaw components of the gyroscope signal. Regarding the four inertial sensor signals five attributes were applied to each respective signal:MaximumMinimumMeanStandard deviationCoefficient of variationThe experimental data consisting of 10 trials with the deep brain stimulation system set to ‘On’ and 10 trials with the deep brain stimulation system set to ‘Off’. Software automation using Matlab consolidated the experimental data to a feature set for machine learning classification. The Waikato Environment for Knowledge Analysis known as WEKA enabled machine learning classification through the following six machine learning algorithms: multilayer perceptron neural network, support vector machine, K-nearest neighbors, logistic regression, and J-48 decision tree, and random forest.The machine learning classification accuracy for differentiating between deep brain stimulation set to ‘On’ and ‘Off’ modes using the smartphone as a wearable and wireless inertial sensor to provide quantified feedback for all six machine learning algorithms is summarized in Figure 5. In particular, multilayer perceptron neural network, support vector machine, K-nearest neighbors, logistic regression, and J-48 decision tree achieved 100% classification accuracy. Random forest misclassified one deep brain stimulation ‘Off’ mode instance as ‘On’ mode attaining 95% classification accuracy.The future objective of the use of a wearable and wireless inertial sensor with deep brain stimulation is to enable automated closed loop parameter optimization using the objective and quantified feedback from a wearable and wireless inertial sensor system. The Food and Drug Administration of the United States considers the deep brain stimulation system to be a Class 3 device [38]. The Class 3 device is considered as greatest risk to the patient [39]. Intuitively the machine learning classification response according to multiple inertial sensor quantification scenarios would be warranted. In order to further determine the most appropriate machine learning classification algorithm, further investigation of the classification accuracy of respective of only the accelerometer signal and only the gyroscope signal is warranted. Essentially, this extension of the machine learning classification endeavor seeks to determine the most robust machine learning classification algorithm in the event that one of the sensors has a failure mode issue.Figure 6 provides the classification accuracy of distinguishing between deep brain stimulation ‘On’ and ‘Off’ modes for the Essential tremor subject with the feature set constrained to only the acceleration magnitude attributes. The J-48 decision tree achieved 100% classification accuracy. All of the five other machine learning classification algorithms (multilayer perceptron neural network, support vector machine, K-nearest neighbors, logistic regression, and random forest) attained 95% classification accuracy, as a consequence of misclassifying one deep brain stimulation ‘Off’ mode instance as ‘On’ mode.Using only the attributes derived from the gyroscope signal data further machine learning classification accuracy for differentiating between deep brain stimulation ‘On’ and ‘Off’ modes for the Essential tremor subject was determined as illustrated in figure 7. Interestingly the J-48 decision tree only attained 90% classification accuracy using only the gyroscope signal data. One instance was misclassified as deep brain stimulation ‘Off’ mode instead of ‘On’ mode, and another instance was misclassified as ‘On’ mode instead of ‘Off’ mode. The five other machine learning classification algorithms (multilayer perceptron neural network, support vector machine, K-nearest neighbors, logistic regression, and random forest) attained 100% classification accuracy.Given the machine learning classification accuracy findings presented in figures 5, 6, and 7 the multilayer perceptron neural network, support vector machine, K- nearest neighbors, and logistic regression achieve the highest classification accuracy for three feature set scenarios: both accelerometer and gyroscope, accelerometer, and gyroscope. The future goal is to integrate one of these machine learning classifiers in to an embedded system using the objective quantified feedback from a wearable and wireless inertial sensor system for the close-loop and automated parameter optimization of deep brain stimulation for the treatment of movement disorder, such as Essential tremor. Future studies are also recommended for the more refined selection of an appropriate machine learning algorithm using a feature set derived from quantified feedback from a wearable and wireless inertial sensor system, such as computational efficiency in consideration of the processing capability of the embedded system of close-loop and automated parameter optimization of deep brain stimulation.V.  ConclusionFig 5. Machine learning classification accuracy for differentiating between deep brain stimulation set to ‘On’ and ‘Off’ modes using both smartphone accelerometer and gyroscope.Fig 6. Machine learning classification accuracy for differentiating between deep brain stimulation set to ‘On’ and ‘Off’ modes using smartphone accelerometer.Fig 7. Machine learning classification accuracy for differentiating between deep brain stimulation set to ‘On’ and ‘Off’ modes using smartphone gyroscope.Essential tremor is a prevalent form of neurodegenerative movement disorder. The research objective is to assess the capabilities of six machine learning algorithms for the classification of deep brain stimulation using ‘On’ and ‘Off’ status for a subject with Essential tremor conducting a reach and grasp task. The future goal is to integrate the machine learning algorithm into a closed-loop system for automated parameter optimization of the deep brain stimulation for the broad treatment of movement disorders. Six machine learning algorithms were evaluated: multilayer perceptron neural network, support vector machine, K-nearest neighbors, logistic regression, J-48 decision tree, and random forest.  The nature of the movement disorder was objectively quantified through a smartphone representing a wearable and wireless inertial sensor equipped with both an accelerometer and gyroscope.  The inertial sensor signal data was consolidated into a feature set for machine learning classification. The research addressed three feature set scenarios, such as both the accelerometer and gyroscope, accelerometer, and gyroscope. Addressing these feature set scenarios enables a more informative determination of the most robust machine learning algorithms available. Four machine learning algorithms attained the highest classification accuracy regarding the three feature set scenarios: multilayer perceptron neural network, support vector machine, K-nearest neighbors, logistic regression.In pending studies, it is recommended that the machine learning algorithms are also contrasted for computational efficiency in consideration of the available embedded system processing capacity. The future goal is to integrate machine learning with an embedded system while using quantified feedback from the wearable and wireless system to objectively assess the response to the closed-loop automation of optimizing a parameter configuration for deep brain stimulation treatment of movement disorder, such as Essential tremor.