The Voight-Kampff Machine (VKM) is an approach to select a rejection threshold for custom gestures. When users want to customize their gesture set, training data is sparse. So, learning a reliable rejection threshold for malformed and accidental input is a challenging problem. At the same time, high activity (HA) data has no periods of low activity which usually indicate end points of gestures. This leads to a high number of false positives, unless the rejection threshold is tight enough to reject such input. Our approach is a 5-step process for selecting a rejection threshold based on synthetically generated positive and negative gesture samples. In our evaluation we compared VKM to alternative threshold selection techniques and found VKM to be the only approach which consistently achieved high accuracy on 4 high activity datasets: Kinect, Mouse, Vive Position and Vive Quaternion. For details on the method, please refer to our paper, GitHub repository, and download the datasets (done automatically with the script in repository).

Reference Implementation: VKM Code Repository
Download Datasets: HA Datasets
Paper: PDF
DOI: On ACM DL

Voight-Kampff Machine as a 5-step process:

1. Form positive synthetic scores distribution using GPSR [3]
2. Form negative synthetic scores distribution using Mincer [this work]
3. Select rejection threshold using F1-accuracy (score that best separates false positives and false negatives)
4. Inflate threshold to account for future data variability and input device type
5. Reduce threshold through a Monte Carlo simulation based on the number of training samples and observed variability

Gesture Path Stochastic Resampling (GPSR) [3] is a technique to generate synthethic gesture samples, by first stochastically resampling the original trajectory to a fixed number of points, and then normalizing the distances between each point. Mincer is a negative synthetic data generation technique which replaces portions of the original sample's trajectory with sub-trajectory of samples from another class. F1-accuracy minimizes the number of false positives and false negatives by balancing precision and recall.

To evaluate VKM, we created a continuous gesture data processing pipeline, called The Dollar General (TDG). TDG is a collection of device-agnostic techniques as illustrated in the figure below.

Machete [2] proposes regions that might be gestures, Jackknife [1] classifies proposed regions, and VKM [this work] rejects input that does not cross the similarity threshold.
Further details can be found in the VKM paper, below references, and in the reference implementation.

• [1] Taranta II, E. M., Samiei, A., Maghoumi, M., Khaloo, P., Pittman, C. R., and LaViola Jr, J. J. " Jackknife: A reliable recognizer with few samples and many modalities." Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 2017.
• [2] Taranta II, E. M., Pittman, C. R., Maghoumi, M., Maslych, M., Moolenaar, Y. M., and Laviola Jr, J. J. " Machete: Easy, Efficient, and Precise Continuous Custom Gesture Segmentation." ACM Transactions on Computer-Human Interaction (TOCHI) 28.1 (2021): 1-46.
• [3] Eugene M. Taranta II, Mehran Maghoumi, Corey R. Pittman, and Joseph J. LaViola Jr. "A Rapid Prototyping Approach to Synthetic Data Generation For Improved 2D Gesture Recognition." Proceedings of the 29th Annual Symposium on User Interface Software and Technology. ACM, 2016.
• [4] Taranta II, E. M., Maslych, M., Ghamandi, R., and Joseph J. LaViola, Jr. "The Voight-Kampff Machine for Automatic Custom Gesture Rejection Threshold Selection." CHI Conference on Human Factors in Computing Systems. 2022.

If you end up using our VKM approach in your work, please cite:

                    @inproceedings{taranta2022_VKM,
                        author = {Taranta, Eugene Matthew and Maslych, Mykola and Ghamandi, Ryan and LaViola, Joseph},
                        title = {The Voight-Kampff Machine for Automatic Custom Gesture Rejection Threshold Selection},
                        year = {2022},
                        isbn = {9781450391573},
                        publisher = {Association for Computing Machinery},
                        address = {New York, NY, USA},
                        url = {https://doi.org/10.1145/3491102.3502000},
                        doi = {10.1145/3491102.3502000},
                        booktitle = {CHI Conference on Human Factors in Computing Systems},
                        articleno = {556},
                        numpages = {15},
                        keywords = {rejection, customization, gesture, recognition},
                        location = {New Orleans, LA, USA},
                        series = {CHI '22}
                    }