ControlXR: a universal haptic interface for AR/VR devices

Introduction

There are currently 4 main interactions and controllers in virtual and augmented reality systems: VR controllers, haptic gloves, and keyboards (virtual and physical).

Virtual keyboards are slow and input bandwidth is very limited. Controllers work great for games and other interactions but are severely limited in text input. Physical keyboards are faster than virtual ones but require a keyboard to be brought around, adding to clunkiness, and haptic gloves are great for natural interaction but also suffer from limited text inputs and lack the speed of input or button pressing of VR controllers.

Based on this context, ControlXR is a new set of VR gloves that combine all four of these interactions into one, universal controller which has high text input bandwidth, can act as VR controller, and in addition as haptic feedback gloves that can feel dozens of different virtual materials.

We use piezoelectric sensors and actuators on the tips of the finger, and through the computer vision of the VR headset we determine the keys being typed.

Progress

Exploring different novel interface of keyboards within VR and through haptic feedback, we tested different keyboard designs throughout the semester.

Aghyad testing out an interaction design in which the keys of the keyboard are mapped on the fingers

The Haptic glove function of ControlXR is functional: when touching virtual objects, the glove gives haptic feedback on the correct fingers. To implement this, we used Unity's XR Interaction Toolkits new library XR Hands. The Haptic feedback is customized based on different frequencies, amplitudes, and waveforms.

Currently our team of 6 is working on designing a custom PCB to drive piezos using Boreas haptic actuators, and reverse engineering the BOS1901 kit to be controlled using an ESP32 microcontroller. Now, the team is focusing on improving a novel typing interface that integrates this piezoelectric feedback.

To enable wireless communication, we successfully connected an ESP32 to a VR headset with two-way communication, allowing any custom message to be sent back and forth for driving and sensing.

Early glove prototype with PCB powering piezoelectric actuators when touching virtual objects.

Visitor at SEAS Design Fair trying the early glove prototype.

Future Goals

• Custom PCB design to communicate between Quest and piezo-electric actuators

• Implementation of VR controller

• Development of waveform based tactile sensations

• Improvement and extension of range of tasks that can be done with the gloves