PliARs
Prototype of Force Feedback Tool for Mixed Reality Applications
I. Gonsher and Z. Lei, "Prototype of Force Feedback Tool for Mixed Reality Applications," 2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), 2021, pp. 508-509, doi: 10.1109/ISMAR-Adjunct54149.2021.00123.
Context
Physical tools are dedicated to physical objects in physical environments.
Virtual tools are dedicated to virtual objects in virtual environments.
Virtual tools are dedicated to virtual objects in virtual environments.
What is the new paradigm that allows one users to move fluently between the virtual and the physical world?
Ideation
The tool integrates the pliers as a functional part and the phone as an interaction input. To achieve an ergonomic composition, we tested out different compositions regarding the relative position of the two parts, such as a phone on the front/backside and horizontal/vertical orientation.
It allows the user to both feel and sees in both modalities. This allows users to see the virtual object and manipulate it accordingly; it also offers opportunities for other affordances, including collaboration at a distance through video conferencing, for example. There is also potential to use these kinds of tools in scenarios where collaboration and instruction occur at a distance.
Prototyping
The 3d-printed casing houses the servo motor. This motor simulates the physical resistance of clamping on objects. It also operates the opening of the pair of pliers as if they were bounced back by springs.
Screen Wireframe
We designed two affordances: the instructor mode and the apprentice mode.
In both modes the user can use the phone to control the tool’s functionality while seeing through the phone’s camera of the actual pair of pliers.
In both modes the user can use the phone to control the tool’s functionality while seeing through the phone’s camera of the actual pair of pliers.
Implementation
The prototype uses an Arduino Bluetooth LE Nano board to control the servo motor. It allows the tool to communicate with the Unity interface on the attached smartphone wirelessly via Bluetooth.
The analog feedback from the servo motor provides positional data of the pliers’ movement. This movement is mapped in the virtual interface in real-time.
In turn, this virtual model determines when the motor should provide force feedback by locking its position.
Buttons on the exterior of the handle mirror the same functionality of the virtual controls, making elements of the interface easily discoverable.
The analog feedback from the servo motor provides positional data of the pliers’ movement. This movement is mapped in the virtual interface in real-time.
In turn, this virtual model determines when the motor should provide force feedback by locking its position.
Buttons on the exterior of the handle mirror the same functionality of the virtual controls, making elements of the interface easily discoverable.
Scenario 1: Instructor records manipulations
Scenario 2: Apprentice follows to recreates manipulations
