Article Preview
TopIntroduction
Tangible User Interfaces (TUIs) are known for enhancing the user experience with an interactive system by providing a more natural way to manipulate information (Hinckley et al., 1994).
There are various technologies can be used to create tangible objects, from electronically instrumented active objects to passive ones. As examples of active tangibles, (Johnson et al., 1999) implemented plush toys with various sensors including pitch and roll sensors, gyroscope, and magnetometers as well flexion and squeeze sensors, which enabled them to sense various kinds of interactions with the toys; (Sajjadi et al., 2014) employed Sifteo Cubes (small computers – cubes with about 4 cm – with screens and sensors that enable them to react to movements and proximity with one another) as tangible objects in a VR game – Maze Commander – where two players, one using a VR headset and another manipulating the Sifteo Cubes, collaborate to escape a maze.
On the other hand, even simple passive tangibles have been shown to have a significant positive effect on the VR experience (Insko, 2001) and they are generally cheaper and easier to build. (Aguerreche et al., 2010) for example, created a reconfigurable object with the shape of a triangle with extendable edges. This passive object is detected by an external sensing infrastructure (motion capture studio) and can be associated with various kinds of virtual objects, allowing their manipulation. Passive objects can also be detected by external depth sensing cameras. In the Annexing Reality system (Hettiarachchi & Wigdor, 2016), for example, a Kinect sensor was use to identify physical objects and map them to virtual objects with similar shape.
Another solution for passive tangibles is the use of structured 2D markers embedded in the objects, which can then be tracked through computer-vision techniques. When combined with smartphones, this allows the development of TUIs for smartphone-based, accessible, eXtended Reality (XR) experiences. While the use of visual markers in TUIs is not new ((Cardoso & Ribeiro, 2021; Drogemuller et al., 2021; Lee et al., 2011; Zheng et al., 2020)), there is little research into their use with 3D printed objects. 3D printing is an attractive alternative because the 3D printing market has reached a state where anyone can now buy, easy to use, relatively cheap, 3D printers.
In this work, we explore the possibility of creating 3D printed modular objects with embedded visual markers and ways to make 3D printed dynamic markers, i.e., markers that change their value through user interaction. We aim to achieve an accessible, tangible system with low-cost materials appropriate for smartphone-based XR. We also aim at developing a system that appeals to the maker community. Markers can, of course, be used in 3D printed objects as stickers. However, we were interested in exploring richer alternatives for embedding the markers directly into the 3D printing process and turning them into interactive elements.
Our main contribution is a set of solutions for 3D printed marker-based TUIs that can then be used in various ways. Although our focus was on creating toys, we believe some of the solutions we describe next could be applicable also in non-game-based systems. Some of the solutions we explore in this paper could be created with other fabrication techniques beyond 3D-printing. However, the purpose of this work is not to explore generic fabrication techniques for marker-based TUIs, but to focus specifically on 3D printing to understand the potential design space of 3D printed marker-based TUIs.