On-Body Tangible Interaction: Using the Body to Support Tangible Manipulations for Immersive Environments

. Recent technological advances in immersive devices open up many opportunities for users to visualize data in their environments. However, current interactive solutions fail at providing a convenient approach to manipulate such complex immersive visualizations. In this article, we present a new approach to interact in these environments, that we call On-Body Tangible interaction (OBT): using the body to physically support the manipulation of an input device. The use of the body to support the interaction allows the user to move in his environment and avoid the inherent fatigue of mid-air interactions. In this paper, we explore the use of a rolling device, which ﬁ ts well on-body interaction thanks to its form factor and offers enough degrees of freedom (DoF) for data manipulation. We ﬁ rst propose a new design space for OBT interactions, and speci ﬁ cally on the forearm. Then we validate the feasibility of such an approach through an experiment aimed at establishing the range, stability and comfort of gestures performed with the device on the forearm. Our results reveal that on-body tangible interaction on the forearm is stable and offers multiple DoFs with little fatigue. We illustrate the bene ﬁ ts of our approach through sample applications where OBT interactions are used to select and execute space-time cube operations.


Introduction
Immersive systems such as the Hololens 1 , MetaVision 2 or Moverio 3 allow the user to display data and visualizations directly on the physical world by attaching them to a fixed physical anchor; we hereafter refer to these as immersive visualizations.
Immersive visualization offers a large set of advantages, such as spatialization of data, physical exploration of data, wide display area, etc. It thus constitutes a compelling alternative to visualizing multidimensional data on 2D displays. However, data exploration involves many tasks, such as filter, select, adjust or annotate the dataset, which are not entirely covered in existing solutions.
Indeed, a large interaction vocabulary is required and approaches based on mouse, touch and mid-air interaction fail to offer enough degrees of freedom (DoF) [17,34,39]; other solutions are often ambiguous and tiring (especially mid-air gestures [7,34,35]); finally many restrict the user's interaction to a defined place, usually a desktop, to use the input device (3D mouse or other [30]). The challenge then is to provide an interactive solution for immersive environments that preserves the freedom of movement of mid-air interaction and the DoFs of tangible interaction.
To this end, we propose to explore On-Body Tangible (OBT) interactions, i.e. a new approach using the body as a physical support for manipulating a tangible object to interact with immersive environments. This approach combines: (1) the use of a connected physical object, that offers multiple degrees of freedom and, (2) the use of the body to guide the physical manipulations of the tangible object and exploit the user's proprioception (i.e. sensing its own body parts). While the first aspect offers the multi-DoF of tangibles, the second aspect ensures that the solution can be used anywhere and can potentially contribute to reduce muscle fatigue, two inherent characteristics of mid-air interactions.
Our contribution is both conceptual and experimental. First, we propose a design space that encompasses the physical properties of the body (support) and the interactions that can be performed on it. Second, we validate the fundamental advantages of the OBT interactive approach (the feasibility of the gestures, their amplitudes, the comfort and fatigue related to the interaction) through a three-part experiment: the first part of the study focuses on on-body translations of the object, the second on rotations and the third evaluates rolls. Finally, we discuss the advantages and disadvantages of on-body tangible interactions, and we present two illustrative applications where onbody tangible interaction is used to select and execute space-time cube operations.

Related Work
This section synthesizes the main interaction solutions proposed for immersive environments, i.e. tactile, mid-air, tangible and on-body interactions.

Tactile Approaches
Whether the interaction is integrated directly on the immersive device itself [39], or deported to an external device (mobile device, interactive tabletop) [14,31], different forms of tactile interaction have been applied to manipulate immersive visualizations [18,20,31,32,39]. However, Bergé's [7] and Besançon et al. [10] works established that tactile interaction is neither the most efficient, nor the preferred interaction modality for 3D data manipulation. Moreover, tactile interaction requires a dedicated surface [7], which can divert the attention of the user from the task to perform, and may constrain the movements if using fixed tactile display [30,32].

Mid-Air Interactions
Mid-air interaction offers obvious advantages for immersive environments, such as unconstrained mobility, and are light and easy to perform [13,38,43,51]. These advantages led to multiple developments, such as the interactive immersive experience proposed by Benko et al. [5] where users could employ mid-air gestures to interact with an augmented dome. Interestingly, authors evaluated the mid-air gestures and found that the proposed interaction was simple to perform but understanding how to perform it was not easy. Other well-known problems with mid-air interactions are fatigue [12], ambiguity when no appropriate feedback is provided [7] and discoverability, i.e. finding the right mid-air interaction for a given task/action [7].

Tangible Objects
Tangible interaction provides multiple degrees of freedom for interaction with immersive environments [1,8,50]. Its physicality suggests how to perform the interaction, minimizing the learning process. Used for the design of DNA (Deoxyribonucleic acid) components, tangible 3D input devices in immersive interface appear to be more satisfying than a 2D interface [41]: the direct 3D mappings between the 3D space and the tangible device manipulation facilitates the understanding. Besançon et al. [10] also established that tangible interactions perform better than its mouse and tactile counterparts, and that tangible's affordance limits the need for a learning phase. However, they pointed out that the mouse was more precise than tangibles used in midair. Interestingly, TUI (Tangible user interface) for have been tested in many different contexts such as interactive visualization of thin fiber structures [26], 3D visualization [15], exploration of volumetric data [25], etc.
Along the same lines, multi-DoF devices have been explored through the augmentation of a mouse in order to overcome the mere 2 degrees of freedom offered. The Rockin' Mouse [3] and the VideoMouse [23] are based on a semi-rounded shapes that allows tilting the device but do not totally support compound gestures. The Roly-Poly Mouse (RPM) [37] uses a completely rounded bottom. It has been shown to provide larger amplitude of movement than previous tilting devices and efficient in 3D contexts and Multi-Display Environments [40].

On-Body Interaction Techniques
As opposed to the previous solutions, on-body interactions have been scarcely used in immersive environments. Serrano et al. [42] explored hand-to-face gestures arguing that they are well suited for HWDs. Dobbelstein et al. [17] proposed the use of the belt as a tactile surface to interact with HWDs. Wang et al. [47] proposed PalmType, an interaction technique that enables users to type with their fingers on the palm to input text in smart-glasses. These works illustrate that on-body interaction techniques allow eyes-free interactions by exploiting the proprioception capabilities of users and do not divert their attention from the task at hand. However, the explored approaches offer a limited set of possible gestures, making it unsuitable for complex data visualization.

On-Body Tangible Interactions
In this section, we present a new interaction approach for immersive environments based on the use of the body to support tangible interactions. We detail the main requirements to interact with immersive environments, our choices of body parts and tangible objects to use, before presenting the design space.

Requirements for Immersive Environments
We illustrate the requirements for immersive environments through the specific case of immersive visualization. There are different types of immersive visualizations according to the data to visualize, which ranges from simple 3D objects to complex multidimensional data. All these immersive visualizations share a set of basic interaction requirements: • Unconstrained mobility (R1): it has been demonstrated that the spatial exploration of data allows for a better spatial understanding of the visualization [27]. The user can have an overview of the visualization from afar, a more detailed view by getting closer, as well as different views from different angles [27]. It is thus important that the interaction techniques do not constrain the mobility of the user. • Multiple degrees of freedom (R2): the multidimensional nature of data in immersive systems requires enough degrees of freedom to tackle its manipulation [4] through multiple commands (filter, sort, annotate, highlight, clusterize, aggregate, etc.). • Visual access (R3): the interaction techniques should not occult the data visualization. They should also allow the user to interact with data without diverting his attention from the visualization [27]. • Stability (R4): the interaction techniques should offer enough stability to properly tackle data visualization tasks in mobility.
While these requirements apply for immersive visualizations, they hold true for several other interaction environments, such as VR caves, augmented maintenance, gaming, etc. Therefore, the solution presented in this paper can be applied to such other interaction environments. We leave the investigation of these other environments for future work.

Combining Tangible and On-Body Interaction
After clarifying the requirements for immersive visualizations, we now describe and justify the properties of On-Body Tangible interaction: the tangible object used in the interaction, the body part used to support it and their combination.
Tangible Object. Among the plethora of existing tangible objects introduced in the literature, we decided to use a semi-spherical object. The rationale behind this choice is manifold. (1) It has been demonstrated [37] that such a form factor offers up to six degrees of freedom (R2), in the form of three types of manipulations (translations, rotations and rolls), facilitating the manipulation of multidimensional data [37]. (2) As opposed to other forms (that have flat surfaces), the contact of a rounded object with the interaction surface is minimal (i.e. a point), which will easily adapt to the outline of any part of the body. Other small sharp objects (such as a pen) have also a minimal contact point, but their sharpness is detrimental as they may hang onto the clothes or hurt on the skin. (3) As opposed to objects that includes flat surfaces, the manipulation of a round object remains continuous, does not interrupt the interaction flow and do not artificially promote the use and need of modes in the application. (4) The choice of a semi-spherical form rather than full spherical one is motivated by the fact that it is easier to hold [37].
Body-Parts Used to Support the Interaction. Many research works have focused on interaction on or with the body [9,21,28,46,49]. Arms and hands were the preferred body parts in most works. These body parts offer numerous advantages: they are easily accessible for interaction; they are in the user's field of vision and generate less social discomfort than other body parts [28,45]. Karrer et al. experimentally identified the most appropriate region of the body to perform interaction with clothes [28]. Among their observations, the non-dominant arm as well as the hip are the preferred body-parts for interaction. Other parts of the body, such as the stomach and legs, have been rejected for social or personal reasons in the same study.
We decided to focus on the forearm of the non-dominant arm to support the interaction for several reasons. It is an always-available physical support that favors physical exploration of data and it does not constrain the movement of the user (R1). Thanks to the body's natural capacity to sense its own body parts (proprioception), the user can perform tangible interactions on the body without having to switch his attention from the data visualization to the interaction tool (R3). Using the forearm offers a large surface on which the tangible object can be laid, and it is effortlessly accessible by the dominant hand as opposed to the upper arm which needs a consequent effort to be touched by the dominant hand (R1, R2). Furthermore, its combination with a tangible object (which use can be extended to mid-air manipulations, as suggested by the Lift-Up option in [40]) does not constrain the user's movement (R1). Finally, several stable poses can be adopted with the forearm (R4), which increases the range of possibilities (R2) (Fig. 1).

Mapping the Input Interaction Space with the Data Visualization Space
While the user's gestures are performed in a 3D physical environment, they trigger actions in a 3D virtual environment. It is therefore important to choose the right frame of reference for the interaction. The frame of reference can be allocentric (external: it can be world-centered, data centered…) or egocentric (relative to the body). In an egocentric frame of reference, the output of a given manipulation is determined by how it is performed with regards to the body, and will have the same effect regardless of the body position and orientation in the world. In our approach, we adopt an egocentric frame of reference to allow the user to interact from anywhere with geographically anchored data in the physical world [33]. Indeed, previous research found that a lack of a logical relationship between the manipulated data's frame of reference and the user's frame of reference can affect performances negatively [36,48].

Design Space for On-Body Tangible Interaction
Based on the main design properties of OBT interactions introduced above, we now present a design space describing the possible use of the forearm as interaction support. It is composed of 3 dimensions: The Pose, the Place of motion and the Tangible action.
Restricting interaction to the forearm -more socially acceptable, accessible and with a large interaction area-has the disadvantage of reducing the available interaction vocabulary. To address this limitation and increase the interaction vocabulary, we propose to consider the posture of the arm (Pose) as well as the use of sub-regions on the forearm (places of motion) and the tangible action (TA) that can be applied.
Pose (inPOS). The pose describes the position of the forearm with respect to the user's body. We identified three main poses for the forearm. In the Vertical pose, the forearm is vertical, the hand points upwards. In the Forward pose, the forearm is perpendicular to the shoulders. In the Parallel pose, the forearm is parallel to the shoulders (Fig. 1). We chose these three poses because they are the most comfortable poses, the most accessible with the non-dominant hand and they are significantly different from each other, facilitating their distinction by both the system and the user.
Place of Motion (POM). The Place of motion represents the surface of the forearm on which the tangible object can beused. We identified two types of places: the first one extends over the length of the forearm, from the elbow to the wrist (length POM); the second one, called width POM, divide the forearm into several sub-regions. The number of sub-regions was defined according to the following criteria: (1) the subregions have to be large enough to accommodate interaction with the tangible object (diameter = 8 cm); (2) the sub-regions have to be easily distinguishable. Applying these criteria, we divided the forearm in three sub-regions: close to the Elbow (Elbow POM), in the middle of the forearm (Middle POM) or close to the wrist (Wrist POM) (Fig. 1). This results into 12 different interaction supports (3 poses Â 4 places), which increases the possibilities of interactions exploiting the proprioception of the user and contributes to avoid the fatigue of a mid-air usage.
Tangible Action (TA). It stands for the physical actions than can be performed with the tangible object. The round shape of the tangible object offers three physical manipulations: translation, roll and rotate. Rotations and rolls along the length of the forearm (Length POM) were not considered. Indeed, they are performed with the tangible object motionless, in the same spot, thus, rendering them too similar to rotations and rolls on one of the other three places of motion. The following experiments will analyze the benefits and limits of each of them separately.

Study: Translations, Rotation, Rolls on the Forearm
We present a three parts experiment in which we explore the feasibility and potential of on-body tangible interaction in the different configurations described above. We explore the potential along three aspects: (1) the maximum range of motion for each of the three physical manipulations; (2) the stability of the interaction; and (3) the width of places of motions We define the range of motion (RoM) as the maximum comfortable amplitude of each Tangible Action (translation, rotation and roll) for each pair of Pose and Place of motion. Concretely, it represents (1) the maximum distance that can be covered with the tangible object when translating it; (2) the total amplitudes of clockwise and counterclockwise rotations; (3) the total amplitude of forward and backward rolls. The greater the range of motion, the greater the range of values that can be manipulated. Stability concerns the forearm (i.e. is it steady enough) and the tangible object manipulation: in the first case, a stable support is required; in the second case, it is important to know if unexpected tangible actions occur while performing one specific tangible actions [37,40].
Width of Place of Motions corresponds to the space of the forearm used by the users when performing a tangible action in this PoM. It is important to determine whether the PoMs are large enough to hold the interaction without overlapping.

Task
Our study was divided into three parts. During the first part, we asked participants to perform translations on the four different places of motion (PoM): the full length of the forearm (length POM), as well as on the Elbow, Middle and Wrist PoMs, as illustrated in Fig. 1. A trial is defined as a back-and-forth translation on the forearm. The starting points of the translations were chosen by the participants at the beginning of each trial. It was not necessary to control the starting points as each trial was composed by 10 consecutive gestures.
In the second and third part of the study, we examined rotations and rolls respectively. A rotation (resp. roll) trial consisted in a rotation (resp. roll) towards the wrist (clockwise) followed by a rotation towards the elbow (counterclockwise) and a return to the initial state. The participants were asked to perform the maximum comfortable rotation (resp. roll) in both directions (clockwise, counterclockwise). Rolls perpendicular to the forearm would be too difficult to distinguish from translation in the width of the forearm and were not considered in this experiment.
Participants were asked to manipulate the device with the dominant hand and to perform the gestures on the forearm of the non-dominant hand. The physical manipulations were performed for each Pose (Vertical, Parallel, Forward, see Fig. 1). The poses and places of motion were explained and illustrated to the participants at the beginning of the experiment. Participants were free to grasp the object as they wished. Finally, since the purpose of the experiment was to study the use of the tangible on the forearm, no feedback was provided to the participants.

Apparatus
The diameter of the rounded tangible object used for the experiment was 8 cm. In order to detect rotations and rolls of the tangible object, we used an IMU of X-io Technologies (x-IMU: 48 g, 57 mm Â 38 mm Â 21 mm). The IMU is composed of a triple-axis gyroscope, accelerometer and magnetometer, offering an angular precision of 1°. The refresh rate of the sensors goes up to 512 Hz and we used Bluetooth to connect the IMU with the computer.
To locate the position of the tangible object and the body parts, we used an OptiTrack system composed of 12 cameras that track infrared reflective markers with a precision of 1 mm. The markers were carefully placed on the tangible object so that they do not influence the participant's grasp (Fig. 2). Additional infrared reflective markers were placed on the main joints of the arm/forearm (Fig. 2). The wrist, elbow of the non-dominant arm and the shoulders were also tracked.

Procedure
The first part of the study (on translations) follows a 3 Â 4 within-subject design with the Pose (Forward, Parallel, Vertical) and the Place of Motion (Length, Elbow, Middle, Wrist) as factors. The second and third parts of the study (on rotation and roll respectively) followed a 3 Â 3 within-subject design, with the same factors but without the Length Place of Motion (which is not in our design space, see Fig. 1).
The Pose factor is counterbalanced using a 3 Â 3 Latin square. Each part of the study is composed of 3 blocks, each block corresponds to one Pose and consists of all the Places of motion in a random order. For each pair of Pose and Place of motion, participants had to do 3 repetitions of each trial (which is composed of 10 consecutive gestures). The participants could take a break between each repetition. Each part of the study lasted between 30 and 40 min. We collected 4320 trials in total for the translations study (360 trials per participant), 3240 trials for the rotation study (270 trials per participant) and 3240 trials for the rolls study (270 trials per participant).

Participants
We recruited 36 (12 participants for each of the 3 studies) (9 female), aged 26.0 on average (SD = 3.4). All the participants were right-handed. Most of the participants were students (PhD, MSc) from the computer science department of the local university; all other participants were familiar with electronic equipment.

Collected Data
We measured the circumference of the forearm near the elbow and the wrist for each participant as well as the inner and outer length of the forearm. Every 10 ms, we recorded the position, rotation and roll of the tangible object, as well as the position of the wrist, the elbow and the shoulders. To evaluate the physical fatigue, we asked participants to fill out a Borg scale [19] for each pair of (Pose, Place of motion).

Study: Results
We report the forearm stability; the device range of motion; the gestures overlapping, the unintentional device movements and the user's fatigue for each tangible action. Regarding the data analysis, we chose to rely on estimation techniques with 95% confidence intervals as recommended by the APA [44].

Forearm Stability: Elbow and Wrist Movements
To study the forearm stability, we analyzed the movements of the elbow and wrist with regards to their starting position, collected at the beginning of each trial. Results show that the elbow was relatively stable: the maximum movement observed, all directions included (along the X, Y and Z axis) and all physical manipulations included (Translations, Rotations, Rolls) did not exceed 1,7 cm on average. This result remain valid when considering each Pose and Place of motion independently. Taking into consideration these findings, we can say that the forearm is sufficiently stable to support tangible interactions.

Range of Motion
Translations. The ROM for translations along the forearm was computed by calculating the average distance covered by the tangible object for each group of 10 successive gestures. It was computed for each pair of Pose and Place of motion. As the size of the forearm differs from one participant to another, we standardized the collected data.
Participants  There was no significant difference between the places of motion.
Rotations. Regarding the rotations, we collected the maximum clockwise (CW) and counterclockwise (CCW) rotations. Overall, CW rotations among the 3 poses were slightly bigger (58°vs 50°) than CCW. This difference between CW and CCW amplitude is in line with those obtained with a device of similar form [37] (CW:73°, CCW:59°). The participants in our study used the "squeeze" hand posture [37]  . Therefore, results tend to establish that rolls towards the elbow had bigger amplitudes for the 3 poses than rolls toward the wrist.

Gestures Overlapping
However, when refining this result through the analysis of the distribution of the device position, results show that the interaction on the Elbow and Middle overlaps over a third of the forearm (Fig. 5A); the overlap between interaction in the Middle and Wrist represents a fifth of the forearm. However, it clearly appears that the translations on the Elbow and Wrist do never overlap. Finally, despite the clear instruction that required participants to perform translation from the elbow to the wrist, we can observe that a fair number of translations were performed beyond the wrist position (i.e. on the hand). During rotations, participants used 23% of the forearm to perform rotations. The surface used was slightly smaller on the Elbow (21%) compared to the Middle and Wrist (23% & 24% respectively). A reason for that could be that interaction near the elbow are more restricted due to the upper arm. Similarly, to the translations results, we observed a large distribution of the exploited surfaces (Fig. 5B).As with translations and rotations, the exploited surfaces when performing rolls on the forearm Fig. 5C) were fairly small (Elbow POM: 19%, Middle POM: 21%, Wrist POM: 20%) but their dispersion was large enough for them to overlap.

Unintentional Device Movements
As underlined in [21], rolls are the most probable unintentional manipulations that can occur during translations, which could lead to the activation of unwanted commands. This is particularly true in our case, since users perform translations over the circumference of the forearm, unlike the flat surface used in [21]. Therefore, we analyzed the stability of the device during interaction by looking at unintentional rolls during translations, similar to [21].

Fatigue
Fatigue was measured using a 6-20 Borg scale [11]. The average Borg score obtained for the poses ranged from 'very light' for the Forward (9) and Parallel (10) pose, to 'light' for the Vertical pose (11). The pose does not appear to affect the fatigue scores and overall, participants did not consider tangible on-body interaction tiring. However, with an average of 13, rotations in the Vertical pose were found relatively tiring to perform ('somewhat hard'). It should also be noted that while participants had the opportunity to take breaks during the experiment between each group of 10 gestures, only one participant asked for a break.

Discussion
The findings presented above consolidate our hypothesis that on-body tangible interaction is a promising approach for use in immersive environments. It is stable, fatigue free, and offers a very rich set of interactions. Multiples actions, poses and places can be combined to best fit the user's set of tasks to perform, offering a wide input vocabulary to manipulate immersive environment.
The forearm stability did not diminish over time. The forearm position was steady for the entire study (all physical manipulations included), making this body part a reliable support for interaction. The interaction itself did not induce fatigue according to the users' feedback, even if the sessions took more than 30 min.
The surfaces covered while performing physical manipulations with the tangible object on the Elbow, Middle and Wrist POM were small enough (less than a fourth of the forearm at most) to consider three distinct regions of the forearm for interaction. However, the dispersion of these surfaces showed that they overlap (all physical manipulations included), making it difficult to employ more than 2 distinct regions of the forearm in practice. We want to point out that our experimental setup did not offer any feedback regarding the available places of motion. We believe that with a visual feedback showing the position of each region on the forearm in the immersive environment, the three regions would be more easily distinguishable. This hypothesis should be studied in a complementary study.
From the previous results, we can draw some design suggestions on how to use onbody tangible interaction for visualization tasks: • The translations performed on the length POM were the most robust in terms of involuntary rotations and rolls (in line with previous results on desktop [37]). In terms of exploitable interaction surfaces, most (at least 73,7%) of the forearm subregions available for interaction was exploited to perform on-body tangible interactions. Due to their stability and large amplitudes, translations on the length POM can be used to control a large set of values, to have a substantial precision or a greater data coverage. For instance, they could be used to manipulate slider type controllers that require a certain degree of precision depending on the manipulated data. Their stability allows them to be used in compound gestures (in combination with a roll or a rotation) which can allow for the design of expert modes. • Translations around the forearm (Elbow, Middle and Wrist POM) exploited at most half of the surface available for interaction. These gestures seem to be better adapted to controllers that do not require a large amplitude, i.e. "rate control" type input. • The amplitudes observed for rolls (ranging from ranged from 46°to 51°) were slightly bigger than those reported for a similar device in [37] on a desktop context (40°on average), while rotations' amplitudes were smaller (66°on average). This shows that on-body tangible interaction offers similar interaction possibilities than its desktop counterpart. • Rotations in the vertical pose were larger than the two other poses at the cost of an increased fatigue. Therefore, these gestures are prime candidates for short interactions, such as controlling menus (limited number of sub-items) or activate toggles (two modes, discrete two-state tasks).
Theoretically, our design space describes a vocabulary composed of 30 continuous interactions: 12 1D-Translation gestures (3 poses * 4 POM), 9 1D-Rotation gestures and 9 1D-Roll gestures (3 poses * 3 POM each). The three-part study conducted in this work showed that it is possible to perform all the translations and rotations identified in our design space. Thus, our approach interaction vocabulary offers a rich and large set of possibilities to carry data visualization tasks. However, it is necessary to choose the right mapping between the possible On-Body Tangible controls and the tasks, which raises several questions: What is the optimal mapping that allows the user to keep a good balance between exploiting a great number of the possible interactions and memorizing them? Is it better to allow the user to create his own mapping, or should a predefined mapping be proposed to him? Answering these questions is not possible without an evaluation of the two approaches, which is one of the perspectives of this work. In the following sections, we introduce both of these approaches through a usage scenario for controlling space-time cube operations, and by introducing a meta-UI developed to help users assign OBT interactions to tasks.

Illustrative Application
In this section, we present an illustrative use of OBT interaction in the context of spacetime cube data visualization. It is firstly used to select the appropriate space-time cube operation (e.g. scale); Then it is used to execute the selected operation (e.g. adjust the scaling ratio). In the first case we illustrate how the operations can be assigned to OBT interactions in a predefined manner; In the second case we illustrate how OBT interactions can be dynamically assigned to different aspects of the operation execution.

Space-Time Cube Operations
Visualizing and interacting with spatio-temporal datasets is rather difficult, as illustrated by the continuous research dedicated to this topic. One of the reasons for this difficulty is the important number of operations that can be performed on these datasets as well as the complexity of these operations. Bach et al. [2] introduced a taxonomy of elementary space-time cube operations to organize them. Bach's taxonomy encompasses interactive operations that are relevant to all visualizations representing a combination of time (1D) and space (2D). The taxonomy is composed of four main groups of operations: extraction; flattening; geometry transformation and content transformation. As per Bach's descriptions [2]: • Extraction: consists in selecting a subset of a space-time object (e.g., extracting a line or cut from a volume). • Flattening: consists in aggregating a space-time object into a lower-dimensional space-time object (e.g., projecting a volume onto a surface). • Geometry transformation: consists in transforming a space-time object spatially without change of content, i.e. without affecting the data presented. • Content transformation: consists in changing the content of a space-time object without affecting its geometry.
In the following subsection, we demonstrate how our OBT interaction approach can be used to support the selection of one of these operations (Sect. 7.2) and we present a different use of OBT interaction to support their execution (Sect. 7.3).

Mapping of OBT Interactions to Elementary Space-Time Cube Operations
In this section, we illustrate how a predefined mapping can be built between a set of operations and the 30 available OBT interactions, to allow selecting one of the spacetime cube operations.
Available OBT Interactions. We used the following subset of gestures from of our interaction vocabulary: • The three main poses: Forward, Vertical and Parallel, each corresponding to one line on Fig. 6 • Each pose has three places of motion: Elbow, Middle and Wrist, each corresponding to one column in Fig. 6 • On each Place of motion, we decided to use discrete instead of continuous tangible actions and retained the following tangible actions: -2 Translations: a translation can occur in two directions around the forearm (clockwise, counterclockwise) (red arrows in each cell of Fig. 6).
-4 Rolls: a roll can be performed in two directions (towards the hand or towards the elbow). In each direction, 2 thresholds have been defined so that a roll in each direction can be assigned to 2 different commands (green dots in each cell of Fig. 6). • As a validation mechanism, we used a physical button positioned on top of the device. A total of 54 commands can be selected with such vocabulary (6 per place of motion Â 3 Places of Motion Â 3 Poses).
Mapping Space Time Cube Operation to OBT Interactions. Bach's taxonomy [2] lists 4 groups of general operations. Each one contains several specific operations: extraction (16 operations); flattening (4 operations); geometry transformation (9 operations) and content transformation (13 operations). As the flattening operations are performed directly on the cube, we added them to the geometry transformation group (performed directly on the cube too). This resulted in a new group of operations that we called "Geometry transformation and flattening". Regrouping these operations resulted in 3 balanced groups of operations. We mapped each group of operations to a pose (see Next, we mapped each sub-group to a place of motion on the forearm (Fig. 6-POM), and finally, the specific operations to one of the possible tangible actions (translate or roll). As an example, to select the task Rotation pitch (Fig. 6), the user puts his forearm in the Vertical pose, places the tangible object on the wrist POM, and performs a small roll towards the hand before validating by pressing the button on the device. In this case, the mapping is predefined and appropriate feedback will have to be displayed to the user to confirm the selected operation. Of course, one important issue in this process is to adopt a coherent, easy to learn mapping. Alternatively, the choice of the mapping may be left to the user. In the next section we illustrate the use of a MetaUI to help the user establish himself the desired mapping between operations and OBT interactions.

A Meta UI Supporting User's Assignments of OBT Interaction to Operation
The use case in the previous section focused on the use of OBT interaction for the selection of space time cube operations. After the command is selected, the next step consists in applying a tangible action of the device (translation, roll or rotation) to control the execution of the space-time cube operation. To facilitate this second step, we implemented a tool (Fig. 7) that allows the user to quickly define his own mapping between the available OBT interactions and the operations to control. The metaUI includes a mapping panel (Fig. 7A). Here the user chooses a pose, place of motion and gesture, and assigns to it a function i.e. controlling one aspect of the visualization operation selected (e.g. value of the operation filtering; starting time of the operation Volume-Chopping; …). The MetaUI also includes a simulation panel (Fig. 7B) that allows to simulate the defined mapping through a set of UI elements. The results of the simulated OBT interaction are applied directly to the visualization. This tool is demonstrated in the supplementary video.
In this paper, we proposed, described and studied a new approach for interaction with immersive environments: on-body tangible interactions. This approach is based on the use of the forearm to support tangible interactions using a rounded object that offers multiple degrees of freedom. It takes advantage of the body's natural capacity to sense its own body parts (proprioception) without switching user's attention from the data and minimizing the fatigue. We proposed a design space describing the Pose and the Place of Motion of interaction. We conducted a study to confirm the stability of the forearm to support tangible interaction, to measure the range of motion of the physical manipulations and measure user fatigue. The results showed that on-body tangible interactions are a promising approach to interact with immersive environments since the interaction support (forearm) is stable, the tangible object combined with the different poses and places of motions offers a large number of degrees of freedom, and users found the approach comfortable. Finally, we illustrated the possible usage of this approach through a concrete illustrative application concerning the selection of spatiotemporal operations. This example was thought to demonstrate several ways of using OBT: with a predefined mapping or with a user defined mapping (i.e. with the Meta UI). Our hypothesis is that the user defined mapping will help to recall the commands and ultimately, improve data visualization exploration. However, further experiments need to be conducted to measure the impact and benefits of each approach.
The next step of this work consists in exploring the technological feasibility of our approach. As it is, the tangible object integrates an IMU to detect rolls and rotations autonomously, while the arm pose and the object translation are detected with a camera-based tracking system. This approach is robust and satisfied the objective of our work, i.e. evaluate the advantages of OBT interaction. To implement a viable technical solution, we plan to explore the use of a bracelet to detect the arm pose, and the use of the accelerometer of the IMU to detect the object translation.
Once we develop this integrated solution, we plan to investigate other relevant research questions, such as the performance (in terms of precision and time) or the memorization of OBT gestures. Indeed, while our current approach allows to map a very large number of commands, it increases the difficulty of learning them. To solve this limitation, one solution is to provide suitable feedforward and feedback mechanisms, which will help with both discoverability and recall of mappings. Moreover, the memorization of OBT gestures could also benefit from semantic and spatial aids of the body places considered in our design space, as demonstrated in previous work [19]. Finally, it will be interesting to explore the usage of this approach under different frames of reference: while the egocentric approach seems to better fit this type of interaction, the effects of using a world-centric reference should be evaluated in a controlled study.