A Wearable Robot As A Behavioral Interface Without Powered Assist

The Parasitic Humanoid (PH) is a wearable robot for modeling nonverbal human behavior. This anthropomorphic robot senses the behavior of the wearer and has the internal models to learn the process of human sensory motor integration continuously, thereafter it begins to predict the next behavior of the wearer using the learned models. When the reliability of the prediction is sufficient, the PH outputs the errors from the actual behavior as a request for motion to the wearer. Through symbiotic interaction, the internal model and the process of human sensory motor integration approximate each other asymptotically.

Fig.1 Wearable sensory devices construct a wearable humanoid without muscle or skeleton.

Fig.2 The symbiotic relationship between the wearer and the Parasitic Humanoid

Strategy for Adaptation to Individuals:

• 1. Do continuous motion analysis.
• 2. Adapt internal model for prediction to the behavior.
• 3. Request for the wearer also to adapt to the model.

Perhaps the ideal interface is one that completely adapts to each individual user. However, it is intractable for artificial devices to learn parameters for adaptation automatically. PH provides a solution by not only adapting to the user but also requesting the user to adapt to its internal model. As a result, the internal model does not require so many complex and unstable parameters.

Self-Organized Mapping for Motion Analysis

In essence, we are designing the nervous system in PH. It uses sensory information to produce a model of body structure by performing self-organized learning. This model performs information integration between motor systems and sensory systems. PH requires human action according to the model. We apply a neural oscillator as the motor command generator, multi-layer perceptrons for the sensory integration of spatial information to generate the next behavior, and Self-Organized Mapping (SOM) neural networks for the segregation of sensory signals in behaviors (Fig.3, 4). Fig.4 is an example of self-organized mapping to segregate walking behaviors in four speeds, such as walking slowly (WS), walking normally (WN), running slowly (RS), and running normally (RN). The inputs for the SOM are joint angles of the knee and hip in the right and left legs. In this case, the cells of the center area have been coding the two states of walking, and the other cells around them have been coding the two states of running.

Fig.3 Paradigm of nervous system of Parasitic Humanoid

Fig.4 Example of self-organized mapping to segregate walking behaviors in four speeds

Symbiotic Behavior by Motion Induction

A request for user motion is not enforced but rather induced by PH. The method of motion induction is based on the concept of entrainment in the neural oscillator [4,5] (Fig.5). The parade march is the typical stimulus. Therefore, our research focuses especially on walking movement based on a neural oscillator model [6]. Using this model, we can induce the walking cycle of the wearer. If arms or insteps are stimulated by cyclic vibration, usually we won't pay any attention to it. But if the cycle of vibration is similar to a walking cycle, the vibration influences the walking rhythm to mimic that of the vibration. Fig.6 is an example of motion induction of a walking sequence. The normal human walking cycle is very stable. The average of the deviation is approximately under 1-2 % of the cycle. In this experiment, the vibration stimuli to the insteps started just after the 30th step. Until the 60th step, the stimuli synchronized the steps of the wearer. Then, the tempo of the stimuli was increased and the cycle of walking followed it. In this experiment, the effective induction succeeded up to approximately 5% of the cycle.

Fig.5 Paradigm of motion induction in a neural oscillator model of Parasitic Humanoid

Fig.6 Example of motion induction in walking sequence: after the 30th step

Prototype Hardware:

The prototype of Parasitic Humanoid is shown in Figure 7. This system consists of vibration motors and electrodes for stimuli as output to the wearer and sensors as input to PH in Table 1, which observe sensory inputs and behavior outputs of the wearer. The total weight of motors, electrodes, sensors and wirings is less than 500g. Signal transmission to the mobile PC uses USB standards. The CPU performances of typical mobile PC's are sufficient for logging the sensory data, controlling the vibrators, and learning the internal neural network models, when the usages are limited to experiments with partial systems of PH (Fig.8).
This system has over one hundred inputs, and less than thirty outputs. As such, the balance of I/O is clearly input heavy, because the internal model is under development and the optimal arrangement of vibrators to attach to the wearer is not clear yet. Several methods of stimulus for motion induction are currently under development and evaluation.

Fig.7 The Prototype of Parasitic Humanoid

Table 1. Sensors in Prototype of Parasitic Humanoid

Wearable Limb Motion Measurement System:

For measurements of limb motion, the PH uses the 3-axis postural sensor manufactured by TOKIN Corporation. This sensor includes three rate gyros, two axis sensors for the direction of gravity, and two magnetic compasses. This sensor has an advantage that the wearer has no devices attached to the joints because this sensor senses not relative postures but absolute postures of limbs. As a result, the influences to behavior by the presence of motion sensors must be minimized. The sampling rate is 100 [Hz], and the angular resolution is 0.1[degree]. The sensors are fixed on the electrodes for stimuli to muscles. Each set of sensor and electrode is sticked on the skin with each supporter band. This installation method makes the position deviations of sensors from the skin by behavioral motions smaller.

Fig.8 Example of Information Monitoring from Sensors

Wearable Eye Movement Measurement System:

Before picking up a cup, we typically look at it first. Moreover, before we move somewhere on foot, we usually look at our destination. Thus, by catching both eye movement and behavior, perhaps we can understand the relation of both. Further, if that relation is known, the next action could possibly be predicted through eye movement. The most important point of PH is to not hinder human behavior during measurement. Our goal is to measure the gaze position in 3-dimensional space from the center of both eyes up to the hand's reaching limits (about 0.7 [m]), according to the angle of convergence, and within an error of 20 [mm]. The sampling rate was set to 100 [Hz], because a rapid sequence of real-time measurements of eye movement and other movement systems (arm position) is important for behavior analysis [7]. From the relation between convergence and gaze position, the accuracy of eye movement detection must be high, i.e. around 0.1 [deg], especially if the gaze position becomes distant.

Fig.9 3-axis Postural Sensors for Limb Motion Measurement

Fig.10 Wearable Eye Movement Measurement System

Fingernail Sensor: To Measure Direct Touch with Fingertip:

We propose a novel fingernail sensor for detecting the touching and bending of the finger. This sensor is placed on the fingernail to avoid hindering the direct touch of the fingertip. The sensor consists of miniature light emitting diodes (LEDs) and photo-detectors, which measure the force at the fingertip and the bend of the finger through the intensity of scattering reflection as the finger's color changes. This instrument is wearable and practical for daily use. Furthermore, since the reflection's measurement is taken from the side of the fingernail, it does not obstruct natural movement of the wearer [8,9,10]. Fig.12 is an example of the response to cyclic contacts (cycle: 1.0[Hz], interval: 0.5[s], contact force: 0.5[N]). The response has sufficient repeatability without delay to detect contacts of the fingertip. We have already applied this technology to Smart Finger [11], which is a new type of display that provides supplementary tactile sensations for augmented reality. For example, someone wearing this small device on his or her fingernail can touch a drawing on a flat sheet of paper and feel a virtual edge along the drawing's line, and virtual roughness from its texture pattern. In this device, the fingernail sensor extracts a force vector due to the various color patterns of the nail corresponding to the contact force direction. It can measure force up to approximately 2[N].

Fig.11 Fingernail Sensor

Fig.12 Example of the response of Fingernail sensor to cyclic contacts

Shoe-Shaped Sensor: To Measure Walking and Standing

Presented here is a shoe-shaped interface designed to induce a specific walking cycle. An individual's walking cycle is measured with a pressure sensor on the sole of the shoe, and a vibration motor attached on the instep stimulates the foot with cyclic vibration. We found that stimulating the instep with cyclic vibration, does not obstruct normal walking movement, and that if the cycle of vibration is similar to that of a walking cycle, the vibration influences the walking rhythm to mimic that of the vibration.

Fig.13 Shoe-Shaped Sensor

Galvanic Vestibular Stimulation:

The vestibular system is sensitive to galvanic vestibular stimulation (GVS) intensity changes and responds by altering the magnitude of the response accordingly [14]. When GVS is delivered to the mastoid through electrodes (Fig.14) during human walking, and subjects respond by deviating towards the anode while walking. For the motion induction in PH, GVS is available to induce the behavioral motion of the direction during human walking and the posture of human standing.

Fig.14 Electrode on mastoid for galvanic vestibular stimulation

Scenarios for Symbiotic Interaction with Parasitic Humanoid :

Ex1. Capturing and Retrieving the Best Behavior

The most direct usage of PH as a behavioral interface is capturing and retrieving behavior. It is not only for scientific research to analyze human behaviors, but also to enhance daily life. When you play golf, you may want to capture and retrieve your best shot. Otherwise you may download the data of the swing from the PH of Tiger Woods. You may exchange data for dance steps like name cards in a ballroom. Behavioral interfaces will make up the new style of communication (Fig.15).

Ex2. Dual Consciousness After Continuous Capture of Sensory Stimuli and Behavior

Consider PH's recording a continuous history of each wearer, and adapting to their individual habits and styles. When the history includes sufficient patterns of the wearer's daily behavior, he can behave as if he has dual consciousness. For example, he may be induced to avoid oncoming cars from behind unconsciously when the PH is aware of their presence (Fig.16, 17). If the PH is linked to a traffic information site, his driving may avoid and suppress traffic jams unconsciously. The wearer doesn't have to pay attention to any annoying verbal information in his continuing behaviors.

Fig.15 Capturing and retrieving behavior with PH

Fig.16 The PH wearer can avoid oncoming cars from behind unconsciously.

Fig. 17 Example of the situation avoiding oncoming a motorbike from behind unconsciously

Conclusion and Future Direction:

We have presented a survey of several interaction techniques and interface technologies that are useful for the design of Parasitic Humanoid. The concept of the design on PH is that the most valuable resource in wearable technology is the body of the wearer. The body is the structural frame work, the resource of sensory information, the display devices for perception, and the actuator for behavior simultaneously. Desktop computers come equipped with large monitors providing high bandwidth output and essentially obviate the need for nonverbal output. Similarly the keyboard does not accept the need for nonverbal input. Wearable computing changes the picture dramatically, since traditional interfacing components are not incorporated as easily. We suggest that this is an ideal area to integrate behavioral information as a primary interface medium. Most of the work we have reviewed in this paper are areas of ongoing research. To conclude we would like to highlight two areas which are particularly challenging but important to consider. Given the personal nature of wearable computers, adaptive interfaces will be especially important. If the computer is to be as natural as clothing, it must be as malleable to the particular whims and idiosyncrasies of each user. Natural interaction methods for interfaces that adapt to the habits and styles of each user is an open area of research, and Parasitic Humanoid will be a solution for the problem. Finally, in the future, PH must not only have an interface to the wearer and the environment, but must have an interface to another computer or network. PH becomes a more adaptive interface to the continuing wearer day by day, because PH is learning the behavioral model of the wearer. As the result, PH is suitable as a medium of nonverbal and behavioral communication. For example a PH could capture and record the swinging motion of a best shot. The behavioral data is useful not only for the wearer to remember the swing, but also for another wearer of PH to learn the better play. Thus the PH must have some behavioral translation to generalize the skill of the wearer.

References:

[1] S. Jacobsen: Wearable Energetically Autonomous Robots, DARPA Exoskeletons for Human Performance Kick Off Meeting, 2001
[2] W. W. Mayol, B. Tordoff and D. W. Murray: Wearable Visual Robots, International Symposium on Wearable Computing, 2000.
[3] S.Tachi, H.Arai, T.Maeda : Tele-Existence Simulator with Artificial Reality(1) - Design and Evaluation of a Binocular Visual Display Using Solid Models, IEEE International Workshop on Intelligent Robot and Systems (IROS'88), Oct. 1988, Tokyo,Japan
[4] S. Grillner: Neurobiological bases of rhythmic motor acts in vertebrates, Science Vol.228 pp. 143-149, 1985
[5] K. Matsuoka: Mechanisms of Frequency and Pattern Control in the Neural Rhythm Generators, Biological Cybernetics Vol.56 pp. 345-353,1987
[6] G. Taga,: A model of the neuro-musculo-skeletal system for human locomotion, Biolog. Cybern. Vol. 73, pp. 97-111,1995
[7] K.Kansaku, S.Kitazawa, K.Kawano: Sequential hemodimamic activation of motor areas and draining veins during finger movements revealed by cross- correlation signals from fMRI, Neuroreport9: 1969- 1974,1998.
[8] R.S.Johansson and G.Westling: Role of glabrous skin receptors and sensorimotor@memory in automatic control of precision grip when lifting rougher or more slippery objects. Exp. Brain Res Vol.56, pp.550-564(1984)
[9] Mascaro.S and Asada.H: Distributed Photo-Plethysmograph Fingernail Sensors : Finger Force Measurement without Haptic ObstructionCDSC-Vol.67, Proceedings of the ASME Dynamic Systems and Control DivisionCpp. 73-80(1999)
[10] Mascaro.S and Asada.H: Photoplethysmograph Fingernail Sensors for Measuring Finger Forces without Haptic ObstructionCSubmitted to the IEEE Transactions on Robotics and Automation (2000)
[11] H. Ando, T. Miki, M. Inami, T. Maeda: The Nail-Mounted Tactile Display for Behavior Modeling, Conference Abstracts and Applications of SIGGRAPH 2002, San Antonio, U.S.A., (2002.7)
[12] T. Kohonen, J. Hymminen, J. Kangras, and J. Laaksonan. SOM PAK: The Self-Organizing Map program package. Technical Report A31, Helsinki University of Technology, 1996.
[13] Yoshikuni Nomura*1 and Taro Maeda*2: The Study of Fingernail Sensors for Measuring Finger Forces and Bending, Journal of The Virtual Reality Society of Japan, Vol..6 No.3 (In Japanese)
[14]Bent LR, McFadyen BJ, Merkley VF, Kennedy PM, Inglis JT. Magnitude effects of galvanic vestibular stimulation on the trajectory of human gait. Neurosci Lett. 2000 Feb 4;279(3):157-60