Successful development of body extension projection mapping technology for the use in everyday space
Touch-screen terminal-operable projected magic hands
A group of researchers led by Professor SATO Kosuke and Associate Professor IWAI Daisuke (Graduate School of Engineering Science, Osaka University) developed “body extension technology” which can be applied to everyday situations by making projection mapping of computer graphic hands in the real world and moving the images by operating a touch-screen tablet.
In most conventional body extension technologies, a user wearing a special camera and gloves activates limbs of a character in a computer game according to the user’s movements, so usage was limited to virtual reality space. However, this group’s technology makes it possible to operate hand models projected indoors with a tablet, making it easy for one to experience body extension in the real world.
Body extension using body models has been studied in the field of virtual reality and is now seen in situations in which a user intuitively operates a character in a computer game according to the body movement of the user. Particularly, in researching the operation of virtual character’s hand motions in response to the user’s hand movement, special gloves have been used to detect how fingers are bent, and infrared sensors have been used to recognize hand gestures.
This research group, led by Prof. Sato, developed a body-extension type interface called ExtendedHand, an interface available by making project mapping hand models in the real world and using them as the user’s extended hands. Prof. Sato’s research in the past showed that users felt like they were moving their own body when they operated projected hand images on a desk through gesture recognition with a camera.
This group developed technology to estimate a hand’s location and how much fingers are bent by using a tablet touch panel. This technology has made it possible for one to operate hand models with a general device without being influenced by an indoor light source.
This technology has no positional limits. Users neither need to operate while limiting movements of hands within the detection range of camera nor wear gloves or set infrared sensors, allowing them to operate hand models with a general terminal. This technology will make it possible to provide a new type of communication aid by getting at distant objects or to remotely controlling sensor-equipped home electrical appliances and -room environment by using projected hand images. It is anticipated that this technology will play active role in society where the IoT (Internet of Things) offers benefits.
This group made a presentation on this technology on March 4, 2016 at the Information Processing Society of Japan Symposium Interaction 2016, which was held at Science Museum in Tokyo from March 2 through March 4, 2016.
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Related links
