In Proceedings of the IEEE/ASME Intl Conf on Advanced Intelligent Mechatronics (AIM'99), Atlanta, GA, September 1999.
Abstract– As embedded microprocessor intelligence penetrates ever deeper into the world, it is often accompanied and represented by user interfaces with limited control and display capabilities. One of the possibilities offered by haptic (force and tactile) feedback is a more direct and human mode of bidirectional information transfer: when well designed and situated it can provide an aesthetically satisfying as well as functional handle onto elements of the digital world. This paper describes the design of an experimental platform,“Aladdin”, which integrates a haptic display into a commonplace manual control, a doorknob. The system’s mechatronic design includes a haptic knob with torque and thermal display, a high quality auditory display, and sensing and actuation of other door elements. Low-latency integration of the haptic and auditory displays, crafting of both the haptic and auditory content and retaining a high degree of control over auditory output were emphasized. Scenarios for the door’s use are described, and results of preliminary experiments with our prototype are discussed.
Index Terms – haptic interface, tangible, multimodal, multisensory, manual control, smart user interface
I.M OTIVATION AND B ACKGROUND
The blinking 12:00 on the home VCR long ago became an icon for the union of microprocessors and consumer product user interfaces. During the last fifteen years, compute power has grown according to Wolfe’s law, but the majority of so-called “smart” consumer devices for sale today are not easier to use. The usability problem may become even more severe with the networking of embedded computing. When the refrigerator and the thermostat start talking to each other, their owner will have to comprehend and manage parameters ever more abstract and distant.
The correlation between impenetrable interfaces and powerful computing is no coincidence. Given a faster engine and more RAM, it is easy to make an appliance do more; widening and clarifying the window of human communication with that appliance may be harder. More information must travel through an inadequate funnel of membrane keypads and hierarchical touchscreen menus. The limited cues that these interfaces offer often do not help users in forming mental models of how devices work.
The authors are Members of Research Staff at Interval Research Corporation, 1801 C Page Mill Road, Palo Alto, CA 94306, USA, {maclean, roderick}@interval, www.interval.The haptic sense relates to touch, and includes the perception of force, texture, temperature and moisture. It is coordinated and physically, neurally co-located with motor functions and allows manipulation as well as sensation, such that it is sometimes described as the one “bidirectional” sense. Haptic feedback provid
es an avenue through which some kinds of information and cueing can be most effectively offered. In addition to improving the ease-of-use of some kinds of tasks, it can be an aesthetic and engaging experience, bringing lost tangibility back into the new electronified world. It is particularly useful in situations where there is a need for continuous or dexterous manipulation; other perceptual channels are unavailable; there is a desire to notify with discretion; or the communication is of an affective or fuzzy nature.
Most commercial haptic displays today, as well as the bulk of academic research, are interfaced to desktop computers for navigating graphical environments and computer games [5, 7, 14]; surgical simulators [12]; and aids for manufacturing and design [2, 6, 8]. There has been less attention to the integration of active haptics into everyday manual controls [4, 8], and it has been largely devoted to simulating systems that exist in the real world. Our vision is to embed small, high quality, customized haptic interactions into everyday environments, increasing usability and pleasure in many points of contact with the digital world.
II.A LADDIN: O BJECTIVES FOR P ROTOTYPE
This project was inspired by questions encountered by Interval’s haptics group. How will people react t
o “live”objects in their environments, and will initial oddness eventually become accepted? Is there a tactile language that people will understand as they do visual grammar? People are highly tuned to reading visual images and understanding their underlying message – could our haptic sense become thus tuned with practice? Can movement through mediated touch be as engaging and as fun as other modes of expression? How finely can we craft an interface’s feel? We think of Aladdin as an enchanted doorknob, a connection to another space. It can communicate something of what is on the other side and put its own personality into the experience. It combines sophisticated one-dimensional haptic feedback with crafted auditory and visual design to convey information about an environment. However, Aladdin is also a platform for multiple agendas:
Smart Tangible Displays in the Everyday World:
A Haptic Door Knob
Karon E. MacLean and Jayne B. Roderick
Embedded Physical Interaction – lodging active haptic displays in architecture and objects, to make the technology behind the interface more usable and engaging. Multihaptics– extending the haptic display palette beyond force and vibration, to resemble the richness of skin connecting with the real ph
ysical world.
Multisensory – merging haptic feedback intimately with displays for other senses, to enhance the expressiveness of the whole experience.
Tactile Language – communicating abstract, expressive and narrative qualities as well as shape rendering.
Gestural display – conveying information visually through knob movement when it is not being touched.
Immersion – achieving presence in or attention to the other space without an exoskeleton or head-mounted display.
III.A LADDIN S YSTEM D ESCRIPTION
Aladdin is a sensed and actuated door and knob providing a threshold to an interior or remote space, which might be a private or shared room, a home or professional building. This space can also become part of the system and must be sensed according to the door’s communicative function. A.Overview
The engineering prototype built here and pictured in Figure 1 is a subset of the desired functionality. Prior to building a full-sized door mediating access to a real space, we wished to explore the technical issues in haptic embedding and some elements of the ensuing interaction experience. Space constraints imposed two principal deviations from an ideal experimental prototype, and these made it less useful as a behavioral experiment. Since it is a half rather than a full door, a user can feel the knob, open the door and listen, but cannot walk through. Secondly, the door is not the entrance to a real space, and the real space is not sensed. We were nevertheless able to get a sense for user reactions by creating scenarios and using them to generate new ideas for how it or other embedded haptic interfaces might work. Figure 2 shows the prototype’s functional layout. A custom-built half-door on a pedestal is conventionally hinged. Its angle of opening is sensed with an encoder in the hinge, and it is locked with a computer-controlled solenoid latch. Its knob is the active haptic display with two components: torque feedback through actuation with position readback on the knob’s axis, and thermal display in the body of the knob with a thermoelectric module. The auditory display is a pair of speakers mounted at head level on either side of the door. Power and signal lines are routed from the computer and power supply in the pedestal, up into the frame and through the hinge to reach the knob.
A single Pentium II CPU orchestrates sensing and control while traversing a programmed interactive n
arrative. Transitions and branches are triggered by a user’s actions –e.g., his first touch, his push against a dynamic resistance, or how sharply he moves at a key moment.
With the exception of the thermal display, building these display and sensory elements into the door was a straightforward albeit nontrivial job. However, we simplified the problem by housing the computer in a
pedestal and custom-building the door: to retrofit such a Figure 1: The Aladdin
engineering prototype is a
half-door with a fully
functional knob.
Figure 2: Schematic layout of prototype.
system into an existing door set in a wall (truly embedding it) would be much more involved. For example, we did not have to tear into existing walls to access the installation. We were able to use a desktop computer, which speeded development and reduced the number of elements we had to locate within the door and walls themselves. Likewise, we did not in this prototype worry about power usage and point of supply; but in a real installation, would want to minimize power needs for cost, cooling and ecological impact.
B.Haptic and Gestural Display
Haptic and gestural interaction occur through the controlled force, motion and temperature of the knob. Aladdin’s haptic output is created with torque applied to the knob’s rotational axis combined with a temperature-controlled knob surface. The display can be operated with the thermal knob or with arbitrary and much simpler non-thermal knobs that permit a greater range of motion. The knob can also be moved when untouched, as a visual or gestural display.
Figure 3 is an exploded CAD drawing showing how the mechanical elements fit together.
Actuation of Knob vs. Door: Haptic feedback is provided through the knob rather than at its hinge. An actuated knob can give interesting and perceptible feedback with relatively small torques levels, whereas the door would require much more power. More importantly, a knob is safer: gone unstable it merely whirls, whereas an errant swinging door can cause real damage.
1)Torque Display
Torque feedback occurs through a direct-drive DC motor with a position encoder, servoed at 1 kHz (Figure 3-5). The principal challenges in locating it within a door were disguising its length, greater than a normal door’s depth; and routing electrical connections from the computer in the pedestal to the motor. The motor’s length is accommodated with curved, spun aluminum plates on both sides.
Choice of Motor and Transmission: The entire project becomes more realistic as actuation size, expense and amplification needs shrink. Output torque of 250 mNm generates a stiffness of about 10 N/mm at the knob circumference. Other specifications were high bandwidth force response, low backlash and a compact and simple design. Their combination steered the design away from a gearbox reduction, and precluded the use of a cable drive. Thus this prototype employs a direct drive configuration with an oversized DC motor (20W Maxon).
Position Sensing: Position is read with an encoder on the motor’s shaft for a knob angular resolution of .090°. Range of Motion: When the thermal display knob is installed, the handle’s range of motion is limited to 180° to accommodate electrical connections using software stops backed up by mechanical stops. The thermal knob and mechanical stops can easily be removed and replaced with an arbitrary knob with unlimited range of motion. Stability: Because of their different inertia, we use different control gainsets for the aluminum thermal knob and for other knobs, constructed of wood or foam. The thermal knob is harder to stabilize and its haptic performance sometimes lower. Since it seems important to keep the thermal display on the moving element, this situation continues to be a constraint and needs to be revisited.
2)Thermal Display
The thermal element is the most experimental aspect of the haptic display, since this medium has been less used than force/torque or vibration in haptic research. As have others [1, 13], we use a thermo-electric module (TEM) as a heat pump and integrate it directly into the handle (Figure 4). A TEM operates as a reverse thermocouple, creating a temperature gradient across its surface in response to a voltage; one surface can thus be made colder than ambient temperature, but the other surface then becomes warmer. The TEM was chosen to raise the display surface at least 15°F above
and below ambient temperature within 30 seconds peak-to-peak. In informal tests, this is achieved with an annular model manufactured by Melcor that produces a peak transfer of 3.7W at 3.9A.
Heatsinking and Grasp Surfaces: The TEM’s warm side must be cooled with either airflow or heat sinking. This posed a challenge, since we wanted the user to feel only one temperature rather than the two juxtaposed; and we could not put a finned heat sink on the knob for safety reasons. The solution is a compromise. The TEM is buried in the tip of a thermally conductive aluminum body with a smooth heatsink (less effective than finned, but safer) on the proximal knob surface (Figure 3-3). Part of the heatsink surface can be touched by a user’s hand; but we designed the shape and dimpled features of the knob (Figure 4b) to encourage a fingertip grasp on the front, controlled surface. The distal, temperature-targeted region of the thermal knob (Figure 3-1) can be driven about 15 degrees F above and below ambient within 15-30 seconds, registering as clearly “hot” or “cold” without burning or frosting a user’s hand. Since we intend for temperature to represent slow room environment changes, this response time was adequate.
active下载1
5
3
2
4
6
Figure 3: Exploded drawing of thermal knob, actuator and cutaway mounting in door. (1) is the thermal controlled surface.
(2) is the heat pump, source of thermal output. (3) is the TEM’s rear heatsink. (4) is the heat pump’s power and signal conduit.
(5) is the DC torque motor. (6) is a cutaway of the front door surface. The entire knob downstream of the motor shaft (elements 1-4) turns as a single rigid unit.
Electrical Path: The TEM’s location on the rotating knob increases complexity and limits range of motion. Current and sensor leads must travel inconspicuously from the rotating knob to the door. We limit rotation to 180 degrees and route the wires through a conduit in the shaft (Figure 3-3). Power mag
nitude, number and quality of signals and expense ruled out slip rings. We are investigating other non-contact means of signal transmission.
TEM Sensing and Control: Two thermocouples aid in the TEM’s control. The first, mounted directly on the TEM inner (heatsink) surface, is a threshold alarm: when its temperature exceeds a dangerous level, the computer controller cuts power. The second is located on the touched (display) side of the TEM, bonded to the inside of the aluminum knob shell. It registers the temperature of the touched knob surface, and closes the TEM control loop.
3) Knob Shape Design
The knob’s shape and texture are critical to the prototype’s success. We have designed many non-thermal knobs for different contexts, adhering to the following criteria.Grip Orientation: shape and texture should encourage the user to approach and grasp it in the position intended by the designer, for correct perception of the haptic feedback.Load Minimization: the knob should not invite power grips that might induce the user to fight the motor.
Safety: no external protrusions that might abrade or cut the user’s hand in the event of instability; and it should have no pinch points. This implies a symmetric knob shape.Visual Asymmetry: some knobs u
se asymmetric texture,color patterns or small concave shape features to emphasize programmed motion when viewed from a distance.C. Additional Sensing and Control
Door Position: The door’s position is monitored with a low-resolution (9°) encoder in one of the door’s hinges.Door Latch Control: The door’s locked and unlocked state is computer-controlled with a solenoid latch, allowing the doorknob to mediate entry to the space behind the door.Knob Touched: It is important to know whether the knob is being touched, both for control mode switching and for shifting context. This is presently inferred by observing the knob’s non-actuated motion; but the group is working on other more reliable and versatile hardware methods.
D. Auditory Display
The auditory display is crucial to the Aladdin narratives. It relies on tight realtime coupling between the haptic and auditory displays; small (1024Kb) audio buffers are swapped directly by the main control CPU before routing through an audio board. The buffers are updated at 88 Hz and the audio process communicates with the main control process at the same rate. This results in low latency in modifying the auditory signal in response to manual user interactions. The CPU limits this rate; but while 88 Hz is well below perceivable auditory frequencies (44 kHz), we have found it an acceptable response time fo
r modifying an auditory event. The audio process communicates with and reacts to the haptics process at its own update rate, 88 Hz.A number of effects (volume, speed of play, equalization,distortion, reverberation) are applied to the auditory output in realtime. Together with the auditory and haptic content,they are expressive design tools for crafting the overall interaction. For example, turning the knob against a textured resistance might reduce an audio feed’s distortion,eliciting the impression that the knob brings the sound into focus. In the current implementation, we aesthetically pre-process the auditory material, culled from a variety of sources. In later prototypes, the audio may be processed
with dynamic filters from live pickups inside the space.
(a) assembled to show surface detail
1
23
(b) disassembled to reveal the (1) touch surface, (2)thermoelectric module and (3) internal heatsink.
Figure 4: Thermal display in haptic knob.
E. Realtime Software Architecture
Our software architecture is designed to enable tight synchronization between haptic sensing and display and other continuous media such as audio and video, providing the sensation of direct control and rich, dynamically configurable behavior, and is used in numerous applications in our lab. In addition to achieving high performance, we wanted to make it easier to author complex and interesting haptic experiences. We use the QNX realtime operating system to guarantee response times (sub-millisecond for the haptics process) while also permitting multiple clocked threads and interprocess communication.The architecture has the following key elements:• Multiple processes running at different refresh rates.• Two-way communication via shared memory, message
passing and remote serial/Ethernet protocols.• Bulk of computation on the haptics side of the model
for tightest feedback with user.Figure 5 shows the architecture’s general structure. Event management – e.g. transitions in the narrative, cued in response to user actions – is handled within Aladdin’s haptics process, because it runs fastest (1 kHz) and has earliest access to sensory information on which transitions are based. The audio control process is treated as a client since it does not trigger control events. This location of “master” control within the haptics process, rather than a slower-running graphics or audio process, is where our model departs from most current convention for functional distribution in multi-process haptics control code. It is a direct consequence of our priority of tight coupling; and enabled by employing simple virtual models. Further details of the software architecture may be found in [11].
Hardware and Servo Details: This prototype is implemented on a 350 MHz P2 desktop computer. In later implementations, we envision embedding the control functions used here in local processors. The principal CPU user here is the audio controller, which could be migrated to a DSP to reduce system load.
The torque feedback control loop is updated at 1 kHz, the auditory output at 88 kHz and the remaining sensing and control functions at 20 Hz. In these early implementations,
the torque feedback is accomplished simply using PI force control when the handle is restrained by a user’s hand, and PID position control when it is not. The thermal display is closed with a PID loop.F. Room Instrumentation
With this prototype, the user must imagine the room behind the door. In the future we plan to sense properties of the room it guards so a version of this information can be conveyed or otherwise used by a visitor. What we want to know depends on the kind of a room it is, what tends to happen inside, and who “owns” the space. For example, the door might be the transition from a hallway to a studio, a private office or a company meeting room; or from the street to the front door to a home or public building.Interesting parameters might include the number of people inside; their collective mood (listening, arguing, laughing,working quietly); the physical environment (temperature and air quality); time since last activity or presence; or the wishes of the person or group who own the space (do not disturb, visitors welcome). In the future some of this may be derived from analysis of audio pickups, probably the simplest and least disruptive method of sensing available.
IV. A LADDIN N ARRATIVES
We implemented a set of narratives to situate Aladdin,ranging from the pragmatic to the whimsical. A few are described here: they have been implemented to various degrees, and offered informally to users to gauge reaction and input. No formal user studies have been performed at this time, but a mor
e extensive discussion of the tactile language we sought to develop can be found in [10].Our approach was inspired by observing how a dance refers to the world, combining and connecting different modes [3]. One mode usually predominates, and these fluidly exchanged contexts, conveyed by the haptic and auditory displays in conversation with the user, can assume many forms. As in dance, they might directly mimic something in the world, resemble one aspect or replicate dynamic relationships between real or virtual elements. We seek the underlying elements of a haptic vocabulary that is comprehensible alone or augmented by other senses.The Well House: The doorknob of your hotel room purrs comfortably or sharply jars you to indicate a) on exiting,whether the lights and heat have been switched off and b)on entry, if the room has been disturbed; reassuring you that the room is “at rest”. This feedback occurs at a time and location you are most likely to wonder if you have switched everything off.
Personalized Keyless Entry: A smart doorknob is the ideal spot to check the identity of the person entering a space. A gestural password is used to identify oneself and gain entry.Butler: The Butler prohibits or welcomes entry and accepts a gestural message from the visitor, according to the room owner’s presence or wish; when the room’s owner returns,
she perceives through the handle that someone has been by.
Figure 5: Software architecture, illustrating the primary processes
(haptic and audio managers) and shared memory.
A visitor receives information about activity in a room that can help him decide whether and how to enter – for instance, telling him a presentation is underway which his entry might disrupt.
This scenario explores the language of greeting. We encode and read much about an encounter through body language: the sly glance, the slight hesitation, the warm, firm handshake. Turning the knob is akin to a handshake and can indicate the room’s mood. This strategy most embodies our research into the vocabulary, syntax, stylistic traits, and representations of the world of non-verbal communication. The Malleable Knob: This door is situated at the entrance to an active public space. At the beginning of the day the knob feels strong and cold and sounds metallic. As people pass through the door and handle the knob, it begins to deform and soften. Activity levels through the space are recorded by the changing shape and suggested materiality of the knob, such that the knob is continuously sculptured. After heavy usage the knob feels hot and gummy and sounds like a wet chamois leather; but on a quiet day it remains fresh and crisp late in the afternoon.
V.O BSERVATIONS AND C ONCLUSIONS
This implementation of the haptic doorknob was successful as a preliminary engineering and user-interface experiment. It demonstrated feasibility of integrating a haptic display into a commonplace manual control and pointed out areas that will require engineering iteration. It allowed us to gauge user reaction and generate insight in where the concept might be used. However, to effectively test the concept as a user interface, it must be more appropriately situated as a walk-through entrance to at least one real space used by a variety of people. We hope to implement this in the future.
With this and with other haptic displays created in our lab, we have become attuned to the issue of user perception of “live” interfaces. This becomes an even more serious concern when it is located outside the realm of the desktop computer, where it may be encountered more frequently or casually. Active haptic displays are new and strange enough that the uncertainty of how the interface might respond can make users unwilling to touch it. It is a research and design goal to determine how to alleviate this concern, either by better design or by allowing the user to develop confidence as such interfaces becomes more familiar [9]. People were afraid of automobiles and airplanes early on; in such cases, the value offered by the technology induced users to overcome their trepidation, while manufacturers had incentive to improve safety and perception of safety. This is the situation embedded haptic displays face today.
VI.A CKNOWLEDGEMENTS
The authors are grateful to Oliver Bayley for key conceptual input; and to Jesse Dorogusker, Becky Fusion, John Ananny, Brad Niven and Kristina Force for various assistance in construction. The Interval haptics team provided ongoing support and input to the project, and many Interval colleagues helped with informal testing.
VII.R EFERENCES
[1]D. G. Caldwell, N. Tsagarakis, and A. Wardle,“Mechano Thermo and Proprioceptor Feedback for Integrated Haptic Feedbac,” in Proc. of IEEE Int'l Conf. on Robotics and Automation (ICRA), Albuquerque, NM, 1997.
[2]J. E. Colgate, W. Wannasuphoprasit, and M. A. Peshkin, “Cobots: Robots for Collaboration with Human Operators,” in Proc. of the 5th Ann. Symp. on Haptic Interfaces for Virtual Environment and Teleoperator Systems, ASME/IMECE, Atlanta, GA, DSC:58, pp. 443-39, 1996.
[3]S. L. Foster, Reading Dancing: Bodies and Subjects in Contemporary American Dance. Berkeley, CA: University of California Press, 1986.
[4]B. Gillespie, “The Virtual Piano Action: Design and Implementation,” in Proc. of Int'l Computer Music Conf., Auhus, Denmark, pp. 167-170, 1994.
[5]Haptic Technologies, PenCATä High Performance Haptic Pen. Montreal, Canada, 1998.
[6]J. M. Hollerbach, E. C. Cohen, W. B. Thompson, and S.
C. Jacobsen, “Rapid Virtual Prototyping of Mechanical Assemblies,” in Proc. of NSF Design and Manufacturing Grantees Conf., Albuquerque, NM, pp. 477-478, 1996. [7]Immersion Corporation, The Immersion Feel-It Mouse & I-Force Game Controllers. San Jose, CA, 1999.
[8]K. E. MacLean, Emulation of Haptic Feedback for Manual Interfaces, Ph.D. Thesis, MIT, February 1996. [9]K. E. MacLean, “Application-Centered Haptic Interface Design,” in Human and Machine Haptics, M. Srinivasan and M. Cutkosky, Eds.: MIT Press, 1999 (in press). [10]K. E. MacLean and J. B. Roderick, “Aladdin: Exploring Language with a Haptic Door Knob,” in Proc. of the 12th Ann. ACM Symp. on User Interface Software and Technology (UIST'99), Asheville NC, 1999.
[11]K. E. MacLean and S. S. Snibbe, “An Architecture for Haptic Control of Media,” in Proc. of the 8th Ann. Symp. on Haptic Interfaces for Virtual Environment and Teleoperator Systems, ASME / IMECE, Nashville, TN, DSC-5B-3, 1999.
[12] A. Madhani, G. Niemeyer, and J. K. Salisbury,“The Black Falcon: A Teleoperated Surgical Instrument for Minimally Invasive Surgery,” in Proc. of IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), Victoria, BC, Canada, 1998.
[13]M. Ottensmeyer, “Hot and Cold Running VR: Adding Thermal Stimuli to the Haptic Experience,” in Proc. of 2nd Phantom Users Group Workshop, Dedham, MA, MIT AI Lab TR 1617, 1997.
[14]Sensable Technologies, PHANTOM Desktop 3D Touch System. Cambridge, MA, 1998.
版权声明：本站内容均来自互联网，仅供演示用，请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系QQ:729038198，我们将在24小时内删除。
发表评论