Professional Documents
Culture Documents
I. INTRODUCTION
T is possible to communicate with a virtual environment
via a haptic interface. An operator using the haptic interface can feel force sensations from the virtual environment and
can in turn provide force and position information to the virtual environment. Unlike the traditional interface using visual
and audio cues, the haptic interface is unique, as it provides a
bidirectional interaction between a human being and the virtual environment [1][3]. Therefore, the haptic interface is a
key input/output device for communication with highly realistic sensations and has the potential for use in many application
areas.
One of the application areas for the haptic interface is virtual
training systems in the medical field, manufacturing industry,
and other fields. For example, during surgical training, medical
doctors use various surgical tools, such as scissors, tweezers, and
surgical knives, and they must train with these tools to master
specific procedures or techniques. However, it is neither easy
nor safe for this training with surgical tools to occur in a real
environment, and thus, a training system that uses virtual reality
(VR) and a haptic interface technology has been researched
aggressively. With the construction of a virtual training system,
training can be safely carried out and a trainee can practice
in various situations that might be difficult to experience in the
real world. Furthermore, the results of some studies indicate that
such a system could increase the skill with which real surgery
can be performed [4] and contribute to the learning of real motor
skills [5], [6].
Based on the need for such systems, many researchers have
been developing tool-type haptic interfaces [7][13]. For example, Okamura et al. [7] developed a scissors-type haptic interface. It has two degree of freedom (DOF) of motion and force
feedback: one for cutting, namely, the single blade rotation, and
one for translational motion of the device. Their group also presented an analytical model to compute force applied to scissors
during cutting of a slab of material [8] and evaluated the cutting
model using the aforementioned scissors-type haptic interface.
Sato et al. [9] developed a brain retractor-type haptic interface
to train surgeons in brain surgery and investigated the soft tissue pushing operation using the haptic device for simulation
of brain tumor resection. In another study [10], a microscissortype haptic device was developed, which presented the cutting
resistance forces to the operator. Goksel et al. [11] developed a
needle-type haptic device and a probe-type haptic device, and
a haptic simulator for prostate brachytherapy with simulated
needle and probe interaction. The use of haptic technology in
medical simulators has attracted attention for many years, and
various commercialized products are already available, such as
LapSim [12], LAP mentor [13], and others. By using a tool-type
haptic interface in a virtual environment, an operator can carry
out virtual training, while feeling force sensations; however,
these tool-type haptic interfaces present the force sensation of
only the corresponding single type of tool. To present the force
sensations of a variety of tools, many tool-type haptic interfaces
are required, which requires multiple installation locations and
costs a great deal.
For this reason, we previously developed a haptic system
that presents the force sensations of a variety of tools [14].
This system consists of a multifingered haptic interface robot
named Haptic Interface Robot (HIRO) and numerous tool-type
devices, including a surgical knife, scissors, and syringe. HIRO
has five haptic fingers, and a variety of tool-type devices can
easily be attached to and removed from HIROs haptic fingers,
enabling the system to present the force sensations of many
tool-type devices. However, we must consider how many haptic
fingers need to be connected to the tool-type device and which
ENDO et al.: DEVELOPMENT OF TOOL-TYPE DEVICES FOR A MULTIFINGERED HAPTIC INTERFACE ROBOT
69
TABLE I
SPECIFICATIONS OF HIRO III
Fig. 1.
holder.
Multifingered haptic interface robot. (a) HIRO III and (b) a finger
TABLE II
SPECIFICATIONS OF HIRO IIIS FORCE SENSOR
70
Fig. 3.
Fig. 2. HIRO with surgical knife. (a) HIRO with surgical knife. (b) Tool
devices steel sphere. (c) Surgical knife and operators hand.
not correspond to the DOF of the overall system. In this case, the
mobility index M [19] implies the DOF of the overall system,
as follows:
m
li
(1)
M = 6(N m 1) +
i=1
(2)
For item 2), to control all DOF of the overall system, the
number of active joints D must satisfy
D M.
(3)
ENDO et al.: DEVELOPMENT OF TOOL-TYPE DEVICES FOR A MULTIFINGERED HAPTIC INTERFACE ROBOT
p
WB =
(4)
det(B T B)
i [e(q i a i ) + e(q i b i ) ]
71
= 12, and m
i=1 li = 18 (nine revolute joints and three spherical joints). The DOF of the knife-type device Fto ol is 6, and
condition (2) is satisfied. The number of active joints D is nine,
because three haptic fingers are used to hold the device, and
thus, condition (3) is satisfied. Although conditions (2) and (3)
are satisfied when the number of haptic fingers is four or more,
we used three haptic fingers in our device, which is the minimal
acceptable number, at the connection.
(5)
B. Kinematics of the Knife-Type Device
(6)
i=1
pto ol + b Rto ol to ol pC i = b pF i + b RF i F i pC i
(7)
(8)
D F i = [I 33 , ( RF i
b
Fi
pC i )] R
36
(9)
(10)
(12)
72
TABLE III
PARAMETERS IN CONNECTION ANALYSIS OF THE KNIFE-TYPE DEVICE
TABLE IV
JOINT ANGLES FOR THE KNIFE-TYPE DEVICE
ol
ol
ol
where D to ol = col[D to
, D to
, D to
] R96 , v to ol =
1
2
3
b
b
6
col[ p to ol , to ol ] R is the velocity vector of the knife-type
device, J C F = diag[D F 1 J F 1 ,D F 2 J F 2 ,D F 3 J F 3 ] R99 ,
and q = col[q 1 , q 2 , q 3 ] R9 .
On the other hand, from the principle of virtual work, the
following relation is obtained:
F to ol = D Tto ol f C
(13)
3
3
(14)
det(D Tto ol D to ol )
(15)
j [e(q i j a i j ) + e(q i j b i j ) ]
(16)
i=1 j =1
Fig. 4. Developed knife device. (a) Knife-type device. (b) Knife-type device
that connects to HIRO.
ENDO et al.: DEVELOPMENT OF TOOL-TYPE DEVICES FOR A MULTIFINGERED HAPTIC INTERFACE ROBOT
Fig. 6.
Fig. 7.
73
74
(17)
Here, note that we cannot obtain the same relation for C1 because of . For the contact at C1 , we can obtain the following
constraint:
b
(19)
(20)
R37
ol
D to
i
(21)
= [I 33 , [( Rto ol2
b
to ol2
pC i )], O 31 ] R
for i = 2, 3
37
(22)
where O 31 R is a zero matrix. As in the case of the knifetype device, we can obtain the relation (11) between the ith
haptic fingertip velocity and the joint angle velocity. Thus, from
(17), (20)(22), (10), and (11), we can obtain the following
kinematics:
3
D to ol v to ol = J C F q
(23)
where D to ol =
R , v to ol =
col[b p to ol , b to ol , ] R is the velocity vector of the tweezerstype device, and J C F and q are the same as in the case of the
knife-type device.
ol
ol
ol
col[D to
, D to
, D to
]
1
2
3
7
97
TABLE V
PARAMETERS IN CONNECTION ANALYSIS OF THE TWEEZERS-TYPE DEVICE
(24)
ENDO et al.: DEVELOPMENT OF TOOL-TYPE DEVICES FOR A MULTIFINGERED HAPTIC INTERFACE ROBOT
75
TABLE VI
JOINT ANGLES FOR THE TWEEZERS-TYPE DEVICE
0.086 0.888
b
Rto ol = 0.286 0.457
0.954 0.057
0.453
0.842
0.293
76
VI. EXPERIMENTS
Fig. 10.
To evaluate the developed devices, we carried out two experiments. One was to manipulate the knife-type device, and
the other was to manipulate the tweezers-type device. In each
experiment, the manipulability-optimized control was used as
the control law of HIRO [25]. This is a mixed control method
consisting of a haptic finger-force control and an arm position
control intended to maximize the control performance index
(27). The force control of the haptic finger is given by
t
F e (s)ds
F (t) = K 1 J TF F e (t) + K 2 J TF
0
is connected to the joint, and all joints are revolute joints, and
its rotational axes are all in the same direction. Let ljto ol be the
DOF of the jth joint of the tool. Then, we obtain kj=1 ljto ol = k.
Now, we set n haptic fingers to be connected to the tool. In this
case, the number of links of the haptic fingers is given as Nn =
3n, and the number of joints of the haptic fingers is given as
n
be the DOF of the jth joint of the
mn = 3n. In addition, let lij
n
ith haptic finger, and we obtain ni=1 3j =1 lij
= 3n.
The haptic fingers and the tool are connected through the passive spherical joint described in Section II, and the number of
n. Thus, the DOF
joints at n contact points is given as mcont =
of the joint at the ith contact point licont is ni=1 licont = 3n.
By substituting these values into (1), we obtain M = k + 6 be1, m = k + m
+ mcont , and
cause of N
= (k + 1) + N
n +
n
m
k
n
3
n
to ol
n
cont
l
=
l
+
l
+
. From
i=1 i
j =1 j
i=1
j =1 ij
i=1 li
this, we found that condition (2) is always satisfied. By considering the relationship between D active joints and k tool
joints, which satisfies condition (3) (namely, D k + 6), we
obtain the following results: 1) When three haptic fingers are
used in the connection, D = 9, and thus, we can connect a tool
with up to three joints to HIRO; 2) when four haptic fingers are
used in the connection, D = 12, and thus, we can connect a tool
with up to six joints to HIRO; and 3) when five haptic fingers
are used in the connection, D = 15, and thus, we can connect
a tool with up to nine joints to HIRO. In addition, if we obtain
the concrete form of the tool and if we derive the kinematics
of the overall system, we can clarify which fingers should be
employed with the given tool by using the performance index
(14).
In the newly developed knife-type and tweezers-type devices,
the use of three haptic fingers is sufficient for the connection
between HIRO and the tool-type device. Thus, the remaining
two haptic fingers that have not connected to the tool-type deice
are fixed in the straight state (for example, see Fig. 4 as the case
of the knife-type device). However, for devices other than the
knife-type or the tweezers-type device, there is a possibility that
four or more haptic fingers are needed for connection between
HIRO and the tool-type device. Since HIRO has five fingers, it
can respond to such a situation, and we believe there is a big
merit and the potential to use a five-fingered hand to manipulate
tool-type devices.
J TF
F d K 3 q f (t)
(25)
3
F di
i=1
T
(26)
+ JA
3
(pi phb ) F di
i=1
CPI = WA + Wq + QA
(27)
1
QA = (q A d q A )T (q A d q A )
2
where and are weighting coefficients, WA is a manipulability measure of the haptic finger (15), Wq is a penalty function
to keep the finger joint angles within the movement range (16),
QA is the penalty function to prevent a large change of the arm
angle, and > 0 is a weighting matrix. Here, a finger/arm that
reaches the limit of the movable range is switched to a position
control to keep the joint angle within the movable range, and the
rest joints of the fingers/arm are controlled by (25) and (26). After returning to within the movable range, the control is switched
back again to (25) and (26). Thus, the force display of HIRO
is accomplished by the haptic finger. Details of the control law
have been shown in [25]. For the experiment, the control PC
ENDO et al.: DEVELOPMENT OF TOOL-TYPE DEVICES FOR A MULTIFINGERED HAPTIC INTERFACE ROBOT
Fig. 11.
Fig. 12.
77
78
Fig. 14.
Fig. 15. Norms of fingertip force in the knife-type devices. (a) Previously
developed knife-type device. (b) Newly developed knife-type device.
Fig. 16.
Fig. 17.
ENDO et al.: DEVELOPMENT OF TOOL-TYPE DEVICES FOR A MULTIFINGERED HAPTIC INTERFACE ROBOT
Fig. 20.
Fig. 18.
Fig. 19. Experimental results for the tweezers-type device. (a) Torque n .
(b) Angle . (c) Times responses of PI. (d) Norms of fingertip forces.
79
80
Fig. 21.
not move. If the participant felt the vibration, the stiffness coefficient K before one-step was the maximum displayable stiffness
coefficient at the damping coefficient, which was set in step 1).
Then, the participant returned to step 1) and set the damping
coefficient D to the next value. Here, the step size of D was
5 Ns/m. The displayable stiffness levels for four participants
are shown in Fig. 21. We observed no large differences among
the different subjects curves, and the maximum displayable
stiffness was about 6 kN/m.
VII. CONCLUSION
We have described a knife-type device and a tweezers-type
device that we developed for the multifingered haptic interface
robot HIRO. The knife-type device represents a tool device
with no joints, and the tweezers-type device represents a tool
with joints. To determine the optimal connection between HIRO
and the tool-type devices, we have proposed the optimal connection method from mobility and singularity points of view.
After we analyzed the kinematics of the knife-type device and
the tweezers-type device, we have developed the devices for
HIRO that satisfied the optimal connections and tested them experimentally. In the experiment with the knife-type device, we
have compared the newly developed knife-type device with a
previously developed one and found that the newly developed
knife-type device has good force transmission ability. In the experiment with the tweezers-type device, we have confirmed that
the operator feels the grasping sensation of the object through
the tweezers-type device, and device has good operability. These
results show the great potential of our tool-type haptic interface
system that is able to present the force sensation of many tooltype devices, as well as the validity of the proposed connection
method.
In this paper, we have developed a knife-type and a tweezerstype device. However, human beings work with many tools. For
example, in the medical field, there are many tools of different shapes and uses, such as surgical knives, scissors, syringes,
and many others; therefore, a haptic interface that can create
the force sensation of many different tools is important for virtual training systems. We plan to develop many other tool-type
devices for use in a virtual training system in future.
REFERENCES
[1] K. Hale and K. Stanney, Deriving haptic design guidelines from human physiological, psychophysical, and neurological foundations, IEEE
Comput. Graph. Appl., vol. 24, no. 2, pp. 3339, Mar./Apr. 2004.
[2] V. Hayward, O. Astley, M. Cruz-Hernandez, D. Grant, and G. Robles-DeLa-Torre, Haptic interfaces and devices, Sens. Rev., vol. 24, pp. 1629,
2004.
[3] B. Hannaford and A. Okamura, Haptics, in Handbook of Robotics,
B. Siciliano and O. Khatib, Eds. New York: Springer, 2008, pp. 719
739, ch. 30.
[4] N. Seymour, A. Gallagher, S. Roman, M. OBrien, V. Bansal, D. Andersen,
and R. Satava, Virtual reality training improves operating room performance: Results of a randomized, double-blinded study, Ann. Surg.,
vol. 236, pp. 458464, 2002.
[5] P. Youngblood, S. Srivastava, M. Curet, W. Heinrichs, P. Dev, and S. Wren,
Comparison of training on two laparoscopic simulators and assessment
of skills transfer to surgical performance, J. Am. Coll. Surgeons, vol. 200,
pp. 546551, 2005.
[6] D. Morris, T. Hong, F. Barbagli, T. Chang, and K. Salisbury, Haptic
feedback enhances force skill learning, in Proc. WorldHaptics, 2007,
pp. 2126.
[7] A. Okamura, R. Webster III, J. Nolin, K. Johnson, and H. Jafry, The
haptic scissors: Cutting virtual environments, in Proc. IEEE Int. Conf.
Robot. Autom., Sep. 2003, pp. 828833.
[8] M. Mahvash, L. Voo, D. Kim, K. Jeung, J. Wainer, and A. Okamura,
Modeling the forces of cutting with scissors, IEEE Trans. Biomed. Eng.,
vol. 55, no. 3, pp. 848856, Mar. 2008.
[9] D. Sato, R. Kobayashi, A. Kobayashi, S. Fujino, and M. Uchiyama, Soft
tissue pushing operation using a haptic interface for simulation of brain
tumor resection, J. Robot. Mechatronics, vol. 18, pp. 634642, 2006.
[10] S. Fujino, D. Sato, K. Abe, A. Konno, and M. Uchiyama, Displaying
feeling of cutting by a micro-scissors type haptic device, in Proc. IEEE
Int. Conf. Robot. Autom., May 2008, pp. 20672072.
[11] O. Goksel, K. Sapchunk, and S. Salcudean, Haptic simulator for prostate
brachytherapy with simulated needle and probe interaction, IEEE Trans.
Haptics, vol. 4, no. 3, pp. 188198, May/Jun. 2011.
[12] Surgical Science Ltd., LapSim. (2012). [Online]. Available: http://www.
surgical-science.com/productsmain/
[13] Simbionix, LAP mentor. (2012). [Online]. Available: http://simbionix.
com/simulators/lap-mentor
[14] H. Kawasaki, T. Mouri, S. Ikenohata, Y. Otsuka, and T. Endo, Multifingered haptic interface robot handling plural tool devices, in Proc.
WorldHaptics, 2007, pp. 397402.
[15] T. Endo, S. Tanimura, and H. Kawasaki, Development of a surgical knife
device for a multi-fingered haptic interface robot, in Proc. 18th Int. Feder.
Autom. Control World Congr., 2011, pp. 64606465.
[16] T. Endo, S. Tanimura, and H. Kawasaki, Development of a tweezers-type
device for a multi-fingered haptic interface robot, in Proc. IEEE/SICE
Int. Symp. Syst. Integr., Dec. 2011, pp. 10061011.
[17] T. Endo, H. Kawasaki, T. Mouri, Y. Ishigure, H. Shimomura,
M. Matsumura, and K. Koketsu, Five-fingered haptic interface robot:
HIRO III, IEEE Trans. Haptics, vol. 4, no. 1, pp. 458463, Jan./Feb.
2011.
[18] J. Colgate and J. Brown, Factors affecting the z-width of a haptic display,
in Proc. IEEE Int. Conf. Robot. Autom., May 1994, pp. 32053210.
[19] K. Hunt, Kinematic Geometry of Mechanisms. Oxford, U.K.: Oxford
Univ. Press, 1978.
[20] L-W. Tsai, Robot Analysis: The Mechanism of Serial and Parallel Manipulators. New York: Wiley, 1999.
[21] J. Merlet and C. Gosselin, Parallel mechanisms and robots, in Handbook
of Robotics, B. Siciliano and O. Khatib, Eds. New York: Springer, 2008,
pp. 269285, ch. 12.
[22] X. Liu, J. Jeong, and J. Kim, A three translational DOFs parallel cubemanipulator, Robotica, vol. 21, pp. 645653, 2003.
[23] C. Gosselin and J. Angles, Singularity analysis of closed loop kinematic
chains, IEEE Trans. Robot. Autom., vol. 6, no. 3, pp. 281290, Jun. 1990.
[24] Z. Li and S. Sastry, Task-oriented optimal grasping by multifingered
robot hands, IEEE J. Robot. Autom., vol. 4, no. 1, pp. 3244, Feb. 1988.
[25] H. Kawasaki and T. Mouri, Design and control of five-fingered haptic
interface opposite to human hand, IEEE Trans. Robot., vol. 23, no. 5,
pp. 909918, Oct. 2007.
[26] H. Kawasaki, Y. Ohtuka, M.O. Alhalabi, and T. Mouri, Haptic rendering
and perception of frictional moment, in Proc. EuroHaptics Conf., 2006,
pp. 201206.
[27] M. Mahvash and V. Hayward, Haptic rendering of cutting: A fracture
mechanics approach, Haptics-e, vol. 2, no. 3, 2001.
[28] T. Chanthasopeephan, J. P. Desai, and A. C. W. Lau, Measuring forces
in liver cutting for reality-based haptic display, in Proc. IEEE/RSJ Intl.
Conf. Intell. Robots Syst., Oct. 2003, pp. 30833088.
ENDO et al.: DEVELOPMENT OF TOOL-TYPE DEVICES FOR A MULTIFINGERED HAPTIC INTERFACE ROBOT
[29] A. Rafiq, F. Tamariz, C. Boanca, V. Lavrentyev, and R. C. Merrell, Objective assessment of training surgical skills using simulated tissue interface
with real-time feedback, J. Surg. Edu., vol. 65, no. 4, pp. 270274, 2008.
Takahiro Endo (M06) received the Dr. Eng. degree from the Tokyo Institute of Technology, Tokyo,
Japan, in 2006.
Since April 2006, he has been with the faculty
of Engineering, Gifu University, Gifu, Japan, where
he is currently an Assistant Professor. His research
interests include haptic interfaces, robotics, and the
control of infinite dimensional systems.
Satoshi Tanimura received the B.S. degree in human and information systems engineering from Gifu
University, Gifu, Japan, where he is currently working toward the M.S. degree in human and information
systems engineering.
His research interests focus on haptic interfaces in
virtual reality and robot control.
81