RSD-TR-9-87 The Compliance of an Object in Tactile Sensing by Yi Lu Department of Electrical Engineering and Computer Science The University of Michigan Ann Arbor, Michigan 48109 July 1987 Center for Research on Integrated Manufacturing Robot Systems Division College of Engineering The University of Michigan Ann Arbor, Michigan 48109-2110

The Compliance of an Object in Tactile Sensing by Yi Lu July 10, 1987 Robotics Research Laboratory Electrical Engineering and Computer Science Department The University of Michigan Ann Arbor, Michigan 48109 This work was supported by NASA under contract NAG 2-350.

RSD-TR-9-87 TABLE OF CONTENTS 1. Introduction............................................... 1 2. Sensor Environment............................................... 2 3. The Compliance of an Object......................... 4......... 4 4. An Algorithm for Computing the Compliance of an Object..... 8 5. Experiments and Discussion............................................... 11 6. Conclusion................................................................................... 13 7. Acknowledgement............................................... 14 Appendix 1..................................... 16 Appendix 2............................................................................................ 17 Appendix 3.......................................18 Appendix 4.......................................18

Abstract Using a tactile sensor to detect the hardness of an object is discussed in the report. The hardness of an object can be represented by its compliance measurement. A softer object has a greater compliance value than a harder object. An algorithm for computing the compliance of objects by a tactile sensor is presented. The algorithm was tested on a wide range of different objects. The results show that the algorithm is very promising.

RSD-TR-9-87 1. Introduction Recently, the tactile perception has received increasing attention in robotics research. Robotics systems are being built to perform complex tasks such as object recognition, grasping, manipulation and assembly. Robots which are to operate in an unknown or a partially specified environment will require sophisticated perceptual abilities. The most concerted effort in machine perception to date has been in machine vision. Research into the integration of vision and touch has shown that the accuracy and efficiency of machine vision can be greatly improved by adding touch as a verifier [1,2,3]. Psychological experiments suggest that the human tactile system is both fast and accurate to recognize real world objects. A robotics tactile perception system will be an important component in an integrated robotics system and a concentrated effort in the development of robot tactile perception is warranted. Tactile sensing is a good choice for a complementary sensor to vision in a generalized robotics environment for a number of reasons. It has the following advantages over visual techniques [4]: (1) They do not suffer from perspective distortion and most of the other difficulties associated with visual data. Vision is often affected by imaging geometry, surface reflectance, lighting and some other environmental conditions. (2) Tactile sensors are relatively small and light so that they do not interfere with the operation of the robot. (3) Tactile sensors do not have the problem of being obscured by the robot arm or other objects in the workspace. tactile sensing 2

RSD-TR-9-87 The active tactile sensing acquires the information by touching the surface of an object directly. It is capable of detecting the texture, compliance, temperature and slippage of a grasped object [5,6,7]. Such information is useful for locating, identifying and handling objects. The Tactile sensing has some limitations. It is low resolution and can only be applied locally. This report presents our research result on computing the compliance of an object through a tactile sensor system. The hardness of the surface of an object can be measured by its compliance. The compliance of a harder object is expected to be smaller than a softer object. Comparing the compliance values of objects, we are able to determine the types of the material of the objects. Stanfield in University of Pennsylvania introduced an approach for computing the compliance of an object. His approach is very simple, but, it does not work in many situations. [7] The work we presented here is part of the ongoing sensor integration project at the Robotics Research Laboratory in The University of Michigan. 2. Sensor Environment The tactile sensor used in our Robotics Research Laboratory is the LTS-200 from Lord Corporation. This device contains both a tactile sensor array and a gross load sensor. The tactile sensor array is composed of 160 sensitive sites. The sites are organized as a 10x16 orthogonal array with 0.071 inches center-to-center spacing between each site. Each site monitors the deflection of a small portion of the touch surface. The gross load sensor measures the forces and torques being applied to the touch surface. tactile sensing 3

RSD-TR-9-87 The force and torque are measured along X, Y and Z axes. The load capacity is no more than 160 lbs at the center of the touch surface and 401bs at the edge. The LTS200 tactile sensor system has a microprocessor based data acquisition system to support the sensors. It accepts the simplified commands from the host and takes the necessary step to complete the desired operation. The sensor is capable of the following commands: Scan Array: to measure the deflection of the tactile surface at each sensitive site. Scan vector: to measure the forces and torques acting on the tactile surface. Scan site: to measure the deflection of a single site on the tactile surface. Vector thresholding: to scan the vector continuously until the force components reach the specified threshold. It returns the current value of the vector. The tactile sensor is mounted on one of the two fingers of a gripper on a PUMA robot arm. An interface between the sensor and the Apollo work station is available. All data processing and high level computation can be done in C under UNIX operating system on Apollo. The sensor interface program on Apollo provides user all the sensor commands and gives user the control of the motion of the gripper on the PUMA robot arm. The gripper can open up to 4.5 inches wide. The current opening distance of the gripper can be read directly from the sensor interface. There is a pressure control applied on the gripper. The pressure is controlled manually. tactile sensing 4

RSD-TR-9-87 3. The Compliance of an Object The hardness of an object can be measured by its compliance. The compliance of a harder object is expected to be smaller than a softer object. In the tactile sensor application, we can measure the deflection at each sensitive site of the touching surface of the sensor, the gross forces and the torques over the touching surface of the sensor. Because of the inaccuracy of the hardware environment, the deflection, the forces and torques read at a different time for the same object surface may not have the same value. Hence, a tactile sensing based method for computing the compliance value of an object bears the following problems: 1. a tactile sensing based method for computing compliance can't give a constant compliance value to the same object at every measuring; 2. for the two objects with the same surface material, a tactile sensing based method may not give the same compliance value to these two objects. However in robotics environment, we only want to know the hardness of an object in comparison with a set of other objects. Even in the human environment, human touching can only sense one object being harder(softer) than others. We want to design an algorithm which computes the compliance values for all objects in a set. By comparing the compliance values of the objects, we are able to rank the hardness of these objects. The algorithm should satisfy the following two criteria: 1. The rank of the hardness of the objects in the set should coincide with human touching sense. tactile sensing 5

RSD-TR-9-87 2. We should get the consistent result from the different runs on the same set of object. Let 0 = ( 01, 02, **. Ok ) be a set of objects with different surface materials, C(Oi) be the compliance value of object Oi, i=l,...,k. Criterion 1 tells us that if we have C(Ojl) > C(Oj2) >... > C(O1j), then object Oj i1l should be softer than 0J,, where 2<ik. Criterion 2 assures that when the algorithm runs on the same set of objects repeatedly, or a set of objects with the same surface material as the other set, we should have the consistent result from the algorithm. In another words, let 1 = { O l, 0 1,..., Ok1 } be another set of objects where Oil has the same surface material as i,where'l<i:. If the algorithm produces the result on O as C(Ojl) > C(Oj2) >... > C(Ojk), then the algorithm should produce the following result on 01 C(ojI ) > C(Oij2) >... > c(oL ). From the study of many experimental results, we have found some factors which are influenced strongly by the hardness of an object. These factors constitute our measurement of the compliance for an object. We shall start with the analysis of these components. The complete algorithm for computing the compliance for an object based on a set of objects will be presented. The relaxation component C 1. For every object, we define an initial distance, do, and a stablized distance, d6. do is defined as the greatest opening distance of the gripper that it touches more than tactile sensing 6

RSD-TR-9-87 50% of the object surface. When the opening distance of the two fingers of the gripper decreases, the gripper holds the object tighter, and more pressure is applied on the object. In general the force we read from the sensor increases with the applied pressure. When the gripper reaches a certain opening distance, the force will stop increasing. This opening distance of the gripper is the stablized distance, d,, of the object. The relaxation component, C 1, is defined as the difference of ds and do, C1 = do- ds. If an object has a harder surface, it has a smaller value of C 1; a softer object has a greater value of C 1. We use this fact to recognize the material of an object. If the hardness of two different materials are close, C1 is not able to give a good measurement. When two objects have different thicmkness, C1 alone sometime cannot give us an accurate measurement on the hardness of the surface. But C1 can be considered as an important component for measuring the compliance of an object. The force component C2. Let Fo indicate the force we read from the initial opening distance of the two fingers do, and Fs indicate the force we read from the stablized distance ds. Fo and F. may also be referred to as initial force and stablized force correspondingly. We define the force component C2 as the difference of Fs and Fo, C2 =Fs - Fo. Many experimental results show that usually the initial force of an harder object is closer to its stablized force, hence an softer object has a greater C2 value than a harder object. C2 is not much effected by the thickness of an object. tactile sensing 7

RSD-TR-9-87 The force and relaxation component C 12. The force and relaxation component is a combination of C1 and C2. We define C 12 as follows: C12 = w 12C1 + C2, where w 12 is a weight and can be computed in the the following way: C2 W 12 = where C m is the minimal value of C2 and C 1m is the minimal value of C1. The initial force component C3. Many experimental data indicate that the initial force, Fo, is a very good measurement for the hardness of an object. The initial force for a softer object is smaller than a harder object. The initial force component C3 is defined as the reciprocal of Fo: C3 F Fo The compliance C Both C12 and C3 are good components for measuring the hardness of an object. We can define the compliance of an object by these two components. But we notice that the difference between the magnitude of C12 and C3 is very large, a standard normalization is required. Let C 1m be the minimal value of C12 obtained in the experiment, C3 in be the minimal value of C3 obtained in the experiment, n1 and n2 are the two integers such that tactile sensing 8

RSD-TR-9-87 1 < C *10' < 10, 1 < Cmin *10n2 < 10. The compliance of an object can be defined as: C = 10n'C12 + 10"C3 4. An algorithm for computing the compliance of an object The following algorithm computes the values of the compliance for a set of objects. The output of the algorithm are the compliance values of all the objects in the set and the weights, W12, nl and n2, which are used in the computation. The compliance values rank the hardness of the objects in the set. An object has a smaller compliance value has a harder surface. The weights, W12, n 1 and n2 can be used late to compute the compliance value of a new object. Let's call the object set from which we computed these weights as the base object set. When we want to compare the hardness of an unknown object with the objects in the base object set, we can use the tactile sensor to get do, d., Fo and Fs values of the unknown object. From do, ds, Fo, F. and the weights, W12, nl and n2, from the base object set. From these data, we are able to compute the compliance of the unknown object. We can determine the hardness of the unknown object by comparing its compliance value with the compliance value of each object in the base object set. The algorithm tactile sensing 9

RSD-TR-9-87 Input: A set of objects, O = {o1, 0 2, —.9 Ok) Output: weights: W12, nl and n2; the compliance of each object oi. Step 1. [Compute do and ds, F[ and Fs for each object oi.1 For each object oi, do the following steps: Step 1.1. Open the two fingers of the gripper to its maximum distance and initialize the sensor. Step 1.2. Close the gripper gradually until the sensor touches more than 50% of the object surface. Copy this distance as do and the force as Fl. Step 1.3. Close the gripper again until it reaches a distance that the force does not increase any more. Copy this distance as ds and the force as Fs. Step 2. [Compute the relaxation component C' and the force component C/ for each object oi.] For i=l to k compute: = - C' = Fs - Fo, Step 3. [Compute the weight W12.] Step 3.1. For i=l to k: search the minimum value in C1 and store it in C1 mn; search the minimum value in Ci and store it in C2ri Step 3.2. W12 = tactile sensing 10

RSD-TR-9-87 Step 4. [Compute C12 for each object oi.] For i=l to k compute: C12 =W12C1 +C2 Step 5. [Compute C' for each object oi.] For i=l to k compute: Step 6. [Compute weights nl and n2.] Step 6. [Compute weights nl and n2.] Step 6.1. For i=l to k search for the minimum value in C12 and store it in C12; search for the minimum value in C 3 and store it in C 3 Step 6.1. Compute nl and n2 such that: 1 < C " *10n < 10, 1 < Cin*10n2 < 10. Step 7. [Compute the compliance C' for each object oi.] For i=1 to k compute Ci = 0n 1*C12 + 10n2*C3. Step 8. Print out the weights, W12, nl and n2. Step 9. Print out every C' value of object oi in the decreasing order. A discussion on the implementation of the algorithm. tactile sensing 11

RSD-TR-9-87 The force we read from the tactile sensor is a vector with X, Y and Z components. In the implementation of the algorithm, we compute the force by averaging the absolute values of the three force components. We use the deflection array to determine the initialized distance do. The fingers of the gripper initially is at its maximum distance and then they close to each other gradually. Before the sensor touches the surface of the object, every element in the deflection array is 0. When more than 50% elements in the deflection array are greater 1, we define this distance of the two fingers as the initial distance do. The stablized distance is a little more complicated. Since the measurement of the force is coarse, sometimes the force at distance di smaller than the force at dil; but at distance di+*, the force may jump to a much greater value, where di > di+l > di+,. Such situation happens particularly when the object is soft and Fi is close to Fo. One example is the sponge in experiment 2 in Fig. 2. Our experiments show that we always have F, - Fo > 200. So we compute the d3 by the following statements: if (Fi+1-F, )<3 and (Fi -Fo) > 200, then the distance at Fi is ds and store this Fi as F,. 5. Experiments and discussion We have performed some experiments on the following objects: a sponge, a folded towel, an eraser, a hard cover book(h-book), the same book without the hard cover(book), a piece of wood and a piece of metal. All these objects have flat surface and the surface is bigger than the sensor surface. The algorithm has been applied on this set of objects twice. The results are shown in the tables of Appendix 1, 2, 3 and 4. Appendix 1 and Appendix 2 show the force measurement for each object from the tactile sensing 12

RSD-TR-9-87 first and second run respectively. In these tables, di is the opening distance of the two fingers of the gripper and d4Sdi~do, and Fi is the force read at di, and F~Fi:~F,. Appendix 3 and Appendix 4 show the results of each components of the compliance and the compliance for each object from the first run and second run respectively. In the first run of the algorithm, we have weights, W12 3= - 23, 10i' 10-2, 1O" = 103. In the second run of the algorithm, we 429 = 28, o = l02, 10n2 = 103. The tables in Appendix 3 have weights, W12 = -1 and Appendix 4 give the values of the different compliance components and the compliance from the two runs. If component C1 is used to measure the hardness of the object, we will get the following result from the first run: C l[sponge]>C l[towel]>C l[eraser] —C jh-book]>C 1 Ibook]C l[wood]>C l[metal]. From the second run, we have: C l[sponge]>C I[towel]>C I[eraser]>C l[book]=C l[h-book]>C l[metal]>C l[wood]. From the results of the two experiments, component C1 alone tells us that metal and wood are the harder than the rest of the objects; book and eraser are harder than folded towel and sponge and sponge is the softest object. C1 can't tell any further information, so it is not a sufficient measurement. Upon the values of component C2, we can tell metal is the hardest object, h-book is harder than the book, metal, h-book and the book are harder than eraser, sponge and towel. C2 alone is not a good measurement for the hardness of an object. The force and relaxation component gives us a satisfactory result. The results from both run show: C l[sponge]>C 2[towel]C 3[eraser]>C 3[book]>C 3h-book>C 3[wood]>C 3[metal]; tactile sensing 13

RSD-TR-9-87 at the second run, C 3[sponge]>C 3[towel]>C 3[eraser]>C 3[book]>C 3[wood]>C 3[h-book]>C [metal]; On the whole, C3 does give us quite good result except the little confusion between'wood' and'h-book'. The results from both runs show that compliance C is a good measurement for the hardness of an object. The results are consistent in both runs and coincide with human sense: C[sponge]>C[towel]>C[eraser]>C[book]>C[h-book]>C[wood]>C[metal]. 6. Conclusion This report has given a detailed analysis on the measurements of the hardness of objects and presented an algorithm for measuring the compliance values for a set of objects by the tactile sensing data. The experimental results show that the algorithm works well over a wide range of various objects and it meets the criteria proposed in section 3. Frequently in robotics environment, we have a set of objects as the models in the system. By applying the proposed algorithm, We can achieve the compliance values of the system models along with the weights W12, nl and n2. We can use these weights and perform Step 1.1, 1.2, 1.3, Step 2, Step 4, Step 5 and Step 7 to get the compliance value for an unknown object. By comparing the compliance value of the unknown object with the compliance values of the system models, we can determine the hardness of this unknown object. tactile sensing 14

RSD-TR-9-87 7. Acknowledgement I would like to thank those people who contributed their time and effort to support the sensor environment within which this research was done, especially Chris Born, Robert Giles and Ron Theriault. I would also like to thank Prof. Ramesh Jain and Dan Murphey for their advice and comments concerning this research endeavor. tactile sensing 15

RSD-TR-9-87 Appendix 1. The result of experiment 1 Object: metal Position di 100 95 90 85 80 Force Fi 519 752 819 869 873 Obiect: wood Position di 200 195 19 1185 180 175 Force Fi 415 943 1064 1091 1101 1098 Object: h-book Position di 195 190 185 180 125 120 115 160 155 Force Fi 362 1 633 859 969 1 1016 1 1021 1029 1034 1033 - Object: book Position di 175 170 165 160 155 150 145 Force Fi 98 354 697 919 957 968 972 Object: eraser Position di 145 1405 130 15 30 125 120 115 110 105 Force Fi 52 210 383 591 795 898 926 934 936 Obiect: towel Position di 160 155 150 145 140 135 130 125 120 115 Force Fi 49 36 61 82 114 162 220 309 419 565 Position di 110 105 100 95 90 Force Fi 725 864 912 924 927 Obiect: sponge, Force Fi 16 34 34 37 37 59 68 66 73 86 Position di 80 75 70 65 60 55 50

RSD-TR-9-87 Appendix 2. The result of experiment 2 Obiect metal Position di 100 95 90 85 80 75 Force Fi 459 817 870 879 888 9887 _ Object: wood Position di 200 195 190 185 180 Force Fi 329 943 1116 1153 1157 Obiect: h-book Position di 195 190 185 180 175 170 165 160 Force Fi 395 703 902 962 979 986 996 997 Obiect book Position di 180 175 170 165 160 155_i 150 145 Force Fi 156 309 623 871 930 943 } 950 950 Object: eraser Position di 145 140 135 130 125 120 115 110 105 Force Fi 74 253 424 622 784 881 902 910 910 Obiect: towel Position di 150 145 140 135 130 125 120 115 110 105 Force Fi 33 45 74 116 168 250 361 500 677 840 Position di 100 95 90 85 Force Fi 909 930 938 939 Position di 135 130 125 120 115 110 105 100 95 90 85 Force Fi 19 30 29 33 45 41 59 55 62 62 99 Position di 80 75 70 65 60 55 50 45

RSD-TR-9-87 Appendix 3 CI, C2, C12, C3 and C from experiment 1. Pressure=40 metal wood towel eraser sponge h-book book C1 15 20 65 35 75 35 25 C2 350 686 875 882 803 672 870 C12 710 1166 2435 1722 2603 1512 1470 C3 0.00193 0.00241 0.02041 0.01923 0.0625 0.00276 0.0102 C 9.03 14.07 44.76 36.45 88.53 17.88 24.9 Appendix 4 C1, C2, C12, C3 and C from experiment 2. Pressure=40 metal wood towel eraser sponge h-book book C1 20 15 60 35 85 30 30 C2 429 824 905 836 814 601 794 C12 909 1184 2345 1676 2854 1321 1514 C3 0.002179 0.00304 0.0303 0.0135 0.05263 0.00253 0.00641 C 11.27 14.88 53.75 30.27 81.17 15.74 21.55 REFERENCE [1] Peter K. Allen, "Sensing and Describing 3-D Structure," International Conference on Robotics and Automation, pp. 126-131, 1986. [2] Peter C. Gaston and Tomas Lozano-Perez, "Tactile Recognition and Localization Using Object Models: The Case of Polyhedra on a Plane," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-6, No. 3, pp. 257-266, May, 1984. [3] John L. Schneiter, "An Objective Tactile Sensing Strategy for Object Recognition andLocalization," International Conference on Robotics and Automation, pp. 1262-1267, 1986. tactile sensing 18

3 9015 03483 8279 RSD-TR-9-87 [4] M. R. Driels, "Pose Estimation Using Tactile Sensor Data for Assembly Operations," IEEE International Conference on Robotics and Automation, pp. 1255-1261, 1986. [5] David Siegel, Inaki Garabieta and John M. Hollerbach, "An Integrated Tactile and Thermal Sensor," International Conference on Robotics and Automation, pp. 1286-1291, 1986. [6] R. E. Ellis, "A Multiple-Scale Measure of Static Tactile Texture," International Conference on Robotics and Automation, pp. 1280-1285, 1986. [7] S. A. Stansfield, "Primitives, Features, and Exploratory Procedures: Building A Robot Tactile Perception System," International Conference on Robotics and Automation, pp. 1274-1279, 1986. tactile sensing 19