Vision system with parameter calculus for gripper correction.

Nica, Marius ; Ganea, Macedon

Abstract: Vision systems work in real time in order to rapidly solve problems in flexible manufacturing systems. The paper presents some image processing strategies, which are used to compute a robot gripper position in order to handle large objects as for example wooden pallets, which are usually manipulated in automated packaging lines.

Keywords: artificial vision, image convolution, Fast Fourier transform, Radon transform, robot gripper.

1. INTRODUCTION

In a previous work by the same authors, the main issues for a vision system were stated, which has the goal to calculate rotation of a robot gripper for a proper grip of a wooden pallet in a flexible packaging line (Barabas & Vesselenyi, 2004), (Vesselenyi, 2004). This has been achieved using the Radon transform. In this paper the second major problem, the translation or position correction of the gripper, is solved.

Methods had been considered, such as convolution in space domain, frequency domain and some special designed methods adapted to solve this particular problem.

In order to pick up the pallets in a proper way, the gripper has to be positioned at the middle of the pallet, above the center of gravity. After solving the rotation of the gripper, the image of the pallet is represented in figure 1.a and 1.b.

The pallets are considered to be in a stack of 4 pieces one on top of the other. The most probable case is that the pallets are not properly positioned one on top of the other, but with some rotation and translation differences. There is also a possibility that the whole stack is not in the right place.

For a good grip of the robot arm on the pallets, piece by piece, it is necessary to know the position of the center of gravity of the pallet in two coordinates x and y of a Cartesian coordinate system, which in image processing is assimilated with the index on lines and columns of the image and is translated further in coordinates after the image is calibrated. So the aim of this work is to find a method to calculate the position of the center of gravity of a pallet from its acquisitioned image after the rotation of the pallet had been found. (figure 1.a and 1.b).

In a future research, the authors would like to establish a visual method by which they can determine the possible overlay of the objects that will be gripped by the robot arm, and also, vertically, the calculus of the parameter correction.

[FIGURE 1 OMITTED]

2. SPACE AND FREQUENCY DOMAIN CONVOLUTION CONSIDERATIONS

Linear image processing is based on the same two techniques as conventional DSP: convolution and Fourier analysis (Otsu, 1979), (Parker, 1997), (Castleman, 1996).

A serious problem with image convolution is the enormous number of calculations that need to be performed, often resulting in unacceptably long execution times. In some works on image processing, strategies for designing filter kernels for various image processing tasks are presented. Two important techniques for reducing the execution time are also described: space domain convolution and FFT (Fast Fourier Transform) convolution. Even though the Fourier transform is slow, it is still the fastest way to convolve an image with a large kernel filter. The two-dimensional version of FFT is a simple extension of the one dimensional FFT. Given an acquired image and a known pattern, the goal is to find out what is the most effective way to locate where the pattern appears in the image. The solution to this problem is correlation (a matched filter) and it can be implemented by using convolution.

Before performing the actual convolution, there are some modifications that need to be made to turn the target image into a kernel. This is the rotation by 180[degrees] to undo the rotation inherent in convolution, allowing correlation to be performed.

3. IMAGE PROCESSING ALGORITHMS

In order to perform the pallet center computation, 3 methods had been tested.

At first we have selected two images with different pallet positions in order to be able to test the algorithms (figure 1.a and 1.b).

The first algorithm to be tested was the space domain convolution. For this method the kernel showed in figure 2.a had been generated and 2D convolution had been applied to the test images with this kernel.

For this purpose, the 'conv2' MATLAB function was used. The convolution result for test image in figure 1.b, is shown in figure 2.a. It's clear that this method can not produce good results because the white area in the resulted images covers a large space, and on this basis, there is no chance to compute the coordinates of the center of pallet.

[FIGURE 2 OMITTED]

[FIGURE 3 OMITTED]

Frequency domain convolution is a little bit harder to achieve. At first we have to apply the 2D fast Fourier transform for the test image and also for the kernel (shown in figure 2.b). This kernel is the same as the one for space domain convolution, except that it's padded with zeros, to obtain the same size as the test image. After obtaining the transformed images the results are multiplied following the rules of complex matrix multiplication. A MATLAB program was developed to do the job. The results are presented in figure 3 for test images. This result shows clearly that the positioning of pallet image is somehow achieved on y direction (line wise), but it's nearly impossible to compute on x direction (column wise).

So, we have concluded that the presented methods are failing to achieve good results in pallet position calculation.

In order to solve the problem, we have adopted another approach, which seemed to be more simple. We tried to locate the center of the pallet separately on each direction. So computing the sum of pixels column-wise we can obtain the 1D graph presented in figure 4.a) and computing the sum of pixels in the same direction of the kernel the graph in figure 4.b. had been obtained.

We have then computed the convolution product of the two sums resulting the graph in figure 5. It can be observed that the graph is showing a clear maximum at point x=166 (after the convolution, half the length of kernel signal must be extracted both from the beginning and end of the graph).

[FIGURE 4 OMITTED]

[FIGURE 5 OMITTED]

In order to find the center point of the graph line-wise we had also computed the sum of pixels on that direction, resulting the graph presented in figure 5.a. Appling a threshold of value 10 we can compute the vector of values that are higher than this, [v.sub.rez], with the MATLAB expression (1).

[v.sub.rez] = find(sslin>10) (1)

To find the value of the center of this region, which matches the center of the pallet on this direction we use (2):

[y.sub.cen] = [v.sub.rez](1) + ([v.sub.rez](length([v.sub.rez]))-[v.sub.rez](1))/2 (2)

we find the coordinate y = 137.

Marking the point x=166 and y=137 on the test image, will yield the point presented in figure 7, which demonstrates that the computed point is indeed in the center of the pallet.

[FIGURE 6 OMITTED]

[FIGURE 7 OMITTED]

4. CONCLUSIONS

Comparing results obtained by applying the shown algorithms we can conclude that the space domain and frequency domain convolution are not suitable to find the center of the pallet in such images.

More adequate to solve this problem is the method of summed projection developed by the authors, which can find the center of the pallet with a fair enough accuracy to be used for robot gripper corrections.

For industrial use, the algorithms found and tested in MATLAB language can be translated in other languages like C++ or Delphi.

5. REFERENCES

Barabas, T., Vesselenyi, T.(2004). Drive and programming industrial robots, University of Oradea, ISBN 973-613-497-0, Oradea

Castleman, K. R. (1996). Digital Image Processing, Prentice Hall, ISBN 0133980588, Englewood Cliffs, NJ.

Otsu, N. (1979). Systems, Man, and Cybernetics, vol. 9, no. 1, pp. 62-66

Parker, James R. (1997). Algorithms for Image Processing and Computer Vision, pp. 23-29, John Wiley & Sons, Inc., ISBN: 0-471-14056-2, New York

Vesselenyi Tiberiu (2004). Automations of Operations for Metallographic Analysis, Phd Thesis, Polytechnic University of Timisoara, Timisoara