Low cost vison system for machine attendance.

Staroveski, Tomislav ; Majetic, Dubravko ; Udiljak, Toma 等

Abstract: The paper presents the development of the low cost machine vision system based on open source libraries. The system is to be used for pick and place tasks in machine attendance or assembly operations. Application, which runs under Linux based operating systems, and uses hierarchical edge matching algorithm, is developed for this purpose. Experimental results show that object recognition robust to illumination changes can be achieved in partially occluded images, with average computation time of 6 seconds, using inexpensive hardware components. Results also show potential of open source systems as platform for similar industrial applications.

Key words: Machine Vision, Object Recognition, Open Source, Pick and Place

1. INTRODUCTION

In production engineering, machine vision (MV) systems are widely used for two different tasks: inspection (quality control), and "pick and place" tasks, applied in machine attendance or assembly operations.

There are many commercial MV systems available in the market today, with adequate overall performance for those tasks, but their cost, along with relatively narrow selection of required hardware during selection phase is usually the limiting factor. The main goal behind this work is to develop inexpensive MV system, based on open source libraries, for machine attendance or assembly purposes, with reliability yet sufficient for industrial applications. In order to achieve such goal, this paper is focused not only to selection of adequate algorithm, but also to the industrial implementation of such system in whole.

2. TASK ANALYSIS

The problem which we selected for the MV system is briefly described as follows. Parts are cyclically being transported to the robot workspace using conveyor belt or similar transport device. At that moment, a vision system is triggered, which performs recognition and localization of arbitrary number of rigid parts. Results of scene analysis, which consist of information about part positions and orientations is then sent to the robot control unit running application specific program. Finally, robot picks parts and puts them on their designated positions.

Parts may differ by their geometric and surface characteristics. They may also be in contact with each other, or overlap to some extent, which is often a case in boxed packaging or on a palette. Because the MV system is to be applied in a real industrial environment, movements of other objects can cause shadows and hence, influence the light conditions. In addition, optical surface characteristics of the same type of part may differ, e.g. as an effect of corrosion.

Motivation for this particular task is found in fact that, unless the parts (work pieces) to be picked are in ordered state, machine attendance systems still require human intervention at least at some point of process, which can be eliminated by implementation of adequate MV system.

3. SELECTION OF MV LIBRARIES

MV libraries provide support for image processing and analysis. As such, they are essential for application development. There are several open source MV libraries available for download under GPL license. It is important to state that those libraries come without any warranty that they will work adequately, as stated in (GPL, 2007).

We did not manage to test all available libraries. Instead, only libraries which have been maintained for longer period of time were considered, since it is reasonable to assume that those would have been in advanced state of development and as such, with lesser bugs. Considered libraries are shown in table 1.

Final selection was made with respect to similar projects in which one of those libraries have already been implemented. In that sense, Mimas MV library was selected, as Mimas was originally developed as a part of Miniman project (Miniman, 2007).

4. SELECTION OF RECOGNITION ALGORITHM

Many strategies have been developed for recognition tasks. These approaches can roughly be divided into 2D and 3D. 2D approaches are preferred, because it is usually too costly or time consuming to create 3D CAD model. In addition, when compared to 2D, computational time for 3D techniques is usually much longer, and they generally require additional image acquisition hardware. For this reason, our consideration was narrowed down only to 2D approaches.

In most 2D approaches, recognition process is conducted by matching 2D models to scene images. Such process is usually conducted in two phases: offline and online. The model is generated from image of object during offline phase. This approach provides fast and practical way of configuring MV system for object recognition of arbitrary rigid shaped objects. In online phase, the model is systematically compared to the image using all allowable degrees of freedom of the chosen class of transformations, e.g. affine, or arbitrary projective transformations (Steger, 2001). The comparison is then based on suitable similarity measure whose maxima or minima are used to decide weather an object is present in the image and to determine its pose. In order to speed up recognition process, search is usually done in coarse-to-fine manner, e.g. by using image pyramids.

The simplest class of object recognition methods is based on gray values of the model and image itself, and uses cross correlation (Brown, 1992) or sum of absolute differences as similarity measure (Lai & Fang, 1999). A more complex class of object recognition methods does not use the gray values of the model or object itself, but uses object's edges for matching. Two example representatives of this class are hierarchical chamfer matching (HCM) (Borgefors, 1988), and the Hausdorff distance (Rucklidge, 1997). Finally, another class of edge based object recognition algorithms is based on the generalized Hough transform (GHT) (Ballard, 1981). More detailed review of these methods can be found in (Urlich & Steger, 2001).

The algorithm robust to illumination changes, which we selected for this task, is called Multiresolution chamfer matching (MCM), originally proposed in (Miniman, 2007), where it was successfully used for recognition of lens. This technique is similar to HCM algorithm, with exception of similarity measure, which is in this case cross-correlation. Canny's algorithm is used for edge detection, (Canny, 1986).

5. EXPERIMENTAL SYSTEM

Industrial robot AdeptSix 300 is used for part manipulation. Standard PC (P4@2GHz with 512MB RAM) running Linux 2.6 kernel (Ubuntu 6.06), equipped with inexpensive consumer 0.8 megapixel webcam is used as platform for MV system. Camera is mounted on top of robot gripper, as shown in figure 1. Advantage of mounting camera on robot is in possibility of expanding system to perform multiple tasks using single camera.

[FIGURE 1 OMITTED]

Experimental software based on MCM algorithm for recognition of arbitrary number of objects is developed. In order to achieve optimal results for each separate part, edge detection and recognition parameters are separately loaded for each object. This information is provided in configuration file, and is being processed during offline phase of process. In online phase, triggered cyclically every time when new parts are in position, recognition process is done in successive manner, i.e. one part is being recognized after another. Communication between robot and MV system is established using network interface (NFS protocol). This process is shown in figure 2.

[FIGURE 2 OMITTED]

6. CONCLUSION

Inexpensive MV system, which is entirely based on open source has been presented. Although this work is still in early stage, overall stability of the process, robustness to illumination changes, as well as to occlusion in some extent shows promising results.

In general terms, inability to detect multiple objects of the same type can be considered as disadvantage, but since robot can manipulate only one object at the time, it is possible to reconfigure the system in a way that recognition process is triggered each time robot takes away recognized part, and thus overcome this drawback. There is also a good chance that the first part to be detected is also the one which is least occluded. Picking that particular part first reduces possibility of damaging robot during gripping operation.

The future work will be conducted on testing of currently implemented algorithm on a larger number of objects and will include investigations of other methods and algorithms for object recognition.

7. REFERENCES

Ballard, D. (1981). Generalizing the Hough transform to detect arbitrary shapes. Pattern Recognition 13(2), pp. 111-122.

Borgefors, G. (1988). Hierarchical chamfer matching: A parametric edge matching algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence 10(6), pp. 849-865.

Brown, L. (1992). A survey of image registration techniques, ACM Computing Surveys 24(4), pp. 325-376.

Canny, J., (1986). A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 8(6), pp. 679-698.

GNU General Public License (2007), Available from: http://www.gnu.org/licenses/gpl.html Accessed: 2007-08-10.

Intel Open Source Computer Vision Library, Available from: http://www.intel.com/technology/computing/mtl/index.htm, Accessed: 2007-05-10

Lai, S. & Fang, M. (1999). Robust and efficient image alignment with spatially varying illumination models, IEEE Proceedings of Computer Vision and Pattern Recognition, Vol. II, pp. 167-172.

Mimas C++ Real-time Computer Vision Library, Available from: http://vision.eng.shu.ac.uk/mmvlwiki/index.php/Mimas Accessed: 2007-07-01.

Miniman Public Final Report (2002), Technical Report (Esprit Project No. 33915), University of Karlsruhe, Institute for Process Control and Robotics, Germany

Rucklidge, W. (1997). Efficiently locating objects using the Hausdorff distance. International Journal of Computer Vision 24(3), pp. 251-270.

Staroveski, T. (2007), Machine vision system for robot guidance, Diploma thesis, University of Zagreb

Steger, C. (2001). Similarity Measures for Occlusion, Clutter, and Illumination Invariant object Recognition, Proceedings of Pattern Recognition: 23rd DAGM Symposium, Radig, B and Florczyk, S, (Ed.), pp. 148-154, 0302-9743, Munich, Germany, September, 2001, Springer, Berlin

Ulrich, M. & Steger, C., (2001). Empirical performance evaluation of object recognition methods In: Empirical Evaluation Methods in Computer Vision, Christensen H. and Phillips, P. (Ed.), pp. 62-76, 9810249535, IEEE Computer Society Press, Los Alamitos, CA

TINA Computer Vision, Available from: http://www.tinavision.net, Accessed: 2007-05-10

Table 1. Considered CV libraries.

Library Name Supported OS Programming lang.

OpenCV Windows, Linux C++
TINA Linux, Unix C
Mimas Linux, Unix C++