KNOWLEDGE-BASED ROBOT VISION SYSTEM FOR AUTOMATED PART HANDLING

This paper discusses an algorithm incorporating a knowledge-based vision system into an industrial robot system for handling parts intelligently. A continuous fuzzy controller was employed to extract boundary information in a computationally efficient way. The developed algorithm for on-line part recognition using fuzzy logic is shown to be an effective solution to extract the geometric features of objects. The proposed edge vector representation method provides enough geometric information and facilitates the object geometric reconstruction for gripping planning. Furthermore, a part-handling model was created by extracting the grasp features from the geometric features.


INTRODUCTION
PC-based intelligent gripping systems are characterized by sensor-based perception and fusion, the incorporation of intelligent algorithms into the control systems, as well as the integration of individual subsystem modules.A robot that can 'see' and 'feel' is easier to train in the performance of complex tasks while, at the same time, it requires a less stringent control mechanism than pre-programmed machines do.A sensory, trainable system is also adaptable to a much larger variety of tasks -thus achieving a degree of universality that ultimately translates into lower production and maintenance costs.
Robot vision is a multidisciplinary field of study, and it plays a critical role in robot intelligence.Vision capabilities endow a robot with a sophisticated sensing mechanism that allows it to respond to its environment in an 'intelligent' and flexible manner.Part recognition is a pivotal process in the multi-sensor robot system.The function of recognition algorithms is to identify each segmented object in a scene and to assign a label to that object.Recognition approaches in use today can be divided into two principal categories: decision-theoretic and structural.Decision-theoretic methods are based on quantitative descriptions (e.g.statistical texture) while structural methods rely on symbolic descriptions and their relationships (e.g.sequences of directions in a chain-coded boundary).
For an intelligent gripping system, the object position and orientation, as well as the object's profile, must be identified and represented accurately in real time.An intelligent gripping process is usually accomplished with the involvement of sensors, such as vision sensor, force sensor, and tactile sensors, so that the grasping status is kept in a stable condition.An automatic gripping planning system must be capable of reasoning out the shape of objects within the workspace [1,2].All possible faces must be reasoned out, based on the boundary model extracted from the partrecognition module.A gripping model is accordingly constructed in terms of those faces.Heuristic approaches are effective and efficient to determine the optimal faces to be grasped.This project is focused on the extraction, description, and interpretation of those features that are significant to the gripping-oriented application and capable of facilitating gripping planning.To represent the objects efficiently and effectively, an edge vector expression method was developed such that computational efficiency is increased dramatically.To extract edge vectors, a fuzzy vector tracing method was developed, based on a continuous fuzzy controller and heuristic vector tracing method.Profile traversing and feature extracting were conducted with respect to the closed vector chain generated.The classification algorithm was based on structural technique -also known as syntactic pattern recognition -which considers relationships between features or edges of an object, and deduces the geometry ID in terms of the topology of edge vectors.

ROBOT VISION SYSTEM
Figure 1 illustrates the generic intelligent gripper system, which consists of a robot vision module, a robot control module, and an intelligent gripping module based on http://sajie.journals.ac.za industrial robots.The core of this type of system features machine vision and gripping intelligence.The architecture of a robot vision system may be regarded as hierarchical.The robot vision functionalities may be partitioned into, first, low-level vision, which includes those processes that are primitive and require no intelligence such as image acquisition and image preprocessing; and, second, high-level vision, which includes extracting, modeling, recognition, description, and cognition.In this project, the objectives of the vision system are to produce data that represent the grippingoriented profile, and to identify the geometry.The architecture of the robot vision system implemented in this project is illustrated in Figure 2.
It can be seen in Figure 2 that the robot vision system starts with visual data collection, and ends with a description and interpretation of the object's geometric features.In robot vision systems, geometric feature extraction and representation are the two most important issues to which a solution must be found in terms of the application requirements.
To constitute an intelligent gripping-oriented robot vision system, the following issues must be taken into consideration: • Image acquisition: includes selection of visual sensors in terms of image resolution which the vision system must possess, illumination, and other specifications.
• Image processing techniques: encompass methodologies for real-time http://sajie.journals.ac.za image processing, complexity of computation, and precision.Furthermore, determine whether an object's contour or entire image should be exploited to identify the object, and whether colour or greyscale image will be employed to parse the features of the object.• Irregularity handling functions: deal with uncertain irregularities that may occur on the scene or during the computation.
Computational efficiency is the key to a real-time imaging system.Different forms of the conventional deterministic and programmed models, a good number of algorithms -based upon artificial intelligence, such as neural networks, fuzzy logic, and neuro-fuzzy techniques [4] -have been presented.Some of these have been applied and proven to be significantly computationally efficient in multiple fields of real-time applications, ranging from low-level thresholding to high-level interpretation [5].
Boundary detection is used in the vision system to simplify and greatly accelerate image processing.Owing to certain discontinuities at the boundaries resulting from noise, shallow, and other interferences, edge-linking and edge-thinning were applied, after edge detection, in order to threshold an edge magnitude image and thin the resulting binary image down to a single pixel-wide closed boundary.
Feature recognition and representation is core in object recognition.To grip objects intelligently, object recognition must provide all necessary geometric features, such as area, length, width, centre of gravity, chain of vectors, and object geometry ID for the successive gripping planning module.Inherent features of objects are always invariant with regard to translation, rotation, and scale.Extraction and representation of an object profile are the crucial issues to which a gripping http://sajie.journals.ac.za oriented application must find a solution.In this project, the object profile is represented by the proposed edge vector representation (EVR) -i.e., using a closed chain of vectors to represent the object profile.In practice, objects with different types of geometry have been tested.The EVR has proved to be a very effective way to represent the object profile.Moreover, it is capable of providing the necessary information for object geometry reconstruction, which is crucial to gripping planning.
To extract the closed chain of vectors, a continuous fuzzy controller (CFC) was developed to achieve this goal.

CONTINUOUS FUZZY CONTROLLER
Fuzzy logic is one of the fastest-growing technologies in the world since the inception of the computer era.Fuzzy logic is based on natural language, which can model nonlinear functions of arbitrary complexity, and can be further built on the expertise of experts.The fuzzy controller is capable of giving excellent and robust solutions to complicated systems, if adequate, precise, and reliable expertise has been accumulated.Fuzzy pattern recognition is a very active branch in the field of pattern recognition, which may result in systems with high computational efficiency and achieve the desired accuracy.A schematic fuzzy system architecture is illustrated in Figure 3.

Figure 3: Basic architecture of a fuzzy system
To represent the object profile with a closed chain of vectors, locating the corner points, with given tolerance, is the kernel.As far as the corner tracing is concerned, a continuous fuzzy controller was developed in this application on the basis of testing various corners and collecting enough knowledge, which covered all known possibilities of corner appearance.
To judge whether a corner appears, two determinative factors are considered: one is the length of orthogonal vector Len to the base vector v base from the M th point; the other is the distance Prj from the project point that is perpendicular to the base vector to the endpoint of v base ; Node, the output of this Continuous Fuzzy Controller (CFC), is the membership of the corner point.The implementation procedure of the developed CFC algorithm is described as follows:

Normalization of universe of discourse
For the sake of simplification, x, y, and z are used to represent Len, Prj, and Node Len, Prj and Node to [-1, 1], the following normalization formulae are applied:

Definition of fuzzy set and membership functions
For the normalized input and output variable x, y, and z, five fuzzy sets are defined as: NL, NS, ZE, PS, and PL, which are represented by the symmetric and entirelyoverlapped triangle membership functions A1~A5, B1~B5, C1~C5 respectively.The five membership functions can be further expressed as: Negative Small (NS): Positive Small (PS):  In terms of fuzzy logic theory, the following relations are given [6]: (10) http://sajie.journals.ac.za

Fuzzy rule base
If the input is singleton, the following consequents are calculated as: Where the firing strength α i is given as:

Defuzzification
Weighted Mean Method is used for defuzzification.The defuzzification result z 0 is defined as: In this application, the proposed CFC gives a good solution to edge vector tracing and yielding the chain of vectors.In particular, curves can be approximated with adequate precision.

GEOMETRIC FEATURE EXTRACTION AND PART GEOMETRY RECOGNITION
In terms of the configuration of the developed three-fingered gripper, the geometry ID -which is further employed to generate an appropriate grasping configurationmust be figured out.In this project, syntactical pattern recognition is utilized to identify the object geometry.Identification of edges can be achieved by counting the edges that satisfy certain criteria.To extract geometric features from the obtained edges, calculation of the angle between two adjacent vectors must first be made.
Fuzzy set classification (or identification), which has been proven to extract geometric features effectively, uses fuzzy membership rules to classify images.A membership rule is described as: The technique that is utilized to create a membership function is the key to fuzzy set classification.As far as the triangle classification is concerned, triangles can be classified as an isosceles triangle, equilateral triangle, right-angle triangle, or http://sajie.journals.ac.za normal triangle.Fuzzy set I, E, R, and N can be used to describe the above triangles respectively.
The universe of discourse of triangle is given as: where A, B, C are three angles of a triangle.
The membership functions of I, E, R, and N can then be defined as: Any input (A, B, C) can be classified with reference to the above membership rule.
In this way, the exact shapes of round, block, wedge, parallel-piped, and polyhedron objects are recognized and classified, as shown in Figure 6.http://sajie.journals.ac.za gripper are completely determined by the configuration of the gripper, which is represented by three finger vectors (primary finger vector v m , and other two finger vectors v g1 , v g2 ), grasp radius r g , and grasp centre C g .Figure 7 depicts the configuration of the gripper while gripping a round object.The grasp centre C g coincides with the centre of the object, while the primary finger vector v m points to the east.If the profile is an equilateral triangle, C g coincides with the COG of the object.The configuration of the gripper is illustrated in Figure 8. 6 / 3a r g = , the direction of v m , coincides with the orientation of the object, which is determined in the object recognition module.v 1 , v 2 , and v 3 are edge vectors of the object.
Grasp models may result from the object grasp features, which can be extracted from the geometric features.When object geometry matches a predefined grasp model, the aforementioned configuration of the gripper is derived. http://sajie.journals.ac.za

EXPERIMENT AND DISCUSSION
To implement an intelligent gripping system, the proposed knowledge-based vision system was integrated into an industrial robot system for handling parts.As illustrated in Figure 9, the intelligent gripping system is made up of the following: • An ABB IRB-1400 industrial robot with an S4 controller, which can be accessed through ethernet or serial communication [7].
• A servo linear actuator, mounted on the wrist of the robot manipulator, which is driven by a Powermax hybrid stepper motor [8] under closed-loop control.The stepper motor controller and its optical encoder are interfaced to a PMAC board [9] through a relevant voltage-frequency converter [10].
• A three-fingered gripper with one flat finger and two round fingers is attached to the linear actuator such that the position and speed of each finger can be accurately controlled.

•
A robot vision, which consists of a CCD camera [11] and an AGP bus frame grabber [12].The camera is fixed over the work cell and takes images of the object.It is interfaced to the frame grabber that provides live images and is controlled through APIs.

•
Robot touch, achieved by attaching the tactile sensor to the flat finger of the gripper, which is capable of detecting the grasp force, tactile image, and any slipping that may result during the handling procedure. http://sajie.journals.ac.za • An IBM-compatible PC is utilized as the host computer aimed at constructing an open system.The PC bus interface cards are used as the interface; modular software design, based on object-oriented programming, facilitates compatibility and reconfigurability.
To test the proposed continuous fuzzy logic algorithm and edge vector representation method for on-line part recognition and feature extraction, a complex shaped object (as shown in Figure 10) is gripped by using the developed integrated intelligent gripping system.

Figure 9: The experimental setup for the intelligent gripping system
Objects to be http://sajie.journals.ac.za  means certain interference exists.As a result, gripping planning should be remadethat is, another featured planar face pair is examined, until no interference exists.Figure 12 illustrates the grasp model created for the object in the application program.Figure 13 demonstrates the object being grasped by the robot gripper.
Model-based gripping planning is an effective solution to build up the grasp model.
No prior knowledge on this object is needed.It is a significant exploration and attempt to solve the gripping planning problem intelligently.The proposed algorithm succeeds in generating the grasp model by extracting the geometric features arising from a vision system, and reasoning out the grasp features from the geometric features.The gripping planning process is fully automated; no interaction is required during this process.The result of gripping planning is visualized by illustrating the three finger positions, orientations, and grasp centre.The constraint examiner and interference examiner ensure that the gripping process will be performed reliably and robustly.In addition, the visualized gripping planning model can be used as a complement to the interference examiner.The geometry categories that can be handled by this application cover basic part geometry to some extent.The proposed system may find application in automatic assembly or component handling systems.In this project, an intelligent gripping system was integrated with an industrial robot.
Hardware components (servo actuator, sensors, robot and interface boards) and software components were used to perform the integration task in a modular fashion.It has been shown that edge vector representation can represent different objects appropriately.Moreover, edge vector representation is capable of providing necessary information for object geometry reconstruction, which is necessary for gripping planning.
In this application, the developed continuous fuzzy controller finds a good solution to edge vector tracking and yielding the chain of vectors robustly by locating the corner points.The fuzzy inference engine was proven to be an ideal solution to trace out the chain of edge vectors approximating and representing the profile of the objects.On the other hand, the developed continuous fuzzy controller traced the chain codes and checked out the corner points heuristically.Fuzzy set classification has been proven to be able to extract geometric features effectively.In this way, the exact shapes of objects -whether round, block, wedge, parallelpiped, or polyhedron -are recognized and classified.Grasp models may result from the object grasp features, which can be extracted from the geometric features.

Figure 2 :
Figure 2: Schematic architecture of robot vision system • Representation and description of object's geometric features: dependent on the application in which the vision system is going to be used.Common techniques are B-Rep, CSG, or feature-based representation.In addition, classification or model-based object matching may be made with regard to the object model.

Figure 5 :
Figure 5: Triangle membership functions of Len, Prj, and Node

Figure 6 :
Figure 6: Object classificationWith the determination of the part geometry ID, the grasp models of the threefingered gripper can be deduced.The position and orientation of the three-fingered

Figure 7 :
Figure 7: Configuration of the gripper while gripping round object r g is equal to the radius of the fingertip contact circle, as shown in Figure 7.

1 VFigure 8 :
Figure 8: Configuration of the gripper while gripping a wedge with profile of equilateral triangle

Figure 10 :
Figure 10: Object with complex shape to be grasped Figure 11 illustrates the result of object recognition.With developed image processing and object recognition algorithms, the chain of vectors displayed in the main window of the application program indicates its geometry ID: polygon with parallel edges.

Figure 11 :
Figure 11: The result of object recognition The three finger vectors are used to check out any possible geometric interference between the gripper and the object.The finger vectors are then extended in the normal directions, pointing to the exterior of the object.If any intersection points between these vectors and the edge vectors of the object are checked out, that

Figure 12 :
Figure 12: Grasp model generated for the object

Figure 13 :
Figure 13: The object being grasped by the robot gripper

Table 1
gives all 25 rules for fuzzy reasoning.

Table 1 : Rules for fuzzy reasoning 3.4 Fuzzy reasoning methods
According to Table 1, the fuzzy rule set can be described as: Rule 1 (R1): IF Len is negative large && Prj is negative large, THEN Node is positive large; Rule 2 (R2): IF Len is negative small && Prj is negative small, THEN Node is positive small; Rule 24 (R24): IF Len is positive small && Prj is positive large, THEN Node is negative small; Rule 25 (R25): IF Len is positive large && and Prj is positive large, THEN Node is negative small.