Prototype machine for testing camera, lens, and lighting combinations for Machine Vision (MVDeep) under a Thai government project

How the brief came about

The challenge in this case was to reduce the time spent choosing cameras, lenses and lights for camera-based inspection. Whether a Machine Vision (camera-based inspection system) job performs well depends on matching this equipment to the workpiece from the outset. The Thai private company participating in the project did this step by trying equipment one set at a time and checking the results with statistical values.

Most tests took more than 1 day per job, some over 1 month, and required people with knowledge of lighting, imaging, industrial statistics and image processing. As a result, the company could take on fewer jobs than it received. The bottleneck was testing time and the number of people able to do the work.

We developed this machine under a government machinery prototype development support project in fiscal year 2019. The project's full name is Automated Machine Vision Laboratory By Deep Learning (an automatic workpiece inspection machine using an intelligent deep learning camera). The word Laboratory here means a machine for testing equipment combinations on real workpieces, not a classroom.

Requirements and constraints

The machine had to produce no fewer than 100 Cpk reports per 8 hours and summarise the results of every equipment combination as comparable statistical values. Other requirements and constraints are as follows.

• Build 1 prototype machine under a government support project
• Start with colour shade sorting inspection, the first task set by the participating company
• Decide equipment selection by statistical values instead of estimates based on human experience
• The workpiece must be placed on the XY Table inside the cabinet, where space is limited

Why use robots to switch equipment, then decide by Cpk?

We had robots switch the cameras, lenses and lights, then decided by Cpk, because the bottleneck was the time needed to test equipment one set at a time and the small number of specialists. A machine that can repeat tests in the same sequence and summarise every combination by the same criteria allows equipment selection to be based on figures.

The machine is divided into an upper cabinet for testing and a lower cabinet for the electrical control unit. An NVIDIA Jetson Xavier processing board uses ROS (Robot Operating System, a robot control software suite) to control all the robots. The workflow is as follows.

1. 1

   Place the workpiece and enter details

   The user places the workpiece on the XY Table in the upper cabinet, then selects the inspection type and enters the workpiece details, centre position and size on the machine's screen.

2. 2

   Robots arrange the camera, lenses and lights

   When Start is pressed, the first robotic arm set moves the camera to the centre of the workpiece, takes images, then changes the distance. The other two sets hold the lights and change the lens and light models, so that images are obtained from several equipment combinations.

3. 3

   AI sorts colour shades and calculates Cpk

   AI groups the workpiece's colour shades from images (Clustering means grouping similar data), then a statistics module calculates the Cpk of each equipment combination.

4. 4

   Produce reports and send them to the Cloud

   The machine summarises the 5 equipment combinations with the highest Cpk values in PDF reports and can send the reports to a Cloud system from the machine's program.

Deliverables

We delivered 1 MVDeep prototype machine with software, documents and transfer of operating knowledge. The main items are as follows.

• A test cabinet with an XY Table and 3 robotic arm sets for positioning the camera, holding the lights and changing lenses and lights
• Several types of light: Bar Lights, multi-angle Ring Lights and Coaxial Lights (lighting in line with the lens)
• FA lenses (industrial lenses) with several focal lengths, and telecentric lenses (lenses for measurement work that reduce image size changes with distance)
• Electrical control unit in the lower cabinet, and an NVIDIA Jetson Xavier board running ROS
• Software that takes in workpiece details, groups colour shades, calculates Cpk, produces PDF reports and sends the reports to the Cloud
• A work plan report, a progress report, a final report, and an operation and maintenance manual

Confirmed results from the final report

The project's final report confirms MVDeep's test results in 3 areas: report output speed, sending reports to the Cloud, and cabinet safety. In addition to the test results, this work also delivered documents and transfer of operating knowledge to the participating company.

• Speed: the machine met the target of producing no fewer than 100 Cpk reports per 8 hours
• Report delivery: the machine's program can send reports to a Cloud system
• Safety: the machine has a Safety Sensor and stops when the cabinet cover is opened
• Documents: a work plan report, a progress report, a final report, and an operation and maintenance manual
• Knowledge transfer: we transferred operating knowledge of the machine to the participating company

What we learned

This work offers 3 principles for selecting equipment for camera-based inspection systems.

• Select equipment by the same statistical criteria for every combination, rather than by looking at images or by human experience. Cpk allows many camera, lens and lighting combinations to be compared on the same basis.
• Start from one clear inspection task. This work started with shade sorting, which made it possible to define clearly how results are measured before extending to other types of inspection.
• Separate what can be repeated from what has to be searched for. Preset positions let the machine repeat tests in the same sequence, while automatic finding of camera and lighting positions is another layer of work that must be developed and tested separately.

How this approach can apply to your work

If you have to select cameras, lenses and lights for several camera-based inspection jobs, you can use the same approach as this case: test several equipment combinations in the same sequence, then compare the results by Cpk. What has to be adjusted is the inspection criteria, the workpiece size and the positioning method, when your work differs from shade sorting.

• A machine of this kind is for testing to find an equipment combination, not for inspecting workpieces on a production line. Once the combination is chosen, the on-site inspection system still has to be designed separately.
• Prepare electrical power, compressed air and space for opening the cabinet for maintenance
• The robotic arms have many motors, so spare motors should be kept and the electrical system checked at the interval given in the manual (usually 1 check per year)