Camera-based parts counting system on a vibratory feeder (Machine Vision Counter) for an industrial machinery customer

How the brief came about

The challenge in this case was to count parts on a vibrating-track part feeder to exactly the quantity set, then have the machine stop feeding once that quantity was reached. The customer is in the industrial machinery sector. There are 5 part types on the track, running together across multiple lanes.

The program's test criterion required counting with “no shortfall and no excess” for every part type. If the count falls short or goes over, the work of dividing parts by quantity has to be recounted or corrected afterwards.

Requirements and constraints

The system had to work on the customer's machine and be usable at the machine without changing the program. The main requirements and constraints were as follows.

• Count 5 part types: flat round parts in several sizes, parts with a central hole, and plug-shaped parts
• Parts run together across multiple lanes, so counting must be separated by track lane
• Users at the machine must be able to select the part model, set the count quantity (Set Count) and add new models themselves
• The system must connect to the machine through 1 Input channel and 2 Output channels
• The equipment set must use a 24V power system connected to the machine's main power system

How does the system count multiple part models across multiple lanes with a camera?

The system counts multiple part models across multiple lanes by splitting the work into 2 tasks. The detection model for each part model finds the parts in the image, while counting uses a counting line (Line Counter) on an image divided into lanes following the track. When a part crosses the line, the program adds one to the count.

We chose this approach because there are many part types and part models must be switched at the machine. Giving each part model its own detection model lets users add new part models from the screen, while counting and signalling still follow the counting line and the quantity set by the user. Users can therefore check which rules the system counts by.

1. 1

   Add part models and train the model

   On the Training page, the user sets the number of images and the interval for the camera to take sample images (Snapshot), then trains the detection model from images of the parts and background images of the track with no parts on it. The manual recommends at least 200 part images per part model. The result is then saved as a new part model with a sample picture on the Add Product page.

2. 2

   Set the image area and counting line to match the track

   The person setting up the system defines the part of the image used for counting, divides the image into lanes matching the lanes of the vibrating track, and places the counting line (Line Counter) on the image.

3. 3

   Select the part model and set the count quantity

   The user selects the part model from the list, enters the required quantity (Set Count) and presses Process. The screen shows 2 counter channels and the total, and has Stop and Reset buttons for use during counting.

4. 4

   Signal the machine to stop feeding

   When the count is nearly complete, the system sends a signal on the first Output channel for the machine to stop the main track. When the count is complete, the system sends a signal on the second Output channel for the machine to stop the top-up track. The Tower Light turns green for about 4 seconds, and the program then starts a new counting cycle.

The Reset button on the machine is wired as 1 Input channel, so users can reset the counter from the machine. Users set the count quantity and press reset themselves. The system counts and signals according to the values the user has set.

Deliverables

We delivered the Vision Counting program together with the equipment set installed on the customer's machine and documentation for users. The deliverables were as follows.

• Vision Counting program: a part counting page, a page for adding, editing and deleting part models, and a Training page for taking sample images and training the model
• Hardware: Global Shutter camera, lens, 5 lighting sets, processing unit, display and router
• Installation: camera on a post above the vibrating track, camera position adjustment, and connection of the 24V power system to the machine's main power system
• Machine connection: 1 Input channel from the Reset button and 2 Output channels for stopping the feed track, along with the Tower Light
• Documentation: a user manual and an 8-item work checklist covering switching the machine, program and camera on and off, counting sample parts, and Input/Output signals

Work steps

This job ran from 2016 to 2017. It began with developing and testing the program, then test installation on the actual machine, followed by delivery of the equipment set and adjustment of the system to suit the machine. The work sequence was as follows.

1. September–November 2016: developed the Vision Counting program, prepared the user manual and tested counting with the 5 part types
2. December 2016: test installation of the system on the machine
3. January 2017: delivered the equipment set, adjusted the lanes in the image to match the track, and defined how the Input/Output signals work with the machine
4. May 2017: connected the 24V power system to the machine's main power system, prepared the 8-item work checklist and closed out the final phase

Confirmed results from the installation and job close-out

We installed and tested the Machine Vision Counter on the actual machine, delivered the equipment set in January 2017 and closed out the final phase in May 2017. Users at the machine can select the part model, set the count quantity and add new models themselves.

This page shows no accuracy figure because no numerical measurement was recorded. What was delivered is the equipment set, the user manual and the 8-item work checklist for checking the system.

What we learned

This job offers the following principles for camera-based part counting on machines.

• Parts must be in a lane before the camera can count them correctly. During on-site testing, parts that strayed out of their lane were not counted, so the machine's guide rails and the lane division in the image have to be designed together with the machine builder.
• Camera position and lighting must be finalised on the actual machine. On this job, several site visits were needed before the camera mounting position was finalised, so project plans should allow time for on-site adjustment and define the work checklist with the customer before testing begins.
• Separate the model's role from the counting rules. The model finds parts in the image, while the counting line, the set quantity and the Input/Output signals are rules that can be checked. Users therefore know why the machine stopped and can reset it from the machine.

How can you apply this approach to your machine?

If your machine feeds parts in lanes and must stop when the quantity is reached, you can use the same approach as this case: find the parts with a model, count with a counting line on an image divided into lanes, then send a signal to the machine. What has to be adapted is the lighting, the camera position and your machine's signals.

• Prepare enough samples of every part model for taking images to train the model, as well as a track with no parts on it for background images
• Check whether the machine's guide rails can keep parts within their lanes
• Agree the Input/Output signals and the machine's power system with the machine builder from the outset