Face recognition customer greeting kiosk software (Greeting Kiosk) and a promotion management back-office system for a retail business

How the brief came about

The brief for this case was to trial an advertising screen that could interact with customers, in a retail business's pilot project. A typical advertising screen shows the same content to everyone who walks past, so the marketing team did not know which promotion suited whom, or how much interest the screen attracted.

The marketing team wanted a kiosk that would greet customers and offer them the right promotions, let the team change advertisements and promotions themselves without waiting for a programmer, and show kiosk usage statistics. We joined the project as a subcontractor to a consulting firm, responsible for developing the kiosk software and the back-office system.

Requirements and constraints

The kiosk had to work with the project's existing systems, and the use of face images had to be restricted from the design stage. The main requirements and constraints are as follows.

• Screens designed for a 55-inch portrait touchscreen
• Face recognition only for members who have registered their face and confirmed the display of their information at the registration kiosk. For non-members, only the gender and age range estimated from the face may be used.
• Stored data must be encrypted, and data transmission must use secure protocols.
• The kiosk must connect to the project's AI engine (the AI program used for face recognition) and Membership API (the interface to the membership system).
• If the camera detects several faces at once, the system must select the person closest to the kiosk.
• Promotion product categories were defined from the outset, because adding or removing categories later affects the screen design.
• Face recognition can be scheduled to switch on and off according to store opening hours, to reduce server load while the store is closed.

How the kiosk selects a promotion for each customer

The kiosk selects promotions by comparing the customer profile with the condition tags the marketing team applies to each promotion: gender, age range, time period, and member or non-member. The AI engine handles face recognition and estimates gender and age range, while promotion selection uses rules based on the tags set by the marketing team.

This approach separates responsibilities clearly. The AI provides information on who is standing in front of the kiosk, the marketing team decides which promotion is shown to whom, and the team can change the rules itself from the back-office system. The kiosk works in the following steps.

1. 1

   Advertising mode (Ads Mode) while waiting for customers

   The kiosk loops advertising still images or videos in the order set in the back-office system, in either full-screen or non-full-screen format.

2. 2

   Camera detects faces

   When someone comes within the camera's capture range, the system looks for a face. If it finds several people at once, it selects the person closest to the kiosk.

3. 3

   Recognise the member or estimate the customer group

   The face image is sent to the AI engine and compared with registered member faces. For a member, the system retrieves data through the Membership API to build the customer profile. For a non-member, the system uses the gender and age range estimated by the AI engine instead.

4. 4

   Greeting and promotions (Promotion Mode)

   The screen shows a welcome page with a spoken greeting, then displays the promotions whose condition tags match the profile. Customers can browse promotions by product category. For non-members, the screen invites them to register for special privileges.

5. 5

   End of use and statistics recording

   When the customer presses to end the session or walks away from the screen, the kiosk shows a thank-you page, records the usage history (Activity Log) in the database, then returns to playing advertisements as before.

Deliverables

We delivered a web application in 2 parts, developed with PHP, Python, HTML, JavaScript and a MySQL database, and installed and tested it. The delivered components are as follows.

• Greeting Kiosk screens: advertising mode, promotion mode, welcome page, thank-you page and Activity Log recording
• Advertising management: add, edit and delete image or video advertisements, in both full-screen and non-full-screen formats
• Promotion management: apply condition tags for gender, age range, time period, and member or non-member, by product category
• Kiosk settings: morning and afternoon screen themes, and a preview of the kiosk screen before publishing
• Statistics dashboard: number of users split by member and non-member, time spent at the kiosk, how often members return, and the number of times each promotion was viewed, with data export to CSV files
• User management for the back-office system
• Design documents and acceptance test scripts (UAT is the testing the customer uses to decide whether to accept the work)

Confirmed results

We delivered, installed and tested the software according to the scope of work. The customer finished testing the system on 27 December 2019, and the project was closed as completed work in 2020.

This case was a pilot project. We have no marketing performance figures, such as sales or kiosk user numbers, so we report only the work status supported by documents.

What we learned

This work offers the following principles for systems that use customers' face images and must connect to other parties' systems.

• Privacy must be in the requirements from the outset. The system should recognise only people who have registered and consented, using only broad group information such as gender and age range for everyone else, and stored data must be encrypted.
• Test cases must be tied to requirements agreed before development starts. The problem encountered in this work was that test cases did not match the specification. When test cases are tied to requirements from the outset, acceptance testing does not need to be reinterpreted later.
• The kiosk works to its full potential when all external systems are ready. In this work, another vendor's marketing-side software could not be used in practice, so it is advisable to assign an owner for each system and check readiness from the early stage of the project.

How this approach can apply to your work

If your business already has a membership system and you want an in-store screen that offers promotions by customer group, you can use the same approach as this case: separate the kiosk, the face recognition system and the back-office system, and let the marketing team control the promotion selection rules itself. What needs adapting depends on your existing systems and the screen size you use.

• Have the marketing team first summarise who each promotion suits, because these conditions are what the system uses to select promotions.
• Define promotion product categories from the start.
• Agree who is responsible for the AI engine, the membership system, the registration kiosk and the kiosk software, then write test cases that match the requirements before development.
• Schedule face recognition to switch on and off according to store opening hours.