Overview

This is P24-C4; he plays Connect4. He was built for the Sidmouth Science Festival. Actually, he was built for the 2024 festival (hence “P24…”) and I have only just got round to putting details on the web. The 2026 festival has now closed; so here are some more details about the robot before he gets put away for another year.

Here he is in action…

Hardware

P24-C4 stands about 370mm tall, is about 300mm long and 300mm wide.  He has 16 degrees of freedom (2 tracks, 5 servos per arm, a shoulder rotation servo, neck rotation and elevation servos and a rear antenna motor).  There is a further servo for elevation one of the cameras, but this is redundant with the current configuration.

The robot has a Raspberry Pi camera mounted in his torso which is used to interpret the state of the game.  The camera is connected to a Raspberry Pi 5 which is mounted in the back of the hull.  The camera is a wide view angle variety, but it must still sit back from the board by about 300mm to be able to see the whole board at once.  Hence the robot moves forward to place a counter in the board (so that the arms do not need to be excessively long), then reverses again.

The main RPi5 has an SSD fitted for OS storage. It connects to the robot via the 40pin header onto a control board like that fitted to Firefly. The base of the robot is actually Firefly’s hull, although the sprockets have been reprinted for the original colour scheme.  Also, Firefly’s rear motors were removed as they are redundant when tracks are fitted.  Firefly’s batteries are not high enough capacity to run the robot for a whole day, so we opted for a mains supply instead.

The robot has a second camera and RPi 5 mounted in his head.  This one has a Hailo AI accelerator fitted.  The original robot implementation used the face tracking model that comes with the Hailo to track faces in front of the robot.  When waiting for his opponent to play, the robot would sometimes look at the board and sometimes look at his opponent; this was to make the robot appear more animated.  This works well when there is only one or two people in front of the robot.  But it became very jittery when a crowd gathers round which is common at the festival.  For subsequent years I haven’t enabled face tracking; some day I hope to have time to re-instate it so that he looks almost exclusively at his opponent rather than flitting from face to face in the crowd.

Software

A custom program, c4main.py, has been written to run on the primary RPi to play the game.  It takes the video feed from the camera then uses OpenCV to find the board and read which counters have been played.  This is not easy; the vision algorithm must cope with people sitting close to the board wearing all colours of clothes as well as varying light levels that change through the course of the day.  This isn’t thoroughly solved; the system needs recalibration from time to time to keep running.  I hope to revisit this aspect of the system before the next festival.

C4main publishes a web page showing a marked-up view form the camera.  The page is opened on a PC browser so that the robot can be directed from the PC.

On the view note that the program has drawn a blue boarder round the whole board, yellow lines around the board holes and has marked the counters that it has located.  This shows the program’s interpretation of the board. P24-C4’s control code must ignore the clutter that it can see through the holes in the board.

When it’s the robot’s turn, the program sends the game state to a neural network that was supplied by Colin Walls of Sidmouth Robotics.  The net decides on the best column for the robot to play in.  It also indicates if playing in that column will result in the robot winning the game.  c4main passes the column to be played to the robot controller (which appears as a serial port to the RPi).  The robot controller then choreographs the robot’s motors and servos to move forwards, take a counter from one of the holsters, place the counter into the board, then back away so that the camera can see the whole board again.  The robot informs c4main that the move is complete.  If the game has completed, c4main can direct the robot to “celebrate” or “sulk” depending on the outcome.

P24-C4 is not a good loser!

Conclusion

P24-C4 is pretty popular at the festival, playing (and mostly winning) dozens of games through the course of the day.  Hopefully it can come back next year, hopefully with some improvements.