I’m a bit late to the voice control party. But that’s good news because everyone else has done all the hard work. In particular, Neil Stevenson from Team Dangle has blogged how to install Vosk (https://dangle.blogsyte.com/?p=199) on a Pi. I copied Neil’s instructions and had a voice control system up and running (on a RPi400) in minutes.
The good news is that that my wife’s posh Bose noise cancelling headphones work beautifully with it too: I was worried that they might not be Pi-compatible. Actually, setting them up wasn’t easy; I used the instructions on https://howchoo.com/pi/bluetooth-raspberry-pi but had to be very careful not to let them configure as “headphones” (the rather insistent default). They must be a set up as a “headset” to work with Vosk.
I then tried to connect the headphones to P21’s Pi. This didn’t go to plan. The problem is that I redirected the Pi’s hardware UART to the Pi’s header ages ago. This is used to pass messages between the Pi and the robot’s Arduino hardware controller and is fundamental to the robot’s design. The problem is that the UART is normally required by the Pi’s Bluetooth system.
I thought of the following options to counter this:
- Revert the robot interface to the Pi’s software UART. I’m not keen on this; on https://howchoo.com/pi/bluetooth-raspberry-pi it says that the software UART timing fails if the Pi is heavily loaded. The Pi is certainly heavily loaded when the vision system is running.
- I could try connecting a Bluetooth dongle to a USB port on P21’s RPi. I have no idea if this will help so I will probably look into this in time. I would probably have to reprint P21’s lid to fit a dongle in.
- I could place another RPi on the robot and connect it to the implement connector. I might run into UART problems again though. Vosk probably loads the Pi and I would not be able to use the hardware UART (obviously!).
- I could place another RPi on the robot and connect it via IP and WiFi.
I decided to go with the last option. I decided to use the RPi and touchscreen from P19 since it is currently redundant. This setup was appealing since it can be created with minimal changes to P21.
So, here’s what I did:
- I took the Pi and touchscreen off of P19. I updated the OS then installed the Bluetooth manager software as described in https://howchoo.com/pi/bluetooth-raspberry-pi . This time the headphones connected without problem.
- I added a new socket interface into PiDrogen’s software framework. A client can now connect to it and issue words. I updated the general framework to understand some words like “go”, “stop” and “pause”. I updated the line follower game to understand “straight” (go straight on at the next junction), “right” and “left”. I also added “return” which has the robot do an about turn, reverse the directions it followed before, follow the line (back to the start) then do another about turn.
- I installed Vosk on the RPi following Neil’s instructions. I then added a socket client to glue Vosk to P21’s framework. This is awkward since Vosk often repeats itself; the glue code clears the repetition.
- I designed a carrier for the RPi, touchscreen and a battery pack, see below.
Does it work?
Yes. But it took quite a few “takes” to get a complete video of the whole system working. Three errors made the system unreliable:
- The line follower code made a number of errors where it either lost the line or it took the wrong turn at junctions. At the time I countered this by down-tuning the robot speed but I now believe I should have adjusted the black threshold.
- Sometimes Vosk is too slow. I start the robot by saying the sequence “go straight right left” (those being the directions to follow for Up the Garden Path). If the word “straight” reaches P21 after the robot has reached the first junction then the robot defaults the first junction to straight on, but then the sequence is out of step with the robot’s location. As a result, the robot takes a wrong turn at a subsequent junction. I can bypass this problem by omitting the word “straight”, but I would prefer to get it working nicely.
- Sometimes Vosk misses a word. If one of the direction words is missed then the robot might make a wrong turn.
At time of writing I have a video that could be submitted to the competition. The robot is voice controlled and uses the camera to follow the line (which is the maximum scoring option). But the video was made with down-tuned speed and I forgot to return the robot back to the start by saying “return” (which is not part of the game, but it’s fun).
I think I’m going to address some of the issues above then have another go!
Team PiDrogen 