When the robot perceives and speaks

When Thymio avoids an obstacle, it applies a rule someone wrote: if the front sensor sees a nearby object, turn. You can read the rule, predict the behaviour and correct the program. A robot that recognises a gesture or an image works differently. Nobody wrote the rule. A model learned it from examples. It succeeds on cases close to its examples and can get the others wrong.

The micro:bit lets you bring this to life in class. The free micro:bit CreateAI tool records gestures with the motion sensor of the V2 board, trains a model, lets you test it, then sends it into a MakeCode program on the board. Students see that with few examples, or with gestures that are too similar, the model gets confused.

Robots and toys now hold conversations thanks to a language model. Their answers are fluent, but they can be wrong, and the same message does not always get the same answer. When the model runs on a server, the child’s voice is sent there to be transcribed and processed. The review by Belpaeme and colleagues, in 2018, already noted that understanding young children’s speech remained difficult for robots.

1. Programming a rule: Thymio follows a line or avoids an obstacle. The students write the rule in plain English.
2. Training a model: with CreateAI, three gestures, a few examples of each, then a test by another student.
3. Comparing: who decided the behaviour? Why does it get things wrong? What has to change to correct it, the program or the examples?