Why voice orders waited 3 seconds

esp32-pie-asr runs a 65.7M parameter streaming speech model on an ESP32 chip, with no PC and no language model. Its design doc has a section on voice control demos where spoken orders drive a simulated device, and one entry in it is a good debugging story.

The beam search keeps several hypotheses and commits a token only when all of them agree on it. That is fine for a transcript. For a voice order it was slow: the last words of an order waited until the hypotheses agreed. A twin hypothesis that differs only in case ("C" for "c") could keep them apart until the 3 s limit, after which the best one is committed and the rest dropped.

The measured result: from the end of an order to its action took 3.1 s at the median.

The fix is a pause commit. Once the best hypothesis has had no token for 0.5 s, it is committed whole and the others are dropped. With it, the same measurement gives 1.9 s at the median. It is off by default (pieasr_beam_set_pause) and only the voice control pages turn it on, because the plain transcriber is better off waiting for agreement.

The doc also explains what is left of the 1.9 s: an order ends with a pause of 0.5 s, and the last chunk, up to 1.28 s of audio, must then come in and be computed in about 0.9 s.

Source: docs/design.md in BoumedineBillal/esp32-pie-asr.