aharrison-fullerandClaude Opus 5
4286e88344
Loads parakeet-mlx in-process (no HTTP hop) and serves it over the Wyoming
protocol. On an M4 Mac mini this transcribes typical voice commands in ~110ms
versus ~1150ms for a whisper.cpp large-v3 setup, with identical accuracy on a
ten-command benchmark.
Two behaviours matter beyond speed: silence returns an empty string rather
than whisper's "Thank you." hallucination, and there is no decoder context
carried between requests.
Notable implementation details, all covered by mutation-checked regression
tests:
- MLX streams are thread-local, so the model is loaded and evaluated on a
single dedicated worker thread. Splitting those raises
"There is no Stream(cpu, 1) in current thread".
- parakeet_mlx.load_audio() shells out to ffmpeg, which is unnecessary here
since Wyoming delivers 16kHz mono PCM. The mel is built directly via
get_logmel(), whose input must be float32 -- it views the complex STFT
output as the input dtype, so anything narrower doubles the mel bin count.
- Wyoming's run loop has no except clause, so an exception escaping
handle_event closes the connection without sending a Transcript and Home
Assistant waits indefinitely. Failures are caught and returned as an empty
transcript instead.
Defaults to parakeet-tdt-0.6b-v2 rather than the newer multilingual v3
because v2 emits digits ("21 degrees") where v3 spells numbers out, and
Home Assistant's local intent matching expects digits.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
41 lines
1.3 KiB
Bash
Executable File
41 lines
1.3 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Generate the end-to-end test clips using macOS TTS. They are not committed
|
|
# because they are trivially reproducible binaries.
|
|
set -euo pipefail
|
|
|
|
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/clips"
|
|
mkdir -p "$DIR"
|
|
|
|
PHRASES=(
|
|
"turn off the kitchen lights"
|
|
"set the living room thermostat to twenty one degrees"
|
|
"what is the temperature in the bedroom"
|
|
"dim the hallway lights to thirty percent"
|
|
"is the back door locked"
|
|
"turn on the christmas tree in the conservatory"
|
|
"set a timer for twelve minutes"
|
|
"play radio six music in the kitchen"
|
|
"whats the octopus agile rate right now"
|
|
"close the blinds in the study and turn on the desk lamp"
|
|
)
|
|
|
|
i=1
|
|
for phrase in "${PHRASES[@]}"; do
|
|
say -o "$DIR/cmd$i.aiff" "$phrase"
|
|
afconvert -f WAVE -d LEI16@16000 -c 1 "$DIR/cmd$i.aiff" "$DIR/cmd$i.wav"
|
|
rm -f "$DIR/cmd$i.aiff"
|
|
i=$((i + 1))
|
|
done
|
|
|
|
# 3s of digital silence -- a good model returns an empty string here rather
|
|
# than hallucinating "Thank you." the way whisper does.
|
|
python3 - "$DIR/silence.wav" <<'PY'
|
|
import sys, wave
|
|
w = wave.open(sys.argv[1], "wb")
|
|
w.setparams((1, 2, 16000, 0, "NONE", "NONE"))
|
|
w.writeframes(b"\x00\x00" * 16000 * 3)
|
|
w.close()
|
|
PY
|
|
|
|
echo "Wrote $(ls "$DIR"/*.wav | wc -l | tr -d ' ') clips to $DIR"
|