wyoming-parakeet-mlx/README.md

wyoming-parakeet-mlx

A Wyoming protocol speech-to-text server for Home Assistant, backed by NVIDIA's Parakeet TDT running on Apple Silicon via parakeet-mlx.

The model is loaded in-process — there is no HTTP hop between the Wyoming bridge and inference.

Why

Speed is the obvious reason — see Performance below — but two behaviours matter just as much for a voice assistant:

• Silence returns an empty string. Every whisper variant tested hallucinates on digital silence ("Thank you.", or "you" for faster-whisper), which reaches your conversation agent as a real utterance. Parakeet and Moonshine return nothing.
• No cross-request contamination. whisper.cpp's server carries decoder context between requests unless you pass -nc, and will return the previous utterance — in testing, roughly one time in five.

Performance

On an M4 Mac mini, against 48 clips of Home Assistant commands:

Backend	Median	Exact	Silence
parakeet-tdt-0.6b-v2 (this)	102 ms	46/48	""
whisper.cpp large-v3-turbo	518 ms	47/48	"thank you"
whisper.cpp large-v3	941 ms	47/48	"thank you"
faster-whisper base.en	326 ms	44/48	"you"

Roughly 5× faster than the best whisper.cpp configuration, matching it within one clip on accuracy. Full comparison against 11 backends, methodology and caveats → BENCHMARKS.md, including Moonshine (faster still, meaningfully less accurate) and mlx-whisper (the accuracy ceiling, 11× the latency).

Requirements

• An Apple Silicon Mac (M1 or newer) that stays on. This does not run on Intel Macs, a Raspberry Pi, or inside Home Assistant OS.
• Homebrew Python. If you are not sure you have it:

  brew install python@3.13
  

• About 3 GB of free disk — 2.3 GB of model, 600 MB of libraries.

Why Homebrew Python specifically

The installer needs Python 3.10 or newer that includes the lzma module. macOS ships Python 3.9, which is too old, and pyenv frequently builds Python without lzma (it needs xz present at build time). Both fail — the pyenv case unhelpfully, passing every version check and then dying at import time with ModuleNotFoundError: _lzma. Homebrew's builds are fine.

ffmpeg is not required, unlike most speech-to-text tooling. Wyoming already delivers audio in the right format, so this builds the mel spectrogram directly.

Install

1. Download and run the installer.

git clone https://github.com/adamhf/wyoming-parakeet-mlx
cd wyoming-parakeet-mlx
./install.sh

It will ask for your password once, to register the background service.

Expect it to take a few minutes — most of that is downloading the model. It sets up its own Python environment, runs the test suite, downloads the model, and registers a background service on port 7892 that starts automatically when the Mac boots, with no need to log in.

Keep this folder where it is. The installer sets things up inside it, so moving or deleting the folder afterwards breaks the service.

Remove it later with ./uninstall.sh.

Installer options

--port, --model, --user, --python, --no-daemon, --no-download. Run ./install.sh --help for details. See Security for why --user is worth using.

Connect it to Home Assistant

Installing it is not enough on its own — Home Assistant has to be told about it, and then told to use it. That is two separate steps, and the second one is easy to miss.

1. Find your Mac's IP address. System Settings → Network → click your active connection → Details → the "IP address" field. Or in a terminal:

ifconfig | grep "inet " | grep -v 127.0.0.1

That command often lists more addresses than you expect. Ignore anything belonging to virtualisation software (Parallels, VMware, Docker — commonly 10.211.55.x, 10.37.129.x, 172.17.x.x) or a VPN (Tailscale uses 100.x.x.x); those are not reachable from Home Assistant in the usual case. You want the one on the same network as Home Assistant itself — if Home Assistant is on 192.168.1.50, you want the Mac's 192.168.1.x address.

If your Mac has both Wi-Fi and Ethernet active you will see two candidates. Either works, but pick one that will stay put — a wired address, ideally with a DHCP reservation on your router — because if that interface goes away, Home Assistant loses speech-to-text.

2. Add the integration. In Home Assistant: Settings → Devices & Services → Add Integration → Wyoming Protocol. Enter the IP address from step 1 and port 7892.

If it succeeds you get a new entry called parakeet-mlx. If it says it cannot connect, see Troubleshooting.

3. Actually use it. Go to Settings → Voice assistants, click your assistant, and set Speech-to-text to parakeet-mlx. Save.

Until you do this, nothing changes — you will have installed it and Home Assistant will carry on using whatever it used before. If you have more than one assistant configured, change each one you care about.

4. Try it. Click the Assist icon (top right of Home Assistant), press the microphone, and say something like "turn off the kitchen lights".

Check it's working

Is the service running?

sudo launchctl print system/local.wyoming-parakeet | grep "state ="

Expect state = running.

Watch it work — leave this running, then speak to Assist:

tail -f /tmp/local.wyoming-parakeet.stderr

Each command produces a line like 2.70s audio -> 124ms, 45 chars. The 124ms is how long transcription took. The text itself is deliberately not logged; see Security.

Test it without speaking, using generated audio:

./test/make-clips.sh
.venv/bin/python test/wy-test.py test/clips/*.wav

Expect ten commands transcribed correctly, silence.wav coming back empty, and times around 100–150 ms.

Troubleshooting

What you see	What it usually means
Home Assistant says "Failed to connect" when adding the integration	Wrong IP, or the service is not running. Check state = running above, confirm the IP with ifconfig, and make sure both machines are on the same network.
Integration added fine, but Assist is no worse and no faster	You missed step 3 — the assistant is still using its old speech-to-text engine. Settings → Voice assistants → Speech-to-text.
No matching distribution found for parakeet-mlx during install	Wrong Python. Run brew install python@3.13 and try again, or point the installer at it: ./install.sh --python /opt/homebrew/bin/python3.13.
error: service did not start at the end of install	Look at /tmp/local.wyoming-parakeet.stderr for the reason. If you changed --user, re-run ./install.sh rather than editing the service file by hand.
The first command after a reboot is slow (~300 ms), then it is fast	Expected. The graphics framework compiles its kernels on first use. It settles by the second or third command.
Everything works, then stops after a few weeks	Check the Mac has not gone to sleep. System Settings → Displays → Advanced → prevent automatic sleeping.
You want to see what it actually heard	Add --debug, but read Security first — the log then contains everything spoken near your microphones.

Server options

Most people never need these. install.sh bakes them into the service definition, so to change one, re-run the installer rather than editing the service file by hand. Run script/run --help for the full list.

Flag	Default
--uri	—	tcp://0.0.0.0:7892. Bind to one interface to limit exposure.
--model	…parakeet-tdt-0.6b-v2	See Model choice.
--max-audio-seconds	120	Cap on buffered audio per utterance.
--debug	off	Verbose logging, including transcript text.

Model choice: v2, not v3

The default is parakeet-tdt-0.6b-v2 even though v3 is newer and multilingual, because of inverse text normalisation:

Spoken	v2	v3
"twenty one degrees"	21 degrees	twenty-one degrees
"thirty percent"	30%	thirty percent

Home Assistant's local intent matching (hassil) expects digits. If your pipeline has prefer_local_intents enabled, a model that spells numbers out still looks accurate while quietly pushing commands off the fast local path onto your LLM fallback.

Measured on the benchmark corpus, v3 returned digits for only 10 of 21 number-bearing commands, against 21/21 for v2. That single behaviour is most of the gap between their exact-match scores. v3 remains the right choice if you need languages other than English — just size the trade-off first.

Updating the model

HF_HUB_OFFLINE=1 is set in the daemon, so it never silently re-downloads or changes model at boot, and starts fine without a network. Updating is therefore deliberate:

HF_HUB_OFFLINE= .venv/bin/python -c \
  "from parakeet_mlx import from_pretrained; from_pretrained('mlx-community/parakeet-tdt-0.6b-v3')"

Then re-run ./install.sh --model mlx-community/parakeet-tdt-0.6b-v3. The old model stays cached, so reverting is just another ./install.sh.

Tests

.venv/bin/python -m pytest

44 unit tests, well under a second. The model is mocked throughout, so they need no GPU, no network and no 2.3 GB download — they cover the audio marshalling and threading around it, which is where the real bugs were. Every regression test was mutation-checked: the fix reverted, the test confirmed to fail.

Worth knowing about a few:

• test_load_and_inference_share_one_thread — MLX streams are thread-local, so the model must be loaded and evaluated on the same thread or mx.eval() raises There is no Stream(cpu, 1) in current thread. The test deliberately overlaps its calls: sequential calls can coincidentally reuse one thread out of a multi-worker pool and pass a broken implementation.
• test_audio_is_float32_not_bfloat16 — get_logmel views the complex STFT output as the input dtype, so anything narrower silently doubles the mel bin count and the matmul fails.
• test_model_failure_still_sends_a_transcript — Wyoming's run loop is try/finally with no except, so an exception escaping handle_event closes the connection having sent nothing, and Home Assistant waits for a response that never arrives. The handler catches and returns an empty transcript so it fails fast instead.
• test_concurrent_handlers_do_not_share_audio — guards against reintroducing the cross-request contamination described above.
• test_audio_buffer_is_capped and test_transcript_text_is_not_logged_at_info — the two security properties. Both are easy to regress with an innocent-looking refactor and invisible when they break.

End to end

Unit tests never touch the real model, so after any model or library change:

./test/make-clips.sh                                    # generates via macOS TTS
.venv/bin/python test/wy-test.py test/clips/*.wav

Expect all ten commands correct, silence.wav empty, ~100–150 ms each once warm. The first request or two after a restart run slower (~200–350 ms) while Metal compiles its kernels. Check number formatting, not just the words — cmd2, cmd4 and cmd7 are the ones that catch a model with weak ITN.

Updating the library

.venv/bin/pip install -U parakeet-mlx mlx mlx-metal wyoming

Re-run both test suites afterwards. This project calls get_logmel() directly rather than load_audio() (which shells out to ffmpeg), so it depends on two parakeet-mlx internals rather than public API — the tests above are what tell you if either moved.

Logs

tail -f /tmp/local.wyoming-parakeet.stderr

Each request logs audio duration, inference time and the transcript's character count — not the transcript itself. Pass --debug to include the text, bearing in mind that the log then contains everything said to your voice assistant, in a file other local accounts can read. See Security.

The daemon runs as its configured --user, and launchd opens these files as that user; if you change --user, re-run install.sh rather than editing the plist, so the existing log files are chowned across.

Security

The Wyoming protocol has no authentication or transport encryption. Any client that can reach the port can submit audio, and Home Assistant trusts whatever this service returns — a transcript becomes an intent, and an intent unlocks doors. Treat the port as a trust boundary and keep it on a network you control.

Two things this server does about that:

• Buffered audio is capped (--max-audio-seconds, default 120). Audio accumulates until AudioStop, and a client that never sends one — malicious or just stuck — otherwise grows the buffer indefinitely. Measured at ~11 MB/s over loopback, enough to exhaust 32 GB in under an hour from one connection. Over the cap, further audio is dropped with a single warning and whatever was captured is still transcribed.
• Transcripts are not logged at INFO. The log records duration, latency and character count; the text itself is behind --debug. Log files are long-lived, and on macOS /tmp they are world-readable by default — meaning every voice command would otherwise sit in plaintext readable by any local account.

Worth doing yourself, depending on your threat model:

• Bind to one interface. The daemon listens on 0.0.0.0, so it is exposed on every network the host is attached to — including VPN interfaces like Tailscale, which is easy to overlook. Set the URI to a specific address (--uri tcp://192.168.1.10:7892) or firewall the port to your Home Assistant host.

• Do not run it as an admin account. The installer defaults --user to whoever runs it. On a typical macOS setup that account is in admin, and if %admin has a NOPASSWD sudo rule then a compromise of this service is a direct path to root. To use a dedicated service account:

  sudo dscl . -create /Groups/_parakeet PrimaryGroupID 450
  sudo dscl . -create /Users/_parakeet UniqueID 450
  sudo dscl . -create /Users/_parakeet PrimaryGroupID 450
  sudo dscl . -create /Users/_parakeet UserShell /usr/bin/false
  sudo dscl . -create /Users/_parakeet NFSHomeDirectory /usr/local/var/wyoming-parakeet
  sudo dscl . -create /Users/_parakeet Password '*'
  sudo dscl . -create /Users/_parakeet IsHidden 1
  sudo mkdir -p /usr/local/var/wyoming-parakeet
  sudo chown _parakeet:_parakeet /usr/local/var/wyoming-parakeet
  
  ./install.sh --user _parakeet
  

  Two things to get right. Keep the checkout outside your home directory — macOS home directories are drwxr-x---, so a service account that is not in your group cannot traverse into one; /opt/wyoming-parakeet works, owned by you so git pull && ./install.sh still needs no sudo, and read-only to the service. And the model cache follows HOME, which is the service account's home, so either let install.sh download it as that user or move an existing models--mlx-community--parakeet-tdt-0.6b-v2 directory into <service home>/.cache/huggingface/hub/.

• Pin your dependencies if you care about supply chain. requirements.txt is deliberately loose so install.sh picks up fixes; pin exact versions (and ideally hashes) if you would rather audit upgrades. Model weights are safetensors, so loading them does not execute code, but the initial download from HuggingFace is trust-on-first-use — pin a revision if that matters to you. HF_HUB_OFFLINE=1 in the daemon means it never re-fetches after that point.

License

MIT