Wyoming speech-to-text server for Home Assistant using NVIDIA Parakeet

Loads parakeet-mlx in-process (no HTTP hop) and serves it over the Wyoming protocol. On an M4 Mac mini this transcribes typical voice commands in ~110ms versus ~1150ms for a whisper.cpp large-v3 setup, with identical accuracy on a ten-command benchmark. Two behaviours matter beyond speed: silence returns an empty string rather than whisper's "Thank you." hallucination, and there is no decoder context carried between requests. Notable implementation details, all covered by mutation-checked regression tests: - MLX streams are thread-local, so the model is loaded and evaluated on a single dedicated worker thread. Splitting those raises "There is no Stream(cpu, 1) in current thread". - parakeet_mlx.load_audio() shells out to ffmpeg, which is unnecessary here since Wyoming delivers 16kHz mono PCM. The mel is built directly via get_logmel(), whose input must be float32 -- it views the complex STFT output as the input dtype, so anything narrower doubles the mel bin count. - Wyoming's run loop has no except clause, so an exception escaping handle_event closes the connection without sending a Transcript and Home Assistant waits indefinitely. Failures are caught and returned as an empty transcript instead. Defaults to parakeet-tdt-0.6b-v2 rather than the newer multilingual v3 because v2 emits digits ("21 degrees") where v3 spells numbers out, and Home Assistant's local intent matching expects digits. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 03:04:48 +01:00
commit 4286e88344
18 changed files with 1168 additions and 0 deletions
@@ -0,0 +1,91 @@
+#!/usr/bin/env python3
+"""Wyoming ASR server for NVIDIA Parakeet via parakeet-mlx (Apple Silicon)."""
+import argparse
+import asyncio
+import logging
+
+from wyoming.info import AsrModel, AsrProgram, Attribution, Info
+from wyoming.server import AsyncServer
+
+from .engine import ParakeetEngine
+from .handler import ParakeetEventHandler
+
+_LOGGER = logging.getLogger(__name__)
+__version__ = "1.0.0"
+
+# v2 is English-only but does better inverse text normalisation than the
+# multilingual v3 ("21 degrees" / "30%" rather than "twenty-one degrees" /
+# "thirty percent"), which is what Home Assistant's local intent matching
+# expects. Don't switch to v3 without re-checking that.
+DEFAULT_MODEL = "mlx-community/parakeet-tdt-0.6b-v2"
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--uri", required=True, help="unix:// or tcp://")
+    parser.add_argument("--model", default=DEFAULT_MODEL, help="HuggingFace model id")
+    parser.add_argument("--language", default="en", help="Language code reported to HA")
+    parser.add_argument("--debug", action="store_true", help="Log DEBUG messages")
+    parser.add_argument("--log-format", default=logging.BASIC_FORMAT)
+    parser.add_argument("--version", action="version", version=__version__)
+    return parser
+
+
+def build_info(model: str, language: str) -> Info:
+    """Describe this service to Home Assistant's Wyoming config flow."""
+    return Info(
+        asr=[
+            AsrProgram(
+                name="parakeet-mlx",
+                description="NVIDIA Parakeet TDT via MLX",
+                attribution=Attribution(
+                    name="senstella", url="https://github.com/senstella/parakeet-mlx"
+                ),
+                installed=True,
+                version=__version__,
+                models=[
+                    AsrModel(
+                        name=model,
+                        description=model,
+                        attribution=Attribution(
+                            name="NVIDIA",
+                            url="https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2",
+                        ),
+                        installed=True,
+                        version=None,
+                        languages=[language],
+                    )
+                ],
+            )
+        ]
+    )
+
+
+async def main() -> None:
+    args = build_parser().parse_args()
+
+    logging.basicConfig(
+        level=logging.DEBUG if args.debug else logging.INFO, format=args.log_format
+    )
+
+    _LOGGER.info("Loading %s", args.model)
+    engine = ParakeetEngine(args.model)
+    await engine.start()
+
+    wyoming_info = build_info(args.model, args.language)
+    server = AsyncServer.from_uri(args.uri)
+    _LOGGER.info("Ready on %s", args.uri)
+    await server.run(
+        lambda *a, **kw: ParakeetEventHandler(wyoming_info, args, engine, *a, **kw)
+    )
+
+
+def run() -> None:
+    try:
+        asyncio.run(main())
+    except KeyboardInterrupt:
+        pass
+
+
+if __name__ == "__main__":
+    run()