Rewrite setup instructions for non-specialist readers
The audience for this is Home Assistant users, who are not necessarily developers. The instructions assumed too much and buried the step most likely to trip someone up. Biggest fix: connecting to Home Assistant is two steps, not one. Adding the Wyoming integration does nothing on its own -- the assistant keeps using whatever speech-to-text it used before until you change it under Settings -> Voice assistants. That was previously half a sentence at the end of a paragraph, and its failure mode is silent: everything looks installed and nothing improves. It is now its own numbered step with an explanation. Also adds: - "Check it's working" and "Troubleshooting", the latter drawn from failures actually hit while building and deploying this, including the wrong-Python error and the service-did-not-start-after-changing-user case. - IP address guidance that survives contact with a real machine. Checking on the deployment host showed five addresses -- two Parallels interfaces, a Tailscale one, and two real ones -- so the docs now say which to ignore. The obvious `ipconfig getifaddr en0` was rejected: it returns nothing there. - Requirements now lead with `brew install python@3.13` rather than an explanation of lzma; the reasoning moved into a collapsed block. - A plain-English guide to reading the benchmark table, and an explicit note that the numbers are a snapshot nothing re-runs automatically. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -23,6 +23,20 @@ means are skewed by first-request kernel compilation.
|
||||
| mlx-whisper large-v3 | MLX | 1170 ms | 1251 ms | 48/48 | 21/21 | `"thank you"` |
|
||||
| faster-whisper distil-large-v3 | CPU int8 | 4269 ms | 4304 ms | 46/48 | 21/21 | `"thank you"` |
|
||||
|
||||
### How to read this
|
||||
|
||||
- **Median** — the typical time to transcribe one command. This is the number
|
||||
you feel when talking to your assistant.
|
||||
- **p90** — nine times out of ten it was at least this fast. A p90 close to the
|
||||
median means consistent; a long tail is worse than the median suggests.
|
||||
- **Exact** — how many of the 48 clips came back word-for-word correct.
|
||||
- **Digits** — of the 21 clips containing a number, how many wrote it as `21`
|
||||
rather than `twenty one`. This matters more than it looks; see
|
||||
[Model choice](README.md#model-choice-v2-not-v3).
|
||||
- **Silence** — what came back for three seconds of pure silence. Anything
|
||||
other than nothing is the model inventing words, which your assistant then
|
||||
tries to act on.
|
||||
|
||||
**Exact** is a strict string match after normalising case, punctuation and
|
||||
whitespace. **Digits** counts how many of the 21 number-bearing clips came
|
||||
back with digits rather than spelled-out words — see
|
||||
@@ -59,6 +73,11 @@ What the numbers say:
|
||||
- **Latency is raw inference**, excluding Wyoming protocol overhead. End to
|
||||
end through this server, expect roughly 15–35 ms on top.
|
||||
- One machine, one run each. Treat differences of a few percent as noise.
|
||||
- **These are a point-in-time snapshot.** Nothing re-runs them automatically,
|
||||
and the test suite cannot detect drift, so treat them as stale after any
|
||||
change to the audio path, a `parakeet-mlx`/MLX upgrade, or a macOS release
|
||||
(which changes the TTS voices the clips are built from). Re-run the sweep
|
||||
and update the table in the same commit as the change.
|
||||
|
||||
## Reproducing
|
||||
|
||||
|
||||
Reference in New Issue
Block a user