Blog ·

Underdog Bark comes to utt

Two English speech models from Conway Research. Bark Clean leaves out the ums as it writes. Bark Verbatim keeps every word. Both arrive in the next update.

Underdog Bark is a family of English speech models from Conway Research, the company behind the Underdog app. Bark started as NVIDIA's Parakeet v3, the model utt uses by default, and was trained further on English.

utt now offers two of them. Both run on the Mac's Neural Engine, like every Parakeet model in utt, and both download about 440 MB.

Clean or Verbatim

  • Bark Clean writes what you meant to say. It leaves out "um" and "uh", false starts and words you repeated, as it transcribes. Pick it for dictation.
  • Bark Verbatim writes what you said, hesitations included. Pick it for meeting notes, captions or anything that has to match the recording.

How they compare

We ran every Parakeet-engine model utt offers, apart from the Japanese one, on the first five minutes of Ken Robinson's TED talk Do schools kill creativity? on an M2 Pro. Each transcript was scored against TED's own subtitles. Bark is in orange; hover over a model for its numbers.

Accuracy against speed Up is fewer errors, right is faster. Error rate with "um" and "uh" taken out, speed in seconds of audio per second, on a log scale.
30×100×300×5%6%7%8%9%10%11%Parakeet English Classic — NVIDIA, nvidia/parakeet-tdt-0.6b-v2: 6.2%, 176×Parakeet English Classic NVIDIA · parakeet-tdt-0.6b-v2Bark Clean — Conway Research, ConwayResearch/Underdog-Bark-3.0-Clean: 7.1%, 110×Bark Clean Conway Research · Underdog-Bark-3.0-CleanParakeet Quick — NVIDIA, nvidia/parakeet-tdt_ctc-110m: 7.1%, 288×Parakeet Quick NVIDIA · parakeet-tdt_ctc-110mBark Verbatim — Conway Research, ConwayResearch/Underdog-Bark-3.0: 8.8%, 121×Bark Verbatim Conway Research · Underdog-Bark-3.0Parakeet Ultra — Moondream, moondream/parakeet-ultra: 8.8%, 100×Parakeet Ultra Moondream · parakeet-ultraParakeet Standard — NVIDIA, nvidia/parakeet-tdt-0.6b-v3: 9.6%, 135×Parakeet Standard NVIDIA · parakeet-tdt-0.6b-v3Parakeet Redux — Moondream, moondream/parakeet-redux: 10.1%, 34×Parakeet Redux Moondream · parakeet-redux
Word error rate Every word"Um" and "uh" taken out TED's subtitles leave out hesitations, so a model that writes them down loses points for them. The gap between the dot and the ring is how many fillers a model kept.
0%5%10%15%Parakeet English Classic — NVIDIA, nvidia/parakeet-tdt-0.6b-v2: 6.6% / 6.2%Parakeet English Classic NVIDIA · parakeet-tdt-0.6b-v2Bark Clean — Conway Research, ConwayResearch/Underdog-Bark-3.0-Clean: 7.1% / 7.1%Bark Clean Conway Research · Underdog-Bark-3.0-CleanParakeet Quick — NVIDIA, nvidia/parakeet-tdt_ctc-110m: 7.1% / 7.1%Parakeet Quick NVIDIA · parakeet-tdt_ctc-110mBark Verbatim — Conway Research, ConwayResearch/Underdog-Bark-3.0: 11.4% / 8.8%Bark Verbatim Conway Research · Underdog-Bark-3.0Parakeet Ultra — Moondream, moondream/parakeet-ultra: 9.9% / 8.8%Parakeet Ultra Moondream · parakeet-ultraParakeet Standard — NVIDIA, nvidia/parakeet-tdt-0.6b-v3: 12.7% / 9.6%Parakeet Standard NVIDIA · parakeet-tdt-0.6b-v3Parakeet Redux — Moondream, moondream/parakeet-redux: 12.4% / 10.1%Parakeet Redux Moondream · parakeet-redux
Load time First loadEvery load after Seconds, on a log scale. The first load downloads the model and compiles it for the Neural Engine. Parakeet Standard was already compiled on this Mac, so its first load is not a real one.
1 s10 s100 s1000 sParakeet Redux — Moondream, moondream/parakeet-redux: 604 s / 26 sParakeet Redux Moondream · parakeet-reduxBark Verbatim — Conway Research, ConwayResearch/Underdog-Bark-3.0: 140 s / 2 sBark Verbatim Conway Research · Underdog-Bark-3.0Bark Clean — Conway Research, ConwayResearch/Underdog-Bark-3.0-Clean: 96 s / 3 sBark Clean Conway Research · Underdog-Bark-3.0-CleanParakeet Ultra — Moondream, moondream/parakeet-ultra: 85 s / 4 sParakeet Ultra Moondream · parakeet-ultraParakeet English Classic — NVIDIA, nvidia/parakeet-tdt-0.6b-v2: 48 s / 2 sParakeet English Classic NVIDIA · parakeet-tdt-0.6b-v2Parakeet Quick — NVIDIA, nvidia/parakeet-tdt_ctc-110m: 29 s / 2 sParakeet Quick NVIDIA · parakeet-tdt_ctc-110mParakeet Standard — NVIDIA, nvidia/parakeet-tdt-0.6b-v3: 3 s / 2 sParakeet Standard NVIDIA · parakeet-tdt-0.6b-v3
ModelErrorSpeedFirst loadLater
Parakeet English ClassicNVIDIA6.2%176×48 s2 s
Bark CleanConway Research7.1%110×96 s3 s
Parakeet QuickNVIDIA7.1%288×29 s2 s
Bark VerbatimConway Research8.8%121×140 s2 s
Parakeet UltraMoondream8.8%100×85 s4 s
Parakeet StandardNVIDIA9.6%135×3 s2 s
Parakeet ReduxMoondream10.1%34×604 s26 s

Bark Clean is among the most accurate models here, and Bark Verbatim makes fewer errors than Parakeet Standard, the model both were trained from. Parakeet English Classic and Parakeet Quick also do well on this talk. This is one speaker in one clip, so read the order as rough. The Bark model cards report results on seven larger test sets.

Wait for the first load

The first time you pick a Bark model, the Mac compiles it for the Neural Engine. That took a minute and a half to two and a half minutes on the M2 Pro. utt shows Loading on the Model page while it works, and anything you dictate in the meantime waits for it. After that, Bark loads in two or three seconds.

Phonon-2 is gone

The same test showed Phonon-2 running at half the speed of speech on the M2 Pro, with a twelve-minute load every time utt started. FluidAudio, who converted it for the Mac, measured it on an M5 Pro. It is no longer in the model list. If you had it selected, utt switches to Parakeet Standard.

Try it

After the update, open Settings, go to Model and pick Bark Clean or Bark Verbatim. Bark is English only. For other languages, keep a Parakeet or Whisper model.

Underdog Bark 3.0 and Bark 3.0 Clean are by Conway Research, licensed CC-BY-4.0. The Core ML conversion utt runs is by FluidInference.