How Confer cleans up voice dictation without losing your meaning
Confer cleans up voice dictation without losing your meaning. It keeps your final answer and every constraint, and preferences let you choose what it adds.
The Confer Team5 min read
Below is a request to a coding agent, dictated the way people talk while still working out the details.
okay so on the settings page, the save button, can you make it blue, no actually green, to match the rest of the app, sorry, I just got pulled into a Slack thread about the invoice, okay, where was I, move it to the left side, and don't touch the cancel button, I already fixed that one yesterday
Confer pastes the following.
On the settings page, can you make the save button green to match the rest of the app and move it to the left side? Don't touch the cancel button. I already fixed that one yesterday.
Confer kept the final color and dropped the first one, left out the interruption about the invoice, and kept the "don't" together with the reason for it. Many dictation apps now remove filler words and correct grammar. Confer goes further and works out what you meant, so what you paste reads as if you had typed it yourself. Every output in this post is what Confer's current cleanup returned, with all preferences on.
You also decide what Confer does beyond cleanup. Ask a coding agent to redesign a page and leave the look open, and Confer asks for a few versions to compare. Swear while describing a bug, and Confer keeps your point in plain words. Both are preferences you can turn off.
How do prompts and preferences shape the output?
A prompt sets the format of the output, e.g., Clean for plain dictation, Email for a message in Mail, or Engineering Task for a brief to a coding agent. You can edit the built-in prompts and add your own, and Confer chooses the right one from the app you are in and what you say. Preferences are rules that Confer adds after every prompt, so even a custom prompt that only says "write it as a tweet" keeps your constraints and your final answer.
How does Confer decide what to keep?
Confer cleans up the speech model's words on a frontier language model, and its instructions rank four goals in order. When two goals conflict, the earlier one wins.
- Faithful. The text keeps every point, request, name, number, and negation, in the order you said them.
- Correct. Every sentence is complete, with correct spelling, grammar, and punctuation.
- Clear. A vague phrase such as "the thing" becomes what it refers to when the context makes it clear.
- Concise. Confer removes words that carry no meaning, and only those words.
Cleanup also works outside of code, e.g., for a to-do list dictated into a notes app.
okay so things for tomorrow, um, pick up the dry cleaning, call the dentist to move my appointment, actually no, just cancel it, I'll rebook next month, and, uh, email Priya the slides before ten, sorry, the dog is barking at the mail carrier again, and grab groceries, we're out of eggs and coffee, oh and the car registration is due Friday
Confer writes:
Things for tomorrow:
- Pick up the dry cleaning.
- Call the dentist to cancel my appointment. I'll rebook next month.
- Email Priya the slides before ten.
- Grab groceries. We're out of eggs and coffee.
The car registration is due Friday.
What do Confer's preferences do?
Preferences are rules in Confer's settings, and every rule starts on. You can turn off any rule you don't want, or skip one for a single dictation by saying so.
| Preference | What it does |
|---|---|
| Keep every constraint | Keeps each requirement, preference, and reason, including what to avoid |
| Use your final answer | Keeps only the final version when you change your mind |
| Drop tangents | Leaves out unrelated asides, and keeps anything that might matter |
| Write names exactly | Writes files, code, products, and people exactly, and never guesses |
| Never invent details | Never adds anything you didn't say or clearly imply |
| Ask for variations | Asks an agent for several versions when you leave a design open |
| Remove swearing | Replaces swear words with plain words that keep your meaning |
| Write version numbers | Writes a version you read out, like "one zero six seven," as 1.0.67 |
Ask for variations is the one preference that adds something you didn't say. When you ask an agent to redesign a page and leave the look open, or say you're torn between options, Confer asks for several versions to compare side by side. For example, "the pricing page hero feels cramped, can you redesign it, I'm torn between keeping the dot terrain behind the headline or just a plain background" comes out as follows.
The pricing page hero feels cramped. Can you redesign it? I'm torn between keeping the dot terrain behind the headline or using a plain background. Please create several distinct variations to compare side by side, including options with the dot terrain and a plain background.
When you have already decided the change, as with the settings page example at the top, Confer adds nothing.
How do we test changes to the cleanup?
A unit test can check how Confer builds the cleanup instructions, but it can't check what the model writes, so every change to text the model reads goes through a prompt eval before it ships.
- Confer's production code builds the exact request for each test dictation, once for the live version and once for the change.
- Each case runs 5 times on both versions. The cases include real dictations and holdout cases written before any tuning.
- A frontier model grades every output twice, and a defect counts only when both gradings flag it.
- A blind side by side comparison on 50 real dictations checks that ordinary cleanup didn't get worse.
Each round is scored against fixed limits, e.g., at least 98 percent of points kept and at most 2 percent of outputs with a major meaning error.
The current Drop tangents and Ask for variations rules came out of a review of 279 of our own dictations. The table compares the old and new rules over two rounds of 32 cases.
| Check | Old rules | New rules |
|---|---|---|
| Tangents dropped | 55% to 63% | 100% |
| Points kept | 98% to 99% | 99.6% to 100% |
| Variations requested where not wanted | 20% to 26% | 0% |
| Variations requested where wanted | 100% | 94% to 98% |
| Outputs with a major meaning error | 0.6% to 1.2% | 0% |
The eval also shaped Write version numbers. Speech-to-text writes a spoken "one zero six six" as 1066, 10.66, or 1.0.6.6, and without the rule Confer wrote 33 percent of versions correctly. With the rule, it writes 95 to 97 percent correctly and leaves every other number alone. The fast model that handles short dictations got as few as 25 of 60 versions right, so a short dictation with a version now runs on the smarter model, which got 45 of 45.
The same cleanup pass also listens for commands you give Confer itself, e.g., "don't send this yet," and a later post covers how those commands work.