heypoul

heypoul turns speech into text with Microsoft's MAI-Transcribe-2 on Azure AI Foundry. It does two things:

• Push-to-talk dictation. Hold a key, speak, and the words appear in the overlay while you talk (MAI-Transcribe-2-Streaming). Release, and the text lands in a draft you can read, extend, undo and clean up before you say "hey poul send" and it is typed into the focused window (see Draft mode). On Windows the key is Right Alt unless you pass --key (a virtual-key code, e.g. 0x78 for F9) or set HEYPOUL_KEY.
• Meeting mode. heypoul meeting transcribes a call live on two tracks, labelled Dem (what your speakers or headset play, captured by WASAPI loopback) and Dig (your microphone). You keep hearing the call as normal; there is no virtual audio device to install.

While it runs, a small pill sits at the bottom of the screen (drag it anywhere): a state orb, the live waveform, and the words as they arrive. Idle it shows which key to hold; after a release it shows what was typed.

Install

Windows (PowerShell):

irm https://agentics.dk/install/heypoul.ps1 | iex

Linux:

curl -fsSL https://agentics.dk/install/heypoul.sh | bash

Released builds are amd64 and cloud-only.

Run it through pks

The recommended way to give heypoul access to Foundry is pks exec. Sign in once with pks foundry init, then:

pks exec heypoul
pks exec heypoul meeting

heypoul never receives an Azure token. pks starts a credential proxy on 127.0.0.1 in front of your Foundry resource and gives heypoul its address and a throwaway token that only opens that proxy. The proxy attaches a fresh Entra token to every request and every WebSocket, so a meeting can run longer than the token's one-hour lifetime.

Without pks, set HEYPOUL_ENDPOINT and either HEYPOUL_API_KEY or HEYPOUL_TOKEN yourself.

Draft mode

Since 0.3.0 dictation is staged: nothing is typed when you release the key. The text goes into a draft, the end of it shows in the pill with a ▴, and you drive it by voice. Say the wake word "hey poul" (also heard as "hej poul", "hey Paul") followed by a command, alone or at the end of a sentence:

"Hey poul" inside a sentence stays text when no command follows it. A command at the end of a hold is split off: "Vi tager det i morgen, hey poul send" adds the sentence and sends.

Without the wake word. A hold that is nothing but a command runs it: "Send.", "vis teksten", "fortryd", "skift model" — up to three words, so "send mig lige en mail" is dictation. Inside a longer hold the same bare words run only when they are reversible (show, hide, undo, clean, the model commands); send, enter and ryd in the middle of speech still need hey poul in front, or they are text.

Read at every pause. The stream completes a run of speech at each pause (about half a second of silence), and heypoul reads each run on its own as it completes, before the key is released: a command in the middle of a hold first lands what was said before it, then runs, so "… [pause] hey poul vis tekst [pause] og så videre" opens the panel between the two pieces of text. A wake word followed by words the grammar does not know ("Hey Paul, vi teksten" is how vis teksten is sometimes heard) is sent to the decision model right away, with the command list as the options and none among them; a confident command runs when the hold lands, anything else stays text, wake word and all. Corrections to the content ("nej, ret det til tre") need no command at all: the cleanup model applies them.

Cleanup. The draft is rewritten by a chat deployment after every hold: fillers and false starts go, and self-corrections are applied, so "to, nej faktisk tre tusind" becomes "tre tusind" and "ret kunden til Andel" replaces the name. Each cleanup reads the previous cleaned text followed by the new hold, not the raw words from the start, so a correction applied once stays applied and the panel never flashes the old words while the model works; the header says renser meanwhile. The panel always shows exactly what send will type. A name the model picker cannot match ("skift mod den") opens the four-option choice instead of failing.

Choices. Whenever heypoul needs a decision from you it asks the same way: a title and at most four numbered options fill the panel, and the next hold answers it, no wake word needed. Say the number ("to", "nummer tre", "den anden"), the name or enough of it ("luna", "gpt fem to"), or "flere" for the next page. Those the grammar reads on its own, and it reads them from the live words, so a "tre" is taken while the key is still down. Anything else ("den nyeste", "nej ikke luna, den anden") goes to the decision model, which answers in a quarter second with a confidence; below 0.85 heypoul says ikke forstået · sig et tal and keeps asking. A confident none (you went on dictating over the question) closes it and keeps your words. "annuller", "luk" or "hey poul skjul" close it. Four options, numbered, is a habit that carries to a car or a phone unchanged.

Which model. Nothing needs configuring: at start heypoul reads the model catalog pks serves at /pks/v1/models (every deployment on your resource, with the default and the enabled set from pks foundry select) and takes pks' default when it can chat, else the newest plain GPT deployment. The panel header names it. hey poul skift model asks with a choice; hey poul brug luna switches by name; either way one probe request first proves the deployment answers (the Claude deployments on a Foundry resource do not on this route; the HUD says svarer ikke and keeps the old model). A spoken switch is saved to config.json; HEYPOUL_TEXT_MODEL overrides it.

The decision model is the deployment pks foundry select names as Voice classifier model when that is a Decision model (Decision-1), else any Decision deployment on the resource, else none, in which case answers must be a number or a name. HEYPOUL_DECISION_MODEL overrides. From the shell:

pks exec heypoul -- models          # numbered, * = in use, [pks default], [judges answers]
pks exec heypoul -- models --json   # OpenAI list shape with enabled, roles, output
pks exec heypoul -- decide --option gpt-5.2 --option gpt-6-astra --option gpt-5.6-luna --option Flere "den nyeste"

With pks ≥ 7.10.0 the list is pks' catalog across every registered provider (Foundry, Gemini, Anthropic, OpenAI, Moonshot), each row saying who serves it, and cleanup goes through pks' chat route so heypoul never holds a key. The switch flow in pictures, the voice answers and the fallback for older pks are in the tutorial Skift model med stemmen. The catalog's shape, where it comes from and the measured Decision-1 verdicts are in docs/model-catalog.md.

Try a deployment first:

pks exec heypoul -- clean --model gpt-5.5 "øhm vi sender det mandag, nej tirsdag"

gpt-5.5 and gpt-5.6-luna both answer in 1.5–2 s on a 40-word draft.

Direct mode. HEYPOUL_MODE=direct (or "mode": "direct" in config.json) types the text on release as before 0.3.0; voice commands are off there.

Clicks

The panel is clickable since 0.9.0: ▴ opens it, the button at the header's right closes it, a row of a choice or of the model list picks it, the settings rows open their choices, × on a word-list chip removes the word, stop in the meeting view stops the meeting. A click does exactly what the spoken command does. The overlay should not take the focus from the window you dictate into (WS_EX_NOACTIVATE); if it does on your desktop, start with HEYPOUL_RESTORE_FOCUS=1 and the focus is put back before each injection, and say so.

Settings

hey poul indstillinger, or Settings… in the tray, opens the settings view in the panel. Each row is a four-option choice, so it is answered by voice or by click; tryk ny tast waits for the next key you press. Changes land in config.json at once and apply without a restart.

Panel height and idle. Drag the panel's top edge (the small bar) to make it taller, up to the top of the screen, or shorter, down to 240 dp; the height is saved as panelH in config.json (0 = the built-in 420). After eight seconds with nothing happening and the pointer elsewhere the panel dims to about half; any change, a spoken word or a hover brings it back. The pill never dims.

Meeting mode

In the daemon: hey poul lyt med, or Lyt med in the tray, or a click on stop / the green dot to end it. The meeting view shows the newest sixteen lines of both tracks, the unfinished sentence of each in orange, and the header carries a green lytter dot in every view. Push-to-talk keeps working; what you dictate also lands in the Dig track. The same engine runs standalone for scripts:

pks exec heypoul meeting

Each meeting gets a folder (Documents\heypoul\meetings\<date_time> on Windows) with:

The live text is fast but has no phrase list. For names and domain terms, transcribe the WAV files again afterwards in batch mode.

On loudspeakers the microphone also picks up the other side, so on Windows the "Dig" track is opened through the operating system's own echo canceller (the Voice Capture DSP, in Windows since Vista); it subtracts what the output device plays and keeps the two device clocks in step itself. If it cannot be opened the header says why and the raw microphone is used; HEYPOUL_AEC=0 skips it, and heypoul --aec-probe 10 prints a JSON diagnosis with the endpoints, the samples that came out and a WAV to listen to. Loopback captures everything the output device plays, including notification sounds and music.

When live words do not appear

With engine=stream the words should show in the pill while the key is held. If they only arrive after release, the stream failed and heypoul fell back to batch transcription. Since 0.2.1 the console (and %TEMP%\heypoul.log on Windows, /tmp/heypoul.log elsewhere) says why, one line per attempt:

stream connected in 412ms (ws://127.0.0.1:53342/mai/v1/realtime?intent=transcription)
text from stream (23 chars, final after 180ms)

or

stream dial failed after 10s: realtime dial ws://…: context deadline exceeded
stream gave no result after 5s (…) — batch transcription instead

A stream rejected the session line means the service refused the session.update; heypoul then falls back to batch at once. The common case is misleading on the wire:

stream rejected the session: invalid_value: PCM input rate must be 24000, or 16000 for MAI transcription. — the resource behind the endpoint does not know MAI-Transcribe-2-Streaming; …

The rate is not the problem. The service answers this when the model name does not resolve on the Foundry resource, which happens when MAI-Transcribe-2-Streaming is not deployed there or the resource is in a region that does not serve it (served from swedencentral, centralus, eastus2 and southeastasia). Batch MAI-Transcribe-2 is available more widely, which is why push-to-talk still works. Fix it on the resource: deploy the model, or run pks foundry init and pick a resource in one of those regions. heypoul meeting --replay-them <wav> exercises the same stream without a microphone and prints the same lines.

Environment