Sketchnote
Show your coding agent the problem instead of typing it out.
2026 · Website · Source on GitHub
Overview
Press a hotkey, talk through a bug, and draw or click on the screen. One Ctrl+V then pastes timestamped notes and annotated screenshots into your coding agent. It's a GNOME Shell extension with a Rust service behind it, installed with one command. Speech is transcribed on your own machine by default, so nothing leaves it unless you paste it.
How a capture becomes notes
Capture
You circle a footer, click a button, and say what's wrong with each.
Record
The screen, the mic, and every stroke, pin and click with its time and position.
Transcribe
whisper.cpp turns the speech into punctuated text with a timestamp for every word.
Match
Each sentence goes to the mark it overlaps most in time. A click nobody talked about stays in the notes but gets no screenshot.
- stroke → #1
- click, no speech → unnumbered
- click + speech → #2
Screenshots
A keyframe for each mark, cropped around the activity with the annotations drawn in, laid out as one numbered contact sheet.
Pasted notes
#1 (0:04) “This footer has too much text, let's cut it down.” (0:08) [clicked] #2 (0:11) [clicked] “This button should open the settings instead.”
The contact sheet is pasted right after the text, so the agent gets both from one key press.
Results
- Install
- One command, no Rust toolchain needed
- Transcription
- ~10 s for ~21 s of speech, on CPU
- Shipping
- 21 PRs, 2 releases, 31 tests
Highlights
- Each sentence you say is attached to the mark it was about, so the notes read the way you meant them. Mid-sentence splits went from 5 of 6 to 0 of 6.
- Transcription runs locally with whisper.cpp, or through Groq or any compatible API. If the API fails, the local model takes over, so a capture is never lost.
- Ships as a prebuilt binary with first-run setup and a safe self-update that never interrupts a recording.
- An end-to-end test drives real keyboard and mouse input into a headless GNOME Shell, and caught a clipboard bug before release.