Building an MCP server for an app the AI can't hear: lessons from Instant DJ
Instant DJ, our Mac DJ app and studio, runs a local MCP server so an AI assistant can play, write and mix music in it. The assistant can't hear a note, so the design is about maps, numbers, musical time and a tight local boundary.
Instant DJ, our DJ app and music studio for the Mac, runs a Model Context Protocol (MCP) server so an AI assistant can play it, write songs in it and mix them. The hard part wasn’t the protocol. It was that the assistant on the other end can’t hear a single note, so everything it needs to make music has to arrive as structure and numbers. Here is what we built, and what we would tell anyone adding MCP to an app.
Keep it local, and closed to web pages
The server runs inside the app on 127.0.0.1:47231 and speaks JSON-RPC 2.0 over streamable HTTP. It is off until the user switches it on under Setup › AI tools, where the connect command waits with a Copy button:
claude mcp add --scope user --transport http instantdj http://127.0.0.1:47231/mcp
Listening on loopback keeps the network out, but not the browser: any web page can send requests to localhost. So the server refuses three things:
- A foreign
Originheader. Anything buthttp://127.0.0.1orhttp://localhostmeans a web page, and gets a 403. - A
Hostheader that isn’t the server. That is how DNS rebinding gets in; also a 403. - A body that isn’t
Content-Type: application/json. A browser can posttext/plainwithout asking first, so plain text gets a 415.
No Access-Control-Allow-Origin header is ever sent. Other programs run by the same user on the same Mac can still connect, and that is the point: the user’s own assistant is exactly such a program.
Hand the model a map before anything else
An assistant that guesses at an interface wastes calls and makes mistakes. The first tool, describe_app, returns the whole map in one answer: every panel and what it is for, every instrument, every key of the keyboard with what it plays right now, every shortcut, and every parameter by name with its range, default and current value. It ends with recipes, such as making a loop from nothing or writing a whole song and rendering it.
Parameters are set by name, like reverb.size or bass.cutoff, never by position on the screen. Names survive redesigns, and a model can read them.
Give it numbers for what it can’t hear
A person mixing a track listens. The assistant reads. So every tool that changes the sound can tell it what changed:
- The meters.
get_statereports the output peak, a 300 ms RMS in dBFS and the ducking gain alongside the transport and the tracks. - The render.
bouncerenders offline, about 30 times faster than real time, and returns the file with its peak, its RMS and where each section starts. - The ears. In the Studio,
analysereads every track, the mix and the master: loudness (ITU BS.1770, gated), peak, the share of energy in six bands, how correlated the two sides are, and loudness per section. Then it says what stands out, in words: a drop no louder than what came before it, a melody buried under the drums, a part crowding the bass, a limiter asked to take off more than 4 dB. It names the loudest 100 ms of the song and the tracks playing there. - The master.
mastermeasures the mix and sets the gain for a loudness target such as −14 LUFS, and holds back, saying why, where the limiter would have to work too hard.
The loop becomes: write, render, measure, fix. The human listens and steers, but the assistant no longer needs the human to tell it the drop is quiet.
Work in musical time
Music runs on bars, not milliseconds, and an assistant can’t react on the beat over HTTP. So the app does the timing. Changing the playing loop takes effect on the next bar line, so a call made anywhere in the bar lands on the one. wait_for_bar returns during the last sixteenth before a downbeat, so a change made right after it lands on that bar.
There is an honest edge: the next bar is decided as its downbeat is rendered, so a call in the last 10 ms or so before a downbeat lands a bar later. The documentation says so, rather than letting the assistant discover it.
Fewer, bigger calls
Round trips are slow and every one is a chance to drift. set_track writes whole rows of a bar in one call, in the same shape get_track returns, with note names a model already knows: C2, Eb2, chords like C3+Eb3+G3. Every editing tool takes a track argument, so editing one track never changes what is playing. And undo takes back the last edit, 32 deep, so a wrong guess is cheap.
Let it write data, and let the app render
The Studio, new in 1.6, takes this to whole songs. A song is a JSON document: sections, tempo, key, tracks with their sounds, patterns of notes and drum steps, clips placing them on bars, the mix and automation. The assistant writes the document; the app renders every track with its own engine into a stem, about 28 times faster than it plays, and caches stems so a change re-renders only the track it touched. Our 15-track, 3-minute showcase renders all its stems in about 15 seconds on an M-series MacBook Pro.
Reading the document is strict about what can’t be played, and kind about what merely looks odd. A clip naming a pattern that isn’t there is refused, with the place it went wrong. A field the app doesn’t know, or a note outside an instrument’s range, comes back as a warning without refusing the song. Error messages are plain sentences, because the reader is a model deciding what to try next.
The small things that matter
- Annotations. Every tool carries MCP annotations, with
readOnlyHinton the reads, so a client can skip asking the user before a harmless call. - The sandbox. The App Store build is sandboxed, so the tools write under
~/Music/Instant DJand read only what the user opened or anything under~/Music. The server tells the assistant this, instead of letting it fail on a path. - A self-test. One command exercises the server end to end: connect, list the tools, read the map, set and read parameters, write a track, undo, wait for a bar, render, save a project, and all three refusals.
What we would tell anyone adding MCP to an app
- Bind to loopback, and still refuse web pages by
Origin,Hostand content type. - Make the first tool a complete map, and name everything.
- Measure what the model can’t perceive, and say what stands out in words.
- Let the app own timing, rendering and anything else that needs to be exact.
- Prefer one call that does a whole job to many small ones, and make undo cheap.
- Write errors for a model: what went wrong, and where.
This is what made the first real use possible: an assistant scoring the background music and sound effects for a product video. Instant DJ is on the Mac App Store, free for 7 days, and more about it is on its page.
Questions, answered
- What is an MCP server in a desktop app?
- A Model Context Protocol server is a small endpoint inside the app that exposes its actions as tools an AI assistant can call. Instant DJ's runs on the Mac itself, at 127.0.0.1:47231, and speaks JSON-RPC 2.0 over streamable HTTP.
- Is a local MCP server safe?
- Instant DJ's is off by default, listens only on the Mac's loopback address, and refuses requests from web pages: a foreign Origin header, a Host header that isn't the server, and bodies that aren't JSON are all refused. Other programs of the same user on the same Mac can connect, which is the point.
- How does an AI assistant make music it can't hear?
- The app measures for it. It reports levels and loudness, the energy in each frequency band, and notes on what stands out, such as a drop no louder than what came before it. The assistant edits until the numbers say the mix is right, and you listen and steer.
- Which AI assistants work with Instant DJ?
- Any MCP client, such as Claude Code or Codex. Instant DJ has no AI inside; your own assistant connects to it on your Mac after you switch the AI tools on.