The Language That Does Not Yet Exist

My Tokenese project began with what I thought was a reasonable premise: a language designed for inter-model communication need not inherit every human ambiguity, inefficiency, and historical accident of our legacy use of language. It could be friendlier to tokenization, denser in meaning, more explicit about relations, and portable across model families. It could offer machines a medium suited to their probabilistic needs rather than endlessly translating through our primitive monkey chatter.

It took six weeks to build, and one afternoon to falsify.

I’ve written the full post-mortem and archived the repo. What follows is the part I care about here, which is not that the project failed. Plenty of my projects fail. It’s that I spent a few weeks afterward writing about how we measure writing quality, and recognized my own handwriting on the crime scene.

The number that killed it

On 6/18, during pre-publish validation of the announcement post, I re-counted the flagship example on the certified tokenizers. But it proved the opposite of my point. Oops!

The claimed figures, English around 55 against Tokenese around 22, had been eyeballed. Never measured. The spec had seven design invariants and the sixth one was literally “measured, not asserted,” which I had violated in the headline example on the landing page.

So I measured the premise properly. Same semantic task, encoded five ways:

Read that table slowly, because it says three things at once.

Most of the apparent savings was terseness, not language design. Verbose 36 to terse 18 is a 2x win available to anyone, right now, with zero spec and zero learning cost and full readability. The fair baseline for a designed language was never polite English. It was terse English, and against terse English the designed forms are a wash.

The designed syntax actively hurt. Dotted handles and sigil clusters are out of distribution for BPE tokenizers, so they shatter. @svc.logs.first-error is five tokens. !@svc.ok? is five. edge-fn is three. I had chosen, with great care, precisely the constructs that tokenize worst.

And a narrow honest win did survive: roughly 20 to 25 percent over terse English on structured conditional payloads built from operators that audit to a single token. Real, modest, fragile, and nowhere near the 2.5x to 4x the spec targeted.

The instrument worked, which is the good news

Every element in the Tokenese lexicon had to pass a reproducible cross-tokenizer audit before admission. Seven columns: OpenAI, Anthropic, Gemini, Qwen, DeepSeek, Llama, Gemma. CI-gated, so the alphabet couldn’t quietly expand.

That audit is what killed the project. The thing I built to defend the claim is the thing that disproved it, on schedule, before anyone else had to.

I want to sit in that for a second, because it’s the only part of this I’d repeat exactly. A project with an instrument sharp enough to falsify itself is a project that can end cleanly. Most of the ideas in this lineage didn’t end. They just stopped updating.

The steelman, and why it doesn’t hold

Here is where I have to argue with an earlier draft of myself.

The obvious defense, and the one I reached for first, is that token count is one dimension standing in for a vector. It answers a question about encoding efficiency, not about whether meaning survives the trip. A notation could lose that comparison by ten percent and still be the better medium, if what arrives at the other end is less ambiguous, more repairable, or cheaper to act on.

There’s a second move behind it, which I still think is true on its own terms. A language isn’t merely a syntax placed in a prompt. It becomes a language when its users learn it, develop expectations inside it, compress recurring structures into it, repair misunderstandings through it, and build a community of use around it. A model handed an unfamiliar notation and a specification is translating. It is not yet thinking in that notation. By that standard I never measured Tokenese at all. I measured translation overhead, once, on a system that had never used the language for anything.

Both of those are fair. Neither of them saves the project, and it took writing the post-mortem to see why.

The vector argument assumes the other dimensions are still up for grabs. They aren’t. Precision got solved orthogonally: JSON Schema plus constrained decoding makes invalid output unrepresentable at the sampler, which is better determinism than any grammar spec can offer, enforced at generation time rather than checked afterward, at zero acquisition cost. Compression got solved orthogonally too, by prompt caching, falling token prices, growing context windows, statistical pruning, and latent exchange between models that share an architecture. My 20 to 25 percent residual on narrow payload classes is smaller than a typical quarter of token price decline.

And the community-of-use argument turns out to be the strongest argument against the project rather than for it. A designed language charges a teaching payload to every participant, per session, per model, forever. The incumbent charges nothing. FIPA-ACL paid that cost in developer learning in 2000 and died of it. Tokenese would pay it in prompt tokens. Same ledger, same sign.

Seventy years of the same idea

I thought I was early. I was late by about seven decades.

Interlingua machine translation in the 1950s. Loglan and Lojban from 1955 on. KQML in the early 1990s, then FIPA-ACL around 2000 to fix KQML. The Semantic Web’s reasoning layer from 2001. Controlled natural languages like Attempto and ASD-STE100. Then the recent wave: SynthLang’s glyphs, Gibberlink’s audio handshake, TOON’s slimmed JSON, LLMLingua’s prompt pruning.

Four intents recur across all of it, close to verbatim. Ambiguity is waste and formal semantics will fix it. The machine channel shouldn’t pay human-language overhead. A neutral interlingua beats N-squared pairwise mappings. Machine communication should be auditable. All four are legitimate. That is exactly the trap. Each intent is real, and each has since been satisfied by a mechanism that is not a designed language.

Which leaves a law, and I’ll state it plainly because it cost me six weeks:

Every project that created a new surface form died. Every survivor compressed within the incumbent substrate.

LLMLingua prunes English and survived. TOON slims JSON and survived, in a niche. Constrained decoding restricts English and JSON emission and won outright. KV-cache and latent exchange skip text entirely where both ends share an architecture, and are thriving in research. KQML, FIPA-ACL, Lojban, the Semantic Web’s reasoners, SynthLang, Gibberlink, and Tokenese all introduced new surfaces. All dead or dying.

The mechanism underneath is what I’d now call substrate gravity. Tokenizers are trained on English and code and JSON, so the incumbent is already near-optimal in the exact currency that challengers compete on. A designed language fights the training prior, the tokenizer economics, and the tooling ecosystem simultaneously. My own audit measured that effect precisely: 1.3x larger than the prose it replaced.

What I’d want to be wrong about

The post-mortem includes revival conditions, pre-registered so the document can be proven wrong rather than argued with. The designed-interlingua idea is worth revisiting if a major vendor ships a tokenizer deliberately trained on a designed agent-language corpus, or a vendor-neutral latent-exchange standard emerges and needs a plain-text fallback layer, or someone demonstrates with reproducible cross-tokenizer counts a designed language beating terse English by more than 2x on a general task distribution, or inference pricing inverts its trend for years.

Absent those, the answer to “should we design a language for the machines?” is no. Be terse, emit JSON against a schema, cache your prompts, and let the sampler enforce your grammar.

I’d still like to be wrong. Perhaps the eventual thing looks less like a spoken language and more like a semantic intermediate representation, combining explicit symbols, learned embeddings, typed relations, confidence, provenance, and compact natural-language residues. Models might evolve several registers, just as humans do. We can’t know in advance whether it would translate, or even be perceptible to us as language. As a singer and beatboxer and deep listener, I love playing edges like this. It’s rich with opportunity, and none of that affection is evidence.

Which is the older lesson underneath all of this. Arthur Quiller-Couch, lecturing at Cambridge on style, told writers to “murder your darlings,” meaning the passage you are proudest of is the one most likely to be serving you rather than the work. It survives today as “kill your darlings,” usually handed to Faulkner or Stephen King, but the wording is his, from On the Art of Writing.

The advice generalizes well past prose. The cut is the craft in software too: the clearest code, the cleanest architecture, and the most usable product tend to arrive the same way, by removing an elegant idea that wasn’t earning its keep. A designed language for machines is a very elegant idea.

Tokenese was a recent casualty.

What this has to do with writing

The demand evidence was five unique visitors in the final fortnight, zero external adopters, zero inbound interest across six weeks public, and a package I never published. I keep those numbers in the post-mortem because they’re the ones I’d have most liked to leave out.

But the number that actually mattered was the one I never ran before publishing. I had built the instrument. I just didn’t point it at my own headline until validation forced me to, and then it took an afternoon.

That’s the connection to the other piece. The argument there was that a single score standing in for a vector is a bad way to evaluate communication, whether the score is a token count or an AI-detection percentage. This is the version of that mistake I made myself, and the correction isn’t as flattering as I’d hoped. I don’t get to say the project is merely unmeasured. It was measured, by the instrument I built for exactly that purpose, and it lost.

The vector framing doesn’t rescue a project. It tells you which measurement would have been decisive, and how early you could have run it.

Mine was available in week one.