# Field Notes > Field notes from real builds, not vendor marketing. Distilled practice notes on AI agents, context engineering, and building production software as a non-developer. Sourced from published articles and production build logs. Author: Dr. Tali Režun, AI advisor, lecturer and builder, COTRUGLI Business School. Canonical: https://fieldnotes.talirezun.com Licence: CC BY 4.0 ## Contents 1. [Context Engineering](https://fieldnotes.talirezun.com/context-engineering): How do you give an AI agent the context it actually needs? 2. [Agent Memory and Second Brains](https://fieldnotes.talirezun.com/agent-memory): How do you give an AI agent memory that survives the session? 3. [Coding Agents and Harnesses](https://fieldnotes.talirezun.com/coding-agents): What is the difference between a model and the harness you run it in? 4. [Building Without Being a Developer](https://fieldnotes.talirezun.com/building-without-being-a-developer): Can you ship production software without being able to write the code? 5. [Orchestration and Multi-Agent Work](https://fieldnotes.talirezun.com/orchestration): When is more than one agent worth the trouble? 6. [AI Sovereignty and Open Weights](https://fieldnotes.talirezun.com/ai-sovereignty): What does it mean for a European organisation to control its own AI? 7. [Organisational Design for the AI Era](https://fieldnotes.talirezun.com/organisational-design): How should a company reorganise itself around AI, if at all? 8. [Verification](https://fieldnotes.talirezun.com/verification): How do you know an agent's work was right if you did not watch it happen? 9. [What I Shipped and What It Cost](https://fieldnotes.talirezun.com/what-i-shipped): What have you actually built, and what did it take? --- --- title: "Context Engineering" number: 1 slug: "context-engineering" status: "published" question: "How do you give an AI agent the context it actually needs?" summary: > Context engineering is the continuous work of deciding what an agent sees, and when. It runs through every phase of a build rather than just the first one, and it is where most agent work quietly fails. updated: 2026-08-20 sources: - title: "Behind the Curtain: The Three-Phase Process I Use to Build Every AI-Coded Product" url: "https://talirezun.substack.com/p/behind-the-curtain-the-three-phase" publication: "Substack" date: 2026-03-16 sections: ["three-phase-build", "how-it-fails"] - title: "Context is the Code: The Complete Three-Phase Process for Building with AI Agents" url: "https://talirezun.substack.com/p/context-is-the-code-the-complete" publication: "Substack" date: 2026-06-03 sections: ["what-it-is", "three-phase-build", "the-opening-prompt", "what-to-load", "how-it-fails", "continuity"] - title: "Blueprint of a Frontier Coding Agent" url: "https://talirezun.substack.com/p/blueprint-of-a-frontier-coding-agent" publication: "Substack" sections: ["what-it-is", "what-to-load", "how-it-fails"] - title: "From Prompts to Precision: The Art & Science of Context Engineering" url: "https://medium.com/@talirezun/from-prompts-to-precision-the-art-science-of-context-engineering-cebd47462b1c" publication: "Medium" sections: ["what-it-is", "the-opening-prompt"] - title: "Why I Ditched RAG Pipelines for 1M Token Context Windows" url: "https://medium.com/@talirezun/why-i-ditched-rag-pipelines-for-1m-token-context-windows-d5a2982f7cce" publication: "Medium" sections: ["what-to-load"] - title: "The Mixed Fleet" url: "https://talirezun.substack.com/p/the-mixed-fleet" publication: "Substack" date: 2026-08-17 sections: ["how-it-fails", "continuity"] - title: "Lumina: An AI Agent Your Business Can Stand Behind" url: "https://talirezun.substack.com/p/lumina-an-ai-agent-your-business" publication: "Substack" date: 2026-07-31 sections: ["how-it-fails", "continuity"] - title: "From 0 to Dev in One Day: What a Real Coding Harness Actually Looks Like" url: "https://talirezun.substack.com/p/from-0-to-dev-in-one-day-what-a-real" publication: "Substack" date: 2026-07-20 sections: ["the-opening-prompt", "for-non-developers"] - title: "Context as Architecture: Full-Corpus Grounding, Orchestrated Agentic Development, and Compliance-by-Design" url: "https://talirezun.com/download/context-as-architecture/" publication: "Research paper" date: 2026-08-01 sections: ["what-to-load", "continuity"] related: ["agent-memory", "coding-agents"] tags: ["context-engineering", "ai-agents", "methodology"] --- ## What is context engineering, and how is it different from prompting? {#what-it-is} Prompt engineering is what you say in one message. Context engineering is everything else the model can see when it reads that message, and how you manage that across hours, sessions and days: what gets loaded, what gets summarised, what gets dropped, what gets carried forward. Andrej Karpathy called it the delicate art and science of filling the context window with just the right information for the next step, and that is still the cleanest definition I have found. Harness engineering sits one ring further out again, the machinery that does the loading and decides what the agent is allowed to touch. Three concentric levels, each containing the one inside it. I got the framing wrong the first time I published it. In March 2026 I wrote up the three-phase process I use to build every product, and I named the first phase "Context Engineering." Three months later I ran a live workshop with MBA students in the Chasing Jarvis programme at COTRUGLI. I expected tool questions. Which agent, which model, Cursor or Claude Code. What I got instead was a student who put her hand up after I finished walking through the phases and asked what happens to context in phase two and phase three. I did not have a good answer, because there was not one. By calling phase one "context engineering" I had implied it was a thing you do once and then leave behind. Students read the phases as sequential and cleanly separated: do the context work, then build, then deploy. That reading produces real failures, and it was my fault for teaching it that way. Context engineering is not a phase. It is the practice that makes phase one worth doing, keeps phase two coherent, and lets you trust what comes out of phase three. The useful analogy is not giving someone an instruction. It is preparing their workspace before they arrive: the tools, the background, the examples, the constraints, everything they need to do the job without having to guess at any of it. ## How do you structure the first phase of a build? {#three-phase-build} Phase one produces documents, not code. No repository, no environment, nothing committed. The output is a small set of markdown files that become the shared brief for every agent that touches the project afterwards. For a real product it takes three to seven days, and it cannot be compressed, because the slow part is thinking rather than typing. After the workshop I renamed the phases to stop implying otherwise. They are now Research, Design and Foundations, then The Build Phase, then Debug, Audit and Deploy. Phase one starts before the documents, and it does not start with a spec. I open a project with whatever model I am using as my daily driver and I talk, using voice-to-text rather than typing, because I explain an idea more fully out loud than I do on a keyboard. What I ask for first is a foundational concept document: does this idea make sense in the world, what already exists, what would users actually want, and what have I not thought to ask yet. That step exists because the idea on the napkin is almost never the idea you should build, and phase one is the process of finding out what you are actually building. Then four files. - **architecture.md** decides the stack and says why. Services, databases, external APIs, hosting, which model powers which capability. Not a full specification, a set of chosen building blocks with the reasoning attached. - **blueprint.md** is the feature specification split into components, with the boundary between front end and back end made explicit. It also carries the economics. An agent directed by an incoherent business model builds an incoherent product. - **ui_ux.md** describes how a real person moves through the application. Onboarding, navigation, what each screen shows, what each interaction does. Not pixels. Enough that an agent can make sensible decisions without me specifying every layout choice. - **security.md** is the one people skip and it is the one whose absence costs most. Which regulations apply, what authentication is used, what has to be encrypted, what the rate limits are. Written here, the agent builds it in. Left out, you retrofit it after an audit finds the gaps. If the thing you are building is itself an AI system, a reasoning layer or a retrieval pipeline or a multi-agent workflow, add a fifth. An orchestration document that says how the AI components connect, which models power which step, and what data moves between them. Agents are noticeably worse at inferring this than they are at inferring ordinary application structure. The research pays for itself in decisions you do not have to unwind. On an AI avatar assistant I built for a niche platform, phase one surfaced almost immediately that I should be evaluating ElevenLabs and HeyGen for the video avatar streaming rather than building that capability myself. That is weeks of misdirected development that never happened. My rule of thumb is that every hour in phase one saves five in phase two. I should be honest that this is a rule of thumb rather than a measurement. What I can say with more confidence is the inverse, and I have watched it play out on every project where I rushed the front end of the work: chaos in phase two is almost always a phase one failure, and not a failure of effort. A failure of clarity. One thing worth saying plainly, because it changes how you write these files. They are not for you. They are for the agent. ## How do you hand the context over when the build starts? {#the-opening-prompt} With an opening prompt that does four things at once: points the agent at the documents, invites it to find the gaps in them, asks for a plan before any code, and sets the standing rules that will govern the whole build. The quality of that prompt is a direct reflection of the quality of phase one. If the documents are thorough it writes itself. If they are thin you will be able to feel it while typing. Before the prompt, the setup. Project folder named after the project, a `docs` subfolder inside it holding every phase one file, and a GitHub repository connected to the agent. Most frontier harnesses connect over OAuth in one click. This is not optional. Version control here is not tidiness, it is the recovery mechanism for the times an agent does something you disagree with, and you will need it more often than you would like to admit. The prompt itself looks roughly like this, and the shape matters more than the wording: ``` I'm building [what it is, in one sentence]. I've prepared detailed project documentation: @architecture.md, @blueprint.md, @ui_ux.md, @security.md Please read this documentation first. Identify any gaps or improvements you see, and then prepare a detailed build plan. Rules: (1) Push all substantial development milestones to GitHub (2) Maintain and update documentation in parallel with development, including both user guides and technical guides (3) Update the standing context file after every major build milestone ``` Notice what it does not do. It does not say "build me a second brain app." It hands over context, asks the agent to argue with that context before acting on it, and separates planning from execution so you can disagree with the plan while disagreeing is still cheap. The `@` is doing real work. It is how you point an agent at a specific file so it reads that file before doing anything else, rather than inferring what you meant from the folder around it. The three standing rules are the ones I keep on every project, and each one exists because of something that went wrong without it. Push milestones, so there is always a state to roll back to. Maintain the documentation in parallel, because documentation written afterwards never gets written. Update the standing file at every milestone, because the agent that starts tomorrow only knows what that file tells it. ## What belongs in an agent's context, and what does not? {#what-to-load} Load the smallest payload that answers the step in front of you, and load it when that step arrives rather than at the start. What earns its place: architectural decisions, unresolved bugs, the standing rules of the project, and the specific documents the current task actually touches. What does not: redundant tool output, whole files when you need three lines, and tools the agent will not call in this step. More context is not better context, and this took me a while to accept. The frontier harnesses have already worked this out. They read with grep, glob, head and tail rather than loading complete files. When they compact a session they keep architectural decisions and unresolved bugs and throw away redundant tool results. On the tooling side, the effect is counterintuitive enough that it is worth stating: the Vercel v0 team reportedly removed around eighty percent of their available tools and got better results. More tools often means worse performance, because every tool definition is context the model has to hold and reason about. The same principle showed up in a completely different shape when I rebuilt Moj AI, a legal document assistant for Slovenian building regulation. I spent about a year on a conventional retrieval pipeline: chunk the documents, embed the chunks, store the vectors, retrieve the closest matches. It shipped at eighty percent accuracy and nothing I tried got it past eighty-five percent on complex table data. I tried smaller chunks, larger chunks, overlapping chunks, three embedding models, metadata on chunks, hybrid search, multiple retrieval passes, reranking. I even built a second model to check the first one's answers. The failures were specific and they were the same failure each time. A table saying maximum coverage forty percent, maximum height twelve metres, minimum parking one and a half spaces per unit came back as "coverage and height restrictions apply, parking is required." The structure was destroyed and the numbers, which were the entire point, were lost. Cross references broke, because section 10.3 pointing at section 4.2.1 does not survive the two sections landing in different chunks. Worst of all, a query about Ljubljana could come back with rules from Maribor, because the chunks did not know which municipality they belonged to. In a legal application, eighty percent might as well be zero. What fixed it was giving the model more context, not less: a lightweight router that matches a query against structured metadata rather than vector similarity, picks at most three documents, and injects them whole. Every table, every footnote, every cross reference intact. A five hundred page municipal plan converts to roughly four hundred to six hundred thousand tokens, which fits. That sits awkwardly next to everything I have just said about loading less, so let me name the distinction rather than pretend it is not there. A curated payload you hand a model once, for one question, is not the same thing as a session that accumulates. The first can be enormous and stay coherent. The second degrades as it fills with tool output, half-finished attempts and your own messages. Volume is not the problem. Clutter is. ## Why does an agent drift, and where does it start? {#how-it-fails} Drift starts as a context problem and arrives looking like a reasoning problem, and it is almost always silent. The agent does not crash and it does not tell you it has lost the thread. It keeps going, gets more confident, and by the time the output is obviously wrong you have three commits built on a misunderstanding. Silence is the thing to internalise. Early on I watched a tool call fail and the agent carry on as though nothing had happened. Nothing in the transcript said anything was wrong. Four sources of drift, in the order I hit them most often. **The window fills.** Code accumulates, documentation accumulates, your own messages accumulate. A two hundred thousand token window sounds generous until you spend a focused afternoon in one, and then it is gone. My working rule is that I start planning the handover at around eighty percent, because past that the quality degrades before you notice it degrading. I watch the context indicator the way a pilot watches the fuel gauge. **Position, not just volume.** The research I have seen reported on this says model performance drops by more than thirty percent when the important content sits in the middle of the window rather than at either end. I have not run that test myself, so take it as reported rather than verified here, but it matches what I feel in long sessions and it produces a simple rule: important context goes at the beginning or the end, never buried in the middle. **Stale context, which is worse than no context.** This is the one that got me most recently. During the Lumina Gen 2 build I corrected a security statement in the code and did not carry the correction into the documentation. Lumina's own assistant reads that documentation. So for a while, my product was answering prospective customers from the version I had already fixed. Nobody flagged it. I found it while wiring up something else entirely. **Vague input.** Specificity is context, and this is the cheapest fix on the list. Do not tell an agent that something is broken. Tell it that the submit button on the profile screen stops responding after the first file upload and the console shows a 403 from the storage endpoint. Vague bug reports produce vague fixes, reliably, every time. A related trap sits next to that one. If you ask an agent to test something and you do not explicitly authorise it to use live credentials, many will quietly fall back to synthetic responses, and synthetic responses do not reveal real failures. Say out loud that the key is in the environment file and that you want real calls. The pattern under all four is the same: the failure does not announce itself. Two of five enterprise access deploys on that same Lumina build finished with no success message at all, and I only caught it because I checked whether the services were actually live rather than trusting the deploy log. Further back, a worker agent once deployed one of my applications over an entirely different project because the default in my CLI configuration was wrong. Everything was recoverable from GitHub, which is the whole argument for the commit discipline, and the structural fix is to stop handing agents master credentials at all. Scoped, project-specific service account tokens with the minimum permissions the job needs. Agents do exactly what they are configured to do, including the mistakes, so the thing to control is not their intent but their blast radius. Rich context in means coherent output out. Stale context means contradictions in the code. Missing context means the agent assumes, and its assumptions will not be yours. ## How do you carry context across sessions and across agents? {#continuity} With files, not with the model. The model starts from nothing every time and no amount of window size fixes that. Four artefacts do the work: a documentation library kept in sync, a live spec file recording decisions and the reasons behind them, a handoff file written at the end of every session, and the operational rules of the project so a fresh agent knows the house rules before it touches anything. None of it is clever. That is rather the point. The standing file goes first. Claude Code loads CLAUDE.md, OpenCode loads AGENTS.md, Cursor has its own. Whatever the name, it is your agent's permanent brief: architecture decisions, conventions, the rules that should govern every action. Instruct the agent to update it after every meaningful milestone. An unmaintained standing file becomes stale context, and stale context is worse than none. The handoff file does the heavy lifting. Written by the orchestrator at the end of a session, it says what happened, which agent did what, what is still open, and where the relevant documentation is. Crucially, it carries the role instruction forward too: you are the orchestrator, you do not write code yourself, you delegate, you audit. It is the same job you would do for a person joining the team on a Monday. Underneath that there are three layers of memory, and knowing which one a piece of knowledge belongs in is most of the skill. **Markdown files** hold active project context: the current sprint, the current codebase, the current session. Lightweight, readable, native to models. Every project gets this layer, no exceptions. The cost is discipline, because they go stale if you let them. **Retrieval pipelines** are the older paradigm and they still have a place, but a narrow one. Reach for one when you have a genuinely large and heterogeneous corpus, thousands of documents, that cannot be handled any other way. Do not reach for one because it sounds sophisticated. It is meaningful infrastructure and it is not free. **A compiled wiki** is the long-term layer: sources read once and written into an interlinked structure of entities, concepts and summaries, updated rather than duplicated when something new arrives. This is the layer that compounds across projects and across years rather than across a sprint. It is the subject of the next chapter. The scale this has to survive is the argument for taking any of it seriously. Lumina Gen 2 took hundreds of sessions with hundreds of agents over roughly a month, and around eight hundred and sixty commits. The whole system exists so that agent number forty-seven starts with the same understanding of the project that agent number three finished with. I want to be honest about how automated this is not. I still manually remind the agents, every single session, that all documentation must be in sync before we close. Manually. Every time. Some of this is a discipline problem that better tooling should solve and has not solved yet. ## Does any of this change if you cannot read the code? {#for-non-developers} The practice is identical. The verification is different. You still write the brief, still keep the standing file current, still write the handoff at eighty percent. What changes is that you cannot check the work by reading the diff, so you check it by using the thing. I do not write the code. I have not for two years. So this is the version of the practice I actually run. For a small project, one page of markdown is plenty for the brief. What it is, who it is for, what it should look and feel like, and where it will eventually be deployed, so the agent knows the destination on day one. Bigger ideas need the four files from phase one, but do not start there. The single most useful sentence I have found for working with an agent is this: sorry, I do not understand this, I am not a developer, please explain it simply. Use it constantly. No serious agent will think less of you, and every time you use it you are correcting a mismatch between what the agent assumed you knew and what you actually know. That is context engineering too. Watch the window. If you are on a free model the window is smaller than the frontier ones, so keep the scope of a first build small and finish something rather than overreaching and stalling. Then verification, which is where the difference actually bites. What you can check without reading code: everything a user touches. Go through every button, every screen, every feature yourself, the way a real person would, including the error states and the empty form and the interrupted upload. When something breaks, do not tell the agent it is broken. Tell it what you clicked, what you expected, what you got, and hand it a screenshot. What you cannot check: whether the implementation underneath is sound. For that you need something other than your own eyes, and there are two answers depending on scale. On a small build, get someone who did not build it to use it and tell you honestly what happened. They will find things you never will. On a production system, put a real test suite behind a CI pipeline and gate deployment on it going green, which is what Lumina runs now. The honest limit is in between those two. I have experimented with letting agents test themselves, and it works, but it is slow and genuinely expensive in tokens, especially on anything visual. There are still classes of interface bug that a human eye catches instantly and an agent misses completely. So the loop I actually run is hybrid: automated coverage where it is cheap and repetitive, my own hands where it is not. Do not remove yourself from the testing loop yet. --- --- title: "Agent Memory and Second Brains" number: 2 slug: "agent-memory" status: "published" question: "How do you give an AI agent memory that survives the session?" summary: > An agent forgets everything when the window closes. A second brain is the durable half of the system: structured, linked notes the agent reads back on demand, so accumulated thinking outlives any single conversation. updated: 2026-08-20 sources: - title: "The Agent Memory Problem, and Why It Matters" url: "https://talirezun.substack.com/p/the-agent-memory-problem-and-why" publication: "Substack" sections: ["the-problem", "four-layers"] - title: "Building Knowledge Immortality Through the Second Brain Architecture and The Curator App" url: "https://talirezun.substack.com/p/building-knowledge-immortality-through" publication: "Substack" sections: ["what-a-second-brain-is", "knowledge-immortality"] - title: "The Second Brain That Grows Smarter and Lives on Your Computer" url: "https://talirezun.substack.com/p/the-second-brain-that-grows-smarter" publication: "Substack" sections: ["what-a-second-brain-is"] - title: "From Graph to Intelligence: The My Curator MCP and the Art of Querying Your Second Brain" url: "https://talirezun.substack.com/p/from-graph-to-intelligence-the-my" publication: "Substack" sections: ["retrieval"] - title: "The Shared Brain: When Second Brains Start Thinking Together" url: "https://talirezun.substack.com/p/the-shared-brain-when-second-brains" publication: "Substack" sections: ["shared-brain"] - title: "Second Brain to Shared Brain: Building a Neural Network of Your Own Knowledge" url: "https://talirezun.substack.com/p/second-brain-to-shared-brain-building" publication: "Substack" date: 2026-07-03 sections: ["four-layers", "retrieval", "shared-brain", "what-broke"] - title: "The Curator — Product Update" url: "https://talirezun.substack.com/p/the-curator-product-update" publication: "Substack" date: 2026-06-18 sections: ["what-broke"] - title: "The Curator" url: "https://github.com/talirezun/the-curator" publication: "GitHub" sections: ["what-a-second-brain-is", "retrieval"] - title: "Context is the Code: The Complete Three-Phase Process for Building with AI Agents" url: "https://talirezun.substack.com/p/context-is-the-code-the-complete" publication: "Substack" date: 2026-06-03 sections: ["the-problem"] related: ["context-engineering", "coding-agents"] tags: ["agent-memory", "second-brain", "knowledge-management", "mcp"] --- ## Why does an agent forget, and why is that hard to fix? {#the-problem} Because by default every conversation starts from nothing. The model has no idea who you are, what you are building, or what was decided yesterday. For a single question that is fine. For a project that runs for months it is a structural problem, and the reason it is hard to fix is not storage. It is that an agent needs a different shape of memory than a chatbot does. A chatbot answers a question. You ask, it retrieves something relevant, it responds, it is done. An agent runs a task. It opens a file, cross-references a policy, writes a summary, calls a tool, checks the result, and loops. What it needs at any given moment is almost never three semantically similar paragraphs. It is the whole package: the policy and the exception to the policy, the contract clause and the definition section that changes what that clause means. Miss one piece and the agent either fails, or does something worse and produces an answer that is plausible and wrong. The obvious fix is a bigger window, and the obvious fix does not work. In eighteen months of running retrieval systems in production I documented hallucination rates above twenty percent on context-dependent queries. And even with a million-token window and a frontier model, quality degrades noticeably somewhere around eighty to ninety percent utilisation, in ways that are subtle enough that you will not notice them happening. Chroma's research on this is the clearest I have seen reported. So capacity is not the answer. Structure is. ## What kinds of memory are there, and which one do you actually need? {#four-layers} Four, and conflating them is where most of the confusion lives. **In-context memory** is whatever is sitting in the active session, and it evaporates when the session ends. **External memory** is the retrieval pattern: fast, stateless, re-derived on every single query. **Persistent memory** is plain files that survive between sessions but do not inherently connect to each other. **Semantic memory** is a compiled, cross-referenced structure that compounds, where each new source integrates with what is already there. Most projects need three of the four and reach for the wrong one first. I should be straight about the history of this framing, because it is a good example of how the practice moves. When I wrote about the agent memory problem in May 2026, I laid out the landscape of approaches the industry was trying and concluded that no single one was adequate. The clean four-layer split came later, in July, and it is me spelling out something I had described at length without ever naming properly. If you read the two pieces back to back the second one is not a correction. It is the shape finally becoming visible. The layer that matters most for anyone building over months rather than days is the fourth, and the argument for it is about compounding rather than accuracy. Retrieval treats your documents as a lookup table. Chunk everything, embed it, and at query time fetch whatever is mathematically closest. It works, and it starts from zero every single time. Nothing accumulates. The tenth question you ask gets no benefit whatsoever from the fact that you asked nine questions before it. Curation is the opposite bet: read once, integrate permanently. The knowledge is compiled and kept current rather than re-derived on demand. Retrieval fetches. Curation remembers. ## What is a second brain, in concrete terms? {#what-a-second-brain-is} A folder of markdown files on your own machine that an AI keeps organised. You drop in a PDF, an article or a text file, and it reads the source and writes an interlinked wiki out of it: entity pages for the people, tools and companies, concept pages for the ideas, and a summary page for the source itself. Roughly five to fifteen linked pages per source. Drop in something on the same subject a month later and it updates the existing pages rather than creating near-duplicates, so the wiki gets denser rather than just bigger. The tool I built for this is The Curator. It is open source under MIT, it runs locally on `localhost:3333`, and the files it writes are ordinary markdown that Obsidian opens natively. Nothing about the format is proprietary and nothing is locked in. If the project disappeared tomorrow you would still have a folder of readable notes. Two things about it that people get wrong, and I would rather state them plainly than let the marketing version stand. **It needs an API key.** Either Google Gemini or Anthropic. There is no version of this that runs the ingestion pipeline on nothing. Gemini has a free tier, though Google tightened it substantially at the end of 2025 and a single batch of five to ten PDFs will usually exhaust a day's quota. On paid keys, moderate solo use lands around five euros a month. If you would rather nothing left your machine at all, it runs against a local model through LM Studio, and the trade is exactly what you would expect: full privacy, lower quality. **The output is roughly ninety-five percent right, not a hundred.** An AI building a knowledge graph makes mistakes: links that point nowhere, pages that end up orphaned, the same concept written twice under slightly different names. That is why there is a health layer that scans for broken links, orphans and near-duplicates. Every scan is opt-in, priced before you run it, and gated behind a preview, so nothing destructive happens without you signing off on it. ## How does the model actually reach the notes? {#retrieval} Through an MCP server that exposes the wiki to a frontier model as a set of tools. Seventeen of them, ten read and seven write. The model can list domains, pull an index, search across the wiki, read a specific page, and then, if you let it, write findings back. Once that bridge is connected, the model is not being handed a pile of documents. It is navigating a structure. The graph-native tools are the ones that justify the whole architecture, because they answer questions a flat index cannot answer at all. `get_backlinks` returns every page that links *to* the one you are looking at. A search index has no concept of this. `get_connected_nodes` walks the graph outward up to two hops and ranks what it finds by distance, which is how you surface a connection you never consciously made. `get_graph_overview` returns the shape of the whole thing: node and edge counts, a breakdown by page type, the top twenty hubs, the orphans, the most-used tags. The working query pattern is boring and it is the right one: list the domains, pull the index, search, then read the specific nodes that matter. Cheap traversal first, expensive full reads last. It is the same just-in-time discipline from the previous chapter, applied to notes instead of code. For scale, my own articles domain sits at roughly three thousand three hundred nodes and fifteen thousand edges as of mid-2026, built by ingesting sources over about six months. All seventeen tools work identically against a local model with no network, which matters if the material is sensitive. What I cannot tell you is how this behaves at ten times that size. I have not tested it against tens of thousands of nodes, multiple active domains and years of ingestion. Traversal efficiency and token consumption at that scale are open questions, and writes through the MCP are already noticeably slow on large wikis. This is generation one of something. ## What changes once your thinking outlives the session? {#knowledge-immortality} The economics of maintaining a knowledge base invert. Historically, keeping a structured, interlinked, current body of knowledge was a full-time job, which is why it was the preserve of institutions with librarians and archivists on staff. Nobody did it for themselves because the bookkeeping cost more than the knowledge was worth. That constraint is gone, and it is gone for individuals rather than just for organisations. The tedious part was never the reading or the thinking. It was the filing. What that buys, at the personal end, is expertise that does not evaporate. Thirty years of judgement about a domain currently lives in one head and leaves when that head does. Written down as an interlinked structure that a frontier model can traverse, it becomes something you can interrogate, hand over, or keep working with after the person who built it has moved on. The durability bet is markdown, deliberately. Plain text has been readable for decades and will stay readable for decades more, which is not something you can say about any particular application's database format. The whole design follows from wanting the notes to outlive the tool that made them. I would rather not oversell the personal version of this. Compounding is real and I feel it daily in my own work. But the claim I am making is about the mechanism, not about having proven that a lifetime of knowledge survives, because not enough time has passed for anyone to have proven that. ## What happens when more than one person contributes? {#shared-brain} The same architecture works for a group, but only if you solve the privacy problem first, and the solution is that raw notes never leave anyone's machine. A model running locally on each contributor's own computer pre-processes their changed pages into compact delta summaries before anything is sent. The collective wiki receives structured knowledge, not a copy of somebody's private thinking. That one design decision is what the entire feature rests on. Without it, a shared brain is just a folder everyone can read, and nobody sensible would put their working notes in it. The rest follows from it. You opt in specific domains, not your whole wiki, so your journal and your unfinished thinking and your unrelated client work stay where they are. Synthesis reconciles contradictory formulations from different contributors, fixes links that break across the join, and attributes provenance to every fact it integrates. What comes back to you is a separate read-only mirror domain, queryable exactly like any of your own. The principle, stated as plainly as I can: private brains remain private, and shared intelligence is built only from explicit contributions. Now the part that matters more than any of the above. It has not been proven at the only scale that counts. Everything so far has been verified by tests, simulations, and solo or paired use between me and a handful of early testers. That tells me the software behaves correctly under everything we could think to simulate, including real concurrency against real storage and real network conditions. It does not tell me how it behaves when twenty actual people with actual inconsistent habits and actual half-finished contributions use it together for a full semester. That test cannot be simulated. It has to happen, and it has not happened yet. So the feature is beta-labelled, the workflows may change, and anyone deploying it should keep independent backups. I would rather say that than discover it in public. ## What went wrong building this? {#what-broke} Almost every real bug was silent. That is the honest summary of a year of building memory infrastructure, and it is the single most useful thing I can hand over. The failures that hurt were not the ones that threw errors. They were the ones that completed, reported success, and left something quietly broken behind them. The worst of them surfaced during production battle-testing before wider release, and it is the reason that testing existed. Removing a contributor from a shared brain is a routine administrative action. Delete their pages, rebuild the collective wiki from what remains. Under a particular timing condition, the rebuild read stale state, produced a completely empty wiki, and reported success. A green checkmark, no error anywhere, and a semester of collective work gone. Alongside it, a simpler race condition where two contributors' machines creating the same brand new page at the same instant produced an error for whoever lost. That one I am relaxed about, because it was visible. You would know something had gone wrong. Neither bug was catchable offline. Both required testing against real storage, real network conditions and simulated multi-machine concurrency. If I had stopped at "the offline tests pass," a cohort's shared wiki could have been silently deleted on my watch. That release ran on five hundred and forty-seven assertions, five hundred and nineteen of them offline and the remainder running against live services and live credentials. The offline suite was entirely green before the two bugs above were found. I do not think there is a stronger argument anywhere in my work for why offline test coverage is a floor rather than a finish line. The earlier failures rhyme with it. Synthesis could be permanently bricked by a single malformed contribution, so one person's bad page took down the whole run. Contribution tracking originally filtered by wall-clock time, which meant a contributor whose machine clock was skewed could have their work silently dropped from a synthesis. A push operation only ever covered one opted-in domain at a time, quietly, without saying so. Failed operations could render as a false success. The near-duplicate scanner became unreachable behind an interface regression and simply vanished from the screen for anyone whose wiki was structurally clean, which was precisely the people who most needed a semantic scan. And one that belongs in this chapter because it is the most embarrassing: my own test suite was silently writing test data into real user knowledge folders, because the code that resolves which directory to use preferred the production configuration over the test override. Nobody lost anything and the stray data was cleaned up. It should never have been possible. Earlier still, before any of the collaborative work, large wikis produced ghost files: the same entity spawning three slightly different pages because deduplication was not strict enough. That one is unglamorous and it took real work to fix, and it is the sort of thing that decides whether a knowledge graph is usable at size or just impressive at demo scale. There is a pattern in all of it, and it is the same pattern as the last chapter. The failure mode of these systems is not a crash. It is a confident, successful-looking completion with something quietly missing underneath. Build your checks accordingly. --- --- title: "Coding Agents and Harnesses" number: 3 slug: "coding-agents" status: "published" question: "What is the difference between a model and the harness you run it in?" summary: > The model writes the code. The harness decides what it sees, what it may touch, and when it stops. Most of the difference between a good session and a wasted one comes from the harness, not the model underneath it. updated: 2026-08-20 sources: - title: "Blueprint of a Frontier Coding Agent" url: "https://talirezun.substack.com/p/blueprint-of-a-frontier-coding-agent" publication: "Substack" sections: ["what-a-harness-does", "the-twelve-parts", "model-or-harness", "permissions"] - title: "Exploring Early Indicators of AGI in Coding Agents: A Case Study on MCP-Powered Systems" url: "https://talirezun.com/download/exploring-early-indicators-of-agi-in-coding-agents/" publication: "Research paper" sections: ["model-or-harness"] - title: "Chasing Jarvis: Can Technically Sophisticated Non-Programmers Deploy SaaS Applications Using AI Coding Agents?" url: "https://talirezun.com/download/chasing-jarvis-can-technically-sophisticated-non-programmers-deploy-saas-applications-using-ai-coding-agents/" publication: "Research paper" sections: ["model-or-harness", "comparing-them"] - title: "From One Agent to Coding Agent Armies: My 15-Month Journey to AI Orchestration" url: "https://medium.com/@talirezun/from-one-agent-to-coding-agent-armies-my-15-month-journey-to-ai-orchestration-b9138675a075" publication: "Medium" sections: ["comparing-them"] - title: "The Claude Desktop Coding Agent Experiment: Early Results & Context Management Lessons" url: "https://medium.com/@talirezun/from-english-to-code-building-production-saas-with-claude-desktop-3ee9c787f5be" publication: "Medium" sections: ["comparing-them"] - title: "Three Philosophies, One Goal: A Practitioner's Comparison of Augment Code, Claude Code, and Codex CLI" url: "https://talirezun.substack.com/p/three-philosophies-one-goal-a-practitioners" publication: "Substack" sections: ["comparing-them", "permissions"] - title: "From Writing Code to Directing Intelligence: Five Days Inside Augment Code's Intent" url: "https://talirezun.substack.com/p/from-writing-code-to-directing-intelligence" publication: "Substack" sections: ["comparing-them", "permissions"] - title: "The Mixed Fleet" url: "https://talirezun.substack.com/p/the-mixed-fleet" publication: "Substack" date: 2026-08-17 sections: ["comparing-them", "staying-current"] - title: "Anthropic Shipped Two Models This Week. I Only Wanted One Back." url: "https://talirezun.substack.com/p/anthropic-shipped-two-models-this" publication: "Substack" date: 2026-07-01 sections: ["staying-current"] - title: "From 0 to Dev in One Day: What a Real Coding Harness Actually Looks Like" url: "https://talirezun.substack.com/p/from-0-to-dev-in-one-day-what-a-real" publication: "Substack" date: 2026-07-20 sections: ["staying-current"] related: ["context-engineering", "orchestration"] tags: ["coding-agents", "developer-tools", "harness", "claude-code"] --- ## What does a coding harness actually do? {#what-a-harness-does} Everything the model does not. The model is the brain: it reads and it writes text. The harness is the loop that keeps it running, the tools that let it touch your files and execute commands, the memory that stops it starting from zero every session, and the permission system that decides what it is allowed to do at all. Take a frontier model and strip the harness away and you have a very good text predictor that cannot open a file. The term was only formalised in early 2026, which is late given how long the thing itself has existed. The clearest framing I have come across is not mine. Beren Millidge wrote in 2023 that a raw language model is a CPU with no RAM, no disk and no input or output. The context window is the RAM. External storage is the disk. Tool integrations are the device drivers. The harness is the operating system. His line was that we had reinvented the Von Neumann architecture, and once you see it that way the whole field reorganises itself in your head. Vivek Trivedy at LangChain put the same idea more bluntly: if you are not the model, you are the harness. I can read modern harnesses architecturally because I built a bad one first. In late 2024 I assembled Claude Desktop, the Model Context Protocol, and a hand-picked set of servers: GitHub for version control, Playwright for browser automation, Sequential Thinking for multi-step work, Context7 for documentation retrieval. On top of that sat markdown I maintained by hand, architecture documents and blueprints and handoff notes. I did not know I was building a harness. There was no word for it. But every component that a frontier agent ships today, I was assembling manually, badly, and learning exactly why each one exists by living without it. ## What is a harness made of? {#the-twelve-parts} Twelve components, which group into five roles. The **brain** is the orchestration loop and prompt construction. The **hands** are the tools. The **memory** is short-term memory, context management and state. The **safety layer** is output parsing, error handling and guardrails. The **team** is verification loops and sub-agent orchestration. Telemetry sits across all of it. You do not need to build any of these any more, but you need to know they exist, because when a session goes wrong it is almost always one of them. Worth understanding about the loop itself: it is stupid on purpose. Anthropic describes their own runtime as a dumb loop. Assemble the prompt, send it, parse the response, run any tool calls, feed the results back, repeat. All the intelligence lives in the model. This is a deliberate architectural bet, and it is the reason the harness can stay thin while models get better. Three of the twelve carry most of the weight in practice, and I would rank them in this order. **Context management** is where most agents fail silently. It is also the component I have written about more than anything else, which is not a coincidence. The four strategies that matter are compaction, hiding old tool output while keeping the calls visible, just-in-time retrieval instead of preloading, and delegating exploration to sub-agents that come back with a summary of a thousand or two thousand tokens rather than everything they read. **Verification loops** are what separate a production agent from a demo. Three shapes: rules-based checks the agent can run itself, like tests and linters and type checkers; visual checks, usually screenshots through a browser tool; and a separate model acting as judge. Give a model a way to check its own work and the quality of that work goes up sharply. That effect is reported by the people who build these tools rather than measured by me, but it matches everything I see. **Error handling** is a design decision, not a feature you get for free. Four kinds of failure and each needs a different response: transient failures you retry, failures the model can fix itself if you show it the error message, failures that need a human, and failures where the right move is to stop. The arithmetic is unforgiving. A ten-step process that succeeds ninety-nine percent of the time at each step finishes correctly about ninety percent of the time. The most underinvested component is telemetry, and the reason is the theme of this whole site. Agent failures are quiet. The agent did not crash. It took a wrong turn three steps ago and then built confidently in that direction for twenty minutes. Without a record of every tool call and every decision point, you find that out at the end. ## Does the model or the harness decide the outcome? {#model-or-harness} The harness, far more often than the marketing around model releases would suggest. The strongest evidence I have for this is my own, and it is embarrassing in the right direction: a small cheap model with good tooling beat the same class of setup without tooling, decisively, on the same task. In 2025 I ran Cline with Grok 3 Mini, a small and inexpensive model, and connected five MCP servers to it: documentation retrieval, sequential reasoning, a knowledge graph for memory, GitHub, and a database. That configuration took a retrieval-based SaaS application to about ninety percent complete in nine days, for roughly thirty dollars in API costs. The comparable setup without MCP servers failed within forty-eight hours. Same class of model. The difference was entirely the layer around it. The finding held when I looked at it more systematically. Across two years of testing eight platforms for a case study, the pattern in my own development logs was that lower-capability models produced production-quality code when the context was comprehensive, while frontier models produced poor results when the context was inadequate. Failures attributable to insufficient context far exceeded failures attributable to the model not being clever enough. That is the single most useful thing I learned in two years, and it is also the least intuitive. Two more data points, both second-hand and worth treating as reported rather than verified. LangChain moved from outside the top thirty to fifth place on a coding benchmark by changing only the infrastructure around their model, same weights, same training, more than twenty positions. And Manus, one of the more impressive agentic systems of the last two years, was rebuilt five times in six months, with every rewrite removing complexity rather than adding it. Complex tool definitions became general shell execution. Management agents became simple structured handoffs. The practical consequence is that model choice becomes a staffing decision rather than a loyalty decision. Complex reasoning and orchestration go to the strongest model you can afford. Mechanical work goes to something cheaper. Audit goes to a model from a different vendor entirely, for reasons I will come back to. You are not picking a model. You are staffing a team. ## How do the harnesses I have used actually compare? {#comparing-them} They compare badly, in the sense that any ranking I write here has a shelf life of about a quarter. So take everything below as dated observations rather than a recommendation, and note the dates, because at least one of these verdicts has already reversed itself. **Cline, late 2024.** A VS Code extension where I learned context engineering the hard way. Roughly thirty minutes of useful working window before I needed to write a handoff, and no memory whatsoever between sessions unless I created it myself. Everything manual. It taught me more than any tool since, precisely because nothing was done for me. **Claude Code, January 2025 beta.** Immediately and obviously better at reasoning across multiple files than anything I had used. Also, a single day of intensive development cost me over three hundred euros. I stopped using it, and the reason was not capability. Raw capability and practical viability are different things, and the second one is what determines whether you ship. **Claude Desktop with a custom MCP stack, through 2025.** Not a coding agent product, a harness I assembled. Genuinely reliable for projects under about twenty files, and cheap. It took a legal document assistant to roughly seventy percent complete across more than fifty conversations. Where it fell down was complex debugging: database connection problems, integration failures, anything requiring broad visibility across an unfamiliar codebase. The loop became start a conversation, spend tokens loading context, explain the bug, spend more tokens on analysis, begin a fix, hit the conversation limit, start again. I migrated to VS Code and Augment Code. That verdict is now wrong. Claude Code runs inside the Claude Desktop app, which means you can work this way without touching a terminal, and it is currently what I use most. I prefer it to the command line. Same product name, completely different answer, eighteen months apart. This is the clearest illustration I have of why you should distrust any tool comparison with a date on it, including this one. **Augment Code, from summer 2025.** The thing that mattered was the Context Engine: it indexes the codebase into a graph of symbols, dependencies and call trees ahead of time, so a new session starts with structural understanding rather than starting cold. Handoffs stopped being painful. It stayed single-agent and sequential, which is what eventually pushed me on. **Google Antigravity, late 2025.** My first genuine parallel multi-agent work, with a manager view for spawning and monitoring agents. Two things broke. Agents made incompatible assumptions about each other's work, so I would come back to a backend expecting data structures the frontend was not producing, and need a cleanup agent to reconcile them. And my own hardware became the ceiling at around five simultaneous agents. **Augment Intent, early 2026.** A coordinator agent reads the codebase, drafts a living specification, decomposes it into tasks, and dispatches implementor agents that each work in an isolated git branch, with a verifier checking output against the spec before it surfaces. The spec, not the code, is the source of truth. I ran twenty-one agents on one build and a hundred and seventy-seven tasks on another. It is not a prototyping tool, it is expensive for exploration, and long coordinator threads made the interface lag. **Codex, spring 2026.** A different philosophy: delegate and come back. A task runs in a sandboxed cloud environment for up to half an hour and returns a pull request with terminal logs and test citations. Apache-licensed and open source, which matters if you need auditability. Strongest at code review, where it reliably catches logic errors and race conditions. Weaker on frontend work. Its instruction file format is portable across tools, which is a quietly significant advantage. **OpenCode, 2026.** Open source, ships with free models, and it is what I now recommend to anyone starting, partly because you can see how it works rather than being shielded from it. It is also the only harness I have found that genuinely routes different agents to different providers inside a single session. One honesty note about that last point. The compatibility work behind it, which harness can mix local and cloud models and which cannot, I did against official documentation rather than by running every combination myself. I would rather tell you that than publish a wish list dressed as a field test. ## How much should you let an agent do on its own? {#permissions} Quite a lot, if you separate two questions that people tend to merge. The model decides what to attempt. The tool system decides what is permitted. Keep those apart and you can give an agent real autonomy without giving it real reach, which is the only combination that works at production scale. In practice that means scoped credentials, always. Project-specific service account tokens with the minimum permissions the job needs, never master admin access. Agents do exactly what they are configured to do, including the mistakes, so what you are controlling is not their intent but their blast radius. I learned this the way everyone learns it, by having a worker agent deploy an application over an entirely different project because a default in my CLI configuration was wrong. The gates I actually keep: - **A plan before execution.** Get the plan right and an agent can often execute it in a single pass. Argue with the plan while arguing is cheap. - **Isolation per agent.** Separate branches or worktrees, so parallel work cannot collide in the working directory. - **A verifier that is not the author.** Something checks the output against the specification before it reaches me. - **Tests green before deploy.** On anything with real users, a full suite in CI gating the deployment, no exceptions. - **My eyes on the final release.** What I do not automate is the approval to go to production, and the final quality gate before release. Those stay human and I do not expect that to change soon. One thing I want to be precise about, because it gets misreported as a safety position. I do not automate browser-based testing of authenticated flows, and the reason is economic rather than principled. It works. I have Playwright running through MCP and it does what it says. But token consumption on visual testing is high, login walls are awkward for agents to navigate repeatedly, and manual testing plus a specific written report to the agent is currently faster and cheaper. That is a judgement about price, and prices move. The remaining gap has a name in my head: the open loop. An agent that can deploy, test the deployment on real infrastructure, observe what actually happened, and report back without a human in the middle. Nobody has closed it yet, mine included. Worker agents are eager. Until the loop closes, they need boundaries. ## How do you avoid betting on a tool that changes under you? {#staying-current} By betting on the shape rather than the tool. The twelve components do not change. Roles, context and handoffs do not change. What changes, roughly quarterly, is which product implements them best, and if your working method is built around a specific product you will rebuild your method every time the market moves. Look at what happened to my own verdicts. Claude Desktop went from "graduate away from it" to "my daily driver" in about a year. Every model version number in everything I have published is now stale, some of it within weeks of publication. A tool comparison is a photograph, not a map. Two habits keep this from being a problem. **Distrust benchmark numbers, including favourable ones.** Nearly every figure circulating about these tools is vendor-published. In at least one case I looked into, the comparison was not symmetric: the vendor's model got one fixed prompt while the models it was compared against got the better of two attempts. Treat headline scores as a direction of travel rather than a verdict, and re-verify anything you would make a purchasing decision on. **Get good at one thing before you get clever.** Pick one harness. Build one real project end to end with it. Break it, fix it, ship it. That is the whole curriculum, and it transfers completely when you switch. Orchestration, multiple agents, mixed local and cloud fleets, all of that is worth doing and none of it is worth doing first. Orchestration multiplies whatever you already have, and if what you have is chaos, you now have parallel chaos. The version-number discipline follows from the same logic. I stopped putting model version strings in anything durable, because the naming moves faster than the writing and a wrong version number costs more credibility than a vague one. Name the tier and the vendor. Let the reader look up what is current. --- --- title: "Building Without Being a Developer" number: 4 slug: "building-without-being-a-developer" status: "published" question: "Can you ship production software without being able to write the code?" summary: > Yes, with conditions, and the conditions are the interesting part. What you trade is line-level control. What you need in exchange is judgement about systems, an honest test loop, and the discipline to verify what you cannot read. updated: 2026-08-20 sources: - title: "Chasing Jarvis: Can Technically Sophisticated Non-Programmers Deploy SaaS Applications Using AI Coding Agents?" url: "https://talirezun.com/download/chasing-jarvis-can-technically-sophisticated-non-programmers-deploy-saas-applications-using-ai-coding-agents/" publication: "Research paper" sections: ["who-this-is-for", "what-breaks", "how-you-verify", "what-it-costs"] - title: "Chasing Jarvis: The Bottleneck Was Never the Code" url: "https://talirezun.substack.com/p/chasing-jarvis-the-bottleneck-was" publication: "Substack" date: 2026-06-28 sections: ["what-you-need", "the-centaur-model", "what-it-costs"] - title: "A Year in the Review" url: "https://talirezun.substack.com/p/a-year-in-the-review" publication: "Substack" date: 2026-06-23 sections: ["what-breaks", "what-it-costs"] - title: "Six Months After I Shipped Lumina" url: "https://talirezun.substack.com/p/six-months-after-i-shipped-lumina" publication: "Substack" date: 2026-07-10 sections: ["what-breaks"] - title: "I Could Tell You My AI Has Never Lied. I'm Not Going To." url: "https://talirezun.substack.com/p/i-could-tell-you-my-ai-has-never" publication: "Substack" date: 2026-08-06 sections: ["who-this-is-for", "what-breaks", "how-you-verify"] - title: "From Prototype to Production: Building an AI Widget Platform in 30 Days" url: "https://medium.com/@talirezun/from-prototype-to-production-building-an-ai-widget-platform-in-30-days-23c603c91475" publication: "Medium" sections: ["what-it-costs"] - title: "The Year I Started Coding with AI: My Coding Agent Journey" url: "https://medium.com/@talirezun/the-year-i-started-coding-with-ai-my-coding-agent-journey-431f6f25afe1" publication: "Medium" sections: ["what-you-need", "who-this-is-for"] - title: "From 0 to Dev in One Day: What a Real Coding Harness Actually Looks Like" url: "https://talirezun.substack.com/p/from-0-to-dev-in-one-day-what-a-real" publication: "Substack" date: 2026-07-20 sections: ["what-you-need"] related: ["coding-agents", "context-engineering"] tags: ["non-developer", "centaur", "product", "practice"] --- ## What do you actually need to know before you start? {#what-you-need} Not programming. You need enough vocabulary to direct the work: what a database is for, what an API does, what authentication involves, and why deployment is its own discipline rather than a button at the end. That is about a week of deliberate learning. You are not learning to code. You are learning enough to say what you want in terms the agent can act on. I tell students this on day one of every workshop, because the loudest voices online skip it and the skipping is why people bounce off. It does not work out of the box. Nobody types a sentence and receives a business. What actually happens is that you describe an idea, the agent builds something roughly shaped like it, and then you spend the real time on the gap between roughly and actually. Closing that gap needs vocabulary. If you cannot tell the difference between a problem with the database and a problem with the way the front end is calling it, you cannot tell the agent anything useful, and it will guess. Learn the bricks before you try to conduct the orchestra. A week on what the parts are and what they do buys you months of being able to direct rather than hope. The mental shift underneath this is that the ceiling moved. It used to be whether you could write the code. It is now how well you can specify, structure, and supply context. Those are different skills, and thirty years of running businesses turns out to be better preparation for the second one than a computer science degree is. The single most useful sentence I have found, and I use it constantly: sorry, I do not understand this, I am not a developer, please explain it simply. Treat the agent like a very capable colleague who has never met you and does not know what you know. It will not think less of you. Every time you use that sentence you are closing a gap between what it assumed and what is true. ## Who does this actually work for? {#who-this-is-for} Domain experts with substantial technology experience but no programming background. That is a narrower group than "anyone," and I want to be precise about it, because I spent two years studying this question with myself as the subject and the answer that came back was bounded rather than universal. The honest framing is a qualified yes with conditions. Six of them: you master context engineering rather than programming, you have deep domain expertise to compensate for the technical gaps, you scope realistically to what the tools can currently do, you invest serious time in learning and iteration and quality assurance, you can pay for the subscriptions, and you audit systematically for security. Remove any one and the picture changes. I should be clear about my own position in that finding, because it is the main limitation of the research. I am a technologist rather than a developer. I understand systems, I can architect a solution, I have enough coding knowledge to reason about a complex application stack. What I could not do was sit down and write thousands of lines of production code. That sits precisely on the boundary the study was investigating, and it means my results probably overstate how accessible this is to someone starting from zero. The paper says so about itself, along with the fact that it is a single researcher studying a single deployed application, and that a meaningful share of its own citations are my own earlier articles. Complete novices still face substantial barriers. There is a real possibility that these tools amplify existing skill gaps rather than closing them. Which brings me to something I would rather state than bury. My own position on this has moved twice, in public, and it has moved in both directions. In late 2025 I was telling non-technical founders to partner with someone technical for production deployment, because production deployment still required expertise. By mid-2026 I was writing that non-technical founders are not at a disadvantage, they are one context package away from shipping. And in August 2026, after seven months of running a product with real customers, I wrote that I now think the gap between people who work this way and people who do not is widening as models improve, not closing. I had assumed for years that better models would democratise this. I currently think the opposite is happening. Those three positions are not compatible as slogans. What reconciles them is that they answer slightly different questions: can you build it, can you ship it, and does the ability to do so spread. My current answers are yes, yes with conditions, and probably not on its own. The most useful move I made in two years of thinking about this was deciding the original question was the wrong one. "Can non-developers build production software" is not the interesting inquiry. The interesting one is what new forms of expertise make someone effective at collaborating with these tools, because that is the thing that is actually being distributed. ## Where does the human stay in the loop? {#the-centaur-model} The human supplies the specification, the taste, the judgement, and everything that depends on context the model has never seen. The model supplies implementation at speed. Neither half is sufficient alone, and the interesting work is being specific about where exactly human judgement compounds rather than treating "human in the loop" as a reassuring phrase. The evidence for the pattern is better than it usually gets credit for. A controlled field experiment run with Procter and Gamble, with 776 participants, found that individuals working with AI matched the performance of two-person teams working without it, and that AI broke down the silos between research and commercial roles. Karpathy's framing of the same division is the cleanest I have read: the specification is human, the implementation is the model's, the review is human again. Now the caveat, which matters more than the evidence. The word gets watered down. It becomes a slogan for keeping people in roles where they no longer add value, or a polite cover for automating everything and calling the leftover supervision "collaboration." Neither is the thing. The discipline is to identify precisely which decisions get better when a human makes them, and to stop defending the ones that do not. And there is a counterweight in the research that I cite in my own work and should not leave out here. A separate industry study found that a twenty-five percent increase in AI adoption correlated with a seven point two percent decrease in delivery stability. Individual productivity gains do not automatically become better software delivery. Both findings can be true: the individual gets faster, and the system around them gets less stable, because the system was not redesigned around the new speed. That is a chapter of its own, further on. ## What goes wrong when you cannot read the code? {#what-breaks} The first seventy percent goes fast and needs almost no expertise. The last thirty percent demands debugging skill, architectural understanding, and systematic problem solving, which is exactly the set of things a non-developer does not have, arriving at exactly the moment it is needed most. That asymmetry is the central difficulty of working this way, and no tool has removed it. How far you get before you hit it depends heavily on the harness. In my own testing across eight platforms, the browser-based builders started struggling somewhere around fifteen to twenty components. IDE-integrated agents managed forty to fifty with proper context management. The stronger command-line agents reached seventy-five to a hundred. Tools that index the codebase ahead of time went past a hundred, and still needed a human to make the architectural decisions. Then there is the ordinary mess, which nobody puts in a launch post. Agents making conflicting commits. An agent deploying to the wrong production project entirely. My own test infrastructure silently writing test data into real user folders, which is genuinely embarrassing and which I found rather than being told about. The darkest green squares on my contribution graph are not triumph. They are Saturdays spent debugging. The failure that taught me the most was none of those. It was a hair studio in Ljubljana whose assistant started telling real visitors, mid-conversation with real customers, that it was having trouble connecting. The cause was a two thousand character safety limit I had set months earlier and forgotten about, silently rejecting the owner's longer instructions. I traced and fixed it the same day, raised the limit four times over for everyone, and added a live counter so nobody hits an invisible wall again. What that day actually taught me was about thresholds rather than character counts. There is a moment when a product stops being a project and becomes a business, and it is not funding or launch or a revenue number. It is the first time a real person's real customers feel it when something breaks. Later, a bigger version of the same shape. A customer uploaded a document that met every stated requirement and was large enough on its own to consume well over the entire available context budget, and their assistant stopped answering anyone at all. The requirements were not wrong. They were incomplete, and incompleteness in a written rule is invisible until someone walks into it. That points at the deepest failure mode in this way of working, and I only saw it clearly when I ran a multi-agent audit across my own product. Every single finding had the same shape: something asserted in a document had been mistaken for something enforced in the code. A rule written down, checked at runtime against a list nobody could ever have filled in, passing forever. Nothing failed. Nothing logged. The document said the property held and the code agreed, and neither of them was doing anything. When you cannot read the code, written intent is most of what you have. So the gap between what a document claims and what the system does is the specific place your kind of build goes wrong. ## How do you check work you cannot read? {#how-you-verify} By moving verification out of the code and into things you can observe: behaviour, staged deployment, adversarial review by a model that did not write the thing, and tests that fail. The governing rule I ended up with is one sentence long. A comment cannot fail. A test can. That sentence is the whole method compressed. Any property you write down in a document, write down again as a test, so that when it stops being true something breaks loudly instead of quietly staying written. The four checks I actually run: **Use it yourself, exhaustively.** Every button, every screen, every error state, as a real user, including the empty form and the interrupted upload. When something breaks, never say it is broken. Say what you clicked, what you expected, what you got, and attach the screenshot. Specific reports get specific fixes. **Deploy in stages and verify live between them.** On one recent release I shipped fourteen backend functions as five separate production deployments, each one checked against the live service before the next started. It is slower and it means a failure has one obvious cause instead of fourteen candidates. **Audit with a model that did not write the code.** This is the highest-value habit in the whole chapter. An agent reviewing its own work reads what it meant to write. A model from a different vendor reads what is actually on the page. It is exactly like bringing in an external auditor, and it works for the same reason: not because they are smarter, but because they are not invested in the story of how the thing got built. I write with one model and audit with another as a matter of routine. **Get someone who did not build it to use it.** They will find things you never will. This one has never stopped being true and I do not expect it to. Now the limit, plainly. None of this lets me read a diff, and none of it is equivalent to being able to. My own research says exactly that: people without a programming background cannot reliably verify code correctness, and the mitigation it recommends is engaging actual security professionals for a pre-deployment audit, while conceding that even that does not eliminate the underlying risk. There is also a circularity I have not solved. Using AI agents to help audit AI-generated code assumes you know enough to direct the audit. If you do not, you have added a step without adding assurance. The published research on generated code is not comforting either. One large study across more than a hundred models and eighty tasks found only around fifty-five percent of generated code was secure. On my own thirty-day build, the security pass surfaced eighteen issues that needed fixing, which means eighteen vulnerabilities existed in code I had already decided looked finished. I ship anyway, with tests, staged deploys, cross-vendor audits and residual risk I can name. That is not the same as safety. It is a set of defences I can actually operate, which is better than a standard I cannot meet. ## What does this actually cost, in time and money? {#what-it-costs} Far less in money than people expect and far more in time than anyone admits. Lumina went from nothing to a deployed product in thirty days for under a hundred euros in model credits. It also took roughly two hundred hours of my own time inside those thirty days, evenings and weekends included, and about half of that was context engineering rather than building anything. That second number is the one missing from every optimistic version of this story. A few more, all from my own records rather than estimates. The avatar platform came to around seventeen thousand lines across more than three hundred agent sessions, reached a production MVP in about six weeks, and burned close to three hundred dollars a day in credits at peak. Across twelve months I logged 1,278 contributions, ninety-eight percent of them commits, spread across four products that shipped. The client on the avatar project did the counterfactual arithmetic themselves and put it at five developers and six to twelve months against one founder and six weeks. That is their number, not mine, and it is the kind of comparison worth being sceptical about, but they were the ones paying either way. Two structural things about cost that took me too long to learn. **Subscriptions beat metered API billing for development work.** Not because they are cheaper on any given day, but because on metered billing you start optimising for cost instead of quality. You skip the audit run. You do not re-verify. Once, on an early command-line agent billed by usage, a single intensive day cost me over three hundred euros, and the effect was that I worked more cautiously rather than better. **The numbers everyone quotes exclude most of the cost.** Hosting, payment processing fees, domains, and the ongoing operational work of running a live product are not in the hundred euros. Neither is the two-year learning curve that made the thirty days possible, which is the largest cost of all and appears in nobody's accounting including, usually, my own. The claim I will stand behind is narrower than the headline. Testing an idea used to cost a payroll and half a year. It now costs a weekend and a few hundred in credits. That changes who gets to try, which is a genuinely large change. It does not mean the thing you build in that weekend is a business, and it does not mean the last thirty percent got any easier. --- --- title: "Orchestration and Multi-Agent Work" number: 5 slug: "orchestration" status: "published" question: "When is more than one agent worth the trouble?" summary: > Rarely, and then decisively. Most tasks are faster with one agent and a clear brief. Fan-out earns its keep when the work is genuinely independent, or when you need several honest opinions before you commit. updated: 2026-08-20 sources: - title: "From Writing Code to Directing Intelligence: Five Days Inside Augment Code's Intent" url: "https://talirezun.substack.com/p/from-writing-code-to-directing-intelligence" publication: "Substack" sections: ["when-to-fan-out", "the-orchestrator", "what-breaks"] - title: "From One Agent to Coding Agent Armies: My 15-Month Journey to AI Orchestration" url: "https://medium.com/@talirezun/from-one-agent-to-coding-agent-armies-my-15-month-journey-to-ai-orchestration-b9138675a075" publication: "Medium" sections: ["what-breaks", "when-not-to"] - title: "I Built Auto Loops Before They Had a Name" url: "https://talirezun.substack.com/p/i-built-auto-loops-before-they-had" publication: "Substack" date: 2026-06-22 sections: ["loops", "verification"] - title: "I Built a 24/7 AI Email Support Agent" url: "https://talirezun.substack.com/p/i-built-a-247-ai-email-support-agent" publication: "Substack" sections: ["loops"] - title: "Blueprint of a Frontier Coding Agent" url: "https://talirezun.substack.com/p/blueprint-of-a-frontier-coding-agent" publication: "Substack" sections: ["the-orchestrator", "verification"] - title: "The Mixed Fleet" url: "https://talirezun.substack.com/p/the-mixed-fleet" publication: "Substack" date: 2026-08-17 sections: ["the-orchestrator", "verification", "when-not-to"] - title: "Lumina: An AI Agent Your Business Can Stand Behind" url: "https://talirezun.substack.com/p/lumina-an-ai-agent-your-business" publication: "Substack" date: 2026-07-31 sections: ["when-to-fan-out"] - title: "A Year in the Review" url: "https://talirezun.substack.com/p/a-year-in-the-review" publication: "Substack" date: 2026-06-23 sections: ["what-breaks"] - title: "How I Built an AI Marketing Team That Actually Works" url: "https://medium.com/@talirezun/how-i-built-an-ai-marketing-team-that-actually-works-from-memes-to-technical-content-in-minutes-87f646608c60" publication: "Medium" sections: ["when-not-to"] related: ["coding-agents", "context-engineering"] tags: ["multi-agent", "orchestration", "workflow", "ai-agents"] --- ## When does running several agents beat running one? {#when-to-fan-out} When the execution is genuinely separable from the judgement, and the judgement lives somewhere else. That is the whole rule. If you can hand out work that does not need to know what the other work is doing, and something other than the workers decides whether the result is good, fan-out pays. If you cannot, you are buying coordination overhead and calling it parallelism. Look closely at the multi-agent setups that actually work and almost none of them are really arguments about doing more things at once. They are arguments about where the checking happens. An expensive model plans and judges while a cheap swarm executes. A verify engine sits outside the swarm and catches what the swarm produces. The parallelism is incidental. The separation of powers is the point. The scale this reaches is real when it fits. I have run twenty-one agents coordinating on a single build, and a hundred and seventy-seven tasks inside one workspace. The second generation of Lumina took hundreds of sessions with hundreds of agents over roughly a month, producing around eight hundred and sixty commits. None of that was possible with one agent and a long afternoon. And the counterweight, which is reported by another team rather than measured by me, is worth holding next to those numbers. Cursor's engineering team found that twenty agents of equal status produced roughly the throughput of two or three, because they spent their time holding and waiting on file locks. Their fix was not fewer agents. It was three distinct roles: planners, workers and judges. Same conclusion from the other direction. Structure, not headcount. ## What does the orchestrator actually do? {#the-orchestrator} It holds the specification, decomposes the work, dispatches it, reviews what comes back, and does not write code itself. That last clause is not a stylistic preference. An orchestrator that starts implementing loses the overview that made it useful, and you end up with an expensive model doing a cheap model's job badly. The most useful reframe I know for this: orchestration is a management skill, not a technical one. You are hiring, briefing, delegating, reviewing and, when it comes to it, firing. If you have ever run a team you already know most of it, and the parts that are unfamiliar are mechanical rather than conceptual. The pieces that make it work in practice: **A living specification, not the code, as the source of truth.** The orchestrator drafts it, updates it as work completes, and every downstream agent reads the current version rather than whatever it was told at dispatch time. **Isolation per worker.** Separate branches or worktrees, so two agents editing adjacent things cannot collide in the working directory. Without this, parallelism turns into merge conflicts almost immediately. **A verifier that is not the author.** Something checks the output against the specification before it reaches you. **The role instruction carried forward.** When a session ends, the handoff says what happened and also restates the job: you are the orchestrator, you do not write code yourself, you delegate, you audit. Otherwise the next session quietly reverts to being a single agent with ambitions. Model choice becomes staffing. Complex reasoning and decomposition go to the strongest model you have. Mechanical work goes to something cheaper. The audit goes to a model from a different vendor entirely. You are not picking a favourite. You are filling roles. ## Do autonomous loops work, or do they just spend money? {#loops} They work, and they will absolutely spend money if you let them. An auto loop is an agent driving a running application through real user scenarios without a human in the seat: act, observe, diagnose, fix, verify, repeat. It belongs in the debugging and deployment phase, not the coding phase, and the thing that decides whether it earns its keep is not the loop. It is the verifier. I was running these for over a year before anyone called it loop engineering, so what follows is mostly scar tissue. **Prepare the context before you start it, not during.** Application architecture, a full feature inventory, the concrete scenarios you want walked, and explicit success criteria. A loop cannot ask you clarifying questions at three in the morning. **Give it eyes and hands.** Browser automation through MCP, so it can navigate the interface, click things and see what happened. **Run separate loops per role.** I mixed ordinary user scenarios and administrator scenarios in one loop and produced nothing but debugging confusion. Splitting them fixed it. **Solve authentication in phase one, not when the loop hits a login wall.** This is the failure that cost me the most time. Sign-in flows break browser automation reliably. Authentication state gets lost between sessions, and bot detection starts throwing verification prompts at your agent. It is an architecture problem wearing a testing problem's clothes. **Budget tokens and set an iteration cap before you walk away.** Every action and every screenshot analysis accumulates, and browser work burns context faster than almost anything else. My loops used to hit the ceiling mid-task and stop, without ever completing gracefully. Then the part that matters more than all of it. The hard part is not the loop. It is the verifier. Without an independent check against objective criteria, the agent declares success and moves on. The sharpest version of this I have read is that a loop which cannot distinguish good output from bad does not save you work, it produces wrong answers faster. That is the correct way round to think about it: the danger is not that the loop stalls, it is that it accelerates. I should own the obvious criticism here. My own early loops had no independent verifier in them. I was the verifier, by reading the output myself afterwards, which is exactly the personal review that objective criteria are supposed to replace. That was a shortcoming, not a design choice. One more piece of vocabulary that clarified my thinking, and it is not mine. A **loop** repeats while your session is open. A **goal** runs until a verifiable condition is true and then stops. A **schedule** runs in the cloud with your laptop closed. Only the goal has a termination condition, and a termination condition is a verification claim in disguise. If you cannot write the stopping condition, you do not yet know what you are checking for. The scheduled form is worth its own mention because it is the cheapest useful version of any of this. My support inbox runs as a headless agent on a cron job, hourly, on an always-on machine. It fetches unread mail, classifies by intent, and either answers or forwards to me with one sentence explaining why it did not feel able to answer. Roughly a fifth gets forwarded, and forwarding is the correct outcome rather than a failure, because a mediocre answer to a complicated question is worse than no answer at all. That took an afternoon to build and costs a few cents a run. Its weakness is instructive too. It logs how many messages it processed, replied to and forwarded. It logs nothing about whether the replies were any good. I spot-checked the sent replies for the first few weeks and then stopped, which means the quality loop on that system is currently me remembering to look. ## How do you check work you did not watch happen? {#verification} With a model that did not do the work. This is the highest-value habit in this entire chapter and it takes one extra step: write with one model, hand the output to a model from a different vendor, and ask it to audit. The reason it works is not that the second model is smarter. An agent auditing its own code reads what it meant to write. A different model reads what is actually on the page. It is exactly like bringing in an external auditor, and the value is the same: they find things the internal team never will, because they are not invested in the story of how the thing got built. Everything else about verification, the tests, the streaks, the staged deploys, the audits that found things I did not want to find, is the subject of chapter eight. It earned its own chapter. ## What goes wrong when you scale up? {#what-breaks} Agents make incompatible assumptions about each other's work, and nobody tells you. My first parallel runs ended with a backend expecting data structures the frontend was not producing, discovered at integration time, and requiring a cleanup agent whose only job was reconciling two correct implementations of different ideas. The failure list from running this at scale, roughly in order of how often it bites: **Context does not propagate sideways.** Architectural decisions made by one coordinator did not automatically reach another. Agents spun up by a second coordinator lacked context the first one held, and had to be briefed manually. Every coordinator boundary is a place where context has to be carried deliberately. **Commits collide.** Commit messages from different coordinators clashed and overlapped, which is cosmetic until you are trying to read the history to work out what happened. **Wrong-target deploys.** One set of worker agents briefly attempted to deploy to an entirely different production project. My guardrails caught that one. An earlier version of the same mistake, before I had guardrails, overwrote an application, and everything was recoverable only because of the commit discipline. **Your own hardware becomes the ceiling.** Running agents locally, I hit GPU and CPU strain at around five simultaneous agents. That number is specific to my machine and to that year, but the shape of the problem is not. **The interface degrades before the work does.** Long coordinator threads made the tool itself lag noticeably, which is a small thing that makes a long session unpleasant. **Cost is non-linear and arrives late.** A heavy fan-out day consumed roughly a quarter of a month's allocation on a two-hundred-dollar plan, so ten-hour days exhausted a month in two. On the avatar platform, peak burn ran close to three hundred dollars a day in credits. None of that was visible when I designed the workflow. It was visible at the end of the week. None of these were catastrophic and everything was recoverable. But recoverable is doing real work in that sentence, and what makes it true is version control, isolation, and a human paying attention. ## When should you not orchestrate at all? {#when-not-to} If you are new to this, do not orchestrate. Not yet, genuinely. Pick one harness, build one real thing end to end, break it, fix it, ship it. That is the whole curriculum and it transfers completely. Come back to this chapter in three months. The reason is not that orchestration is difficult. It is that orchestration multiplies whatever you already have, and if what you already have is chaos, what you now have is parallel chaos. I watch people hurt themselves with this. They read about a hundred and seventy-seven agents and reach for the largest available structure before they have a specification worth decomposing. Three habits that keep it from going wrong, once you do start: **Start with one coordinator.** Resist adding a second until the first one is genuinely saturated. **Keep the unit of work small.** Roughly the size of a pull request. Large units make failures expensive to unwind. **Treat notes as infrastructure.** Architectural decisions and constraints belong in the specification, not in the chat thread where only the current session can see them. And a piece of honesty about my own writing on this, because it is the most common way these systems get oversold. I once published a piece about building an AI marketing team. What I had actually built was one agent with two different tool configurations, plus two separate generation tools. It produced genuinely good work and I stand by the workflow. But the plurality was in the tooling, not in concurrent agents, and calling it a team was a description of how it felt rather than what it was. That distinction matters, because a great deal of what is described as multi-agent is one agent with several sets of tools. That is often the right architecture. It is just not the same claim. The last thing to be clear about: none of this is autonomous. You need to be present. It reduces the implementation burden enormously and it does not reduce the architectural judgement burden at all. That part is still yours. --- --- title: "AI Sovereignty and Open Weights" number: 6 slug: "ai-sovereignty" status: "published" question: "What does it mean for a European organisation to control its own AI?" summary: > Sovereignty is not a slogan about hosting location. It is a set of concrete answers about where the weights run, who can revoke your access, what leaves your network, and what you would do if a vendor changed terms tomorrow. updated: 2026-08-20 sources: - title: "16x Cheaper, Open Weights, and the Model That Doesn't Disappear on Fridays" url: "https://talirezun.substack.com/p/16x-cheaper-open-weights-and-the" publication: "Substack" date: 2026-06-18 sections: ["the-shutdown", "open-weights", "the-cost-case"] - title: "Anthropic Shipped Two Models This Week. I Only Wanted One Back." url: "https://talirezun.substack.com/p/anthropic-shipped-two-models-this" publication: "Substack" date: 2026-07-01 sections: ["the-shutdown"] - title: "The Fight for Intelligence" url: "https://talirezun.substack.com/p/the-fight-for-intelligence" publication: "Substack" date: 2026-07-08 sections: ["what-sovereignty-means", "the-shutdown"] - title: "The Brain Is Ready. The Body Is the Problem." url: "https://talirezun.substack.com/p/the-brain-is-ready-the-body-is-the" publication: "Substack" date: 2026-05-04 sections: ["what-sovereignty-means", "open-weights", "the-tradeoff", "the-three-rings"] - title: "Data Sovereignty in the AI Age: Building Your Own Private ChatGPT" url: "https://medium.com/@talirezun/data-sovereignty-in-the-ai-age-building-your-own-private-chatgpt-a83c96e342a0" publication: "Medium" sections: ["what-sovereignty-means", "the-tradeoff"] - title: "The Mixed Fleet" url: "https://talirezun.substack.com/p/the-mixed-fleet" publication: "Substack" date: 2026-08-17 sections: ["open-weights", "the-tradeoff"] - title: "The Energy and Water Footprint of Generative AI: A Vanguard Leadership Perspective" url: "https://talirezun.com/download/the-energy-and-water-footprint-of-generative-ai/" publication: "International Leadership Journal" sections: ["the-tradeoff"] related: ["organisational-design", "coding-agents"] tags: ["ai-sovereignty", "open-weights", "europe", "policy"] --- ## What does sovereignty actually mean here? {#what-sovereignty-means} Four questions, none of which is about where the datacentre is. Do you possess the weights, or only permission to call them. Who can switch you off, and how much notice would you get. What actually leaves your network. And what would you do on Monday if the terms changed on Friday. Answer those and you know your position. Answer "our provider has an EU region" and you know almost nothing. My own position on this moved, and it moved because of an event rather than an argument. In 2025 I wrote about sovereignty as physical locality: bring it in-house, run it locally, keep the data on your own hardware. Vendor risk was in the piece, but as a bullet point. Changing terms of service, service limitations, lock-in. The sort of thing you list and then stop thinking about. That is not where I would put the emphasis now. The version I would defend today is that sovereignty is continuity insurance, and the asset that provides it is a weights file you already have plus a work layer that is portable enough to move. Not a contract, not a jurisdiction, not a residency guarantee. Something on a disk that nobody can reach into and remove. The operating rule that falls out of it is short: never let your ability to function depend on only one path existing. I want to be careful about what that does and does not imply, because the sovereignty argument attracts purists and I am not one. I still run frontier cloud models for the hardest work, and I would recommend that to most people building today. The claim is not that you should avoid the good models. It is that you should be able to keep operating in a degraded but functional way if one of them becomes unavailable, and that you should know in advance which mode you would drop into. ## What did the export-control episode actually prove? {#the-shutdown} That no bad actor is required. A model launched on the ninth of June 2026. Three days later, on Friday the twelfth, at twenty past five New York time, a US Department of Commerce export-control directive made it inaccessible to every non-US national worldwide, including the vendor's own employees. It stayed gone for nineteen days. Access began returning on the first of July. I had it for three days. Then it was gone, and there was no process I could participate in, no notice, and no appeal. The stated reason involved a reported technique for getting the model to identify software vulnerabilities. The vendor's own analysis, which they published, was that the technique was narrow, not universal, and reproducible on several other widely available models. Whether you find that reassuring is not really the point. The point is what the episode reveals about the shape of the dependency. Nobody behaved badly here. The vendor did not deprecate a product to push an upgrade. There was no outage, no billing dispute, no acquisition. A government made a decision on a Friday evening on another continent, and a tool that a lot of people had built processes around stopped existing for them. An organisation does not need anyone to act in bad faith for a single-vendor intelligence dependency to become a real operating risk. My reading is that this was the first time a national government applied export controls directly to a deployed model rather than to chips or hardware. I have not been able to verify that independently, so treat it as my reading rather than an established fact. If it is right, it is a category change, because chip controls take years to bite and a model switch-off takes minutes. What I actually changed afterwards was not my model choice. It was making sure there was a second path that already worked, rather than a second path I could theoretically build. ## Are open-weight models good enough yet? {#open-weights} For a large share of real work, yes, and the threshold that matters is not the trillion-parameter flagship. It is the mid-sized model that is genuinely good and fits on one consumer graphics card. That is the point at which "run it yourself" stops being a research project and becomes a Tuesday. Two releases from the summer of 2026 illustrate where the line sits, and I am dating them deliberately because this moves fast enough that an undated claim here is worthless. **Qwen3.8-27B**, released on the fourteenth of August 2026 under Apache 2.0. Roughly twenty-seven billion parameters, dense, with a 262,000-token native context window. At four-bit quantisation the community consensus is that this needs a 24GB card rather than a 16GB one. That is one high-end consumer GPU. Not a rack, not a datacentre, one card. I should be clear that the memory figure is community consensus rather than my own benchmark. **DeepSeek V4 Flash**, in public beta from the thirty-first of July 2026 with MIT-licensed weights. Text only, and it wants 32GB and up to run locally. Earlier in the year, **Kimi K2.7** landed with a modified MIT licence, a trillion total parameters with thirty-two billion active per token in a mixture-of-experts arrangement, and a 256,000-token context window. Its published benchmark score came from its own maker, independent scores were not available when I wrote about it, and practitioners were reporting that headline numbers did not replicate cleanly on real repositories. I said then to treat it as directional rather than definitive and I would say the same now. Which brings me to the caveat that applies to this entire section. Nearly every benchmark number circulating about these models is vendor-published. In at least one comparison I looked into, the prompting was not symmetric: the vendor's model got one fixed prompt while the models it was measured against got the better of two attempts. Read headline scores as a direction of travel. Re-verify anything you would spend money on the strength of. One distinction that gets collapsed constantly and should not be. Open weights and local execution are not the same thing. A model with published weights and six hundred billion parameters is still not running on your laptop. Open weights give you the right to run it and the ability to keep a copy. Whether you can actually run it is a separate question about your hardware. ## What does the cost difference look like in production? {#the-cost-case} Roughly an order of magnitude, on the right kind of work. On a test of twenty-four generated landing pages across twelve prompts, an open-weight model came in at about a dollar fifty in total, against about twenty-four dollars for a frontier model on the same set. Per page, four to fifteen cents against seventy-five cents to a dollar twenty-six. That is sixteen times cheaper, and about seven and a half times cheaper than the mid-tier frontier option on the same task. Two caveats, and the second one is the important one. The outputs were comparable, not identical. On a task like this that is a reasonable basis for comparison, because the point is whether the result is fit for purpose rather than whether it is the same result. On complex multi-step engineering work I would not make that assumption at all, and the benchmark gap between these tiers was real and material. And I could not run the frontier side of that comparison myself. Export controls meant I did not have access to that model when I wrote the piece, so those figures came from external research rather than from my own account. That is a meaningful weakness in a cost comparison and I would rather say so than let the ratio travel without it. What the numbers support, stated conservatively: for high-volume, well-specified, relatively mechanical generation, the cheap open-weight option is not a compromise, it is the correct choice, and the saving is large enough to change what you can afford to attempt. For the hardest reasoning work, the frontier models were still ahead when I measured, and I would still route there. That is the routing strategy in one line. Open weights for volume, frontier for judgement, and a self-hosted fallback that already works. ## What do you give up by running your own? {#the-tradeoff} Capability, and more of it than the enthusiasts admit. The capability ceiling of what you can run on consumer hardware is still well below what the frontier cloud models do. Local deployment of a model good enough to be a genuine daily assistant remains a niche capability rather than a default, and I say that as someone who runs local models regularly and wants the other answer to be true. There is a version of this argument that says the gap closes on its own, because what needed a server farm in 2023 runs on a laptop now. I think that is probably right and I also think "eventually" is not a useful answer for anyone who has to make an infrastructure decision this quarter. The second thing you give up is cleaner economics, and this one surprised me. The obvious move is a mixed fleet: a frontier orchestrator on your subscription, delegating mechanical work down to a local model. Mostly it does not work. Pointing a harness at a local endpoint is usually substitution rather than addition, session-wide, replacing the frontier model rather than letting it call a local worker. One harness genuinely routes different agents to different providers in the same session. And the vendor position is that subscription logins are for native use of their own products, with third-party tools expected to use metered API billing, which one open-source project acted on in March 2026 by removing its subscription integration after a legal request. So the mixed fleet works, and if a frontier model is in it, you pay API rates for that slice. There is currently no supported way to have both the subscription economics and the cross-provider orchestration. I should also say that I established most of that against official documentation rather than by running every combination myself. I did not want to publish a wish list dressed up as a field test. What you gain, beyond the two things already covered, is a smaller and more predictable surface. Nothing leaves the machine. Nothing is retained under a policy you did not write. And there is a modest environmental argument alongside it: in a comparison I co-authored, running a mid-sized open model locally used roughly a third less energy per task than a frontier cloud model on the same job. That figure comes from published literature and, in places, from vendor-reported data rather than from independent measurement, so I would treat it as an argument for the direction rather than a number to quote. ## How do you decide what an agent may touch? {#the-three-rings} Three rings, decided in advance rather than in the moment. Ring one is what you connect freely. Ring two is what you connect with your eyes open. Ring three never goes near a cloud agent at all. Sorting your data into those three buckets before you wire anything up is the single most useful hour you can spend on this, and almost nobody does it. **Ring one, connect freely.** Published work, public-facing material, generic research. The test is simple: data you would be comfortable with your employer or a contractor seeing. **Ring two, connect with awareness.** Email, calendar, project management tools, working documents. Sensitive, but not personally identifying and not financially critical. This is where most useful agent work actually happens, and where you should be reading retention policies rather than assuming them. **Ring three, local only or not at all.** Financial records, medical data, personal correspondence, legally privileged communications, client-confidential material. Do not route this through a cloud agent. If you need AI help with it, use a local model and accept the lower capability as the price. The underlying tension is not going away, and it is worth naming plainly: the more context an agent can reach, the more useful it becomes, and the more of that context passes through infrastructure you do not control, the more of your position you have given up. Those are not independent variables you can optimise separately. Every increase in usefulness is an increase in exposure, and the rings are just a way of deciding where you are willing to make that trade before you are in a hurry. Two practical notes from running this. Use a dedicated machine and a dedicated identity for agent work where you can. If an agent has access to a contained environment with nothing sensitive in it, the risk is proportionate to the access. If it runs on your daily driver, it has access to everything by definition, whether or not you intended that. And watch what vendors do rather than what they say. When one company's screen-capture feature launched unavailable in the EU, the UK and Switzerland, that was a product decision rather than an accident, and it told you more about how the feature handles data than any policy page would have. --- --- title: "Organisational Design for the AI Era" number: 7 slug: "organisational-design" status: "published" question: "How should a company reorganise itself around AI, if at all?" summary: > Most AI initiatives fail on org design rather than on technology. The work lands in a team with no mandate, no budget for failure, and no route to the people whose processes would have to change for any of it to matter. updated: 2026-08-20 sources: - title: "Building the Organization of Tommorow: An Open-Source Framework and Playbook for Partner-Run, AI-Augmented Organisations" url: "https://talirezun.com/download/building-the-organization-of-tommorow/" publication: "Research paper" sections: ["where-it-fails", "the-framework", "compensation"] - title: "The Fight for Intelligence" url: "https://talirezun.substack.com/p/the-fight-for-intelligence" publication: "Substack" date: 2026-07-08 sections: ["the-framework", "the-klarna-test", "the-regulatory-clock"] - title: "The Great Reckoning: Vanguard Leadership in the Age of Intelligent Machines" url: "https://talirezun.com/download/the-great-reckoning-vanguard-leadership-in-the-age-of-intelligent-machines/" publication: "Research paper" sections: ["the-regulatory-clock"] - title: "AI as a Force Multiplier: Leaders for Industry 5.0" url: "https://talirezun.com/download/leaders-for-industry-50/" publication: "International Leadership Journal" sections: ["where-it-fails"] - title: "The ØØT Framework" url: "https://github.com/talirezun/oot-framework" publication: "GitHub" sections: ["the-framework", "the-klarna-test", "compensation"] - title: "The Shared Brain: When Second Brains Start Thinking Together" url: "https://talirezun.substack.com/p/the-shared-brain-when-second-brains" publication: "Substack" sections: ["the-framework"] - title: "Chasing Jarvis: The Bottleneck Was Never the Code" url: "https://talirezun.substack.com/p/chasing-jarvis-the-bottleneck-was" publication: "Substack" date: 2026-06-28 sections: ["teaching-it"] related: ["ai-sovereignty", "agent-memory"] tags: ["organisational-design", "change", "leadership", "adoption"] --- ## Why do AI initiatives stall inside large organisations? {#where-it-fails} Because the problem is organisational and the budget went to technology. The models are ready. The organisations are not, and no amount of model quality fixes a process that nobody has permission to change. The most commonly cited number here is that around ninety-five percent of enterprise generative AI pilots produce no measurable effect on profit and loss, with roughly five percent seeing genuine revenue acceleration. That gap is not a capability gap. The most useful research finding I have come across on this is not about adoption at all. It is about perception. A randomised controlled trial in 2025 put experienced open-source developers on real tasks in their own repositories with the AI coding tools available at the time. Measured, they were about nineteen percent slower. Asked afterwards, they reported being about twenty percent faster. That is a thirty-nine point swing between what happened and what it felt like, in the group you would most expect to judge it accurately. Sit with that for a moment, because it invalidates the way most organisations currently evaluate this. If your rollout decision rests on how the pilot team says it went, you are measuring a feeling. Nobody can self-assess AI-assisted productivity without an external baseline, which means the baseline has to be captured before the rollout, and almost nobody does it. The rest of the evidence points the same direction, at the organisation rather than the tool. Around eighty-eight percent of organisations report using AI in at least one function, but the small minority that attribute meaningful profit to it are several times more likely to have redesigned workflows end to end rather than layering tools onto existing processes. Workflow redesign, not model selection, is what separates them. Meanwhile a large majority of leaders expect agents to be integrated across their organisation within twelve to eighteen months while under half are automating any workflow with agents today, which is the gap between intention and practice written as a statistic. Two findings that get read as technophobia and are not. Developers refuse AI assistance for specific things at high rates: around three quarters decline it for deployment and monitoring, around seven in ten for project planning. That is not fear. Those are precisely the tasks where an error is expensive, the context requirement is highest, and accountability is clearest. The refusal is rational and worth listening to rather than overriding. And AI assistance improves delivery only where the surrounding engineering discipline already exists. With mature continuous integration, code review and observability, it lifts deployment frequency and lead time. Without them, it accelerates the production of bugs. The tool amplifies the system it lands in, which is a general principle disguised as an engineering finding. The practical consequence for anyone running a programme: capture a baseline before you start, run a genuine pilot with a slice of the team rather than a mandate to everyone, and plan for a resistance plateau measured in a year or more rather than a quarter. My own reading of the ninety-five percent figure is that a large share of it is teams that gave up after the first disappointing pilot, which is a different failure from the one the number is usually used to describe. ## What structure actually holds up? {#the-framework} A file-based one. Everything an organisation decides, records, or automates should live as plain text and ordinary spreadsheets in version control, because that is the only format that survives a change of vendor, a change of tool, and a change of staff. Whatever structure you choose on top of that, this is the part that determines whether it is still legible in three years. The framework I built and publish for this is called ØØT, the Organisation of Tomorrow. It is a public repository rather than a methodology I sell, dual licensed with Apache 2.0 on the code and Creative Commons on the documentation. It ships twelve skill packs, nine spreadsheet templates, eight scheduled routines, four governance documents, and installer tooling. It is designed to be taken apart: adopt the compensation model without the rest, or the knowledge layer without the compensation model. Three deployment tracks run identical governance: a cloud track, a privacy track running local models on your own hardware, and a community track for people with no budget at all. Agent-assisted installation takes an hour or two. Doing it by hand takes about sixteen hours across two weekends. A ten-partner firm on the cloud track runs at roughly four hundred and seventy euros a month in subscriptions; a five-partner privacy setup is around two and a half thousand euros of hardware once and about fifty-five euros a month after. Now the part that matters more than the feature list. **Five of the twelve skill packs are scaffolds rather than finished work**, and one of them is governance and compliance. I mention that first because it is the one an organisation would most want to be finished, and describing a framework as regulation-aware while its compliance module is incomplete is exactly the kind of overclaim this site exists to avoid. The finance and treasury layer is in the same state: the shape is there, the operational instructions are being filled in release by release. **The knowledge architecture already failed once.** The original design had a single collective brain, and it leaked individual partners' private notes into company infrastructure. Nobody was harmed and it was caught internally, but it was a design error rather than a bug. The fix was to split it in three. - A **personal second brain** on each individual's own machine, private by default, holding everything they read and decide. - A **firm brain**, shared and version-controlled, containing synthesised summaries rather than raw notes, pseudonymous by default, with names surfacing only when both the firm and the individual opt in. - A **ledger** that only automation writes to. Timestamped, signed, never hand-edited, which is what makes it worth anything as a record. The separation solves a specific problem: institutional memory should survive somebody leaving, without their private thinking having been absorbed into the company along the way. **Nobody outside my own reference organisations has published a deployment of this.** No adopter count, no independent case study, no third-party evaluation. The generation-three material in the repository is research-stage and I say in the documentation that some of it may simply not work. Treat the whole thing as a well-documented proposal that I run myself, not as a validated method. ## How do you decide whether to let AI replace a human role? {#the-klarna-test} With a written test, scored before the decision rather than justified after it, reviewed by somebody whose pay does not go up if it passes. Ten questions, each scored zero, one or two, twenty available, and fourteen required to proceed. Below fourteen you hold, fix the gaps, and score it again. I named it the Klarna Test after the company that publicly cut around seven hundred customer service roles in 2024 with AI credited for the change, saw service quality degrade, and was rehiring on a hybrid model by 2025. The honest version of that story includes a detail people leave out: headcount still fell by roughly forty percent over the period. So the replacement narrative collapsed and the reduction largely stuck. Both things happened, and a framework that only tells you the first half is selling you something. The ten questions, compressed: 1. Has quality been measured against a human baseline, on real production traffic, for at least three months? Vendor benchmarks do not count. 2. Are you still measuring the original success metric, rather than a proxy like model confidence? 3. Is there a pre-committed reversal threshold, written down before go-live? 4. Is the reversal plan operational? People on standby, escalation paths live, capacity restorable inside two weeks. 5. Were the affected people consulted in writing, with a record of what they said and how it was addressed? 6. Do you have a real baseline from before the rollout? 7. Has a non-beneficiary reviewed it? Someone whose compensation does not increase if this goes ahead. 8. Is the public communication posture decided and owned? "We say nothing" is an acceptable answer only if it is a written decision with a review date. 9. Is a ninety-day post-deployment review scheduled and resourced, with a named owner and a decision framework? 10. Would you be willing to defend this decision in two years? Two rules make it work rather than decorate. **Score zero if it is not defined now**, which stops the whole thing being gamed by promising to sort it out in phase two. And **the non-beneficiary reviewer is not optional**, which is the only question in the list that cannot be answered by the person who wants the answer to be yes. The recheck is every ninety days, and proceed is a conditional verdict rather than a permanent one. In practice it runs as a blocking gate: a change labelled as replacing human work does not merge until it has been scored. A worked example, from one of the reference firms. Handing routine customer onboarding emails to AI, with the original partner remaining as the escalation path, scored sixteen out of twenty. The two lost points were a pilot that ran four weeks rather than three months, and a ninety-day review that was scheduled but under-specified. Both were tightened rather than waived, and it went live with those gaps known and written down. That is the realistic outcome for most decisions, and it is more useful than a clean twenty would have been. ## Does the pay model have to change? {#compensation} If the work changes shape and the pay does not, you have created an incentive against the tools. When somebody can do in two hours what used to take eight, paying for eight hours rewards them for pretending it still takes eight. Nobody will say that out loud, and it will still be true in the quiet parts of the organisation where adoption actually dies. The direction I argue for is paying for output rather than attendance, and I want to be careful about how strong that claim is, because it is the most speculative material in this chapter. What I actually ship is four layers. A small guaranteed base draw, so nobody is living on variance. Monthly variable pay against accepted output specifications. A quarterly long-tail entitlement, a share of the value an artefact actually generated over its operational life, which exists specifically to kill the incentive toward shipping features nobody uses. And a role-weighted annual bonus split roughly three ways between personal output, team, and company outcomes. Three further layers are documented and deferred: subscription credits, dividends, and buying back units at a published price. Those need legal scoping and, usually, payment rails that are not settled yet, and rushing them would be irresponsible. Alongside it sits something I would recommend to any partnership regardless of the rest: a written, signed declaration of which reward model each partner is actually on. Pure eat-what-you-kill, pure lockstep, or a declared hybrid with the weights written down. The point is not the choice. It is that the choice is recorded, renegotiable on a known cadence, and not something everyone quietly assumes differently. Now the limits, which are real. Output-based pay is reasonably well grounded for many roles and shaky for others. It handles work with a long gap between input and outcome badly: research, long-cycle sales, regulatory work. It handles hard-to-attribute contribution badly too: creative collaboration, mentorship, the person who makes everyone else better. The team and recognition components exist precisely because attribution alone is incomplete, and if that sounds like a patch, it is one. The legal position varies materially by jurisdiction, and the framework does not solve that. It points at it and runs. Worker classification, the legality of variable pay, securities law around anything resembling an equity entitlement, data protection where an attribution agent is scoring people's output. Local counsel is not a suggestion here. And one number I want to disown before someone quotes it at me. The idea of a firm being five percent humans guiding ninety-five percent agent operations is a design hypothesis, not a measured outcome. It is a limit case, useful as a direction to think in, and there is no evidence it describes a median organisation. ## How much should regulation drive the timeline? {#the-regulatory-clock} Less than most European programmes currently let it. The clearest evidence I have is a deadline I was building against myself: the EU AI Act's high-risk obligations were due to take full effect on the second of August 2026, and via the Digital Omnibus package that moved to December 2027. Sixteen months, after two years of compliance planning across the union had been organised around the earlier date. The detail I find most instructive is not the shift. It is that my own framework's governance documentation, refreshed three days before I wrote about it, had not caught up. I am one of the people tracking this closely, publishing about it, with a repository whose selling point includes regulatory awareness, and I was stale within days. The design conclusion is straightforward once you have been caught by it. Do not build the programme around a date. Build it around a capability that would be worth having whether or not the date moves. Record-keeping, human oversight, transparency about what is automated, a risk process you actually run: all of those pay for themselves in operational terms and none of them become worthless if a deadline slips. A compliance programme that is only rational because of a specific date is a bet, and that bet just lost sixteen months. There is a second-order argument here that I have made in co-authored work and that I would present as my position rather than an established finding. Roughly seven percent of European banking and financial services organisations have implemented AI at scale, against US firms that moved from experimentation to execution across the same period. I attribute the lag to three things that compound: genuine compliance overhead, employment law that makes rapid restructuring slow and costly, and, most dangerously, a cultural disposition to read regulatory friction as a competitive advantage rather than as a delay. My reading is that Europe does not avoid the adjustment, it receives it compressed. A decade of deferred restructuring arriving in two years is not a softer landing. That is a projection rather than a measurement, it is contested by people who think European caution will look wise in hindsight, and I would rather state it as an argument than dress it as a finding. ## What do executives get wrong in the room? {#teaching-it} They expect it to work out of the box, and then they conclude it does not work. That is the single most common failure in a workshop and it is almost always fatal within a fortnight, because the disappointment lands before the practice does. Everything else I could tell you about teaching this is downstream of getting people past the first hour. I teach a five-module course to MBA cohorts and executives. It has run online with close to a hundred and fifty participants, drawn predominantly from across several African countries and partly supported by the UN, and as a condensed two-day edition in Belgrade with two cohorts from different generations of an MBA programme. The measurable outcome from the Belgrade sessions is the one I trust most, and I want to be precise about what it does and does not show. By the end of day two, every group had shipped something real with a live URL: factory-floor optimisation tools, bank-regulation compliance dashboards, workflow automation systems. Many of those people had not touched a developer tool two days earlier. What I do not have is a completion rate, a dropout number, or any follow-up on whether a single one of those artefacts survived the week. I know that people who had never built software shipped a working link in two days. I do not know that any of it mattered a month later, and I am not going to imply that I do. Four mistakes I see repeatedly, in the order they usually appear: **Treating context engineering as an advanced topic.** I originally put it in module four. Feedback made it obvious that was wrong, so it now appears in simple form from module one and deepens through the course. Everything else people struggle with turns out to be a context problem in a costume. **Treating resistance as a communications problem.** It is not. It is a rational response to being asked to change how you work, without a mandate, by someone who has not measured whether the new way is better. The answer is a real pilot with a slice of the team, a baseline captured beforehand, and champions who earned the role rather than being appointed to it. Appointed champions consistently backfire. **Skipping the vocabulary.** People try to direct an agent without knowing what a database is for or why deployment is its own discipline, and the agent fills the gap with assumptions. A week on the parts and what they do is the highest-return investment in the whole curriculum. **Confusing speed with readiness.** The speed gain is real and it lands in the build phase. The discipline requirement in the phase after it has not decreased at all. The claim I hold most loosely and believe most strongly is this. In almost every room I have taught, the technical barrier turned out to be psychological rather than real, and something changes permanently the moment somebody shares a link to a thing they built themselves. That is experience rather than evidence, and I do not have a way to measure it. But it is why I keep doing the two-day version rather than the lecture. --- --- title: "Verification" number: 8 slug: "verification" status: "published" question: "How do you know an agent's work was right if you did not watch it happen?" summary: > Agent failures are quiet. The work completes, reports success, and leaves something broken underneath. Verification is the discipline of making that gap loud, and it is where most of my own worst findings came from. updated: 2026-08-20 sources: - title: "I Built Auto Loops Before They Had a Name" url: "https://talirezun.substack.com/p/i-built-auto-loops-before-they-had" publication: "Substack" date: 2026-06-22 sections: ["why-it-matters", "automated-testing"] - title: "Behind the Curtain: The Three-Phase Process I Use to Build Every AI-Coded Product" url: "https://talirezun.substack.com/p/behind-the-curtain-the-three-phase" publication: "Substack" date: 2026-03-16 sections: ["the-audit"] - title: "Context is the Code: The Complete Three-Phase Process for Building with AI Agents" url: "https://talirezun.substack.com/p/context-is-the-code-the-complete" publication: "Substack" date: 2026-06-03 sections: ["the-audit", "automated-testing"] - title: "The Mixed Fleet" url: "https://talirezun.substack.com/p/the-mixed-fleet" publication: "Substack" date: 2026-08-17 sections: ["the-audit", "tests-not-comments"] - title: "I Could Tell You My AI Has Never Lied. I'm Not Going To." url: "https://talirezun.substack.com/p/i-could-tell-you-my-ai-has-never" publication: "Substack" date: 2026-08-06 sections: ["tests-not-comments", "what-audits-find", "what-you-cannot-check"] - title: "Second Brain to Shared Brain: Building a Neural Network of Your Own Knowledge" url: "https://talirezun.substack.com/p/second-brain-to-shared-brain-building" publication: "Substack" date: 2026-07-03 sections: ["why-it-matters", "tests-not-comments"] - title: "Six Months After I Shipped Lumina" url: "https://talirezun.substack.com/p/six-months-after-i-shipped-lumina" publication: "Substack" date: 2026-07-10 sections: ["tests-not-comments"] - title: "Chasing Jarvis: Can Technically Sophisticated Non-Programmers Deploy SaaS Applications Using AI Coding Agents?" url: "https://talirezun.com/download/chasing-jarvis-can-technically-sophisticated-non-programmers-deploy-saas-applications-using-ai-coding-agents/" publication: "Research paper" sections: ["what-audits-find", "what-you-cannot-check"] - title: "From 0 to Dev in One Day: What a Real Coding Harness Actually Looks Like" url: "https://talirezun.substack.com/p/from-0-to-dev-in-one-day-what-a-real" publication: "Substack" date: 2026-07-20 sections: ["automated-testing"] - title: "Blueprint of a Frontier Coding Agent" url: "https://talirezun.substack.com/p/blueprint-of-a-frontier-coding-agent" publication: "Substack" sections: ["why-it-matters"] related: ["orchestration", "building-without-being-a-developer"] tags: ["verification", "testing", "audit", "quality"] --- ## Why is verification the hard part? {#why-it-matters} Because agent failures do not announce themselves. Software you wrote yourself tends to fail loudly: it throws, it crashes, it returns an error. Work produced by an agent tends to fail quietly. It finishes, it reports success, and something underneath is wrong in a way nothing surfaces. The question you have to answer is therefore not whether the task completed. It is whether what it produced is actually true. Three of my own, from three different systems, all with the same shape. A tool call failed and the agent carried on as though nothing had happened. Nothing in the transcript said otherwise. Two of five deployments finished with no success message at all. I found out because I checked whether the services were live rather than trusting the deploy log. And the one that still bothers me: removing a contributor from a shared knowledge base triggered a rebuild that read stale state, produced a completely empty wiki, and reported success. A green checkmark, no error anywhere, and a semester of collective work gone. The offline test suite was entirely green at the time. The arithmetic makes this worse than it looks. A ten-step process that succeeds ninety-nine percent of the time at each individual step finishes correctly only about ninety percent of the time end to end. Agent work is many steps, most of them unwatched, and the failures compound silently until something visible finally breaks. The counterweight is the most encouraging finding in the field: give a model a way to check its own work and the quality of that work improves substantially. That is reported by the people who build these harnesses rather than measured by me, but everything I have seen is consistent with it, and it means the verification layer is not overhead. It is a capability multiplier. ## Who checks the agent's work? {#the-audit} Not the agent that did it. That is the entire principle and everything else in this chapter is an implementation detail. An agent reviewing its own output reads what it meant to write. A model from a different vendor reads what is actually on the page. The analogy that makes it click is the external auditor. An outside auditor finds things the internal team never will, and it is not because they are smarter. It is because they are not invested in the story of how the thing got built. They have no memory of the constraint that made a compromise reasonable in March, so they see the compromise rather than the reason for it. In practice I write a defined segment with one model and hand that segment to a model from a different vendor with instructions to audit it. Different training, different emphases, different blind spots. The findings are never identical, which is the whole point of running more than one. For a security pass before anything goes live, I run three models from three different vendors over the same codebase, each asked to look specifically at authentication, data validation, exposed credentials, injection paths, unprotected endpoints and performance bottlenecks. I compile what comes back, sort by severity, and have my coding agent fix everything material before deployment. It has surfaced real issues every single time I have run it. Not once has it been wasted effort. I should be honest about one thing, because a reader comparing my published work will notice it. The way I have described this audit stack has changed. In some pieces it is three frontier models running in parallel. In others it is one model writing and a different one auditing. In others again it is review and verifier agents living inside an orchestration workflow. Those are three implementations of the same rule rather than one stable practice I have run for two years, and I would rather say that than pretend at a consistency I did not have. The rule that has been stable is short. The reviewer must not be the author. ## What do you write down, and where? {#tests-not-comments} In a test, not a comment. A comment cannot fail. A test can. Any property you assert in a document, assert again as an automated check, so that the day it stops being true something breaks loudly instead of quietly remaining written. This matters more the less code you read yourself. When written intent is most of what you have, the gap between what a document claims and what the system does is invisible to you by definition, and it will not close on its own. What that looks like in numbers, on my own systems. One product went from a couple of hundred automated tests gating production to a suite carrying several thousand assertions, with almost a thousand new tests added in under a week during one hardening push. Those are different units and I am not going to pretend they are the same metric, but the direction is the point: the ratio of asserted-in-prose to asserted-in-code moved a long way. On the open-source side, one release ran on five hundred and forty-seven battle-test assertions. Five hundred and nineteen of those ran offline. The remainder ran against live storage, live network conditions and live credentials. The offline suite was completely green when the live suite found two serious bugs, one of which could have silently destroyed a shared knowledge base. Offline coverage is a floor. It is not a finish line, and I would not have believed how far apart those two things are if I had not watched it happen. Three habits underneath the tests: **Require a streak, not a pass.** One green run is not evidence. A loop library I looked at requires ten consecutive clean runs before it will call something working, and that scepticism is correct. Systems that fail intermittently pass intermittently. **Deploy in stages and verify live between them.** On one release I shipped fourteen backend functions as five separate production deployments, checking the live service between each. Slower, and a failure has one obvious cause instead of fourteen candidates. **Gate deployment on the pipeline, not on judgement.** Everything pushed runs the full suite. When everything is green, the deployment agent can go to production. When it is not, nothing goes anywhere, including when I am impatient. ## Does automated testing pay for itself? {#automated-testing} It depends on the scale, and the honest answer splits three ways rather than one. On a first small build it usually does not. On a production system with real users it always does. In between, use automation for the repetitive coverage and keep your own hands on the parts where a human eye is still faster. I have experimented properly with letting agents test their own work through browser automation, and I want to be precise about why I do not lean on it more, because it gets misread as a safety position and it is not. It works. The mechanics are fine. The problem is cost and friction. Browser-driven testing burns tokens at a surprisingly high rate, because every interaction and every screenshot analysis accumulates in the context. It is slow. Sign-in flows break it reliably, authentication state gets lost between sessions, and bot detection starts throwing verification prompts at your agent. And there remain whole classes of interface bug that a person catches in a second and an agent misses entirely. So on a small project, your own eyes are still the fastest debugging tool you own. Go through every button, every screen, every error state as a real user would. When something breaks, do not report that it is broken. Report what you clicked, what you expected, what you got, and attach a screenshot. Specific reports get specific fixes. Two traps worth knowing before you try it. **Agents default to synthetic responses.** If you do not explicitly authorise real credentials and real API calls, many will quietly simulate the responses instead, and simulated responses do not reveal real failures. Say out loud that the key is in the environment file and that you want live calls. **Solve authentication in the architecture phase.** The moment a loop hits a login wall is far too late to start thinking about how an automated agent will get past it. None of this is a permanent verdict. It is a judgement about the current price of tokens and the current state of browser tooling, and both of those move. When the economics change I will change my answer. ## What does an audit actually turn up? {#what-audits-find} Not bad code. That is the finding that reorganised how I think about this. When I ran a six-agent audit across my own product, covering security, data handling and stated commitments, every single finding had the same shape: something asserted in a document had been mistaken for something enforced in the code. The pattern, over and over: A restriction that was written down as policy, implemented as a runtime check against a configuration list, and shipped with that list empty. Nobody, including me, had a way to fill it in. So it checked an empty list, passed, and had been passing that way for its entire life. A retention rule stated clearly in writing with no mechanism anywhere that would ever act on it. The first record due to be affected was still months away, so nothing had surfaced. A contract document still describing a piece of tooling as nonexistent, three days after that tooling shipped. And my favourite, in the sense that it is the one I tell against myself. I corrected a statement in the code and did not carry the correction into the documentation. That documentation is what my product's own assistant reads. So for a while, my product was answering prospective customers with the version I had already fixed. Across that audit I logged twenty-one gaps between what was claimed and what was evidenced, and almost none of them were in the code. The code mostly did what it was supposed to do. The gap was between the writing and the code, in both directions, and that is precisely the gap you cannot see if the writing is your main instrument. An earlier and simpler version of the same lesson: on a thirty-day build, the pre-launch security pass surfaced eighteen issues that had to be fixed. Eighteen vulnerabilities in code I had already looked at and decided was finished. For context on how normal that is rather than how careless I am, the published research on generated code is not comforting. One large study across more than a hundred models and eighty tasks found only around fifty-five percent of the generated code was secure. If you ship agent-written code without an audit, you are shipping something in roughly that condition. ## What can you still not verify? {#what-you-cannot-check} Whether the thing is telling the truth. That is the honest end of this chapter. I run a product that answers questions from a customer's own documents, and I am not going to tell you it has never hallucinated, because I cannot know that. What I have is an absence of complaints. No client has reported one. That is a floor, not a number, and the two sentences "no client reported a hallucination" and "the system does not hallucinate" are not the same sentence. The reason it is unknowable in production is the structure of the error. The failures that get reported are the ones that look wrong. The failures that matter are the ones that look right: fluent, plausible, confidently sourced, and false. Human detection of that category is far from reliable, and the automated detectors that score well on benchmarks lose a substantial part of that accuracy under real conditions. So the number I would need to make the claim does not exist, and building a system that produces it is a research project rather than a feature. The second thing I cannot verify is code correctness by reading, because I do not read code. My own research says exactly that about people in my position, and the mitigation it recommends is engaging actual security professionals for a pre-deployment audit, while conceding that even that does not eliminate the underlying risk. There is a circularity in the middle of my own method that I have not solved either. Using AI agents to audit AI-generated code assumes you know enough to direct the audit. If you do not, you have added a step without adding assurance, and it will feel like rigour either way. Which brings the chapter back to where the orchestration one left it. The hard part was never the loop. It is the verifier. Without an independent check against criteria that exist outside the thing being checked, an agent will declare success and move on, and you will have built a machine that produces wrong answers faster than you could have produced them yourself. Everything in this chapter is an attempt to put something outside the loop. A different vendor's model. A failing test. A live service that either responds or does not. A person who did not build it. None of them are complete. Together they are the difference between residual risk you can name and risk you simply cannot see. --- --- title: "What I Shipped and What It Cost" number: 9 slug: "what-i-shipped" status: "published" question: "What have you actually built, and what did it take?" summary: > The products I built and run, described in terms of what they do and what building them cost me. No architecture diagrams and no capability claims I cannot point at a published source for. Just the account. updated: 2026-08-20 sources: - title: "The Curator" url: "https://github.com/talirezun/the-curator" publication: "GitHub" sections: ["the-curator"] - title: "Second Brain to Shared Brain: Building a Neural Network of Your Own Knowledge" url: "https://talirezun.substack.com/p/second-brain-to-shared-brain-building" publication: "Substack" date: 2026-07-03 sections: ["the-curator"] - title: "Lumina" url: "https://luminawidget.xyz" publication: "Product site" sections: ["lumina"] - title: "Lumina: An AI Agent Your Business Can Stand Behind" url: "https://talirezun.substack.com/p/lumina-an-ai-agent-your-business" publication: "Substack" date: 2026-07-31 sections: ["lumina", "what-it-cost"] - title: "Six Months After I Shipped Lumina" url: "https://talirezun.substack.com/p/six-months-after-i-shipped-lumina" publication: "Substack" date: 2026-07-10 sections: ["lumina"] - title: "I Could Tell You My AI Has Never Lied. I'm Not Going To." url: "https://talirezun.substack.com/p/i-could-tell-you-my-ai-has-never" publication: "Substack" date: 2026-08-06 sections: ["lumina", "what-i-would-do-differently"] - title: "Why I Ditched RAG Pipelines for 1M Token Context Windows" url: "https://medium.com/@talirezun/why-i-ditched-rag-pipelines-for-1m-token-context-windows-d5a2982f7cce" publication: "Medium" sections: ["moj-ai"] - title: "How I Built an AI-Powered Vacation MCP Search Tool in 14 Days" url: "https://medium.com/@talirezun/how-i-built-an-ai-powered-vacation-mcp-search-tool-in-14-days-and-why-you-should-care-128e09bf6bd0" publication: "Medium" sections: ["the-smaller-ones"] - title: "Conduit" url: "https://github.com/talirezun/conduit-agent" publication: "GitHub" sections: ["the-smaller-ones"] - title: "A Year in the Review" url: "https://talirezun.substack.com/p/a-year-in-the-review" publication: "Substack" date: 2026-06-23 sections: ["the-smaller-ones", "what-it-cost"] - title: "From Prototype to Production: Building an AI Widget Platform in 30 Days" url: "https://medium.com/@talirezun/from-prototype-to-production-building-an-ai-widget-platform-in-30-days-23c603c91475" publication: "Medium" sections: ["what-it-cost"] related: ["building-without-being-a-developer", "agent-memory"] tags: ["projects", "the-curator", "lumina", "build-log"] --- ## What is The Curator and why did I build it? {#the-curator} A local, open-source second brain. You drop a PDF, an article or a note into it, and it reads the source and writes an interlinked wiki out of it: entity pages, concept pages, and a summary page for the source. Everything stays as plain markdown on your own machine, and an MCP server exposes the whole graph to a frontier model as a set of seventeen tools, so the model can traverse it and write findings back during a live session. I did not build it as a product. I built it because I had a memory problem in my own work and nothing available solved it the way I wanted it solved. Practical facts, and the ones people most often get wrong about it: **It is MIT licensed.** Free to use, fork and modify. **It needs an API key.** Either Google Gemini or Anthropic. There is no version of this that does the ingestion on nothing, and I would rather say that plainly than let a "runs entirely locally" impression stand. Gemini has a free tier, though it was tightened substantially at the end of 2025 and one batch of PDFs will usually exhaust a day of it. On a paid key, moderate solo use runs around five euros a month. If you want nothing to leave the machine at all, it works against a local model, with the quality trade you would expect. **Installation is a one-line script on macOS, or clone and `npm install` elsewhere, on Node 18 or newer.** It runs on `localhost:3333`. Obsidian opens the same folder natively, so the graph view comes free. It is live and actively developed. The collaborative Shared Brain layer is still beta and I would not describe it otherwise. What went wrong is in chapter two in detail, so briefly here: early versions produced duplicate ghost pages once a wiki got large, the same entity spawning three near-identical files. A shared-brain release nearly shipped a bug that could have silently emptied a collective wiki while reporting success. And my own test suite was, for a while, writing test data into real user knowledge folders because a directory resolver preferred the production configuration over the test override. That last one is the most embarrassing thing in this chapter and it belongs here rather than buried. ## What is Lumina? {#lumina} A business uploads its own documents and gets a chat agent that answers from them. When the documents do not cover something, it says so rather than inventing an answer. It deploys as a website widget and across several messaging channels from one engine, with a shared inbox where staff can see every conversation, take over a thread from the AI mid-conversation, and search across channels. The design choice underneath it, which I have written about publicly and which shapes everything else, is that the whole document set goes into the model's context rather than being chunked, embedded and retrieved. That decision came out of the year I spent failing to make retrieval work well enough on a different product, which is the next section. It went live at the start of 2026, built over thirty days in December 2025. A second generation rebuilt substantially more of it in about a month, across roughly eight hundred and sixty commits. It is in production and serving businesses. The story I would keep if I could only keep one is not a technical one. A hair studio in Ljubljana used it daily. Their assistant began telling real visitors, mid-conversation with real customers, that it was having trouble connecting. The cause was a two thousand character limit on personality instructions that I had set months earlier as a safety measure and then completely forgotten about. The owner had written longer instructions; the system silently rejected them. I traced it and fixed it the same day, raised the limit fourfold for everyone rather than just for them, and added a live character counter so nobody walks into an invisible wall again. What that taught me was not about character limits. There is a moment when a project becomes a business, and it is not funding or launch or revenue. It is the first time a real person's real customers feel it when something breaks. I have started calling that the responsibility threshold and it arrived earlier than I expected it to. A later version of the same shape, at a larger scale: a customer uploaded a document that met every stated requirement and was, on its own, far larger than the entire available context budget. Their assistant stopped answering anyone. The requirements were not wrong. They were incomplete, and an incomplete written rule is invisible until somebody walks into it. The behaviour I chose afterwards is that the system refuses rather than degrades, because an assistant that stops is recoverable and an assistant that quietly gets worse is not. And the claim I decline to make. I will not tell you Lumina has never hallucinated. No customer has reported one, and "no customer reported a hallucination" and "the system does not hallucinate" are different sentences. What I have is an absence of complaints, which is a floor rather than a number. ## What is Moj AI, and what did three rewrites teach me? {#moj-ai} A Slovenian-language assistant for building regulation. You ask a question about what you can build on a particular plot in a particular municipality, and it answers from the actual construction acts and the municipal spatial plans, with article numbers and page citations. It is live and openly labelled as early beta. It took a year and three complete architectural rewrites, and it is the project that taught me the most per hour spent. The technical arc is in chapter one, so I will not repeat it: a conventional retrieval pipeline that plateaued at eighty percent accuracy, could not get past eighty-five on complex tables, and destroyed exactly the data that mattered. What replaced it reads whole documents rather than fragments. What I want to record here is the part that was not technical. The hardest thing in that year was not writing anything. It was working out when to throw the current version away and start again. Three times, as it turned out, and I resisted each one for longer than I should have because there was working software in front of me. Some numbers from the version that did not survive, because the cost of being wrong is worth writing down. Around three hundred dollars a month in infrastructure for development and testing alone, before a single customer existed. Eight to ten minutes for every deployment cycle, which sounds trivial until you multiply it by how many times you iterate. And a deployment numbered in the hundreds still failing. One thing that did survive from that period is a method rather than a system. Over that project I did more than a hundred successful handovers between agent sessions using written handoff documents, and that discipline is the reason the work accumulated at all rather than restarting every time a context window filled. ## What else is out there? {#the-smaller-ones} Four smaller things, three of them open source. None of them are businesses. They exist because a specific problem annoyed me enough to fix, and the reason they are worth listing is that the failure modes are more legible on small projects than on large ones. **MountVacation MCP** is an open-source MCP server, MIT licensed, that lets any MCP-compatible assistant search a European mountain accommodation database and hand back real results with booking links. Ten tools. It needs the provider's API key. I built it in fourteen days without writing a line of the TypeScript myself. The day one version worked and was wrong. It returned the first batch of results and stopped, because the underlying API pages results in groups of thirty while a search in a busy region has hundreds. Fixing that took three separate strategies, retry logic with backoff, and partial-result handling so that a failure on the fourth page returns the ninety results already collected rather than an error. Then an automated testing loop found the rest: searches in one country returning nothing at all, children's ages parsed incorrectly, currency conversion failures, and location lookups failing for particular regions. It remains a proof-of-concept rather than a production-grade commercial service, and I say so in the repository. **ARIA**, an avatar platform, is a client project and the largest single thing on this list: around seventeen thousand lines, close to four hundred commits, roughly three hundred distinct agent sessions, about six weeks to a production MVP. I have never published its feature set and I am not going to here, because it is not mine to describe. **Conduit** is an Apache-licensed framework for setting up a governed personal agent that lives in a folder on your own machine, with the setup done by pasting prompts rather than typing terminal commands. It is explicitly aimed at people who are not developers, and it uses The Curator as its memory layer. It is the earliest-stage thing I have published. Its documentation currently lists no known limitations, which at this age means they have not been found yet rather than that they do not exist. **ØØT** is the organisational framework, and it has its own chapter. ## What did building these actually cost? {#what-it-cost} Far less money than anyone expects and far more time than anyone admits. The first generation of Lumina cost under a hundred euros in model credits across thirty days. It also cost around two hundred hours of my own time inside those thirty days, evenings and weekends included, and roughly half of that was context engineering rather than building anything. I need to flag something about that first figure before it travels any further. I have published it as both under a hundred and under a thousand, in different pieces, for the same build. The lower number is the one I believe and the one I have repeated most often since, but I would rather point at the discrepancy than quietly pick the more impressive version and hope nobody checks. The rest of the ledger, all from my own records: Across twelve months I logged 1,278 contributions, ninety-eight percent of them commits, spread across four products that shipped. The avatar platform burned close to three hundred dollars a day in credits at peak. That is by some distance the most expensive thing on this page, and it was expensive because a client deadline made parallelism worth paying for. Moj AI cost about three hundred dollars a month in infrastructure during development, before any customer existed. One intensive day on an early command-line agent, back when it was billed by usage, cost me over three hundred euros. That is the day I stopped using that tool, and the reason was economics rather than capability. Those three hundreds are three unrelated things at three different scales, which is a good illustration of how easy it is to build a misleading story out of true numbers. Two structural points about cost that took me too long to learn. **Subscriptions beat metered billing for development work.** Not because the monthly total is lower, but because on metered billing you start optimising for cost instead of quality. You skip the audit run. You do not re-verify. You stop asking for the second opinion. The bill goes down and the work gets worse. **Every figure above excludes most of the actual cost.** Hosting, payment processing, domains, and the ongoing operational work of keeping a live product alive are not in any of these numbers. Nor is the two-year learning curve that made a thirty-day build possible, which is the largest cost in this chapter and appears in nobody's accounting, including, usually, mine. ## What would I do differently? {#what-i-would-do-differently} Write the test at the same time as the document, rather than discovering months later that the document was the only thing enforcing anything. Almost every serious problem in this chapter traces back to something that was written down and never checked, and the fix is mechanical and boring and I did not do it early enough. The rest of the list, in rough order of how much it would have saved me: **Never let the documentation lag the code.** My own product reads my own documentation. When I corrected something in code and did not carry it across, my assistant kept answering prospective customers from the version I had already fixed. That is a very specific consequence of a very general sloppiness. **Test against live services earlier.** A completely green offline suite told me nothing about the two most serious bugs I have shipped, both of which needed real storage, real network conditions and simulated concurrency to surface. **Solve authentication in the architecture phase.** Every automated testing effort I have run has eventually broken on a login wall, and every time it was foreseeable. **Do not build the second thing until the first one has real users.** Real users find failures you cannot imagine. The forgotten character limit was not going to be caught by any amount of me testing my own product. Then two changes of mind, which are harder to write. I assumed for years that better models would democratise this. Having now run a product with real customers for the better part of a year, I think the opposite is happening: the gap between people who work this way and people who do not is widening as the models improve, not closing. The tools reward people who already know how to specify, structure and verify, and those people pull further ahead. And I contradicted myself in public. I said for a year that the bottleneck was never the model. Then, writing up a rebuild, I said that attempting it a few months earlier would have been close to impossible, not hard, impossible. Those two statements cannot both be true in their strong form. What reconciles them is that they are about different work: for a well-specified build, the bottleneck really is context and always was. For a hard orchestration problem at the edge of what the tools could do, the model was the constraint, and it lifted. I would rather leave both statements standing with that note attached than tidy one of them away. The tidy version would be more quotable and less true.