Cole Medin

Language models keep getting better at building software without getting much better at understanding what people want, they argue, so further model progress will not close that gap.

• Their factory keeps the same core loop but adds human judgment at the planning and acceptance points: up-front planning of a release with an agent, usually 10 to 20 tickets, including what is out of scope; parallel laps that each build one issue, test the app as a user would, run an extensive code review, and merge into staging only when CI is green; a manager agent that checks every couple of hours and pings Medin when something is genuinely stuck; and Medin personally testing staging, with anything off becoming a new issue for the factory.
• They still would not trust a software factory to build production software.

Describing their own factory, Medin says the loop takes a GitHub issue in and returns a pull request. The big change, they say, was everything around the loop: instead of reviewing pull requests one at a time, they test a deployed batch in staging and give feedback on the batch as a whole.

The ambition softens across the posts.

• The “dark factory” post argues that a system with no humans anywhere in the loop would not work even with AGI.
• A later post says Medin handed the factory over to a model that stays on task, with no human in the loop — cautioning that it is not production-ready while pushing autonomy further.

Read original · Original 2 · Original 3

If agents can read the tests, they will engineer the code to pass them rather than make the software actually work, they argue.

• Planning therefore produces holdout testing scenarios kept from the builder, which they credit with a large boost in reliability.
• The broader point: evaluation must account for an agent’s incentives, rather than serve as a neutral checkpoint.

In release planning, Medin lists which tickets are in scope and which are out, along with holdout scenarios the builder never sees. They say that last choice is why reliability improved so much.

Read original

Their reasoning splits the work by difficulty.

• Planning, reviewing, and deciding what not to build need the strongest models; implementation does not, because a model following a well-written plan need not be as capable.
• Implementation also uses the most tokens, making it the biggest source of savings.
• Medin tested the approach by running the factory against the same apps with many model and provider combinations, rather than trusting benchmark rankings.

After hitting rate limits on the $200 Max plans for Claude Code and Codex with three or four days left before reset, Medin settled on using a large model for planning and reviewing while GLM 5.3 Flash or DeepSeek v4.1 Flash wrote every line of code. They say it was about four times cheaper than using the larger models, with basically the same results.

Medin says they have mixed models for a year; by the rate-limit post, they call it a necessity for everyone.

Read original · Original 2

That failure mode, they say, has always capped their work.

• It is a practitioner’s criterion, not a leaderboard score: a model that impresses in short bursts but loses the thread is not usable for a long autonomous run.
• Medin pairs the praise with a cost caveat, splitting the work so the strongest model handles only the highest-leverage decisions.

After running GPT-6 Astra hard for a week, Medin called it by far the best coding model they had used, citing task adherence as the biggest change.

• They then handed the factory to it: a requirements document or issue goes in, and shipped code comes out.
• It runs on a ten-dollar server, so it can work while they are asleep or on a plane.

Read original

They describe being more productive than ever and more tired than ever, as the day fragments into sending a request to one agent, switching to the next while it runs, and returning when it needs attention.

• Even during a wait, they feel guilty that they could start another conversation.
• Their diagnosis is an awkward in-between: agents are not good enough to run unattended for a full day on most work, so people babysit them, but they are good enough that people start more sessions while waiting, worsening the split attention.
• Medin’s response is to invest time in autonomy rather than add more sessions.

Medin contrasts coding with music on for four hours straight a couple of years earlier with constantly cycling between agent sessions all day now.

• They would not go back, they say, because the productivity is too good.
• They predict a future with one primary planning session and factories doing the heavy lifting in the background.

Read original · Original 2

But they judge that more people now want something that works out of the box, and accept that.

• They are building an open-source AI software factory designed to work as soon as you download it, and making it a top priority alongside Archon.
• Medin says building it is also teaching them how to make coding agents more reliable in general, so the two efforts feed each other.

Medin announced the open-source factory as an early alpha and described running a factory all year as an experiment that built and still maintains an AI tutor app — a codebase they say they have never read a single line of. Each post points to a fuller video walkthrough or the repository rather than including the instructions.

Read original

It answers a newcomer's first question — is this person a commentator or a builder? — and names the trade-off that shapes everything else: learn the tool or get one that just works.

Cole Medin describes a software factory as a pipeline: a product requirements document goes in, gets split into tasks, and comes out as shipped code, with nobody reading the code in between.

• They say companies already use this approach for parts of development — proofs of concept and product spikes — and that they have helped a few build it themselves.
• They ran one for a year as an experiment: it built and still maintains an AI tutor app, and they have never read a single line of that codebase.
• The post announces an open-source AI software factory, with the goal of making it work out of the box when downloaded.
• It is described as early alpha and is now a top priority alongside a project called Archon.
• The post names the tension directly: teaching people to build the thing has been the whole approach, but more people just want something that works out of the box.
• The aim is to offer both.

For brands planning a campaign

A developer-tools company could use this post as the spine of a build-along tutorial series or a webinar on what an out-of-the-box factory changes for teams that do not want to assemble their own pipeline. The explicit “learn the thing versus get the thing” framing offers a ready-made audience-segmentation device for content planning.

It answers the practical question a technical reader asks after the vision: what does the thing actually do, step by step, and why is there still a human in it? It is the most inspectable of the five for anyone evaluating whether the workflow is real.

Cole Medin opens by rejecting the fully autonomous “dark factory” pitch: even with AGI, it would not work, because language models keep getting better at building software without getting much better at understanding what people actually want.

• The core loop is a GitHub issue in and a pull request out.
• Each lap builds one issue, tests the app as a user would, runs an extensive code review, and merges into staging only if continuous integration is green.
• Many laps run in parallel; a manager agent checks on them every couple of hours and pings Cole when something is genuinely stuck.
• The bigger change is around the loop: they plan one release up front with an agent, usually 10 to 20 tickets, including what is out of scope and holdout-testing scenarios the builder never gets to see.
• They say that last choice boosted reliability because agents that can see the tests will engineer code just to pass them.
• Cole tests staging personally, and anything off becomes a new issue the factory picks up on its own, so feedback comes on a batch rather than per pull request.

For brands planning a campaign

Strong material for a technical webinar: holdout tests, manager agents, and deliberate human gates presented as design decisions rather than fallbacks. A vendor selling evaluation or testing tools could build an entire session around the holdout-testing idea alone.

It answers “how does a practitioner judge a new coding model?” and shows which failure mode actually limits autonomous pipelines. Useful for readers deciding which model claims to trust.

Cole Medin says GPT-6 Astra is not AGI but is by far the best coding model they have used.

• Its main advantage, they say, is staying on task for long stretches without drifting or making a mess halfway through — the limitation they say has always constrained their AI coding workflows.
• They handed the software factory over to it: a product requirements document or issue goes in, and fully validated, shipped code comes out with zero human input.
• It runs on a ten-dollar server, shipping while they are asleep or on a plane.
• The post is explicit that Astra does not do everything.
• Running it on every step gets expensive fast, and most steps do not need it, so the factory splits the work: cheap models handle the mechanical parts, while Astra handles planning and deciding what should not get built in the first place.
• They add that they would not yet trust a software factory to build production software.

For brands planning a campaign

A model-comparison piece built on a working pipeline rather than a leaderboard. For a vendor whose differentiator is reliability or long-horizon task adherence, this post offers a specific claim to test and the caveat that keeps it credible.

A budget-conscious engineering lead will want to share it with a colleague.

Cole Medin writes that they hit rate limits on both Claude Code and Codex on the $200 Max plans, with three or four days still to go before reset, despite doing little more work than usual.

• They say the limits have been getting worse all year.
• Their conclusion: the most powerful model cannot be used for everything.
• They have mixed models and providers all year and now consider it mandatory, using the factory to test combinations on the same apps and find where the best model is actually needed.
• They say GPT-6 Astra and Opus 5.5 are currently the best models, with different strengths: GPT gives the best visuals and does the best reviewing, while Opus 5.5 does the best planning and is the most efficient.
• Neither is needed for implementation, the most token-heavy part of the workflow, because a model following a well-written plan does not need to be that capable.
• Their favourite combination at the time was a large model planning and reviewing, with GLM 5.3 Flash or DeepSeek v4.1 Flash writing every line of code.
• It came out around four times cheaper with basically the same results.

For brands planning a campaign

A tutorial on tiered model routing with a stated cost multiple is directly useful to teams hitting the same subscription ceilings. A vendor selling orchestration, gateways, or multi-model tools could turn this into a practical cost-engineering session rather than a product pitch.

Read it alongside the build posts to understand the person behind the tooling, not just the architecture.

Cole Medin says AI has made them more productive than ever and more tired than ever at the same time, attributing the fatigue to context switching and split attention all day.

• A couple of years earlier, they would put music on and code for four hours straight.
• Now they send a request to one agent, switch to the next while it works, and jump back when it needs them, all day — feeling guilty when waiting because they could start another conversation.
• They would not go back because the productivity is too good.
• Their diagnosis is an awkward in-between: agents are not good enough to run on their own for a full day for most tasks, so people babysit them, but they are good enough to encourage more sessions while waiting.
• Rather than add sessions, they invest that time in building autonomy.
• That is the stated reason for the AI software factory: a system where the agent knows how to work the way they do, well enough to take on a full day of work by itself.
• They predict a future of one primary session for high-leverage planning, with one or more factories doing the heavy lifting in the background.

For brands planning a campaign

A human-interest hook for a webinar or content series on developer workflow and fatigue: the adoption argument for autonomous tooling, told from the practitioner's side. Pairs well with the architecture posts as the “why” before the “how”.

Several laps run in parallel while a manager agent checks on them every couple of hours and pings Medin when something is genuinely stuck.

• One version runs on a ten-dollar server, shipping while they are asleep or on a plane; the factory is free and open source.
• Medin also describes running one all year that built and still maintains an AI tutor app, without their reading a single line of that codebase.

An agent handles planning up front, one release at a time, usually across 10 to 20 tickets.

• The plan includes out-of-scope items and holdout test scenarios the builder never sees.
• Medin credits the hidden tests with a large reliability gain, reasoning that agents who can read the tests will engineer code just to pass them.
• Feedback comes on a whole deployed batch in staging, rather than pull request by pull request; anything off becomes a new issue for the factory to pick up.

Medin says the factory is not yet trustworthy for production software. A true dark factory, with no human anywhere in the loop, did not work: models keep improving at building software without improving much at understanding what is actually wanted.

For brands planning a campaign

A developer-tools brand can anchor a webinar or tutorial series on the factory’s architecture, loop, holdout tests and manager agent, using the downloadable project as a concrete entry point for an audience weighing autonomous coding workflows.

Examples: Source 1 · Source 2 · Source 3

They say GPT-6 Astra produces the best visuals and does the best reviewing, while Opus 5.5 handles planning best and most efficiently.

• Neither is needed for implementation, the most token-heavy step.
• The current favorite combination uses a large model for planning and reviewing, while GLM 5.3 Flash or DeepSeek v4.1 Flash writes every line of code, at roughly four times less cost than using the larger models for broadly the same result.

Medin tested routing choices by running the factory against the same apps with many different model and provider combinations, identifying where the strongest model is genuinely required and where it is not.

Rate limits on the $200 Max plans for Claude Code and Codex ran out three to four days before reset. Medin says the problem worsened through the year, prompting the switch to mixed providers.

For brands planning a campaign

Cost and model-selection material suits a developer audience hitting usage ceilings. A brand could build a comparison or benchmark-style session around matching the right model to each step of an agentic process.

Examples: Source 1 · Source 2

Agent work fragments the day into sending a request, switching to the next session while it runs, and jumping back when it needs attention.

• Each wait brings guilt that another conversation could be started.
• Rather than adding more parallel sessions, they invest that time in building autonomy: a system that knows how to work in the same way and can take on a full day of work alone.

The predicted end state is one primary session for high-leverage planning, with a factory or set of factories doing the heavy lifting in the background as agents begin running for days like trained employees.

Medin says the current state is an awkward in-between. Agents are not yet good enough to run unattended for a full day on most work, so people babysit them. But they are good enough that people start more sessions while waiting, making the problem worse.

For brands planning a campaign

A brand can lead with this human-work angle in a technical webinar on developer experience, selling orchestration and relief from context switching rather than raw model capability.

Examples: Source 1 · Source 2

The long-standing emphasis on teaching people how to build things themselves continues, but more people want something that works out of the box.

• So they are building an open-source AI software factory intended to be downloaded and run, making it a top priority alongside Archon.
• The stated position is to offer both: sometimes you want to learn the thing; sometimes you just want the thing.

For brands planning a campaign

A brand seeking a launch-partner or co-marketing format can build on the open-source distribution angle and the combination of a scaffold with a tutorial showing how to use it.

Examples: Source 1

The posts carry the conclusions; the prompts and step-by-step builds live in the linked videos and code. Medin uses the factory as a reusable test bed rather than a single demo, so the same system appears across the model-comparison and autonomy work.

For brands planning a campaign

Pairing a short written argument with a full video walkthrough and reusable prompts suits a sponsored tutorial series, where a brand’s tooling can appear inside a working example rather than beside it.

Examples: Source 1 · Source 2 · Source 3 · Source 4

A GitHub issue or PRD goes in; each run builds one issue, tests the app like a user, conducts a code review, and merges only when CI is green.

• They say it is free and open source and that it built and maintains an AI tutor app without anyone hand-reading a line of that codebase.
• Planning, holdout test scenarios the builder never sees, and personal staging tests keep a human involved in key decisions.
• For a commercial engagement, ask which entity would contract.

View example

The post does not explain whether they own Archon, work on it, or have a commercial stake; ask about their relationship to the project.

View example

They name it as part of the stack they are trying to make more autonomous.

View example

The posts show paid tool use and public evaluation.

View example

They say they handed the software factory over to it for the highest-leverage steps: planning the work and deciding what should not be built. Cheaper models handle the mechanical steps.

View example

YouTube is where Cole Medin publishes longer demos and explanations.

View example

Spencer writes about wiring agents into Slack, MCP tools and Agentforce, and about an agency trust problem where delivery matters more than the funnel.

• Medin writes about a factory that ships tested code and about preventing a model from following tests to a passing grade.
• Both are skeptical of AI work that stops at demos.

Spencer focuses on business and platform strategy for an AI agency, including Salesforce's move to a headless, agent-accessible platform. Medin focuses on a working coding factory released as open source and model routing for developers.

Read original · Original 2

Cinovoj writes about shipping production agents and identifies process architecture, harness, sandbox and observability as the difference.

• Medin describes holdout tests, hidden acceptance scenarios and a factory loop as reliability levers.
• Both take direct aim at demo-only AI.

Cinovoj focuses on enterprise failure postmortems and production operations, while Medin offers a downloadable software factory and instructions for coding-agent workflows.

Read original · Original 2

Waxman runs GTM Engineer School on a markdown repo, interviews practitioners about context layers, and publishes a pulse newsletter on agents in CRM and go-to-market work.

• Medin pairs posts with video walkthroughs, prompt sets and an open-source repository.
• Both stress context and harness choices over model choice.

Waxman focuses on go-to-market and sales engineering, taught through cohorts and community. Medin focuses on software development agents and a coding factory.

Read original · Original 2

Spencer's posts make that business case: one attacks the AI agency pattern of selling funnels and courses without the ability to scope or deliver, grounding the argument in running an agency while still managing operations at an appliance repair shop.

• The other reads a platform shift, arguing that Salesforce's headless APIs and Slack's agent plumbing matter more than the CRM interface, and describes building agents on Slack with MCP connections and custom webhooks.
• For a webinar on why AI rollouts stall over delivery rather than model choice, Spencer brings the market argument and an operator's view of where agent work is migrating.

Spencer does not show a coding workflow an audience can follow.

• There is no planning-and-implementation loop, model routing, open-source project to download, or build-along material, so a tutorial series anchored here would need to draw on another source.
• The content is argument and market commentary: the Salesforce-Slack post is explicitly framed as an operator's view, not advice, and leans on platform announcements rather than a reproducible build.
• A sponsor selling orchestration tooling would get a stronger business conversation and a weaker technical demonstration than with Medin.

Examples: Source 1 · Source 2

Both posts start from claimed experience shipping production agents, then catalogue what goes wrong: six-figure retainers and six-month timelines with no deployment; demos that break under auth flows and rate limits; and a fintech agent that ran 847 API calls instead of 12 and billed $6,200 in week one.

• The second post reframes the same material around process architecture, arguing that most failures are harness problems and that the work lies in tools, rules, sandboxes, guardrails, and observability.
• That makes Cinovoj a sharp fit for a webinar arguing that running agents reliably is an engineering discipline, with a natural sponsor position around the verification and observability layer.

The posts diagnose and position; they do not teach a method viewers can reproduce.

• There is no planning loop, holdout testing, routing comparison, or open-source repository to run, so the tutorial half of a build-along series would need another source.
• The supporting numbers in the second post, including the study of SMB rollouts and the company examples, appear as stated claims rather than data the audience can inspect.
• The posts also do not show the work behind the 50-plus agents.
• Cinovoj is the better pick for a hard-hitting reliability keynote and the weaker pick for a build-along series.

Examples: Source 1 · Source 2

Waxman runs GTM Engineer School, and both posts teach that audience: one reports a conversation with two practitioners about context layers, splitting brand, canonical, and deterministic layers, and advising teams to pick one problem with a deadline instead of building a company brain.

• The other summarises a GTM newsletter covering Salesforce and HubSpot opening their CRMs to agents, autonomy earned through acceptance rates, and a capstone programme where students each build one real workflow for one real company.
• That post also carries a concrete offer, with a start date, a cohort size, and a described deliverable.
• For a sponsor whose product sits in sales operations, enrichment, or CRM work, Waxman is closer to the buyer.

The subject changes with the audience.

• Waxman's material covers context layers and GTM workflows, not autonomous coding agents or model routing, so a campaign centred on a software factory would lose its subject.
• The posts also offer second-hand teaching: the context-layer discussion reports someone else's three-layer framework, and the newsletter summarises other people's work.
• A sponsor would be relying on curation rather than firsthand build detail.
• Ask for audience composition and rates; the posts provide neither.
• Commercial information is limited to the capstone programme invitation.

Examples: Source 1 · Source 2