Build Daily

Tinley Park · November 2, 2026

Code got cheap. Shipping it shouldn't be the bottleneck.

An AI coding agent can write a feature in an afternoon, and then the feature waits for someone to build it, deploy it, check it and roll it back if it breaks. When writing code was slow, a slow release process hid behind it; now the code is ready first, and release is the bottleneck.

The bottleneck movedThree timelines for shipping one change, illustrative and not measured. Before AI: writing the code takes most of the time, then four release steps follow: build, deploy, check and undo if needed. With AI writing the code but release still done by hand: writing shrinks to a small block and the same four release steps now take most of the time, marked as the bottleneck. With AI writing the code and release automated: both blocks are small, and the release is the same every time. Before AIwrite the codebuilddeploycheckundoAI writes the code, release still by handwritebuilddeploycheckundothe release is now the bottleneckAI writes the code, release automatedwriteautomatedfast, and the same every timeFIG. 1 — Illustrative, not measured. AI shrank the writing. The release stepsstayed the same size until they were automated.
↗ click to enlarge

GodfreyLabs decided to fix the release side, and the fix is Kubernetes with everything written as code. This post explains that decision in plain terms: what the goal is, how the system works, why AI made it practical, and what it still costs.

§ 01
CHAPTER

The goal

Agility to build and release, with reliable, trustworthy automation.

Each part of the goal has a plain test, and a piece of the system that meets it:

What the goal needsWhat delivers it
A release is routine.Every service ships the same way: a change in git, applied by Argo CD. § 03
A failure recovers without a person.Kubernetes compares what should run to what is running, and restarts or moves apps to close the gap. § 02
A bad change is easy to undo.New versions replace a few copies at a time, and the last good version is one step back. § 02
Every change can be reviewed.The whole system is text in git, and a person presses Sync before it goes live. § 03

Speed and trust are not a trade-off here. Research from DORA has found again and again that "speed and stability are not tradeoffs." Releases that can be trusted get run more often, and small, frequent releases are easier to trust.

§ 02
CHAPTER

What Kubernetes does

Kubernetes is open-source software that runs applications across a group of machines and treats the group as one computer. Google released it in 2014, based on the systems it used to run its own services. A vendor-neutral community now maintains it under the Cloud Native Computing Foundation. Every major cloud sells it as a managed service, and it also runs on a company's own hardware.

A team has several options for running its apps. It can install them on servers by hand, package them as containers and run those on a server, hand short pieces of code to a serverless platform, or run containers on Kubernetes. The options can be mixed.

Ways to run an appFour options side by side. Servers: each app is installed on a named server; simple with full control, but every fix needs a person. Containers: apps packaged with all they need and run on a server with Docker; they run the same everywhere, but someone still places and restarts them. Serverless: the cloud runs code when a request comes; no servers to manage, but limits on run time, state and control. Kubernetes: containers run across a pool of machines and are restarted and moved automatically; the trade-off is that the team owns the platform. ServersEach app installed ona named server.strengthSimple. Full control.trade-offEvery fix needs aperson.ContainersApps packaged with allthey need, run on aserver with Docker.strengthRuns the sameeverywhere.trade-offSomeone still placesand restarts them.ServerlessThe cloud runs codewhen a request comes.strengthNo servers tomanage.trade-offLimits on run time,state and control.KubernetesContainers run acrossa pool of machines.strengthRestarts and movesapps on its own.trade-offThe team owns theplatform.FIG. 2 — Four ways to run an app. Each one hands a different share of the workto the platform, and keeps a different share of the control.
↗ click to enlarge

The idea that makes this work is desired state. The team does not run steps like "start this app on that server." It writes down the end result: "three copies of the web app, version 2.4, always." Kubernetes then compares that to what is actually running and closes any gap. If a copy crashes, it starts another one. If the version changes, it replaces the copies a few at a time.

The desired-state loopThree stages in a loop. Declare: the team writes down what should run, for example three copies of the web app at version 2.4. Compare: Kubernetes checks what is actually running and finds only two copies. Correct: it starts one more copy. The loop then repeats continuously. Declare The team writes down what should be running. web · 3 copies · v2.4 Compare Kubernetes checks what is actually running. 2 running · 1 missing Correct It closes the gap with no one paged. start 1 copy repeats continuously FIG. 3 — Describe the end state once. A control loop keeps reality matching it, around the clock.
↗ click to enlarge

Five words cover most conversations about it:

Container
An app packaged with everything it needs, so it runs the same on a laptop, in test and in production.
Cluster
The pool of machines Kubernetes manages as one.
Node
One machine in that pool. Nodes are interchangeable; none of them is special.
Deployment
The written-down instruction for one app: which version, how many copies, how to roll out a change.
Service
A stable address for an app, wherever its copies happen to be running.
§ 03
CHAPTER

Everything is code, in git

Desired state is text. That makes Kubernetes a natural fit for infrastructure as code: the whole system is described in files, kept in git, and changed the same way application code is changed. Tools like Terraform do the same for cloud resources such as accounts, storage buckets and DNS.

At GodfreyLabs, the services, databases and schedules on the clusters are files in one git repository, with a beta version and a production version of each. Argo CD watches that repository and shows any difference between what git says and what the cluster runs. Container images are built by a workflow on a separate management cluster.

Argo CD can apply changes by itself. GodfreyLabs keeps it on manual: Argo CD shows the change, and a person presses Sync. Automation does the work, and a person decides when it goes live. A bad commit cannot take a service down without someone choosing to apply it.

The infrastructure-as-code loopFour stages from left to right. An AI agent drafts and reviews changes. The git repository holds the desired state for beta and production. Argo CD shows the difference between git and the cluster, and a person presses Sync. The cluster then keeps reality matching git. A dashed line returns from the cluster to the AI agent, carrying logs, events and errors, so the agent can read failures and propose the next change. AI agent Drafts the change. Reads the errors. Proposes the fix. Git Desired state for beta and production. Reviewed as diffs. Argo CD Shows the drift. A person presses Sync. Cluster Keeps reality matching git, around the clock. logs · events · errors FIG. 4 — The whole system is text an AI can read and write. A person decides what goes live.
↗ click to enlarge
§ 04
CHAPTER

Why now: nobody gets stuck

The old argument against Kubernetes was never the concepts. It was getting stuck: a pod that restarts in a loop, an error message that points at the wrong layer, an afternoon gone to a missing setting. The learning curve was a long series of those afternoons, and most small teams decided it was not worth it.

That part changed. Because the whole system is text, an AI agent can work on all of it:

  • Debug. The agent reads the pod's events, the logs and the failing config together, explains what went wrong, and proposes the fix. This is where the learning curve used to hurt most, and it is where AI helps most.
  • Learn. Concepts get explained at the moment they come up, against the real files in question, not a tutorial's example.
  • Write. The agent drafts the manifests and config, and the person reviews the diff.
  • Operate. Upgrades, migrations and runbooks get drafted the same way, then checked and applied step by step.
Getting stuck, before and afterTwo chains of steps after a deploy fails. Before: read the error, search for it, guess a setting, try again, and loop back to reading the error until the afternoon is gone. With an AI agent: the agent reads the logs and config together, names the cause, proposes a diff, and a person reviews and syncs it. Before: stuckdeploy failsread the errorsearch for itguess a settingtry againrepeat until the afternoon is goneWith an AI agentdeploy failsagent reads thelogs and confignames the causeproposes a diffperson reviewsand syncsFIG. 5 — The old learning curve was a loop of guesses. An agent reads the evidence first,and a person still decides.
↗ click to enlarge

GodfreyLabs is the test case. Over a few weeks, its whole backing stack (graph databases, Postgres, Redis, sign-in and object storage) moved from containers on a single machine to Kubernetes, then to separate beta and production clusters. Postgres moved onto an operator that manages it inside the cluster, and backups now run on every cluster. An AI agent wrote the manifests, read the errors and proposed the fixes. Every change still went through git and review.

§ 05
CHAPTER

The ledger, after AI

The benefits of Kubernetes did not change. Half of the costs did. The other half are still exactly where they were.

What it buys

Routine releases
One way to deploy and one way to roll back, for every service.
Recovery without a page
Crashed apps restart and failed machines are routed around.
Smaller, safer changes
New versions replace a few copies at a time, and a bad one goes back in one step.
Beta matches production
Both are described the same way, so "it worked in beta" means something.
A system that can be read
Everything is text in git: reviewable, repeatable, and readable by people and AI alike.
A standard
The same definitions run on any major cloud or on company hardware, with a large open-source ecosystem around them.

What it costs

The learning curve · AI shrinks it
Concepts get explained in context, when they come up.
Getting stuck · AI shrinks it
More layers to search when something fails, but an agent reads them all at once.
Writing the config · AI shrinks it
Manifests and settings get drafted, and reviewed rather than typed.
Ownership · still on the team
Someone is accountable for the clusters, their upgrades and their guardrails.
Security · still on the team
Access, network rules and images all need policy. The defaults are permissive.
Upgrades · still on the team
About three Kubernetes releases a year, each supported for roughly a year.
Data · still on the team
Stateless apps move easily. Databases need backups, restores and care.
FIG. 6 — AI shrinks the costs of learning and doing. The costs of owning stay.
§ 06
CHAPTER

Where it doesn't fit

Kubernetes is not the answer for every workload, and GodfreyLabs does not run everything on it. The rule is to use the simplest option that fits each workload:

Static sites
Files on a storage bucket behind a CDN. Nothing to run, nothing to restart.
Small backends
Serverless functions that run when a request arrives. The cloud handles the rest.
The stateful stack
Databases, sign-in, storage, schedulers and builds run on Kubernetes, where self-healing, matching environments and one deploy method pay off.

The GodfreyLabs clusters are self-hosted on virtual machines, which adds more to own. For most teams, a managed service such as Amazon EKS, Azure AKS or Google GKE is the easier way in: the cloud runs the control plane, and the team runs the apps. For a single small app with no one to own a platform, a managed app platform or serverless is still the better call.

§ 07
CHAPTER

What doesn't change

AI writes confident, wrong config. It can miss a security setting, pick the wrong default, or fix a symptom instead of the cause. That is why every change goes through git and review, and why a person presses Sync.

More control means more responsibility. AI changes how fast the work gets done, not who owns it.

Code is cheap now. The release system should not be the bottleneck. Kubernetes and infrastructure as code make releases routine and recovery automatic, and AI made both practical to learn and run. Everything that goes live is still the team's responsibility.

Further reading

  • #kubernetes
  • #infrastructure-as-code
  • #gitops
  • #argocd
  • #ai
  • #platform-engineering
  • #architecture

Continue reading