Published on April 17, 2026

When a developer hands a task to an autonomous coding assistant, the expectation is that the agent will finish the job quickly and hand back clean code. In reality, many real‑world problems—such as large code‑base migrations, multi‑step refactors, or continuous integration pipelines—require an AI‑driven worker to stay alive for hours, even days. This post walks through the architectural patterns, state‑management tricks, and operational safeguards that let an AI coding agent survive a two‑week marathon without "losing the plot."

---

Why Typical Agents Stall After an Hour

Most off‑the‑shelf LLM wrappers impose a hard token limit per request (often 4 k–8 k tokens). When a developer builds a loop that repeatedly prompts the model, each iteration consumes part of the allocated context, and after a handful of calls the conversation history exceeds the model's window. The result is truncated prompts, forgotten variables, and eventually a broken workflow.

Common failure modes

• Context overflow – The prompt grows until the model discards older messages.
• Rate‑limit throttling – Cloud providers cap the number of calls per minute.
• State drift – Intermediate files or temporary data disappear after a container restart.

Addressing these issues requires a deliberate separation between transient LLM calls and persistent execution state.

---

Architectural Blueprint for a Long‑Running Agent

Below is a high‑level diagram you can replicate in any cloud environment:

1. Orchestrator Service – A lightweight Node.js or Python process that schedules work, monitors health, and retries failures.
2. Stateless LLM Wrapper – A thin client that sends a single prompt to the model and returns the raw completion.
3. Persistent Store – A durable database (PostgreSQL, DynamoDB, or even a Git repo) that records:
   • The full conversation history broken into chunks.
   • Intermediate artefacts such as generated files, test results, and version tags.
4. File System Cache – A mounted volume (EFS, GCS Fuse, or a Docker volume) that survives container restarts.
5. Watchdog Scheduler – A cron‑like job that pings the orchestrator every few minutes to ensure the process is alive.

Data flow

Orchestrator → LLM Wrapper → Model ↑ ↓ Persistent Store ←─→ File Cache

The orchestrator never holds the entire conversation in memory. Instead, it pulls the latest N messages from the store, appends the new user instruction, and pushes the response back. By chunking history, you stay within the model's token window while preserving full context for later audits.

---

Implementing Context Chunking

The estimate_tokens helper can be a simple heuristic (e.g., 1 token ≈ 4 characters) or a call to the provider's token‑count API. By always pulling the most recent slice, the agent keeps the conversation fresh without overwhelming the model.

---

Persisting Generated Code Safely

When an AI agent writes a file, you should never trust the in‑memory representation alone. Store the artefact in two places:

• Version‑controlled repo – Commit each generated file with a descriptive message. This gives you a full audit trail and makes roll‑backs trivial.
• Object storage – Upload the same blob to S3/Google Cloud Storage for quick retrieval by other services.

If the orchestrator crashes, the next run can resume by cloning the repo at the last successful commit and re‑hydrating the file cache.

---

Handling Rate Limits and Retries

Most providers return a 429 status when you exceed the per‑minute quota. A robust back‑off strategy looks like this:

Combine this with the watchdog scheduler so that a temporary stall does not halt the entire pipeline.

---

Real‑World Example: Migrating a Legacy Codebase

Imagine you need to rename a core utility across 12 k files. A naïve script would run for hours and risk losing state if the process crashes. With the long‑running agent pattern:

1. Chunk the migration – Split the file list into batches of 200.
2. Prompt the model – Ask it to produce a diff for each batch, feeding only the relevant file names and the previous diff as context.
3. Persist each diff – Commit to Git, push to storage, and mark the batch as completed in the database.
4. Resume on failure – The orchestrator queries the DB for the next unfinished batch and continues.

Running this loop for 13 days (as demonstrated by Simon Last) is simply a matter of keeping the orchestrator alive and letting the persistent store do the heavy lifting.

---

Key Takeaways

• Separate state from the LLM: Store conversation history, generated files, and progress in durable services.
• Chunk context: Pull only the most recent tokens to stay within model limits.
• Version control every artefact: Guarantees reproducibility and simplifies debugging.
• Implement exponential back‑off: Protects against rate‑limit throttling.
• Use a watchdog: Detects crashes early and restarts the orchestrator automatically.

---

Conclusion

Running AI coding agents for days is no longer a sci‑fi fantasy. By treating the LLM as a stateless worker and offloading persistence to databases, version control, and cloud storage, you can build pipelines that survive weeks of continuous operation. The pattern scales from a single developer’s laptop to a fleet of Kubernetes pods handling enterprise‑grade migrations. Start small, instrument your orchestrator, and watch your AI assistants stay on script for the long haul.

---

Source: How to Run AI Coding Agents Continuously for Days Without Losing the Plot