Open Weight AI Models Are Demolishing Walled Gardens

The Token Tectonic Shift: Enterprises Move Workloads to Open Models

Cost is the immediate catalyst, but as Morgan emphasizes, it is only the surface symptom. The deeper driver is control, security, and customization. Enterprises have realized that paying per million token rates of $15 to $30 for closed APIs is unsustainable for high frequency, structured tasks. This is especially true when open models can deliver comparable results at a fraction of the cost, often just $0.20 to $2 per million tokens for hosted open models, and effectively zero once local hardware is amortized.

Take AT&T as a bellwether. In a recent disclosure, the telecom giant revealed that it has already moved 40% of its total LLM token consumption to open weight models, and the trajectory is clear. Morgan predicts a steady state split where 80 to 90% of enterprise tokens run through open models. Only the most demanding reasoning tasks, like multistep strategic planning or complex mathematical proofs, will continue to be routed to premium closed systems.

This shift unlocks a state of token abundance. Developers can now design autonomous, multistep agents without constantly monitoring API bills. Morgan notes that while cost is the pain point you solve today, the real North Star is the ability to own your data, adapt the model to your domain, and deploy anywhere, whether on premises, in cloud, or at the edge.

The Workload Explosion: Coding Agents and Cowork Bots

What are all these tokens actually doing? The consumption curve is not linear; it is exponential, driven by two major agentic paradigms.

First, coding agents are doing the heavy lifting. Today, the overwhelming majority of developer tokens are consumed by coding agents that operate in tight feedback loops. They do not just generate snippets. They write code, compile it, run tests, diagnose errors, and iterate until a suite passes. This loop can burn millions of tokens per feature. With open models now matching or exceeding closed models on coding benchmarks, teams are rapidly migrating. For example, Qwen 2.5 Coder 32B scores over 85% on HumanEval, rivaling Claude 3.5 Opus.

Second, cowork agents are providing automation for non developers. Frameworks like OpenClaw and the Hermes Agent Project have democratized long running autonomy. These agents can handle complete business workflows like processing invoices, triaging support tickets, or synthesizing sales intelligence over hours or days. Morgan's data shows that the OpenClaw surge in April 2025 caused a massive jump in average per user token consumption, driven largely by non technical teams in finance, HR, and marketing.

Two technical enablers made this possible. The first is massive context windows. From 8K to 128K, and now beyond 1 million tokens, models can ingest entire codebases, legal contracts, or product documentation in one pass. The second is native tool calling. Open models are now trained specifically for function calling and JSON structured outputs, making them reliable for querying databases, invoking APIs, and executing shell commands with high accuracy.

Morgan observed that moving to a million plus context with open models unlocked entirely new classes of applications. It is no longer about simple Q&A; it is about agents that can act on your behalf for hours.

The Desktop Data Center and the Return of Local Compute

One of the most surprising developments is the reemergence of local execution as a first class citizen. While coding agents currently lean on cloud due to compute demands, the hardware pipeline is catching up fast.

Apple's unified memory architecture has turned even mid range MacBook Pros into credible LLM hosts. Models in the 20B to 40B parameter range, such as Qwen 38B, run smoothly on a machine with 36 GB of unified memory. For document processing, semantic search, and data extraction, local execution offers sub 50 ms latency, zero egress costs, and complete data privacy.

For professional workstations, Nvidia is rolling out the DGX Spark. This is a desktop unit with 128 GB of unified memory, capable of running models up to 120B parameters entirely on device. These machines also feature high speed NVLink interconnects, allowing multiple units to be clustered into a mini data center on your desk. Many enterprises in banking, defense, and engineering already have Nvidia RTX workstations for rendering, making the upgrade to local LLM inference a natural incremental step.

The ultimate developer experience demands a sub 100 ms feedback loop, which is the kind of instantaneous autocomplete developers expect from GitHub Copilot. To achieve that, local hardware will handle immediate, repetitive tasks, while a smart router offloads heavy batch jobs to cloud hosted open models. This hybrid model cuts costs further: you pay for the hardware once and use it alongside cloud capacity, effectively reducing marginal token cost to near zero for a large fraction of workloads.

Morgan explains that this hybrid model is already emerging. Document workflows run locally, while hard reasoning tasks go to the cloud. The router decides, and the developer doesn't even notice.

The Unbundling of the AI Stack: Beyond the Five Layer Cake

In the proprietary world, frontier labs own the entire stack, from silicon partnerships to application frameworks. Nvidia CEO Jensen Huang famously described this as a Five Layer Cake: Applications, Models, Infrastructure & Inference, Silicon, and Energy. Closed providers want developers to live entirely inside their walled gardens, using their proprietary context, memory, and security layers. Open source, by contrast, is forcibly unbundling this cake. Developers refuse to be caged in. Instead, a best of breed ecosystem is forming around the layers above the raw token generation. This unbundled open AI stack includes:

• Knowledge Layer: Vector databases like Pinecone and Milvus, RAG engines, document chunkers, and embedding models.
• Coordination Layer: Agent orchestrators like LangGraph and AutoGen, local and cloud routers, and MCP servers.
• Execution Layer: Secure sandboxes like Firecracker and gVisor, and cloud runtimes like Ollama Cloud and Together AI.
• Operating System Layer: Ollama handling hardware drivers, inference runtime, API gateways, and model registries.

Each of these layers is now a competitive category, with multiple vendors and open source projects. Developers can swap models, for instance replacing a US trained model with a Chinese one like DeepSeek V3 for cost sensitive tasks, while keeping their knowledge base and orchestration logic intact. This modularity is a game changer for both agility and negotiating power. Morgan summarizes that the scarcity is no longer the model weights; it's the intelligence above them, namely the coordination, the memory, and the tooling. That is where the next wave of engineering innovation will happen.

Ollama's Day Zero Playbook: Orchestrating the Chaos

With new open weight models dropping every few weeks, and sometimes multiple times a month, the challenge of integrating them into a seamless developer experience is immense. Ollama has developed a rigorous Day Zero launch playbook that brings three pillars together in a tight 24 hour window.

First is Inference Engine Integration. Each new model has a unique architecture. The inference engine must be updated to support it, often requiring modifications to the CUDA kernels or MLX ops, to ensure speed and numerical accuracy that match the reference implementation.

Second is Harness & Framework Compatibility. New models often introduce novel capabilities like vision, multi lingual support, or extended context. The Ollama team works with framework maintainers to ensure that the model can be called via standard APIs and that popular agent harnesses can exploit its features on day one.

Third is Hardware Optimization. Collaboration with hardware partners like Apple, Nvidia, and AMD is critical. Without optimized kernels, a capable model can be unusably slow. Ollama's systems engineering heritage gives them unique muscle to tune memory layouts, batch sizes, and KV cache strategies across diverse silicon.

It is a fire drill every time, but they have built the internal tooling to test, benchmark, and release reliably.

The Rapid Release Cadence and Its Implications

Unlike the measured six month cycles of frontier labs, open model releases are accelerating. This summer alone, DeepSeek has rolled out three iterations of its Flash model. The gap between closed source breakthroughs and open source parity has shrunk from over a year to mere months, and in some benchmarks, open models now lead.

This cadence creates both opportunities and headaches. Fine tuning becomes a moving target. Enterprises that invest heavily in custom fine tuning may find their investment obsolete within weeks as a better base model drops. Many are pivoting to instruction tuning or adapter based approaches that can be quickly reapplied to newer base weights.

Tooling is catching up. New pipelines for continuous fine tuning, evaluation, and deployment are emerging, making it easier to stay current.

Security and governance remain the bottleneck. Morgan points out that the chief blocker for adopting Chinese origin models, which are often the most performant on coding, is not capability but security vetting. This is creating a new market for model governance platforms that assess safety, bias, and provenance.

Security Testing and the New Frontier of Use

One of the less discussed but critical advantages of open models is their utility in security research. Closed APIs often refuse requests that involve penetration testing, vulnerability scanning, or red team exercises. Open models, especially those fine tuned for cybersecurity, can be run locally to simulate attacks, test defenses, and audit code without triggering ethical guardrails that are overly conservative.

Morgan notes that even out of the box open models are more permissive in distinguishing legitimate security testing from malicious intent, making them indispensable for DevSecOps teams. This has sparked a wave of startups building AI powered security copilots that leverage open weights for continuous compliance and threat hunting.

Founder Lessons: Systems Muscle and the Two Week MVP

The story of Ollama's own genesis offers a masterclass in founder resilience and the importance of playing to core strengths.

Before launching Ollama in July 2023, Morgan and his cofounder spent two frustrating years building and discarding ideas, ranging from Kubernetes security tools to developer SSO platforms. Despite raising a Series A, they lacked product market fit. The turning point came when they forced a simple exercise: Throw everything out. If we started from scratch today, what would we build?

They identified that running open source LLMs locally was a massive, unsolved pain point. Leveraging their deep systems background in VMware and Docker, they set a two week deadline to build a minimal runtime that could pull, optimize, and serve any open weight model on a Mac. On the very last day, Meta released Llama 2. Because they had a working MVP, they rode that wave to explosive adoption, achieving 100,000 GitHub stars faster than either Docker or Kubernetes.

The key lessons for founders are clear. Leverage your core muscle; Ollama won because they returned to systems engineering, not because they became AI researchers. Bias to action over analysis; two years of customer interviews produced little, but two weeks of building produced a category defining product. Finally, the true moat is developer experience. In a world of commoditized weights, the real value is in the glue, which means the runtime, the tooling, and the ecosystem that makes developers productive.

The Road Ahead: Hybrid, Modular, and Abundant

The future of AI is not a single winner. It is a heterogeneous, modular ecosystem where open and closed models coexist, where local and cloud execution blend, and where the layers above the models become the new battleground.

The economics are clear: token costs are collapsing; compute is becoming democratized via desktop supercomputers; and the pace of innovation is accelerating. For developers and enterprises, the message is unambiguous. The era of the locked in, one model fits all API is ending. The new stack is open, unbundled, and infinitely more flexible.

As Jeffrey Morgan puts it, we have an abundance of open tokens. The real challenges now are orchestration, memory, and governance. That is where the next generation of companies will be built.

The walled gardens are crumbling. And from the rubble, a new, more resilient AI economy is rising, one that puts control, cost efficiency, and choice back into the hands of the builders.