Scaling AI agents requires heavy investment
A mid-sized financial services firm running a customer service agent for 50,000 queries monthly faces a monthly total cost of ownership of $27,350. This total includes $8,500 for a 4-node GPU cluster with A10G GPUs, $4,200 for model inference, $350 for storage, $12,500 for a full-time engineer, and $1,800 for evaluation and testing. Most organizations find these costs 30% to 40% higher than their initial budgets because they underestimate the engineering and optimization requirements. Implementation requires two to four months of engineering time and costs between $40,000 and $100,000. For those using DBRX models, inference costs approximately $0.0005 per 1K input tokens and $0.0015 per 1K output tokens. For a production agent processing 1M conversations monthly, this translates to roughly $3,000 to $7,000 monthly.
| Component | Monthly Cost (Example) |
|---|---|
| Compute Infrastructure | $8,500 |
| Model Inference | $4,200 |
| Engineering Support | $12,500 |
| Evaluation and Testing | $1,800 |
| Storage | $350 |
Production agents often require 24/7 availability, and storage costs typically range from $25 to $40 per TB per month. Beyond these direct expenses, users pay cloud providers for virtual machines, storage, and network egress. You should budget for these expenses to be higher than your initial estimates.
Technical friction and leadership changes
Building effective agents requires a team with expertise in Python or Scala for data engineering, SQL for Delta table management, and ML concepts for evaluation and fine-tuning. The requirement for engineers to master Python or Scala for data engineering, SQL for Delta table management, and ML concepts for evaluation creates a steep learning curve that many teams struggle to overcome during initial deployment. Implementation often takes months before a production system is ready. Many teams find the learning curve is a "cliff" without deep technical experience. The departure of Naveen Rao in September 2025, who left to launch Unconventional, Inc., adds uncertainty to the strategic direction of the AI division. Will the platform maintain its current trajectory without his influence? The company still relies on its existing leadership to drive these AI initiatives forward.
The Mosaic AI Agent Framework combines Mosaic AI Vector Search, the Agent Framework SDK, the evaluation suite, and Model Serving. The platform provides unified billing across open-source models like Llama and Mistral and proprietary providers like OpenAI and Anthropic. The framework uses MLflow 3.0 to track agents deployed outside Databricks, including on AWS, GCP, and on-premise systems. While Agent Bricks remains in beta, the platform supports over 250,000 queries per second through Model Serving.
Format convergence and financial debt
The move toward an open lakehouse architecture brings new questions regarding format interoperability. While Delta Lake remains a major player, the industry has converged on Apache Iceberg as the interoperability standard. Databricks supports Iceberg through UniForm and managed Iceberg in Unity Catalog, but the relationship between the two is complex. Delta Lake 4.0 provides features like Coordinated Commits and the Variant type, while subsequent releases like Delta Lake 4.1 and 4.2 have shipped. New proposals suggest Delta Lake 5.0 might adopt the Iceberg v4 metadata tree as its native content metadata. This potential convergence is a strategic uncertainty for organizations making long-term investments in Delta-native architectures.
This uncertainty follows a period of intense capital movement. Databricks closed a $5 billion funding round at a $190 billion valuation in August 2026. The company also manages significant financial obligations, such as the $1.8 billion debt financing led by JPMorgan in January 2026. Databricks previously raised $10 billion in Series J funding led by Thrive Capital in December 2024.
