--- title: "How Do You Govern an AI Budget Across Departments?" description: "Engineering, support, sales, and finance all consume AI differently. Here's how to set, enforce, and optimize AI budgets per department." canonicalUrl: "https://zuplo.com/blog/2026/10/20/govern-ai-budget-across-departments" pageType: "blog" date: "2026-10-20" authors: "billDoerrfeld" tags: "AI Gateway, API Best Practices, ai-agents" image: "https://zuplo.com/og?text=How%20Do%20You%20Govern%20an%20AI%20Budget%20Across%20Departments%3F" --- Enterprise usage of AI is skyrocketing. And, agentic AI isn't stuck in engineering anymore — it's giving a lift to marketing, sales, finance, data analysis, and all other areas of the business. According to a [2026 McKinsey survey](https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai), 40% of respondents at large organizations are actively scaling agentic AI within at least one business function. Most analysts predict AI agent use will continue. But, the thing is, as highly actionable enterprise agents rise into the thousands, or hundreds of thousands [as Gartner predicts](https://www.gartner.com/en/newsroom/press-releases/2026-04-28-gartner-identifies-six-steps-to-manage-artificial-intelligence-agent-sprawl), so do expenses. As all departments build AI reliance into their workflows, AI budgets are becoming substantial. Nearly two-thirds of large organizations budget over $5M and 39% budget $10M or more, found a study by [Wharton Human-AI Research at the University of Pennsylvania](https://ai.wharton.upenn.edu/wp-content/uploads/2025/10/2025-Wharton-GBK-AI-Adoption-Report_Full-Report.pdf). Now, there's a real need to monitor AI use across an organization, and set and enforce caps on internal use. Yet, this can't be a blanket rule — such cost control needs to be flexible and attuned to the needs of each department. Below, we'll consider how to govern an AI budget across departments, what benchmarking and trends are emerging that [enterprise AI governance](/blog/how-to-avoid-shadow-mcp-servers) frameworks should consider, and how to actually put this to work. ## Different Departments Use Agentic AI Differently Different departments have different technical needs, and AI use is not equal across business functions. A 2025 look at enterprise AI spending by [Menlo Ventures](https://menlovc.com/perspective/2025-the-state-of-generative-ai-in-the-enterprise/) found coding alone took 55% of departmental AI spend, followed by IT (10%), marketing (9%), customer success (9%), design (7%), and HR (5%) trailing after. For a specific case study, consider Pylon's forecasted Claude spend. [Marty Kausas](https://www.linkedin.com/in/martykausas/), CEO, recently shared the company's average [per-person monthly spend](https://lnkd.in/p/gGh-xHVh) across 13 different internal teams. Similar to the above report, Engineering topped the list, at $3.1k per person. The Support team followed, at $1.1k per person, followed by Marketing ($740), Customer Success ($440), and Sales ($230). Their Finance team might not trust AI yet — they're just at $8. [Agentic AI](/blog/agentic-tool-calls-least-privilege) is being used for varied tasks, each incurring vastly different computational processing loads depending on the task at hand. An engineering refactoring task might require massive input tokens for upfront context and chain various calls on the backend, for instance, while a sales workflow might involve a simple customer data lookup prior to a prospect meeting. Given this variance across teams, setting an equal token allowance per department isn't effective. ## Setting Internal Limits for LLM Consumption It's apparent that enterprises require limits to avoid runaway tokens and high costs. So, how would you actually enforce internal limits for LLM consumption? First, it'll require deep observability and the ability to [segment monitoring by department](/blog/attribute-ai-spend-teams-apps). This must include input and output token usage, what LLMs are being used, and helpful metrics like average call token and equivalent costs. Next, enterprises will require some sort of [control plane](/blog/mcp-gateway-buy-or-build) to configure limits, and automatically enforce LLM consumption. The ability to aggregate statistics and enforce limits should be flexible: depending on the organizational size and makeup, limits might be by department, or nested within a subgroup, team, or at the individual level. When a grouping reaches its allocated limit, the system would cut access automatically or route to another service. The downside is that not all AI environments have this functionality out of the box. If they do allow transparent visibility, it's usually on a per-individual account basis, and not aggregated across an entire organization with inter-departmental segmenting and enforcement capabilities. ## Using Semantic Caching and LLM Routing There are other ways beyond static limits you can use to optimize AI budgets within an enterprise. One strategy is [semantic caching](https://thenewstack.io/what-is-semantic-caching/), which is similar to traditional caching but for LLMs. If a prompt matches one saved in the cache with enough semantic similarity, it serves the corresponding cached response, avoiding the need to invoke the LLM at all. Semantic caching can equate to real cost savings. For instance, [benchmarking by Amazon](https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/semantic-caching-benchmarks.html) evaluated 63,796 real chatbot queries and found a 51.9% cost savings using Amazon ElastiCache. Semantic caching helps avoid calls to underlying LLMs — but, if no relevant cache exists, you'll hit an LLM. It doesn't help optimize beyond that. However, that's where LLM routing can assist. Token consumption can be highly variable and unpredictable. In April 2026, [Microsoft researchers](https://www.microsoft.com/en-us/research/publication/how-do-ai-agents-spend-your-money-analyzing-and-predicting-token-consumption-in-agentic-coding-tasks/) found that agentic tasks can consume 1,000x more tokens than code reasoning and chat. They also found substantial variation between models. On the same tasks, "Kimi-K2 and Claude-Sonnet-4.5, on average, consume over 1.5 million more tokens than GPT-5." Agentic AI usage is associated with knowledge management, data retrieval, API calls, and workflows. And its associated costs can be massive and hard to predict. Therefore, LLM routing is important to route requests to the appropriate LLM for the job. This could enable a marketing department, for instance, to route queries to less expensive LLMs for easier tasks. [Researchers have found](https://proceedings.mlr.press/v267/ding25d.html) one routing framework reduced costs by up to 60% with less than a 1% performance drop. ## Other Advanced AI Accounting Measures Of course, it's early days, and governing an AI budget throughout a large organization is an emerging art. It'll highly depend on the organizational makeup, industry, and goals of the knowledge workers and their AI agent assistants. Plus, there are other emerging approaches worth considering: - **Shared budget pools**: Allowing remaining token budgets to overflow across departments, similar to computing workload orchestration at scale. - **Time-based limits**: Setting more fluid limits on an hourly, weekly, or monthly basis. - **Chargeback/showback**: Allocating or reporting AI costs to the departments responsible for consuming them. - **Performing deep analysis**: Looking for duplicate calls, bloated prompts, retry loops, overpowered models, and missed caching opportunities. [Traceburn](https://github.com/TommyTranX/traceburn) does this, for instance. ## Example AI Usage Allocation Budget Let's take a theoretical example of allocating AI spend throughout departments. Imagine you're heading a 5,000-employee customer support company serving the financial services industry. Your main departments are IT, human support service desks, sales to expand the business, and other more minor business functions, such as finance and accounting, HR, and R&D. Your AI budget might consist of a high token monthly allocation for your IT department to build generative AI functionality, as well as a high allocation for human support agents to utilize AI in their day-to-day client-facing work. An example budget could be built after monitoring real-world use and average token use. Based on observed usage, an illustrative monthly allocation might look like this: | Department | Headcount | Monthly Budget (Total) | Average Monthly Token Use (Total) | Monthly Budget (Per Employee) | Average Monthly Token Use (Per Employee) | | -------------------------- | --------- | ---------------------- | --------------------------------- | ----------------------------- | ---------------------------------------- | | IT | 800 | $450,000 | 180B | $562.50 | 225M | | IT: AI engineering | 300 | $300,000 | 120B | $1,000 | 400M | | IT: DevOps and maintenance | 500 | $150,000 | 60B | $300 | 120M | | Support agents | 3,200 | $160,000 | 128B | $50 | 40M | | Sales | 400 | $10,000 | 8B | $25 | 20M | | Marketing | 150 | $22,500 | 9B | $150 | 60M | | Finance | 100 | $7,500 | 3B | $75 | 30M | | R&D | 150 | $75,000 | 15B | $500 | 100M | | HR | 200 | $5,000 | 2B | $25 | 10M | | Totals | 5,000 | $730,000 | 345B | $146 | 69M | _These figures are completely illustrative. Actual cost per token will vary considerably by model and provider._ As you can see, R&D and engineering might have a high per-seat monthly budget compared to the budget of HR or marketing, which is typically less technically involved. Subgroups and teams might need to share and [inherit limits from their parent organization](/blog/give-teams-scoped-ai-access-without-separate-infrastructure), as well. For instance, let's say IT's AI engineering team has a larger allocation since they are building more generative AI functionality. The organization has [semantic caching](/blog/what-is-semantic-caching) in play, which is especially useful for eliminating unnecessary LLM calls for customer-facing chatbots. Semantic caching may be less useful for novel engineering tasks, where queries are less likely to repeat. They also are routing their requests to LLMs. For instance, frontier models might be reserved for complex R&D work, while sales queries could be routed to less expensive models for simpler data retrieval and summarization tasks. ## AI Gateway Aids AI Budget Governance AI usage can vary dramatically depending on the user, department, task, or model involved. Some queries are far more complex, some might chain together APIs on the backend, and different LLMs themselves can consume vastly different amounts of tokens. As organizations operationalize AI, measuring token use, latency, and cost is going to become a commonplace aspect of IT budgeting. This is where an [AI gateway](/blog/what-developers-want-from-an-ai-gateway) can help. Since LLM traffic passes through the gateway, it creates a natural control point to monitor usage across departments, set usage limits and tiers, and enforce those limits in practice. For multi-department organizations, that makes the gateway a useful layer for turning AI budget policies into actual controls.