Model access and budgets

Once people and agents call AI models from your product, three questions come up quickly: who may use the expensive models, how do you stop one runaway agent from spending the month's budget overnight, and who holds the provider API keys. Answering them in each application means scattered keys and spend you only see on the invoice. Better IAM answers them in one place:

• Model access is an authorization decision: inference:invoke on model/{name}, written in ordinary policies.
• Provider-run tools are governed per model: web search, code execution, remote MCP servers, and other tools the provider runs can be allowed, refused, or decided one by one with inference:use-tool.
• Budgets cap tokens, cost, and calls per tenant, group, person, or agent, and per delegation.
• Every call is metered, by identity, agent, model, and day, and lands in billing when you use it.
• A gateway lets any Better IAM credential call models in the Anthropic or OpenAI wire format without ever seeing a provider key.

It works for people, service accounts, and AI agents, including agents acting on a person's behalf.

Setup

import { betterIam } from 'better-iam';
import { sqliteAdapter } from 'better-iam/adapter-sqlite';

export const iam = betterIam({
  database: sqliteAdapter({ filename: './iam.db' }),
  secret: process.env.BETTER_IAM_SECRET!,
  baseURL: 'https://app.example.com',
  inference: {
    usageRetentionDays: 90, // per-call usage records (1 to 3650)
    allowCustomBaseUrls: false, // only root administrators may point providers at custom URLs
  },
});

npx better-iam migrate

Usage

Create a provider

A provider is an upstream account: anthropic, openai, or openai-compatible (any endpoint that speaks the OpenAI Chat Completions API, such as vLLM or a router). Its API key is sealed with the deployment secret, never returned, and opened only inside the server when the gateway calls the provider; administrators see its last four characters.

import { iam } from '../iam';

const anthropic = await iam.api.inference.createProvider(admin, {
  tenantId,
  name: 'Anthropic',
  kind: 'anthropic',
  apiKey: process.env.ANTHROPIC_API_KEY!,
});

Route: POST /inference/createProvider

Creating and changing providers needs a recent sign-in. A custom baseUrl (required for openai-compatible) sends the key to that address, so only a root administrator may set one unless the deployment sets inference.allowCustomBaseUrls, and it must be https. Base URLs an organization chose cannot reach private or reserved addresses unless the deployment sets inference.allowPrivateNetworks. Replace a key with updateProvider({ apiKey }); iam.rotateSecrets() re-seals provider keys when the deployment secret changes.

Route: POST /inference/updateProvider, POST /inference/listProviders, POST /inference/deleteProvider

deleteProvider refuses a provider that models still use (RESOURCE_IN_USE).

Publish a model

A model is published under a public name that callers use and policies name (model/{name}), served by a provider of the same tenant or an ancestor as its upstreamModel:

import { iam } from '../iam';

await iam.api.inference.createModel(admin, {
  tenantId,
  name: 'opus',
  providerId: anthropic.id,
  upstreamModel: 'claude-opus-5-5',
  tier: 'frontier',
  family: 'claude',
  inputPricePerMTok: 5,
  outputPricePerMTok: 25,
  cachedInputPricePerMTok: 0.5,
});

Route: POST /inference/createModel, POST /inference/updateModel, POST /inference/listModels, POST /inference/deleteModel

Models are inherited: define providers and models once on the root tenant and every organization sees them. An organization may publish its own model under the same name, which takes precedence for it, and enabled: false stops every call to a model at once. maxOutputTokens caps the output of every call through the gateway. Every model field is listed under Options.

Fall back when a provider fails

Providers have bad days. fallbacks (up to five model names, in order) keeps calls going when one does: when a model's provider cannot be reached or answers 429, 500, 502, 503, 504, or 529 (overloaded), the gateway tries the next fallback. A fallback is used only if the caller may call it (access and budgets are checked like any call) and it speaks the same wire format. Each failed attempt is metered with no tokens against the model that failed, and the answer against the model that served it; responses name that model in x-better-iam-model and the one asked for in x-better-iam-fallback-from.

import { iam } from './iam';

await iam.api.inference.updateModel(admin, {
  tenantId,
  name: 'opus',
  fallbacks: ['opus-eu', 'sonnet'],
});

List the models a caller may use

inference.listMine lists the enabled models the caller may use, for a model picker. It needs no permission beyond the model access itself.

import { iam } from './iam';

const models = await iam.api.inference.listMine(session, { tenantId });

Route: POST /inference/listMine

In React, useModels({ tenantId }) from @better-iam/react returns the same list as models.

Check a call before making it

inference.check answers whether the caller may invoke a model right now, budgets and provider tools included. When the call is allowed it also returns a single-use ticket that an external gateway redeems with inference.record (see Keeping your own gateway).

import { iam } from './iam';

const result = await iam.api.inference.check(agentKey, {
  tenantId,
  model: 'opus',
  estimatedTokens: 4000, // 0 when omitted
  tools: ['web_search'], // provider tools the request will ask for
});
if (!result.allowed) console.log(result.reason); // ACCESS_DENIED, BUDGET_EXCEEDED, MODEL_DISABLED, TOOL_NOT_ALLOWED

Route: POST /inference/check

A refusal carries reason, the model when known, the exhausted budget for BUDGET_EXCEEDED, and the first refused tool for TOOL_NOT_ALLOWED. An allowed answer carries the model and the standing of every budgets entry that covers the caller. Denials are audited like any decision.

Who may call which model

Model access is a policy decision: inference:invoke on model/{name}. The model's attributes are available to conditions:

{
  "version": 1,
  "statements": [
    {
      "effect": "allow",
      "actions": ["inference:invoke"],
      "resources": ["model/*"],
      "conditions": { "StringEquals": { "resource.tier": "small" } }
    },
    {
      "effect": "allow",
      "actions": ["inference:invoke"],
      "resources": ["model/opus"],
      "conditions": { "Bool": { "principal.mfa": true } }
    }
  ]
}

The playground opens with Alice asking for the frontier model without MFA, which is refused: neither statement applies. Set principal.mfa to true and the second statement allows it; set resource.tier to small and the first one does.

Principal keys work as usual, so a delegated agent session can use only the models the person may use, within the delegation's scope. Delegated sessions are never MFA-verified (principal.mfa is false), so a statement like the second one never admits an agent acting for a person.

Tools the provider runs

Some tools run at the model provider, not in the caller's code: web search and web fetch, code execution and code interpreter, file search, image generation, and remote MCP servers the provider connects to. Computer use, bash, and the text editor are defined by the provider for the model to drive. These tools can read the web, run code, or reach outside systems with the organization's provider account, so a prompt-injected agent could use them to leak data. Each model says what it allows with providerTools:

import { iam } from './iam';

await iam.api.inference.updateModel(admin, {
  tenantId,
  name: 'gpt-research',
  providerTools: 'policy',
});

{
  "effect": "allow",
  "actions": ["inference:use-tool"],
  "resources": ["model-tool/web_search", "model-tool/mcp:*.acme.com"]
}

The gateway reads the tools from the request:

• Anthropic tools[].type (anything but a custom tool), and each mcp_servers[].url;
• OpenAI Responses tools[].type (anything but function and custom);
• Chat Completions web_search_options.

Tool kinds are named without their version date or _preview suffix:

Policies see resource.kind and, for MCP servers, resource.host. Tools the caller runs itself (functions and custom tools) are never checked, and token counting runs no tool. A request may list at most 256 tools and 64 MCP servers, so every one of them is read.

A refused request answers 403 TOOL_NOT_ALLOWED, naming the tool, and never reaches the provider. Refusals are audited as inference:use-tool denials. Delegations and agent ceilings apply as usual: an agent acting for a person needs inference:use-tool within the delegation's scope. External gateways pass the tool IDs to inference.check as tools; the gateway's own reading is exported as providerToolsOf(path, body) from better-iam/server:

import { providerToolsOf } from 'better-iam/server';

const tools = providerToolsOf('/v1/responses', {
  model: 'gpt-research',
  tools: [{ type: 'web_search_preview' }, { type: 'mcp', server_url: 'https://kb.acme.com/mcp' }],
});
// pass `tools` to inference.check

Budgets

A budget caps tokens (maxTokens), cost (maxCostUsd), the number of calls (maxRequests), or any mix of them, per minute, hour, day, or month (UTC windows). A minute budget on requests or tokens is a true rate limit, for example 60 calls a minute for an agent that might loop. It needs at least one of the three, alertAtPercent (1 to 100) applies to each limit it sets, and null clears a limit on update.

For tenant and group budgets, scope: 'shared' (the default) makes one pool for everyone covered, and scope: 'each' gives everyone covered the full amount (for example, one million tokens a day per person). models limits a budget to some model name patterns. People can also cap what an agent spends for them on a single delegation (spend; see AI agents).

Set a budget

inference.setBudget (requires iam:inference:manage) creates a budget, or updates one when you pass budgetId:

import { iam } from '../iam';

await iam.api.inference.setBudget(admin, {
  tenantId,
  name: 'Daily per person',
  subjectType: 'tenant',
  scope: 'each',
  period: 'day',
  maxTokens: 1_000_000,
  alertAtPercent: 80,
});
await iam.api.inference.setBudget(admin, {
  tenantId,
  name: 'Triage agent',
  subjectType: 'identity',
  subjectId: agent.id,
  period: 'month',
  maxCostUsd: 200,
});
await iam.api.inference.setBudget(admin, {
  tenantId,
  name: 'Agents: 60 requests a minute',
  subjectType: 'tenant',
  scope: 'each',
  period: 'minute',
  maxRequests: 60,
});

Route: POST /inference/setBudget, POST /inference/listBudgets, POST /inference/deleteBudget

A call is refused with BUDGET_EXCEEDED when a covering budget is spent, or when the call's estimate would not fit in what is left. The gateway estimates from the request (its size in bytes divided by four, plus its output limit: max_tokens, max_completion_tokens, or max_output_tokens); inference.check and iam.inference.authorize use the estimatedTokens you pass (0 when omitted). The first refusal in a window is audited as inference:budget-exceeded, and crossing alertAtPercent is audited once per window as inference:budget-alert, so a webhook can alert on either. Cost is metered in micro-dollars: tokens times the model's price per million tokens.

The gateway

The gateway lets any Better IAM credential call models without holding a provider key. It relays each provider's own wire format, streaming included, changing only what it must:

Mount it on a catch-all route:

import { iam } from '@/lib/iam';

const gateway = iam.inference.gateway({ basePath: '/ai' });
export const POST = gateway;
export const GET = gateway;
// Hono: app.all('/ai/*', (c) => gateway(c.req.raw));

Point an SDK at it with a Better IAM credential as the API key:

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic({
  baseURL: 'https://app.example.com/ai',
  apiKey: process.env.AGENT_KEY!, // an agent key, a delegated session token, or any Better IAM credential
});
await client.messages.create({
  model: 'opus',
  max_tokens: 1024,
  messages: [{ role: 'user', content: 'Hi' }],
});

For each request the gateway:

1. authenticates the caller (a bearer token or x-api-key);
2. checks inference:invoke, the provider tools, and the budgets;
3. swaps the public model name for the upstream one and holds the answer to the model's maxOutputTokens: a larger max_tokens, max_completion_tokens, or max_output_tokens is lowered, and a request without one gets it (max_completion_tokens for openai Chat Completions, max_tokens for anthropic and openai-compatible, max_output_tokens on /v1/responses; embeddings and token counts have none);
4. sends the request with the sealed key and streams the answer back, asking OpenAI streams to report usage (stream_options.include_usage) and swapping the model name back in the answer (except in streams);
5. meters the tokens the provider reports, when the stream ends for streams.

When a stream ends without final counts (the client closed it early, or the provider sent none), the gateway estimates: input from what the provider reported so far, else a token per four bytes of the request; output a token per four generated characters seen. Closing a stream early never makes a call free. Everything else passes through.

Responses API conversations are stored at the provider under the organization's key, so without a check any caller could continue any other caller's conversation by its ID. The gateway records who created each response it relays, and previous_response_id must name one of the caller's own responses (the same person, through the same agent if an agent is acting) at the same provider; any other ID gets 404 RESPONSE_NOT_FOUND and never reaches the provider, and fallbacks to another provider are skipped. These are refused with 400 UNSUPPORTED_PARAMETER, because their state and usage stay out of the gateway's sight or cannot be tied to the caller:

• the conversation parameter and background: true;
• Responses item_reference inputs;
• file_ids and file_ids anywhere in the request, vector_store_ids (file search), and Anthropic file sources;
• an existing code-execution container.

Built-in tools that the provider bills separately from tokens (web search, file search, code interpreter) are not metered. Relayed responses carry x-better-iam-request-id and x-better-iam-model.

Keeping your own gateway

An existing gateway can use Better IAM for the decision and the metering only. It calls inference.check with the caller's credential and receives a single-use ticket (valid one hour) when the call is allowed, makes the call itself, then redeems the ticket with the token counts through inference.record, using its own service account key with iam:inference:record. The call is metered as the caller the check saw, even if that caller's session has ended since, so a short session cannot make an allowed call escape its budgets or a delegation's spending cap.

import { iam } from '../iam';

const check = await iam.api.inference.check(callerCredential, { tenantId, model: 'opus', estimatedTokens: 2000 });
if (check.allowed && check.ticket) {
  // ... call the provider yourself ...
  await iam.api.inference.record(gatewayKey, {
    tenantId,
    ticket: check.ticket,
    inputTokens: 1200,
    outputTokens: 640,
    status: 'ok',
    latencyMs: 2300,
  });
}

Route: POST /inference/check, POST /inference/record

In-process code can use iam.inference.authorize(credential, { model, estimatedTokens?, tools? }), which returns either { denied } or a permit { principal, tenantId, check, upstreamModel, provider: { id, kind, baseUrl, apiKey } } with the opened provider key (never expose it), and then iam.inference.record(permit, usage):

import { iam } from './iam';

export async function callModel(request: Request) {
  const outcome = await iam.inference.authorize({ headers: request.headers }, { model: 'opus' });
  if ('denied' in outcome) return new Response(outcome.denied.reason, { status: 403 });
  // ... call outcome.provider.baseUrl with outcome.provider.apiKey and outcome.upstreamModel ...
  await iam.inference.record(outcome, { model: 'opus', inputTokens: 1200, outputTokens: 640 });
  return new Response('ok');
}

When the deployment has billing, every metered call, through either path, also lands in the spend ledger on the built-in inference meter.

Usage reports

inference.usage (with iam:inference:read) adds up calls between from and to by identity, agent, model, or day (groupBy), with requests, errors, token counts, and cost, optionally for one identityId, agentId, or model. inference.myUsage shows a caller their own usage and the standing of every budget that covers them (tokens, cost, and requests used and remaining), and inference.listBudgets shows the current window of each shared pool.

import { iam } from './iam';

const report = await iam.api.inference.usage(admin, {
  tenantId,
  from: Date.now() - 30 * 86_400_000,
  groupBy: 'agent',
});
// report.rows: [{ key, label, requests, errors, inputTokens, outputTokens, costUsd, ... }], report.totals

Route: POST /inference/usage, POST /inference/myUsage

Usage records are kept for usageRetentionDays and budget counters 35 days past their window; iam.sweepExpired() deletes them after that.

Options

The inference server option (boolean | InferenceOptions):

iam.inference.gateway(options) (InferenceGatewayOptions):

Model fields, on inference.createModel and inference.updateModel (null clears a field on update):

Budget fields, on inference.setBudget:

Schema

Permissions

Errors

API errors:

inference.check and iam.inference.authorize return refusals instead of throwing them, with reason ACCESS_DENIED, BUDGET_EXCEEDED, MODEL_DISABLED, or TOOL_NOT_ALLOWED.

Gateway answers use each API's own error shape:

The provider's own errors pass through with their status.

Next steps