How We Built the Velocity Quota System for AI Agent Rate Limiting

AI agents are fast. One misconfigured loop can generate thousands of API calls per minute — flooding your Stripe account with refund requests, your CRM with duplicate records, or your database with expensive queries. Rate limiting is not enough — you need containment. The velocity quota system stops runaway agents before they cause damage. This is how we built it.

Token bucket: the right algorithm for the job

We chose a token bucket algorithm over simpler approaches (fixed window, sliding window) because it handles bursts gracefully while enforcing a long-term average rate. Each organization gets a bucket of tokens that refills at a configurable rate. Every tool call consumes one token. When the bucket is empty, calls are rejected.

┌─────────────────────────────────┐
│  Token Bucket (per org)         │
│                                 │
│  Capacity: 100 tokens           │
│  Refill rate: 10 tokens/second  │
│                                 │
│  Current: 57 tokens             │
│  Last refill: 2026-07-19T14:31  │
└─────────────────────────────────┘

The key property: the bucket has a maximum capacity. If an agent has been idle, tokens accumulate up to the capacity — providing burst tolerance for legitimate usage spikes. But the long-term rate is capped by the refill rate.

Cross-Lambda state via DynamoDB atomic counters

WardenAuth runs on AWS Lambda — stateless by design. Token bucket state must survive across Lambda invocations and be consistent across concurrent calls. We use DynamoDB atomic counters with conditional updates:

async function consumeToken(orgId: string): Promise<boolean> {
  const now = Date.now()
  const bucket = await getBucket(orgId)

  // Refill: calculate tokens earned since last check
  const elapsed = (now - bucket.lastRefill) / 1000
  const earned = Math.floor(elapsed * bucket.refillRate)
  const current = Math.min(bucket.capacity, bucket.tokens + earned)

  if (current < 1) return false // bucket empty

  // Atomic consume: decrement only if tokens haven't changed
  const result = await dynamodb.update({
    Key: { PK: `ORG#${orgId}`, SK: 'VELOCITY_BUCKET' },
    UpdateExpression: 'SET tokens = tokens - :one, lastRefill = :now',
    ConditionExpression: 'tokens = :expected',
    ExpressionAttributeValues: {
      ':one': 1,
      ':now': now,
      ':expected': current,
    },
  })

  return result.success
}

The conditional update ensures atomicity: if two concurrent Lambda invocations try to consume the last token, exactly one succeeds. The other gets a ConditionalCheckFailedException and the call is rejected. No distributed lock, no external coordination — DynamoDB's conditional writes handle it.

Containment, not throttling

Throttling makes things slower. Containment stops things from breaking. These are fundamentally different goals:

A velocity quota at 100 calls/minute says "this agent cannot make more than 100 Stripe calls per minute, period." A throttle at 100 calls/minute says "please try again later." The quota is a safety boundary; the throttle is a traffic management tool.

Per-org configuration with sensible defaults

Velocity quotas are configurable per organization but ship with conservative defaults:

• Free tier: 10 calls/second, bucket capacity 50
• Starter: 50 calls/second, bucket capacity 200
• Growth+: 100 calls/second, bucket capacity 500 — configurable

Admins can adjust both the refill rate and bucket capacity from the dashboard. A common pattern: set a low rate for production environments, a higher rate for development/testing.

Alerting on sustained near-limit usage

If an organization operates above 80% of its velocity quota for more than 5 minutes, the system fires a warning alert. The assumption is that sustained near-limit usage indicates either a runaway agent or a legitimate scale need — either way, a human should look.

What we learned

The DynamoDB conditional update pattern works beautifully for token buckets but has one sharp edge: contention under high throughput. When dozens of Lambda invocations concurrently try to consume tokens, the ConditionalCheckFailedException rate spikes. For extreme throughput scenarios (500+ calls/second), we recommend sharding the bucket across multiple counters — but for the vast majority of use cases, a single DynamoDB item per organization handles the load without issue.