Capacity

Understand how Radium handles traffic and what to do when you encounter transient capacity issues.

Radium does not currently enforce fixed concurrency limits, queue depth limits, or tier-based capacity quotas. Capacity controls may be introduced in the future.

Detecting throttling

Every response includes these headers so you can monitor infrastructure health:

Header	Description
`X-Queue-Depth`	Current queue depth for this model (always `0` today)
`X-Queue-Wait-Ms`	Estimated wait time if queued (always `0` today)

import requests

response = requests.post(
    "https://api.radium.cloud/v1/chat/completions",
    headers={"Authorization": f"Bearer {key}"},
    json={"model": "hal-1.0", "messages": [{"role": "user", "content": "Hello"}]}
)

print("Queue depth:", response.headers.get("X-Queue-Depth"))
print("Queue wait (ms):", response.headers.get("X-Queue-Wait-Ms"))

Handling transient errors

During traffic spikes or infrastructure maintenance you may receive a 429 Too Many Requests or 503 Service Unavailable. These are retryable.

Status	Meaning	Action
`429`	Rate limit hit (rare today)	Back off with exponential jitter
`503`	Temporary capacity issue	Retry with exponential backoff
TTFT spikes	Model overloaded	Check status page, consider failover

Retry strategy

Use exponential backoff with jitter when you encounter 429 or 503:

import random, time

def with_retry(fn, max_retries=5):
    for attempt in range(max_retries):
        try:
            return fn()
        except APIError as e:
            if e.status_code in (429, 503) and attempt < max_retries - 1:
                delay = (2 ** attempt) + random.random()
                time.sleep(delay)
            else:
                raise

Managing concurrency in your code

Even without fixed limits on our side, you should cap concurrent requests in your own integration to avoid overwhelming downstream dependencies. A semaphore is the standard pattern:

import asyncio

SEMAPHORE = asyncio.Semaphore(10)  # adjust to your own safe concurrency level

async def safe_request(client, ...):
    async with SEMAPHORE:
        return await client.chat.completions.create(...)

Capacity planning

Estimate your required concurrency before going live:

Required concurrent = (requests_per_second × avg_response_time_seconds)

If you expect sustained high-volume traffic, contact sales so we can plan capacity together.

Status page

Real-time capacity and incident information is available at status.radium.cloud. Subscribe to the RSS feed or webhook for automated alerting:

curl https://status.radium.cloud/feed

Getting started

Models

Switching

Core concepts

Tools

Guides

Enterprise

Capacity

Capacity

Detecting throttling

Handling transient errors

Retry strategy

Managing concurrency in your code

Capacity planning

Status page

​Capacity

​Detecting throttling

​Handling transient errors

​Retry strategy

​Managing concurrency in your code

​Capacity planning

​Status page

Capacity

Detecting throttling

Handling transient errors

Retry strategy

Managing concurrency in your code

Capacity planning

Status page