Quickstart Pricing

Home → Help

524 timeout on long API requests

A 524 is produced by a reverse proxy or CDN sitting in front of the API, not by the model. It fires when the origin has not sent the first byte of a response within a fixed window — commonly 100 seconds.

What you are seeing

Why it happens

The important detail is that the clock measures time to first byte, not total duration. A streaming response starts emitting almost immediately, so it can run for many minutes without ever tripping the limit. A non-streaming request with a long reasoning phase sends nothing at all until it is completely finished, and that is exactly the shape that gets cut off.

This is why the failure looks random. The same prompt succeeds when it happens to finish quickly and fails when the model thinks for longer, and it correlates with reasoning depth rather than with anything you changed.

Confirm it is this

Run the same request with streaming on and off. If one survives and the other does not, you have found it:

# non-streaming — vulnerable to the time-to-first-byte limit
curl -sS -X POST 'YOUR_BASE_URL/chat/completions' \
  -H 'Authorization: Bearer YOUR_KEY' \
  -H 'Content-Type: application/json' \
  -d '{"model":"MODEL","stream":false,"messages":[...]}'

# streaming — first byte arrives in well under a second
  -d '{"model":"MODEL","stream":true,"messages":[...]}'

Streaming also gives you a far better failure mode: if something goes wrong midway you still have the partial output, instead of losing the whole call.

How to fix it

  1. Turn streaming onThis fixes the problem outright for almost every workload, and it is a one-line change. Most SDKs and agent tools stream by default already.
  2. Use a direct endpoint if the provider offers oneSome providers publish an API host that bypasses their CDN specifically to avoid this limit. If long non-streaming calls are unavoidable, that is what it is for.
  3. Split the workIf a single call genuinely needs minutes of reasoning before producing anything, it is usually cheaper and more reliable to break it into stages you can checkpoint.
On APICLAN, streaming requests are never buffered, so they are not affected. For long non-streaming calls there is a direct endpoint at https://api.apiclan.us/v1 with no such limit.

Related

Unexpected token '<' when calling an OpenAI-compatible API401 invalid API key — when the key looks right but still fails

Last checked 2026-08-24. Written from problems diagnosed on a live OpenAI-compatible gateway, not collected from other sites.