Home → Help
socket hang up in the middle of a streamed response
The TCP connection was closed before the stream finished. Nothing in the payload tells you who closed it, so the distinction you need is whether a finish_reason ever arrived — without one, treat the answer as incomplete and do not cache it.
What you are seeing
Error: socket hang upECONNRESET / read ECONNRESETaiohttp.ClientPayloadError: Response payload is not completedThe text stops mid-sentence with no finish_reason
Why it happens
Long streams cross more infrastructure than short requests: your network, any proxy, the provider's edge and the model host. An idle-timeout anywhere in that chain closes the connection, and the client reports it as a reset rather than an HTTP error, because the HTTP response already began with a 200.
Mobile networks and laptop sleep produce the same signature. So does a load balancer with a shorter idle timeout than the model's thinking time, which is why reasoning models hit this more than fast ones.
A stream that ends with a proper finish_reason and then errors is a different thing: the answer is complete and only the teardown failed. That one is safe to use.
How to fix it
- Record finish_reason as the completion signalDo not treat 'the stream ended' as success. Mark a response complete only when a finish_reason arrived; otherwise retry or surface it as truncated.
- Retry the request, not the streamThere is no resume point in a streamed completion. Retry from the beginning, ideally with a lower max_tokens so the next attempt finishes sooner.
- Raise client and proxy timeouts togetherRaising only the client timeout does nothing if a proxy in front closes first. Both have to exceed the slowest response you expect.
A stream that dies partway is still billed for the tokens that were generated — the work happened. Your APICLAN usage log shows exactly how many, which is how you tell a half-finished answer from one that never started.
Related
Last checked 2026-10-01. Written from problems diagnosed on a live
OpenAI-compatible gateway, not collected from other sites.