feat: support timeouts in the send_request context helper (#2138) - #2164
feat: support timeouts in the send_request context helper (#2138)#2164aryansk wants to merge 1 commit into
Conversation
Sorry, something went wrong.
Expose the HTTP clients' existing per-request `timeout` support through `SendRequestFunction` so request handlers can bound how long an extra HTTP call may take. Adds a regression test against the slow test-server endpoint.
|
Hi! I've opened a PR that supersedes this one for #2138: #2174. It implements the same feature (exposing timeout through SendRequestFunction / context.send_request()), and additionally fixes a bug your draft would hit: PlaywrightHttpClient.send_request passes timeout.total_seconds() to Playwright's APIRequestContext.fetch, which expects milliseconds — so any timeout passed on a Playwright crawler would be ~1000x too short. The new PR also adds a regression test for that conversion plus the real end-to-end timeout test. Feel free to close this draft in favor of #2174, or let me know if you'd prefer to take the changes over. Thanks for the original implementation! |
Sorry, something went wrong.
Closes #2138
Summary
SendRequestFunction (used via context.send_request() in request handlers) only accepted url, method, payload and headers, so a handler could not bound how long an extra HTTP call may take. All four HTTP client implementations (HttpxHttpClient, CurlImpersonateHttpClient, ImpitHttpClient, PlaywrightHttpClient) already support a per-request timeout: timedelta | None — this PR just exposes it through the public contract.
Changes
Validation
Note: no new dependency; timeout uses the existing timedelta convention across the Python HTTP clients (the JS implementation names it timeoutMillis, but the Python clients are timedelta-based).