fix: Correct pool ownership and close pooled connections synchronously#72
Open
jsonbailey wants to merge 1 commit into
Open
fix: Correct pool ownership and close pooled connections synchronously#72jsonbailey wants to merge 1 commit into
jsonbailey wants to merge 1 commit into
Conversation
_HttpClientImpl owned the wrong pool: the ownership flag was inverted in the 2023 ConnectStrategy refactor (c516f1f), so close() cleared a caller-supplied pool instead of one it created. On urllib3 2.x, PoolManager.clear() no longer closes connections synchronously (sockets close at GC via weakref.finalize), so a caller's streaming connection lingered until garbage collection. - Restore conventional ownership: close only a pool we created (params.pool is None), matching the async client's existing behavior. - Close each HTTPConnectionPool synchronously before clear() so the TCP FIN is sent immediately instead of at GC. - Log close errors at debug instead of swallowing them. Adds regression tests covering both halves of the ownership contract.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
_HttpClientImplowned the wrong pool. The ownership flag was inverted in the 2023 ConnectStrategy refactor (c516f1f8): it was renamed fromhttp_pool is None(close a pool the library created) toparams.pool is not None(close a caller-supplied pool). Combined with urllib3 2.x — wherePoolManager.clear()no longer closes connections synchronously (nodispose_func; sockets close viaweakref.finalizeat GC time) — this meant:clear()-ed out from under the caller onclose(), and the underlying streaming socket's TCP FIN was deferred to garbage collection.The async client (
async_http.py) already uses the correct rule (external_session is None), so this also brings the sync client back into line with it.Changes
__should_close_pool = params.pool is None— own only the pool we created; leave a caller-supplied pool to the caller.close()closes eachHTTPConnectionPoolsynchronously (sends the FIN now) beforeclear().test_http_connect_strategy.pyfor both halves of the ownership contract (there was previously zero coverage, which is why the inversion survived since 2023).Why it matters
The LaunchDarkly server SDKs supply their own pool for FDv2 streaming. On the FDv1 fallback directive the SDK must close the streaming connection within 3s (spec 1.6.3(2)). Because the FIN was GC-deferred, the FDv2 contract test
directive on streaming success applies payload then engages FDv1failed on slow CI runners (Linux + Python 3.10) where GC slipped past the 3s window.Verification
Instrumented FDv1-fallback contract run under Python 3.10:
weakref.finalize(GC)GC-deferred (~560 ms, variable) → synchronous (~0.6 ms, deterministic), well within the 3s window regardless of machine speed. Full unit suite: 126 passed.
Jira: SDK-2600
Note
Medium Risk
Touches connection lifecycle for all sync HTTP SSE clients; behavior change is intentional but affects SDK streaming shutdown timing and caller-supplied pool safety.
Overview
Fixes inverted
PoolManagerownership in_HttpClientImpl:close()now tears down only pools the client created (params.pool is None), matching the async client’sexternal_session is Nonerule. Caller-supplied pools are no longerclear()’d on close.When the client owns the pool,
close()now closes eachHTTPConnectionPoolbeforePoolManager.clear(), so streaming TCP connections get a FIN immediately instead of lingering until urllib3 2.x GC/finalizers. Close failures are logged at debug.Adds regression tests for both ownership paths (caller pool untouched vs. internally created pool closed + cleared).
Reviewed by Cursor Bugbot for commit ce86c6e. Bugbot is set up for automated code reviews on this repo. Configure here.