feat(client): add idle-management methods to the composable pool Cache#295
Open
aajtodd wants to merge 1 commit into
Open
feat(client): add idle-management methods to the composable pool Cache#295aajtodd wants to merge 1 commit into
aajtodd wants to merge 1 commit into
Conversation
Add three methods to `client::pool` for managing idle cached services explicitly: - `Cached::discard(self)` consumes the checkout and drops the inner service without returning it to the pool. - `Cache::try_pop_idle()` removes and returns one idle service raw, with no return-to-pool wrapper. - `Cache::try_checkout_idle()` takes one idle service wrapped so it returns to the pool on drop, and makes no new service when none is idle. `Cache` previously exposed only `call` (take a service, making one when the pool is empty) and `retain` (bulk eviction by predicate). These cover two operations a pool consumer otherwise cannot express: dropping a checked-out service that has become unusable without forcing its `poll_ready` to error, and acting on existing idle capacity without making a new service. The methods are thin wrappers over the existing shared service list, serialized by the same lock as `call` and `retain`. `discard` sets the `is_closed` flag the `Drop` impl already checks. No existing behavior changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add three methods to
client::pool::Cache/Cachedfor managing idle cached services explicitly:Cached::discard(self)— consume the checkout and drop the inner service without returning it to the pool.Cache::try_pop_idle()— remove and return one idle service, raw (no return-to-pool wrapper).Cache::try_checkout_idle()— take one idle service wrapped so it returns to the pool on drop, without making a new one when none is idle.Motivation
Cachecurrently exposescall(take a service, making one if the pool is empty) andretain(bulk eviction by predicate). Two operations a pool consumer needs have no API:Drop a checked-out service that has become unusable. The only way to skip reinsertion on drop is for the inner service's
poll_readyto returnErr. A consumer that learns a service is bad after checkout — for a connection, the peer sentConnection: close, an HTTP/2 GOAWAY arrived on the response, or an error classifier fired — must either force a syntheticpoll_readyerror or let the known-bad service return to the pool.discardmakes the intent direct and leavespoll_readyfor readiness alone.Reach an idle service without making a new one.
callalways produces a service, starting a new one when the pool is empty. A consumer that wants to act on existing idle capacity only — release it to free the resource it holds, or hand it out for a single use — cannot express "idle if present, otherwise nothing."try_pop_idle(release) andtry_checkout_idle(single use, returns on drop) cover the two shapes.Solution
The three methods are thin wrappers over the existing shared service list, serialized by the same lock as
callandretain:Cached::discard(self)is_closedflag theDropimpl already checks, so the inner service is dropped instead of reinserted.Cache::try_pop_idle()Cache::try_checkout_idle()Cached, matching whatcallreturns for an idle hit. ReturnsNonerather than connecting when the pool is empty.No existing behavior changes.
Tests
Added to the
cachetest module, using the existingtower_test::mockharness:discardprevents reinsertion (cache empty after a discarded checkout).try_pop_idleremoves one idle service and does not reinsert it.try_checkout_idlereturns the service to the pool on drop and makes no new service (a single connection is reused).Noneon an empty cache.See also smithy-lang/smithy-rs#4708