Skip to content

feat(client): add idle-management methods to the composable pool Cache#295

Open
aajtodd wants to merge 1 commit into
hyperium:masterfrom
aajtodd:feat/cache-idle-management
Open

feat(client): add idle-management methods to the composable pool Cache#295
aajtodd wants to merge 1 commit into
hyperium:masterfrom
aajtodd:feat/cache-idle-management

Conversation

@aajtodd

@aajtodd aajtodd commented Jun 27, 2026

Copy link
Copy Markdown

Add three methods to client::pool::Cache / Cached for managing idle cached services explicitly:

  • Cached::discard(self) — consume the checkout and drop the inner service without returning it to the pool.
  • Cache::try_pop_idle() — remove and return one idle service, raw (no return-to-pool wrapper).
  • Cache::try_checkout_idle() — take one idle service wrapped so it returns to the pool on drop, without making a new one when none is idle.

Motivation

Cache currently exposes call (take a service, making one if the pool is empty) and retain (bulk eviction by predicate). Two operations a pool consumer needs have no API:

  1. Drop a checked-out service that has become unusable. The only way to skip reinsertion on drop is for the inner service's poll_ready to return Err. A consumer that learns a service is bad after checkout — for a connection, the peer sent Connection: close, an HTTP/2 GOAWAY arrived on the response, or an error classifier fired — must either force a synthetic poll_ready error or let the known-bad service return to the pool. discard makes the intent direct and leaves poll_ready for readiness alone.

  2. Reach an idle service without making a new one. call always produces a service, starting a new one when the pool is empty. A consumer that wants to act on existing idle capacity only — release it to free the resource it holds, or hand it out for a single use — cannot express "idle if present, otherwise nothing." try_pop_idle (release) and try_checkout_idle (single use, returns on drop) cover the two shapes.

Solution

The three methods are thin wrappers over the existing shared service list, serialized by the same lock as call and retain:

Method Behavior
Cached::discard(self) Sets the is_closed flag the Drop impl already checks, so the inner service is dropped instead of reinserted.
Cache::try_pop_idle() Pops one entry from the idle list and returns it raw.
Cache::try_checkout_idle() Pops one entry and wraps it in Cached, matching what call returns for an idle hit. Returns None rather than connecting when the pool is empty.

No existing behavior changes.

Tests

Added to the cache test module, using the existing tower_test::mock harness:

  • discard prevents reinsertion (cache empty after a discarded checkout).
  • try_pop_idle removes one idle service and does not reinsert it.
  • try_checkout_idle returns the service to the pool on drop and makes no new service (a single connection is reused).
  • both takes return None on an empty cache.

See also smithy-lang/smithy-rs#4708

Add three methods to `client::pool` for managing idle cached services
explicitly:

- `Cached::discard(self)` consumes the checkout and drops the inner
  service without returning it to the pool.
- `Cache::try_pop_idle()` removes and returns one idle service raw, with
  no return-to-pool wrapper.
- `Cache::try_checkout_idle()` takes one idle service wrapped so it
  returns to the pool on drop, and makes no new service when none is
  idle.

`Cache` previously exposed only `call` (take a service, making one when
the pool is empty) and `retain` (bulk eviction by predicate). These cover
two operations a pool consumer otherwise cannot express: dropping a
checked-out service that has become unusable without forcing its
`poll_ready` to error, and acting on existing idle capacity without
making a new service.

The methods are thin wrappers over the existing shared service list,
serialized by the same lock as `call` and `retain`. `discard` sets the
`is_closed` flag the `Drop` impl already checks. No existing behavior
changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant