Warn instead of raising when the connection pool is smaller than the thread pool#753
Open
ajaynomics wants to merge 1 commit into
Open
Warn instead of raising when the connection pool is smaller than the thread pool#753ajaynomics wants to merge 1 commit into
ajaynomics wants to merge 1 commit into
Conversation
…thread pool Solid Queue refused to boot when the Active Record connection pool was smaller than the worker thread pool. This blocked legitimate setups, such as I/O-bound queues that run many threads against a deliberately small pool. Per the discussion in rails#736, downgrade the check from a validation error to a SolidQueue.logger warning emitted once on the boot path in Supervisor.start, so undersized configurations boot while still surfacing the advisory. valid? stays purely about whether we can boot. Closes rails#736
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Solid Queue refuses to boot when the database connection pool is smaller than the worker thread pool, raising a
ConfigurationError. This PR makes the check advisory: Solid Queue boots and logs a warning instead.Why
The hard cap rules out configurations that work well in practice. The reporter in #736 runs an I/O-bound queue — 150 threads making slow HTTP calls against a deliberately small pool — where each thread holds a connection only briefly. Today, booting that setup requires monkey-patching the check away.
As suggested in #736, this replaces the hard failure with a warning, which the reporter confirmed fits the need. (The issue also floated a config flag to bypass the check; an unconditional warning is simpler and adds no new configuration surface.)
How
ensure_correctly_sized_thread_poolfrom the validation chain, soConfiguration#valid?reflects only whether the configuration can boot.Configuration#warn_about_undersized_thread_pool, which logs the advisory throughSolidQueue.logger.Supervisor.start, just after thevalid?check. Thebin/jobsCLI and both Puma plugin modes — forked and async — start throughSupervisor.start, so the warning fires once on every boot path.The warning text matches the previous error, so existing log filters and operator expectations still hold.
Tests
async_supervisor_test: booting with more threads than connections logs the warning; booting with enough connections stays silent. (Removing the call fromSupervisor.startfails the first test.)configuration_test: an undersized pool no longer makes the configuration invalid.Closes #736.