[SC 16940] Fix matplotlib colormap API in statsmodels histogram tests

[AnilSorathiya]

Pull Request Description

What and why?

Replaces deprecated matplotlib.cm.get_cmap("viridis") with matplotlib.colormaps["viridis"] in three statsmodels visualization tests:

• PredictionProbabilitiesHistogram
• ScorecardHistogram
• CumulativePredictionProbabilities

Before: Tests failed on newer matplotlib versions with AttributeError: module 'matplotlib.cm' has no attribute 'get_cmap'.

After: Colormaps are resolved via the supported matplotlib API, and the histogram tests run successfully across Python 3.9-3.14.

Additional change: ModelPredictionResiduals KS test

This PR also updates the Kolmogorov-Smirnov normality test in ModelPredictionResiduals from the string-based scipy call:

kstest(residuals, "norm", args=(residuals.mean(), residuals.std()))

to passing an explicit CDF callable:

kstest(residuals, norm(loc=residuals.mean(), scale=residuals.std()).cdf)

This is equivalent for the current scipy API and produced the same statistic and p-value in local verification. It is not required for the matplotlib fix; it is included as a small cleanup to make the test more explicit and align with scipy's preferred callable-style usage.

Dependency fix for python 3.14 build

The failure comes from shap (in the [all] extra), not scorecardpy.

On Linux CI, shap declares numba and llvmlite without a minimum version. The resolver then picks the oldest compatible pair:

• numba==0.53.1
• llvmlite==0.36.0

Those versions only support Python up to 3.9/3.10, so install fails on Python 3.14.

Explicit Python 3.14 floors were added in pyproject.toml for the extras that pull in shap:

"numba (>=0.65.1) ; python_version >= '3.14'",
"llvmlite (>=0.47.0,<0.48) ; python_version >= '3.14'",

Older Python versions (3.9–3.13) are unchanged.

How to test

Run the affected unit tests locally:

python -m unittest tests.unit_tests.model_validation.statsmodels.test_PredictionProbabilitiesHistogram
python -m unittest tests.unit_tests.model_validation.statsmodels.test_ScorecardHistogram

Or run the full statsmodels unit test suite if preferred.

What needs special review?

Please review the dependency notes alongside the ModelPredictionResiduals cleanup. The kstest change is intentional and behavior-preserving; the scipy version constraint should remain aligned with the intended compatibility policy before merge.

Dependencies, breaking changes, and deployment notes

• No new dependencies
• No breaking changes
• Minor scipy API cleanup in ModelPredictionResiduals
• The scipy version constraint should be reviewed separately from the matplotlib API fix so it does not look accidental

Release notes

Fixes statsmodels histogram tests that failed on newer matplotlib versions due to removal of the deprecated cm.get_cmap() API.

Checklist

• What and why
• Screenshots or videos (Frontend)
• How to test
• What needs special review
• Dependencies, breaking changes, and deployment notes
• Labels applied
• PR linked to Shortcut
• Unit tests added (Backend)
• Tested locally
• Documentation updated (if required)
• Environment variable additions/changes documented (if required)

[github-actions]

Pull requests must include at least one of the required labels: internal (no release notes required), highlight, enhancement, bug, deprecation, documentation. Except for internal, pull requests must also include a description in the release notes section.

[github-actions]

Pull requests must include at least one of the required labels: internal (no release notes required), highlight, enhancement, bug, deprecation, documentation. Except for internal, pull requests must also include a description in the release notes section.

[github-actions]

Pull requests must include at least one of the required labels: internal (no release notes required), highlight, enhancement, bug, deprecation, documentation. Except for internal, pull requests must also include a description in the release notes section.

[juanmleng]

The matplotlib fix is correct — matplotlib.colormaps["viridis"] is the right replacement for the removed cm.get_cmap() API, and it's applied consistently across all three affected files. Clean.

One thing worth discussing before merging:

There's a quiet undocumented change in ModelPredictionResiduals.py that's a bit confusing alongside the scipy pin. The PR switches kstest from the old string-based call kstest(residuals, "norm", args=(...)) to kstest(residuals, norm(...).cdf) — which is correct and forward-compatible with scipy 2.0. But pyproject.toml simultaneously pins scipy <2.0. These two things pull in opposite directions: the code is updated as if scipy 2.0 is coming, but the pin keeps it out. Worth clarifying the intent — if scipy 2.0 has other unresolved issues in the library, the pin makes sense but the description should say so. If it doesn't, the <2.0 cap should probably be dropped.

The kstest change itself is fine and the new call is actually cleaner. Just needs a note in the PR description so it doesn't look accidental.

[johnwalz97]

lgtm

[github-actions]

PR Summary

This PR introduces two main sets of changes:

1. Dependency & Lockfile Updates:

   • The dependency resolution markers have been updated in the lock file (uv.lock) to better support Python 3.14. In particular, resolution markers for packages such as llvmlite, numba, and shap have been reworked to include platform and Python version checks. The changes ensure that the correct package version is selected based on whether the platform is macOS (darwin with x86_64), or other environments, and differentiate between Python versions 3.10, 3.12, 3.13, and 3.14+.
   • Unnecessary older package specifications (for example, older numba/llvmlite versions) have been removed and replaced with more up-to-date dependency markers, streamlining the compatibility conditions across multiple environments.

2. Test Code Adjustments:

   • In the test file for ModelPredictionResiduals, the KS test invocation was updated to use the cumulative distribution function (from scipy.stats.norm) directly after computing the mean and standard deviation, rather than passing a tuple of arguments. This change should improve clarity and correctness of the KS normality test.
   • Several test files under the statsmodels folder have been modified to replace the use of matplotlib’s cm.get_cmap("viridis") with the direct reference through matplotlib.colormaps["viridis"]. This change helps standardize the colormap access and possibly aligns with newer versions of matplotlib.

Overall, these changes ensure that dependency resolutions and test implementations are aligned with the new Python version requirements and modern library APIs.

Test Suggestions

• Run the full test suite to ensure that dependency resolution does not break installation; specifically test on Python 3.13 and Python 3.14 environments.
• Validate the output of the KS test in ModelPredictionResiduals to verify that the normality classification works as expected with the new cdf invocation.
• Render and visually inspect plots generated from the statsmodels test files to confirm that the updated colormap access (matplotlib.colormaps) produces correct visual results.
• Perform cross-platform tests (macOS vs Linux) to verify that the dependency markers correctly select the required package versions.

[AnilSorathiya]

This is equivalent for the current scipy API and produced the same statistic and p-value in local verification. It is not required for the matplotlib fix; it is included as a small cleanup to make the test more explicit and align with scipy's preferred callable-style usage.

Removed scipy <2.0 dependency.
This is equivalent for the current scipy API and produced the same statistic and p-value in local verification. It is not required for the matplotlib fix; it is included as a small cleanup to make the test more explicit and align with scipy's preferred callable-style usage.