GH-49831: [Python] Withhold annotations from Python wheel until they are complete#50168
GH-49831: [Python] Withhold annotations from Python wheel until they are complete#50168rok wants to merge 7 commits into
Conversation
|
|
github-actions
Bot
added
Component: Python
awaiting committer review
There was a problem hiding this comment.
Pull request overview
This PR aims to prevent downstream type checkers from trusting PyArrow’s currently-incomplete bundled type stubs by withholding the PEP 561 py.typed marker from the built wheel (per GH-49831), while keeping stubs and py.typed in-tree for ongoing CI/type-checking work.
Changes:
- Configure the wheel build to exclude
pyarrow/py.typedviawheel.exclude. - Add explanatory comments documenting the rationale and removal TODO.
rok
force-pushed
the
broken-annotations-downstream
branch
from
325b6ac to
860479a
Compare
github-actions
Bot
added
awaiting changes
github-actions
Bot
added
awaiting change review
| # TODO(GH-49831): Temporarily do not install pyarrow-stubs into wheels. | ||
| # The stubs are incomplete, and some type checkers consume .pyi files even | ||
| # without the py.typed marker. Re-enable this when the stubs are complete. | ||
| # |
There was a problem hiding this comment.
You might wanna look into this repo as a follow up 🎁
There was a problem hiding this comment.
Oooh, if this works we can get rid of quite some logic, thanks!
There was a problem hiding this comment.
Ah great - hope so too!
If it doesn't, it could still be worth opening an issue since it's being actively developed - they might just not have coverage for cython yet
There was a problem hiding this comment.
I tested it and didn't get any docstrings of cythonic symbols, so I opened an issue astral-sh/docstring-adder#184.
github-actions
Bot
added
awaiting changes
dangotbanned
left a comment
dangotbanned
left a comment
There was a problem hiding this comment.
Thanks @rok
AFAICT, this should unblock narwhals-dev/narwhals#3560
Probably best to have someone familiar with your build process look this over, but LGTM
rok
force-pushed
the
broken-annotations-downstream
branch
from
1c4cec9 to
fe246a4
Compare
github-actions
Bot
added
awaiting change review
rok
changed the title
github-actions
Bot
added
awaiting change review
rok
force-pushed
the
broken-annotations-downstream
branch
from
7cc87e4 to
245beff
Compare
github-actions
Bot
added
awaiting change review
|
@github-actions crossbow submit wheel-manylinux-2-28-cp311-cp311-amd64 |
|
Revision: 245beff Submitted crossbow builds: ursacomputing/crossbow @ actions-25b0103e6b
|
rok
force-pushed
the
broken-annotations-downstream
branch
from
245beff to
5483c2f
Compare
|
@github-actions crossbow submit wheel-manylinux-2-28-cp311-cp311-amd64 |
|
Revision: 5483c2f Submitted crossbow builds: ursacomputing/crossbow @ actions-27d8a83445
|
|
@dangotbanned @raulcd I think this is ready for review. Stubs remain in the repo, they are checked but they are not packed into the wheel. |
dangotbanned
left a comment
dangotbanned
left a comment
There was a problem hiding this comment.
Thanks @rok, I still (#50168 (review)) approve 😉
Rationale for this change
Since 24.0, pyarrow ships annotations and
py.typedmarker while the type stubs are still incomplete. This makes type checkers trust the partial stubs and report false errors in downstream code (e.g. Module has no attribute "all" forpyarrow.compute). Some type checkers also consume bundled.pyifiles even withoutpy.typed, so the stubs need to be withheld from wheels for now. See #49831.What changes are included in this PR?
This temporarily omits both
pyarrow/py.typedand the bundledpyarrow-stubs/.pyifiles from built wheels until the stubs are complete. Wheel-content validation now asserts that neitherpy.typednor.pyifiles are present, and wheel build scripts no longer request stub docstring injection while stubs are not installed.Are these changes tested?
Wheel-content validation has been updated to check the intended absence of
py.typedand.pyifiles.Are there any user-facing changes?
Type checkers no longer pick up pyarrow's incomplete stubs from wheels.