ENG-1634: Add --build to state publish: build, encrypt, wrap, upload#3818
Open
mitchell-as wants to merge 2 commits into
Open
ENG-1634: Add --build to state publish: build, encrypt, wrap, upload#3818mitchell-as wants to merge 2 commits into
mitchell-as wants to merge 2 commits into
Conversation
Add a --build <srcdir> flag to state publish that, on the customer's own machine, packs the source tree into a wheel, encrypts it under the organization key, wraps the ciphertext together with a cleartext runtime.json into a tar.gz, and uploads only that wrapped artifact. Publishing without --build is unchanged. The org key is fetched and validated before anything is built, so a private publish fails closed (nothing is uploaded) when no key service is configured or the key is unavailable. The plaintext wheel lives only in a temp dir that is removed on every path. --build requires a private/<org> namespace. The wheel package is split so resolution and packing are distinct: ResolveMetadata reads pyproject.toml and applies --name/--version overrides, and Pack takes the resolved metadata and builds the wheel. artifactcrypto.Encrypt now rejects an over-long header (driven by the key id length) rather than emitting a payload its own reader would reject. The genesis/timeless publish flag is left as a TODO until the platform supports it (ENG-1641); until then --build is not genesis-stamped. ENG-1634 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds a --build <srcdir> workflow to state publish to build a wheel locally, encrypt it using the organization key, wrap it into an uploadable artifact, and publish it—ensuring the platform never sees plaintext source contents. This also refactors the internal wheel builder API to separate metadata resolution from packing, and hardens encrypted payload header validation.
Changes:
- Add
--buildflag plumbing and runner support for building/encrypting a private ingredient before upload. - Introduce wrapped-artifact build/encrypt implementation + unit tests for artifact contents, plaintext cleanup, and round-trip decryption.
- Refactor
internal/python/wheelto exposeResolveMetadataand require resolved metadata forPack; add artifactcrypto header-size guard + test.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| internal/runners/publish/publish.go | Adds config dependency and invokes encrypted artifact generation when --build is used. |
| internal/runners/publish/build.go | Implements build → encrypt → wrap flow and mutates publish params to point at the generated archive. |
| internal/runners/publish/build_test.go | Verifies wrapped artifact structure, encryption, plaintext cleanup, and decryptability. |
| internal/python/wheel/wheel.go | Changes Pack to require already-resolved metadata; updates docs and internals accordingly. |
| internal/python/wheel/wheel_test.go | Adjusts test variable naming to match updated implementation. |
| internal/python/wheel/metadata.go | Exports ResolveMetadata returning *Metadata and removes prior internal resolved type. |
| internal/python/wheel/metadata_test.go | Updates tests to use ResolveMetadata. |
| internal/python/wheel/distinfo.go | Updates metadata generation to take Metadata directly. |
| internal/artifactcrypto/encrypt.go | Rejects serialized headers that exceed the max readable header length. |
| internal/artifactcrypto/artifactcrypto.go | Adds exported ErrHeaderTooLarge error. |
| internal/artifactcrypto/artifactcrypto_test.go | Adds test ensuring oversize headers are rejected before reading the body. |
| cmd/state/internal/cmdtree/publish.go | Adds the --build flag to the CLI command definition. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
A --build artifact is encrypted under the project org's key, so it must be published under that same org or its consumers cannot decrypt it. Validate the resolved namespace (flag, meta file, or default) against private/<owner> in one place, after namespace resolution, rather than only checking the flag for a private/ prefix. Also wrap the wheel.Pack error with errs.Wrap (it is not an input error), fix a stale cleanup comment, and close the gzip reader in the test helper. ENG-1634 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Collaborator
Author
|
macOS and Windows test failures are sporadic/known and are unrelated to this PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ENG-1634: Add --build to state publish: build, encrypt, wrap, upload
The orchestration that ties the private ingredient publish side together (ENG-1563).
state publish --build <srcdir>packages a customer's source on their own machine, locks it with the organization's key, and uploads only the locked result — the hosting service never sees the unlocked content. If the key is unavailable it fails before uploading anything, and the plaintext only ever lives in a temporary directory. Publishing without--buildbehaves exactly as before.Notes: A private publish isn't genesis-stamped yet — that depends on a platform-side flag (ENG-1641), so it's left as a TODO. And the encrypted payload carries the wheel file as-is; turning it into an installed package is the consume side's job.
Base branch: targets
mitchell/eng-1633so the diff is only this change; GitHub will retarget it toversion/0-48-1-RC2once the upstream PRs land.Tested for the wrapped-artifact contents, round-trip decryption, plaintext cleanup, and metadata resolution.
🤖 Generated with Claude Code