feat(workflows): add --dry-run flag to specify workflow run#3124
Open
fuleinist wants to merge 2 commits into
Open
feat(workflows): add --dry-run flag to specify workflow run#3124fuleinist wants to merge 2 commits into
fuleinist wants to merge 2 commits into
Conversation
Scoped to engine-only changes per maintainer design guidance: - StepContext.dry_run flag - RunState.dry_run field (persisted via save/load, restored on resume) - WorkflowEngine.execute(..., dry_run=...) + resume() restores it - CommandStep / PromptStep / GateStep short-circuit on context.dry_run with synthetic dry_run_message previews - specify workflow run --dry-run CLI flag + preview rendering - _coerce_options / _first_non_sentinel helpers for deterministic GateStep dry-run branch No new CLI commands introduced. --dry-run lives only on the step-based invocation path (specify workflow run), not on scaffolding commands. Closes: github#2661 Ref: github#2704 (closed per maintainer request, split into smaller PRs)
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a --dry-run mode to specify workflow run by propagating a dry-run flag through the workflow engine, persisting it in run state for resume, and teaching key step types (command, prompt, gate) to short-circuit and emit preview output instead of performing side effects.
Changes:
- Persist and propagate
dry_runacrossWorkflowEngine.execute(...),RunState.save/load, andresume(). - Add dry-run short-circuit behavior to
CommandStep,PromptStep, andGateStep. - Add
specify workflow run --dry-runCLI flag plus preview rendering, and introduce a dedicated dry-run test suite.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
tests/test_dry_run.py |
Adds coverage for step-level dry-run behavior, engine persistence/resume, and CLI output expectations (including --json behavior). |
src/specify_cli/workflows/steps/prompt/__init__.py |
Implements dry-run short-circuit for prompt steps and standardizes output fields (dry_run, executed, dispatched, etc.). |
src/specify_cli/workflows/steps/gate/__init__.py |
Implements dry-run non-interactive gate resolution with deterministic choice selection and option coercion helpers. |
src/specify_cli/workflows/steps/command/__init__.py |
Implements dry-run preview generation for command invocations without dispatching integration CLIs. |
src/specify_cli/workflows/engine.py |
Adds dry_run plumbing, persistence, and partial_state attachment for mid-run failures. |
src/specify_cli/workflows/base.py |
Adds dry_run to StepContext and documents intended preview semantics. |
src/specify_cli/__init__.py |
Adds the --dry-run flag, prints previews in non-JSON mode, and adds preview-print helper utilities. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
mnriem
requested changes
Jun 23, 2026
mnriem
left a comment
mnriem
left a comment
Collaborator
There was a problem hiding this comment.
Please address Copilot feedback
1. GateStep: preserve original output['message'] in dry-run (don't overwrite) 2. CommandStep: preserve original message, set dry_run_message separately 3. PromptStep: same pattern — dry_run_message separate from message 4. --dry-run help text: clarify not all step types short-circuit 5. base.py docstring: accurate about message vs dry_run_message contract 6. _print_dry_run_previews: escape Rich markup in preview text 7. JSON error path: emit valid JSON payload on engine exception 8. resume path: remove stale docstring claim about shared preview helper
Comment on lines
+61
to
+74
| # Dry-run short-circuit — emit a synthetic preview without | ||
| # dispatching to the integration CLI. | ||
| if context.dry_run: | ||
| preview = f"DRY RUN: would prompt integration {integration!r}: {prompt!r}" | ||
| output["exit_code"] = 0 | ||
| output["dispatched"] = False | ||
| output["executed"] = False | ||
| output["dry_run"] = True | ||
| output["dry_run_message"] = preview | ||
| # Preserve the original prompt so | ||
| # ``{{ steps.<id>.output.message }}`` keeps resolving to the | ||
| # original prompt text for downstream templates. | ||
| output["message"] = preview | ||
| return StepResult(status=StepStatus.COMPLETED, output=output) |
Comment on lines
+102
to
+111
| output["exit_code"] = 0 | ||
| output["dispatched"] = False | ||
| output["executed"] = False | ||
| output["dry_run"] = True | ||
| output["dry_run_message"] = preview | ||
| # Preserve the original command/integration/input so | ||
| # ``{{ steps.<id>.output.message }}`` keeps resolving to the | ||
| # original command description for downstream templates. | ||
| output["message"] = preview | ||
| output["invoke_command"] = preview_invocation or command |
Comment on lines
40
to
45
| if isinstance(message, str) and "{{" in message: | ||
| message = evaluate_expression(message, context) | ||
|
|
||
| options = config.get("options", ["approve", "reject"]) | ||
| options = self._coerce_options(config.get("options", ["approve", "reject"])) | ||
| on_reject = config.get("on_reject", "abort") | ||
|
|
Comment on lines
+139
to
+142
| for opt in options: | ||
| if opt not in ("reject", "abort"): | ||
| return opt | ||
| return options[0] |
Comment on lines
+77
to
+89
| #: When ``True``, the built-in step implementations | ||
| #: (``command`` / ``prompt`` / ``gate``) short-circuit and return a | ||
| #: synthetic ``StepResult`` carrying a preview of what would have | ||
| #: been dispatched — no subprocess, no CLI call, no network I/O for | ||
| #: those step types. Custom steps and built-in steps that have not | ||
| #: been updated to honor ``dry_run`` may still perform their normal | ||
| #: side effects; the flag is opt-in per step. Step implementations | ||
| #: publish the preview on ``output["dry_run_message"]`` (consumed | ||
| #: by the CLI's preview loop). ``output["message"]`` is also set to | ||
| #: the preview string during dry-run (it may differ from the real | ||
| #: run's ``message``). Downstream templates that need the original | ||
| #: value should reference the step's pre-dispatch config fields | ||
| #: (e.g. ``command``, ``prompt``) instead of ``output.message``. |
Comment on lines
+872
to
+876
| if dry_run and not json_output: | ||
| console.print( | ||
| "\n[bold yellow]DRY RUN:[/bold yellow] previewing without " | ||
| "dispatching any AI or shell commands." | ||
| ) |
Comment on lines
885
to
887
| except ValueError as exc: | ||
| console.print(f"[red]Error:[/red] {exc}") | ||
| raise typer.Exit(1) |
Comment on lines
+953
to
+962
| def _escape_markup(text: str) -> str: | ||
| """Escape Rich markup characters so a step ID can be printed safely. | ||
|
|
||
| Step IDs are user-controlled YAML; without escaping, an ID | ||
| containing ``[`` or ``]`` would raise ``MarkupError`` from Rich. | ||
| """ | ||
| return ( | ||
| text.replace("[", "\\[") | ||
| .replace("]", "\\]") | ||
| ) |
Comment on lines
+926
to
+934
| def _print_dry_run_previews(state: Any) -> None: | ||
| """Print the dry-run preview message emitted by each step. | ||
|
|
||
| Called by ``workflow run`` after a successful dry-run and from | ||
| exception handlers so a mid-run failure still surfaces the | ||
| previews resolved by earlier steps. Skipped silently when | ||
| ``state`` is ``None`` (e.g. the engine raised before any step | ||
| ran) or when the run did not include a dry-run step. | ||
| """ |
Collaborator
|
Please address Copilot feedback |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a --dry-run\ flag to \specify workflow run\ that previews step outputs without dispatching AI or shell commands. Per maintainer design guidance (\#2704), the flag lives only on the step-based invocation path — no new CLI commands introduced.
Changes
Engine layer
esume()\
Step implementations
CLI
Tests
Closes: #2661
Ref: #2704 (closed per maintainer request to split into smaller PRs)