Follow-up to #2154. v2.0.3 telemetry showed the venv BUILD_FAILED bucket
splits into two unexplained groups; this PR instruments both.
## 1. The exc: bucket — exception type + errno
The dominant remaining venv BUILD_FAILED (phase=venv, err=99) is ~99%
sdk_bootstrap_stderr_sig=NULL — Python exceptions caught by the generic
`except Exception` ("exc:<TypeName>"), not CalledProcessErrors with
categorizable stderr. ~56k/30h, all opaque (stderr_sig only covers
"other:<tail>").
- Handler embeds errno for OSError-family: "exc:OSError:28", etc.
- SDK_BOOTSTRAP_EXC_CODES maps the type → sdk_bootstrap_exc
(FileNotFoundError=1 … OSError=6 … 99=other).
- errno decoded → sdk_bootstrap_errno (ENOENT/EACCES/ENOSPC/…).
## 2. venv_ensurepip_fail instrumentation (the other category)
venv_ensurepip_fail (code 11) is the top categorizable venv failure, and
telemetry flipped the naive assumption: it's NOT just Debian/Ubuntu —
macOS has the MOST distinct affected users (466 vs 121 linux), and linux
is a retry storm (~172 fires/user). Before committing to a `pip install
--target` fallback (Option A) we need to know (a) which interpreter these
users run and (b) whether that interpreter even has pip (→ whether
--target would work, vs needing a system package).
- sdk_hook_py (always emitted): interpreter version as major*100+minor
(309/312). Disambiguates Apple-3.9 vs a 3.10+-with-broken-ensurepip,
and also recovers the version for HOOK_PY_INCOMPATIBLE (whose "py_3.9"
err_kind otherwise collapses to err=99).
- sdk_has_pip (only on err==11, to avoid an extra subprocess per healthy
session): whether `<interpreter> -m pip --version` works. has_pip=true
→ the --target fallback would fix them; has_pip=false → they need a
system package (python3-venv / a complete Python).
Both #1 and #2 are purely additive telemetry on the existing BUILD_FAILED
path — no behavior change to the bootstrap. They de-risk the Option A
decision: ship A only if the affected cohort has pip.
Verified locally on macOS Python 3.13:
- py_compile clean.
- 39 tests in test_exc_failure_encoding.py (34 exc/errno + 5 ensurepip
instrumentation): type-code map, errno extraction + round-trip,
APPEND-ONLY stability, handler-embeds-errno, _probe_has_pip returns
bool + true-on-this-machine, sdk_hook_py always-emitted as
major*100+minor, sdk_has_pip gated on err==11.
- Full suite: 503/503 pass + 2 skipped.
Version 2.0.3 -> 2.0.4 per the per-PR-bump policy (#2114).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fixes#2159. The Stop hook emits feedback via
`hookSpecificOutput: {hookEventName: "Stop", additionalContext}`, but
`Stop`/`SubagentStop` are NOT members of CC's `hookSpecificOutput`
discriminated union (coreSchemas.ts — valid members are PreToolUse,
PostToolUse, UserPromptSubmit, SessionStart, etc.). So the emitted shape
violates CC's documented hook-output schema.
Impact is CC-version-dependent — important nuance, established empirically:
- Reporter (array0224-cloud) on CLI 2.1.150 / 2.1.152: CC rejects the
Stop feedback; the block/reason never reaches the model, so the
auto-rewake/fix loop is lost. (Detection still runs + logs.)
- On CLI 2.1.160 (current) the asyncRewake completion path is lenient:
its gate is `isSyncHookJSONOutput` (hooks.ts) which is just
`!(json.async === true)` — NOT a strict schema parse. So the invalid
hookSpecificOutput is tolerated: metrics + rewakeSummary are still
consumed and the model still receives the findings. I could NOT
reproduce the rejection on 2.1.160, and BQ confirms Stop-path
vulns_found metrics are recorded normally (~21k with-vuln fires / 3d),
i.e. NOT dropped. (An earlier draft of this message claimed metrics
were dropped — that was wrong; corrected after checking telemetry +
repro'ing the old plugin on 2.1.160.)
So this is defensive schema-correctness: the plugin should emit output
that conforms to CC's documented union regardless of how strictly a given
CC version validates it. The reporter's environment validates strictly;
relying on the current version's leniency is fragile.
Fix (CC's documented asyncRewake "clean pattern" — hooks.ts: "error text
on stderr, JSON on stdout"):
- For Stop/SubagentStop, emit_metrics writes guidance to stderr (the
asyncRewake body channel CC delivers via `stderr || stdout`) and sets
top-level `decision: "block"` + `reason` (valid SyncHookJSONOutput
fields; also the documented sync Stop-hook contract for the `-p`
fallback). It does NOT emit a Stop hookSpecificOutput.
- PostToolUse (commit-review, push-sweep) is unchanged — valid union
member, keeps the modern hookSpecificOutput protocol.
Verified locally on macOS Python 3.13:
- py_compile clean.
- 11 new tests (test_2159_stop_hook_schema.py) pin the contract: Stop
output carries no hookSpecificOutput, uses top-level decision/reason,
writes guidance to stderr; the emitted hookEventName (when present) is
a valid union member. 2 existing tests that asserted the buggy
Stop->hookSpecificOutput shape were corrected. Full suite 464/464
pass + 2 skipped.
- END-TO-END in /tmux on CLI 2.1.160:
* FIXED plugin (2.0.3): staged pickle.loads + os.system, benign edit
pulls the file into review_set; Stop LLM review found 2 critical
vulns; CC delivered a clean rewake ("Background security review
found issues" + both findings). All hooks (UPS, PostToolUse[Edit]
pattern, PostToolUse[Bash] commit-review, Stop) fired clean; zero
schema rejections / errors / http_err in the debug log.
* OLD plugin (2.0.2) on the SAME 2.1.160: also delivered Stop feedback
(confirming the no-repro-on-latest finding above) — which proves the
fix carries NO regression risk on current CC while making the output
robust for the stricter versions where it actually breaks.
Version bumped 2.0.2 -> 2.0.3 per the per-PR-bump policy (#2114: a bump is
the only way the fix reaches the existing fleet — relevant for users still
on the CC versions where this breaks).
Closes#2159.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR #2112's telemetry visibility surfaced an immediate finding from
the first 3h of v2.0.1 data: **2,406 phase=2 / err=99 sessions** —
"venv stage / uncategorized" — dominating BUILD_FAILED. The original
err_kind detection patterns were all pip-flavored (pip_no_match,
dns_fail, ssl_verify, etc.) and didn't catch venv-creation failure
modes, so they all collapsed to the catch-all _uncategorized (99)
bucket.
This PR fills the gap on two axes.
## 1. Five new venv-specific err_kind categories (codes 11-15)
Each gated on `err_phase == "venv"` so the same substring doesn't
mis-fire in pip-phase failures:
- 11 `venv_ensurepip_fail` — Debian/Ubuntu without python3-venv
installed; stderr matches "ensurepip is not available" or
"ensurepip ... returned non-zero". Predicted to be the biggest
chunk based on Linux distro market share.
- 12 `venv_path_too_long` — Windows MAX_PATH (260) or POSIX
ENAMETOOLONG. Triggered when state_dir + venv layout exceeds
the path limit (deep Lib/site-packages/<pkg>/<...> paths).
- 13 `venv_no_module` — `python3 -m venv` itself missing
("No module named 'venv'"). Rare but distinctive.
- 14 `venv_already_exists` — Errno 17 / "file exists" — sentinel
race past O_EXCL or stale dir survived `--clear`.
- 15 `venv_setup_failed` — generic "virtual environment was not
created successfully" catch-all for venv setup failures that
don't match a more specific category.
All 5 occupy reserved slots in SDK_BOOTSTRAP_ERR_CODES per the
APPEND-ONLY contract from PR #2112.
## 2. `sdk_bootstrap_stderr_sig` integer hash
For "other:<tail>" err_kinds (which encode to _uncategorized = 99),
emit a bounded integer hash (0-999) of the first ~30 chars of the
stderr tail. This restores cardinality to the _uncategorized bucket
in BQ aggregation without unbounded keyspace — same stderr message
always maps to the same bucket, so a real failure mode replicating
across thousands of machines clusters cleanly. Bounded at 1000
buckets: well below any "high cardinality" alarm but wide enough to
distinguish ~30 distinct dominant patterns (birthday-paradox
collision probability ~50% at ~37 distinct inputs).
The field auto-omits (`if sig:` gate) when err_kind is categorized
— no key-budget cost on the common-case categorized failures.
## Version bump 2.0.1 → 2.0.2
PR #2114 confirmed the version-bump mechanism is the only way to
propagate code changes to the existing fleet — without a bump, CC's
plugin updater short-circuits on string-equality of installation
version vs marketplace version. Following the policy we established:
**bump patch on every functional PR**.
By 17:31:42Z on 2026-06-01 (1m22s after #2114 merged), v2.0.1 was
already appearing in BQ. v2.0.2 should follow the same propagation
curve — ~30% adoption within 3 hours, full convergence within a few
days.
## Verified locally
- py_compile clean.
- 15 new tests in test_venv_failure_deepdive.py (added to internal
test suite at sg-staging/tests/, not in this PR):
* 5 parametrized: each new err_kind maps to its expected code (11-15).
* 1 APPEND-ONLY regression: existing codes 1-10 + 99 unchanged.
* 6 stderr_sig: non-other inputs → 0; None/empty → 0; deterministic
same-input → same-output; bounded to 0-999; distinct inputs →
distinct hashes (5/5 with P(collision) ≈ 1%); leading-chars focus
(path-varying stderr with shared 30-char prefix collide as designed).
* 1 static-shape catcher: every new `err_kind = "venv_..."` branch
in main() is guarded by `err_phase == "venv"`. Catches the
regression where someone adds a venv pattern without the phase
gate and starts mis-categorizing pip-phase failures.
* 1 map-coverage: all err_kind strings assigned anywhere in
ensure_agent_sdk.main() are present in SDK_BOOTSTRAP_ERR_CODES
(catches new categories added in code but forgotten in the map).
* 1 emit-shape: the metric block uses `_encode_stderr_sig`, the
`sdk_bootstrap_stderr_sig` key is written conditionally on `if
sig:`. Catches the regression where someone removes the
helper or makes the emit unconditional (would pad every
categorized BUILD_FAILED row with a zero-valued field).
- Full suite: 452/452 pass + 2 skipped (live API tests, opt-in).
## What this unblocks in BQ
```sql
-- For the 2,406 sessions/3h that were phase=2/err=99 on v2.0.1,
-- v2.0.2+ will split them across the new categories. Query:
SELECT
CAST(JSON_VALUE(additional_metadata, "$.sdk_bootstrap_err") AS INT64) AS err,
CAST(JSON_VALUE(additional_metadata, "$.sdk_bootstrap_stderr_sig") AS INT64) AS sig,
COUNT(*) AS sessions
FROM `proj-product-data-nhme.raw_events.claude_code_internal_event`
WHERE _PARTITIONTIME >= ...
AND CAST(JSON_VALUE(additional_metadata, "$.sdk_bootstrap") AS INT64) = 3
AND CAST(JSON_VALUE(additional_metadata, "$.sdk_bootstrap_phase") AS INT64) = 2 -- venv
GROUP BY err, sig
ORDER BY sessions DESC
```
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The 8 PRs we shipped since 2026-05-26 (#2076, #2077, #2078, #2086,
#2091, #2100, #2101, #2105) all changed plugin code without bumping
the version. CC's plugin updater uses string equality for the
freshness check (pluginOperations.ts:1835):
const isUpToDate =
installation.version === newVersion ||
installation.installPath === versionedPath ||
installation.installPath === zipPath
if (isUpToDate) return { alreadyUpToDate: true }
Users who installed v2.0.0 anywhere between 2026-05-26 and 2026-05-31
have `installation.version === "2.0.0"` in their installed_plugins.json.
The marketplace also advertises "2.0.0" (until this commit), so
isUpToDate returns true and the plugin cache directory is never
refreshed — they keep running whatever 2.0.0 code was current on the
day they installed. The marketplace git pull happens; the per-user
cache install does NOT.
Empirical evidence: in BQ today (5/31) on Windows v2.0.0 fires,
**73% emit sdk_bootstrap outcome 4 (SKIP_WIN32)** — a code path
retired in PR #2055's Windows-enable fix. Those users are running a
plugin tree that pre-dates the fix, even though their telemetry
shows pv=20000.
The fix is a one-line version bump. Once the marketplace advertises
2.0.1, every CC autoupdate cycle sees installation.version (2.0.0)
!= newVersion (2.0.1), installs the new version, and the user's next
session loads the fixed code.
This PR:
1. plugins/security-guidance/.claude-plugin/plugin.json: 2.0.0 → 2.0.1
2. .claude-plugin/marketplace.json security-guidance entry: 2.0.0 → 2.0.1
What 2.0.1 carries (versus 2.0.0 as published 5/26):
- #2076 — Graphite gt commit/push detection
- #2077 — hookSpecificOutput.additionalContext on async-rewake exit-2
- #2078 — CLAUDE_CONFIG_DIR support
- #2086 — core.quotePath=false on diff feeders (Arabic/Hebrew/CJK paths)
- #2091 — fix Bash(...|...) if-clause regression from #2076
- #2100 — drop text=True from subprocess.run, bake PYTHONUTF8=1 (Windows non-cp1252 path crash)
- #2101 — core.quotePath=false on GIT_CMD globally
- #2105 — output_format → output_config.format API migration (#2098)
Verified locally:
- plugin.json + marketplace.json both valid JSON.
- _read_plugin_version_int() returns 20001 (was 20000).
- Existing test suite passes — 408 tests, no regressions caused by
the version bump itself. (29 unrelated failures are from
test_telemetry_failure_signals.py which expects PR #2112's
not-yet-merged code.)
Going forward: bumping `patch` on every functional PR closes this
gap entirely. Without that policy, every fix only reaches NEW
installs, never the existing fleet.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
