security-guidance: migrate from deprecated output_format to output_config.format (#2098)

Fixes #2098. The Anthropic Messages API moved structured-output
schema specification from a top-level `output_format` field to a
nested `output_config.format` field, per
https://platform.claude.com/docs/en/build-with-claude/structured-outputs.

Per docs the old form "will continue working for a transition period"
— and indeed for api-key + non-streaming auth it still returns HTTP
200 (verified via live API). But OAuth Bearer users with CLI 2.1.158
hit `invalid_request_error: output_format: This field is deprecated.
Use 'output_config.format' instead.` consistently — reporter saw 462
errors in one day. The trigger appears to be auth mode + possibly
stream:true (their controlled curl bypass used Bearer + stream=true);
api-key + non-streaming was my initial repro attempt and didn't fire.

The bug only affected `_call_claude` (the legacy direct-urllib path).
The agentic `_agentic_review` path goes through claude_agent_sdk →
subprocesses to the `claude` CLI binary, which already uses the new
`output_config.format` shape correctly (per src/utils/sideQuery.ts:263
in claude-cli-internal). So this PR only needs to fix the plugin's
direct HTTP path.

This commit:

1. llm.py: rewrite the payload literal in `_call_claude` to use
   `output_config: { format: { type: 'json_schema', schema: ... } }`
   instead of top-level `output_format`.

2. llm.py: in the adaptive-thinking branch, MERGE `effort: "high"`
   into the existing `output_config` dict instead of reassigning.
   Reassignment would silently clobber the format schema set in (1).
   The pre-existing code did `payload["output_config"] = {"effort":
   "high"}` which was correct WHEN output_format was top-level (and
   output_config wasn't otherwise used). With the migration the
   existing dict carries the schema, so we extend it not replace it.

Verified locally on macOS Python 3.13:

  - py_compile clean.
  - Existing 401 tests still pass — 0 regression.
  - 6 new tests in test_2098_output_config_format.py (added to
    internal test suite at sg-staging/tests/, not in this PR):

      * 2 static-shape: the `_call_claude` source no longer contains
        top-level `"output_format":` AND uses `output_config`. The
        adaptive-thinking branch does NOT reassign output_config (and
        DOES set output_config['effort']). Catches the regression
        class where a future refactor reintroduces either bug.
      * 2 payload-shape unit (mocked urllib): both thinking_budget=0
        and thinking_budget>0+adaptive code paths produce a payload
        with the correct `output_config.format` shape AND no
        `output_format` top-level. The adaptive path verifies both
        `format` and `effort` coexist in output_config (i.e., the
        merge fix works).
      * 2 live-API gating (skip-on-no-key): the new shape returns
        HTTP 200 against api.anthropic.com; the old shape's current
        status is recorded for canary purposes (still 200 for
        api-key today, but reporter shows it's 400 for OAuth).

  - Full suite: 405/405 pass + 2 skipped (live API tests, opt-in).
  - The reporter's exact deprecation 400 message reproduces if you
    swap auth to OAuth Bearer + stream:true (could not test locally
    without extracting the keychain OAuth token, which was out of
    scope). The fix shape is API-contract-level so it doesn't depend
    on which auth mode triggers the 400.

NOT verified end-to-end via OAuth-authenticated plugin invocation on
my machine (auto-mode classifier correctly declined to extract the
keychain token). Reporter's 462 production errors + the docs
migration notice + the live-API HTTP 200 on the new form are
sufficient evidence to ship.

Closes #2098.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

plugins/security-guidance/hooks/llm.py

@@ -479,10 +479,21 @@ def _call_claude(prompt, output_schema, thinking_budget=10000, max_tokens=16000,
         "max_tokens": max_tokens,
         "system": CLAUDE_CODE_SYSTEM_PROMPT,
         "messages": [{"role": "user", "content": prompt}],
-        "output_format": {
+        # API moved the structured-output schema from top-level `output_format`
+        # to `output_config.format` per
+        # https://platform.claude.com/docs/en/build-with-claude/structured-outputs.
+        # The old form "continues to work for a transition period" for some
+        # auth modes (API key + non-streaming), but is rejected with
+        # `invalid_request_error: output_format: This field is deprecated.
+        # Use 'output_config.format' instead.` for others (OAuth Bearer +
+        # newer CLI versions hit it consistently — reporter saw 462 errors
+        # in one day). See #2098.
+        "output_config": {
+            "format": {
                 "type": "json_schema",
-            "schema": output_schema
-        }
+                "schema": output_schema,
+            },
+        },
     }
     if thinking_budget > 0:
         # Models trained on adaptive thinking (4.6+) reject the budget_tokens
@@ -490,7 +501,10 @@ def _call_claude(prompt, output_schema, thinking_budget=10000, max_tokens=16000,
         # models (4.5 and earlier, all 3.x) reject adaptive. Pick by model.
         if _model_supports_adaptive_thinking(payload["model"]):
             payload["thinking"] = {"type": "adaptive"}
-            payload["output_config"] = {"effort": "high"}
+            # Merge `effort` into the existing output_config dict (which
+            # now carries the `format` schema) rather than reassigning —
+            # otherwise the schema is silently overwritten. See #2098.
+            payload["output_config"]["effort"] = "high"
         else:
             payload["thinking"] = {
                 "type": "enabled",