fix(triage): withdraw the disproven false positives and render deployed-vs-pinned versions #22

Merged

Philipp Baumann merged 1 commit from fm/dse-security-reports-defender-evidence-correction into main

2026-10-28 08:54:19 +00:00

Philipp Baumann commented

2026-10-28 08:53:43 +00:00

Owner

Summary

The 28.10.2026 verification round (firstmate report dse-security-reports-pyjwt-qms-verify) found Defender's pyjwt claim real: packages the report declared absent are installed in root-only Ricochet deployment stores, so the report reassured about a real finding and the "unreliable Linux evidence" reasoning built on that had to be reworked, not trimmed. This PR retracts three of the five 26.10 false-positive confirmations on that evidence (pyjwt, anyio, h11), re-titles and re-sorts the affected section, and renders the deployed-versus-pinned package versions deterministically from the pipeline instead of as a hand-read snippet — plus the two folded-in corrections from the same verification round: the security-check failure cause per unit instead of a collapsed "error", and the WireGuard sensor facts instead of the "coverage gap" inference.

What changed

.local/samples/triage.csv (not committed; re-applied per round) — the FalsePositive annotations for CVE-2026-102268 (pyjwt), CVE-2026-63374 (anyio) and CVE-2025-43859 (h11) are removed; pillow and OpenSSL-Windows stay.
reports/defender-triage.typ — the section «Warum die Linux-Evidenz von Defender unzuverlässig ist» is replaced by «Linux-Paketevidenz: Korrektur der Bewertung (Verifikation 28.10.2026)»: per-package verified position, mechanisms re-sorted with our own collection coverage failure as the root mechanism, and the index-accumulation mechanism kept but its failed inference named. The FP section carries the withdrawal record (the 26.10 confirmations rested on analysis since disproven; remaining candidates: pillow, OpenSSL). New section «Deployt vs. gepinnt» renders the per-unit/per-package table from the derive. New subsections under Evidenzabdeckung: «Sicherheits-Update-Check je Einheit» (status, reason, repo; error carries no count) and «Defender-Sensor je Einheit» (mdatp health, WireGuard onboarding verified, stale definitions and passive mode as explicit findings). The ranking column becomes «Linux-Boden-Evidenz» with inverted polarity, P15's security-check sentence and the P6/P8/P9/P10 analyses are corrected, the WireGuard "coverage gap" framing is replaced by the verified sensor facts, and the Fan-out rows 5/8 are updated.
scripts/derive_findings.py + src/dse_security_reports/derive.py (mirrored, byte-identical on the fixture) — store normalization across the three cache shapes (legacy pyjwt paths, per-package dist-info paths, verified aggregated counts with a swept list), a partial sweep makes no claim; ricochet_aggregate joins Defender claims, the committed lock pins, store versions and running deployments into the payload-level ricochet block; patch_posture records the security-check reason/repo and nulls the update count under error; defender_coverage passes the mdatp health block through.
config/app-locks.json (new, committed) — lock pins of the deployed application projects (tracking-quality, myride_simulation) with provenance (read date, last-committed date, deploy member lock); the drift story (root 2.15.1/1.39.0/1.44.2 vs app 2.13.0/1.36.0/1.41.0, deployment resolves from the June-pinned app lock) and the myride contrast (locks agree, deployment faithful) render from it.
scripts/ssh_evidence.sh — the ricochet sweep covers all four tracked packages (pyjwt, anyio, h11, pillow), the mdatp probe gains a read-only sudo -n mdatp health section (with a health-only refresh for caches that predate it), and the security-check parser records which repository failed and why instead of collapsing to a single word.
scripts/backfill_verified_evidence.py (new) + just backfill-verified — idempotent merge of the verified 28.10 probe results (store facts, mdatp health, sec-check reasons) into a restored 2026-10-27 evidence cache; the collector writes the same shapes in future rounds, so the bridge retires itself.
tests/test_derive.py — the ricochet tests move to the aggregated entry shape, cover the verified-backfill shape (swept list, un-swept packages make no claim), running-deployment resolution, the deployment-vs-pins aggregate with its gap states, the null update count under error, and the health block pass-through.
AGENTS.md, README.md, Justfile — the correction round, the backfill recipe and the app-locks config are documented; the published-round restoration protocol gains the backfill step.

Behaviour change

The re-rendered 27.10 round (published to the firstmate artifacts dir as defender-triage-2026-10-28.pdf / defender-findings-2026-10-28.pdf, previous copies moved to superseded/, the 2026-10-26 set untouched) states the corrected position: three false-positive confirmations withdrawn, the QMS pyjwt row derives trivy-silent from the store evidence (the only scanner-agreement change; verified against an isolated origin/main re-derivation — every other number is unchanged), the deployed-versus-pinned table replaces the hand-read snippet, a refused or un-swept store always renders as a gap never as an absence, and the WireGuard unit renders as sensor-verified onboarded with its index below the export thresholds instead of as a Defender coverage gap.

Section-title decision (asked for in the task)

The old title claimed Defender's Linux evidence was unreliable; the verification shows the opposite where it could check: Defender's package-and-version quotes were real (pyjwt running in production on QMS, anyio/h11 in every myride bundle, the workbench versions in colleague venvs), while our own collection was the unreliable part (root-only stores, silently skipped subtrees, unreadable colleague homes). The new title «Linux-Paketevidenz: Korrektur der Bewertung» therefore names the correction instead of prejudging either side; the same reasoning renamed the ranking column to «Linux-Boden-Evidenz» (polarity inverted: evidence-backed rows read "ja (Evidenz)", index-only rows "nein (Index)"). What remains true about Defender — over- AND under-reporting at the same time (silent on the identical myride stack, silent on the kernel CVEs) — is kept as a mechanism, not a verdict.

Consideration asked for in the follow-up instruction: sudo and the security check

Running dnf --security check-update with read-only sudo would NOT sidestep the failure: the abort is a GPG key-trust question (packages-microsoft-com-prod carries no trusted signing key, dnf attempts an interactive key import, finds no terminal), not a permission question — and auto-importing keys during an evidence run would be a silent trust decision, not evidence. The actual fix is provisioning the repo's signing key via config management (gpgkey in the repo definition), an estate change that belongs to the captain; the check therefore stays exactly as it runs today, and this PR only records what failed, per unit, and never reads the accompanying 0 as a count.

Instructions-PR: none

## Summary The 28.10.2026 verification round (firstmate report `dse-security-reports-pyjwt-qms-verify`) found Defender's pyjwt claim real: packages the report declared absent are installed in root-only Ricochet deployment stores, so the report reassured about a real finding and the "unreliable Linux evidence" reasoning built on that had to be reworked, not trimmed. This PR retracts three of the five 26.10 false-positive confirmations on that evidence (pyjwt, anyio, h11), re-titles and re-sorts the affected section, and renders the deployed-versus-pinned package versions deterministically from the pipeline instead of as a hand-read snippet — plus the two folded-in corrections from the same verification round: the security-check failure cause per unit instead of a collapsed "error", and the WireGuard sensor facts instead of the "coverage gap" inference. ## What changed - `.local/samples/triage.csv` (not committed; re-applied per round) — the FalsePositive annotations for CVE-2026-102268 (pyjwt), CVE-2026-63374 (anyio) and CVE-2025-43859 (h11) are removed; pillow and OpenSSL-Windows stay. - `reports/defender-triage.typ` — the section «Warum die Linux-Evidenz von Defender unzuverlässig ist» is replaced by «Linux-Paketevidenz: Korrektur der Bewertung (Verifikation 28.10.2026)»: per-package verified position, mechanisms re-sorted with our own collection coverage failure as the root mechanism, and the index-accumulation mechanism kept but its failed inference named. The FP section carries the withdrawal record (the 26.10 confirmations rested on analysis since disproven; remaining candidates: pillow, OpenSSL). New section «Deployt vs. gepinnt» renders the per-unit/per-package table from the derive. New subsections under Evidenzabdeckung: «Sicherheits-Update-Check je Einheit» (status, reason, repo; error carries no count) and «Defender-Sensor je Einheit» (mdatp health, WireGuard onboarding verified, stale definitions and passive mode as explicit findings). The ranking column becomes «Linux-Boden-Evidenz» with inverted polarity, P15's security-check sentence and the P6/P8/P9/P10 analyses are corrected, the WireGuard "coverage gap" framing is replaced by the verified sensor facts, and the Fan-out rows 5/8 are updated. - `scripts/derive_findings.py` + `src/dse_security_reports/derive.py` (mirrored, byte-identical on the fixture) — store normalization across the three cache shapes (legacy pyjwt paths, per-package dist-info paths, verified aggregated counts with a `swept` list), a partial sweep makes no claim; `ricochet_aggregate` joins Defender claims, the committed lock pins, store versions and running deployments into the payload-level `ricochet` block; `patch_posture` records the security-check reason/repo and nulls the update count under error; `defender_coverage` passes the mdatp health block through. - `config/app-locks.json` (new, committed) — lock pins of the deployed application projects (tracking-quality, myride_simulation) with provenance (read date, last-committed date, deploy member lock); the drift story (root 2.15.1/1.39.0/1.44.2 vs app 2.13.0/1.36.0/1.41.0, deployment resolves from the June-pinned app lock) and the myride contrast (locks agree, deployment faithful) render from it. - `scripts/ssh_evidence.sh` — the ricochet sweep covers all four tracked packages (pyjwt, anyio, h11, pillow), the mdatp probe gains a read-only `sudo -n mdatp health` section (with a health-only refresh for caches that predate it), and the security-check parser records which repository failed and why instead of collapsing to a single word. - `scripts/backfill_verified_evidence.py` (new) + `just backfill-verified` — idempotent merge of the verified 28.10 probe results (store facts, mdatp health, sec-check reasons) into a restored 2026-10-27 evidence cache; the collector writes the same shapes in future rounds, so the bridge retires itself. - `tests/test_derive.py` — the ricochet tests move to the aggregated entry shape, cover the verified-backfill shape (swept list, un-swept packages make no claim), running-deployment resolution, the deployment-vs-pins aggregate with its gap states, the null update count under error, and the health block pass-through. - `AGENTS.md`, `README.md`, `Justfile` — the correction round, the backfill recipe and the app-locks config are documented; the published-round restoration protocol gains the backfill step. ## Behaviour change The re-rendered 27.10 round (published to the firstmate artifacts dir as `defender-triage-2026-10-28.pdf` / `defender-findings-2026-10-28.pdf`, previous copies moved to `superseded/`, the 2026-10-26 set untouched) states the corrected position: three false-positive confirmations withdrawn, the QMS pyjwt row derives `trivy-silent` from the store evidence (the only scanner-agreement change; verified against an isolated origin/main re-derivation — every other number is unchanged), the deployed-versus-pinned table replaces the hand-read snippet, a refused or un-swept store always renders as a gap never as an absence, and the WireGuard unit renders as sensor-verified onboarded with its index below the export thresholds instead of as a Defender coverage gap. ## Section-title decision (asked for in the task) The old title claimed Defender's Linux evidence was unreliable; the verification shows the opposite where it could check: Defender's package-and-version quotes were real (pyjwt running in production on QMS, anyio/h11 in every myride bundle, the workbench versions in colleague venvs), while our own collection was the unreliable part (root-only stores, silently skipped subtrees, unreadable colleague homes). The new title «Linux-Paketevidenz: Korrektur der Bewertung» therefore names the correction instead of prejudging either side; the same reasoning renamed the ranking column to «Linux-Boden-Evidenz» (polarity inverted: evidence-backed rows read "ja (Evidenz)", index-only rows "nein (Index)"). What remains true about Defender — over- AND under-reporting at the same time (silent on the identical myride stack, silent on the kernel CVEs) — is kept as a mechanism, not a verdict. ## Consideration asked for in the follow-up instruction: sudo and the security check Running `dnf --security check-update` with read-only sudo would NOT sidestep the failure: the abort is a GPG key-trust question (packages-microsoft-com-prod carries no trusted signing key, dnf attempts an interactive key import, finds no terminal), not a permission question — and auto-importing keys during an evidence run would be a silent trust decision, not evidence. The actual fix is provisioning the repo's signing key via config management (gpgkey in the repo definition), an estate change that belongs to the captain; the check therefore stays exactly as it runs today, and this PR only records what failed, per unit, and never reads the accompanying 0 as a count. Instructions-PR: none

Philipp Baumann added 1 commit

2026-10-28 08:53:43 +00:00

fix(triage): withdraw the disproven false positives and render deployed-vs-pinned versions 774b669149

- Retract three of the five 26.10 false-positive confirmations (pyjwt
  CVE-2026-102268, anyio CVE-2026-63374, h11 CVE-2025-43859) on the
  28.10 verification evidence: pyjwt runs as 2.13.0 in the QMS
  production deployment, anyio 4.13.0 and h11 0.16.0 are pinned and
  installed in every myride bundle; pillow stays a candidate (not a
  dependency of either project). The QMS pyjwt row derives trivy-silent
  from the store evidence.
- Retitle the 'Defender Linux evidence unreliable' section to
  'Linux-Paketevidenz: Korrektur der Bewertung': the verified
  unreliability was our own collection (root-only stores, silently
  skipped subtrees, colleague homes), not Defender's index. Re-sort the
  mechanisms accordingly -- the index-accumulation mechanism was
  correct, only the absence inference failed -- and rename the ranking
  column to 'Linux-Boden-Evidenz'.
- Render deployed-vs-pinned versions per unit and tracked package
  (pyjwt, anyio, h11, pillow) from a new payload-level ricochet
  aggregate joined with the committed lock pins (config/app-locks.json):
  root lock vs app lock vs installed store versions, running
  deployments, and the tracking-quality two-lock drift story; a refused
  or un-swept store stays a stated gap, never an absence.
- Record the mdatp health facts (read-only sudo probe): the WireGuard
  unit is verified onboarded to the same Defender org as the reporting
  machines, so zero export rows render as 'index below the export
  thresholds', not a coverage gap; stale definitions at
  up-to-date self-report and passive mode without real-time protection
  carry as explicit sensor findings.
- Record the security-check failure cause per unit instead of collapsing
  to 'error' (packages-microsoft-com-prod GPG signing key not trusted);
  an error status carries no update count (the derive nulls it). The
  check itself is deliberately not run with sudo.
- Collector: the ricochet sweep covers all four tracked packages, the
  mdatp probe gains the health section; scripts/backfill_verified_evidence.py
  (just backfill-verified) merges the verified 28.10 round into a
  restored 2026-10-27 cache, idempotently.

Philipp Baumann merged commit 0a1e64bb02 into main

2026-10-28 08:54:19 +00:00

Philipp Baumann deleted branch fm/dse-security-reports-defender-evidence-correction

2026-10-28 08:54:19 +00:00

Philipp Baumann referenced this pull request from a commit

2026-10-28 08:54:20 +00:00

fix(triage): withdraw the disproven false positives and render deployed-vs-pinned versions (#22)