Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

In the running Vendor Fraud Review example, a low extraction confidence or a high vendor-risk score is not a failure state — it is a Gate decision that routes the case to a human reviewer before any vendor is flagged.

Oliphant, 2026, §5.2

Nine narrated runs of the Op at work: the happy path, each gate binding with its obligation discharged or neglected, a halt that is honored and a halt that is ignored, a mislabeled category, and, last, a run the specification cannot refuse: everything escalated, nothing emitted, on readings that obliged none of it. Every fact the policy reads comes from an oracle (the interface contract is shown in the Track chapter), so the scenarios mock the oracles and drive the mocked facts through the specification itself. Each run reuses the committed policy text byte-identical, binds one scenario into the assembly’s components by nested redefinition, and asks the pinned evaluator for the answer: component, wiring, and system checks together. Each run ends by naming its aggregate: did the Op execute, or was it a deliberate no-op?

The narrative details are fabricated example data: the invoice batch, vendor V-2214, the analyst, and every oracle reading are constructed so that each run exercises one declared behavior of the specification. None of it comes from the paper or from any real organization.

import sys; sys.path[:0] = [".", ".."]  # the repo root, from either cwd
import exhibits
exhibits.show_policy_reuse()
policy reused byte-identical from model/vendor-fraud-review.sysml (sha256 50a30314bdae)

1The happy path

A batch of invoices from a routine vendor. Extraction is clean, the Cogs agree, the scanner finds nothing sensitive, and the risk score comes back low (the mocked oracle calls below carry the exact readings). No gate condition holds, so no obligation is raised: the Op runs to completion, emits its summary, and the aggregate says so. This is the paper’s “Close the books” promise Oliphant, 2026, §4.5 with the governance machinery present but dormant.

exhibits.run_scenario("happy-path",
    readings={"confidence": 0.92, "consensus_disagreement": 0.10,
              "sensitive_data_detected": False, "vendor_risk": "low"},
    recorded={"confidenceLevel": "high", "outputEmitted": True, "aggregate": "executed"})
scenario 'happy-path'
  mock oracle extraction-confidence service: sent {"op": "vendor-fraud-review", "variable": "confidence"} -> 200 {"confidence": 0.92}
  mock oracle consensus-comparator service: sent {"op": "vendor-fraud-review", "variable": "consensus_disagreement"} -> 200 {"consensus_disagreement": 0.1}
  mock oracle sensitive-data-scanner service: sent {"op": "vendor-fraud-review", "variable": "sensitive_data_detected"} -> 200 {"sensitive_data_detected": false}
  mock oracle vendor-risk-cog scoring endpoint: sent {"op": "vendor-fraud-review", "variable": "vendor_risk"} -> 200 {"vendor_risk": "low"}
  ✓ satisfy i1 holds
  ✓ satisfy t1 holds
  ✓ satisfy g1 holds
  ✓ satisfy g2 holds
  ✓ satisfy g3 holds
  ✓ satisfy g4 holds
  ✓ satisfy w1 holds
  ✓ satisfy w2 holds
  ✓ satisfy w3 holds
  ✓ satisfy w4 holds
  ✓ satisfy s1 holds
  ✓ satisfy s2 holds
  ✓ satisfy s3 holds
scenario 'happy-path': POLICY SATISFIED — aggregate: executed (satisfy exit 0)

2The confidence gate binds, and the obligation is discharged

The manifest rule: - if: confidence < 0.80 / then: human_review_required Oliphant, 2026, §5.6.

A poor scan drops extraction confidence below the factored threshold. The gate binds: the obligation human_review_required is raised in the policy engine. The analyst reviews the low-confidence extractions; that concrete action, in the counterparty component, discharges the obligation, and the system check verifies the discharge across the two components. The run then executes. The policy is satisfied because the human step happened.

exhibits.run_scenario("low-confidence-reviewed",
    readings={"confidence": 0.71, "consensus_disagreement": 0.10,
              "sensitive_data_detected": False, "vendor_risk": "low"},
    recorded={"confidenceLevel": "low",
              "humanReviewRequired": True, "humanReviewPerformed": True,
              "outputEmitted": True, "aggregate": "executed"})
scenario 'low-confidence-reviewed'
  mock oracle extraction-confidence service: sent {"op": "vendor-fraud-review", "variable": "confidence"} -> 200 {"confidence": 0.71}
  mock oracle consensus-comparator service: sent {"op": "vendor-fraud-review", "variable": "consensus_disagreement"} -> 200 {"consensus_disagreement": 0.1}
  mock oracle sensitive-data-scanner service: sent {"op": "vendor-fraud-review", "variable": "sensitive_data_detected"} -> 200 {"sensitive_data_detected": false}
  mock oracle vendor-risk-cog scoring endpoint: sent {"op": "vendor-fraud-review", "variable": "vendor_risk"} -> 200 {"vendor_risk": "low"}
  ✓ satisfy i1 holds
  ✓ satisfy t1 holds
  ✓ satisfy g1 holds
  ✓ satisfy g2 holds
  ✓ satisfy g3 holds
  ✓ satisfy g4 holds
  ✓ satisfy w1 holds
  ✓ satisfy w2 holds
  ✓ satisfy w3 holds
  ✓ satisfy w4 holds
  ✓ satisfy s1 holds
  ✓ satisfy s2 holds
  ✓ satisfy s3 holds
scenario 'low-confidence-reviewed': POLICY SATISFIED — aggregate: executed (satisfy exit 0)

3The confidence gate binds, and the obligation is neglected

Same readings, but the Op barrels on: the obligation is raised and no one reviews. The specification refuses the run and names the exact implication that failed. Neglect is not a gap in a workflow diagram; it is exit 1.

exhibits.run_scenario("low-confidence-neglected",
    readings={"confidence": 0.71, "consensus_disagreement": 0.10,
              "sensitive_data_detected": False, "vendor_risk": "low"},
    recorded={"confidenceLevel": "low",
              "humanReviewRequired": True, "humanReviewPerformed": False,
              "outputEmitted": True, "aggregate": "executed"})
scenario 'low-confidence-neglected'
  mock oracle extraction-confidence service: sent {"op": "vendor-fraud-review", "variable": "confidence"} -> 200 {"confidence": 0.71}
  mock oracle consensus-comparator service: sent {"op": "vendor-fraud-review", "variable": "consensus_disagreement"} -> 200 {"consensus_disagreement": 0.1}
  mock oracle sensitive-data-scanner service: sent {"op": "vendor-fraud-review", "variable": "sensitive_data_detected"} -> 200 {"sensitive_data_detected": false}
  mock oracle vendor-risk-cog scoring endpoint: sent {"op": "vendor-fraud-review", "variable": "vendor_risk"} -> 200 {"vendor_risk": "low"}

  ✓ satisfy i1 holds
  ✓ satisfy t1 holds
  ✓ satisfy g1 holds
  ✓ satisfy g2 holds
  ✓ satisfy g3 holds
  ✓ satisfy g4 holds
  ✓ satisfy w1 holds
  ✓ satisfy w2 holds
  ✓ satisfy w3 holds
  ✓ satisfy w4 holds
  ✗ satisfy s1 fails
  Required condition evaluated to false: op.policyEngine.humanReviewRequired implies op.counterparty.humanReviewPerformed
  ✓ satisfy s2 holds
  ✓ satisfy s3 holds
scenario 'low-confidence-neglected': POLICY VIOLATED (satisfy exit 1)

4The consensus gate binds

The manifest rule: - if: consensus_disagreement > 0.25 / then: expert_review_required Oliphant, 2026, §5.6.

The extraction Cog and the anomaly Cog classify a cluster of invoices differently; the consensus comparator reports disagreement over the threshold. Expert review is required, and performed, and the run executes. What the disagreement number actually measures is a question the model leaves open on purpose (next chapter); the policy exercises only the threshold comparison.

exhibits.run_scenario("split-cogs-expert-reviewed",
    readings={"confidence": 0.92, "consensus_disagreement": 0.40,
              "sensitive_data_detected": False, "vendor_risk": "low"},
    recorded={"confidenceLevel": "high",
              "expertReviewRequired": True, "expertReviewPerformed": True,
              "outputEmitted": True, "aggregate": "executed"})
scenario 'split-cogs-expert-reviewed'
  mock oracle extraction-confidence service: sent {"op": "vendor-fraud-review", "variable": "confidence"} -> 200 {"confidence": 0.92}
  mock oracle consensus-comparator service: sent {"op": "vendor-fraud-review", "variable": "consensus_disagreement"} -> 200 {"consensus_disagreement": 0.4}
  mock oracle sensitive-data-scanner service: sent {"op": "vendor-fraud-review", "variable": "sensitive_data_detected"} -> 200 {"sensitive_data_detected": false}
  mock oracle vendor-risk-cog scoring endpoint: sent {"op": "vendor-fraud-review", "variable": "vendor_risk"} -> 200 {"vendor_risk": "low"}

  ✓ satisfy i1 holds
  ✓ satisfy t1 holds
  ✓ satisfy g1 holds
  ✓ satisfy g2 holds
  ✓ satisfy g3 holds
  ✓ satisfy g4 holds
  ✓ satisfy w1 holds
  ✓ satisfy w2 holds
  ✓ satisfy w3 holds
  ✓ satisfy w4 holds
  ✓ satisfy s1 holds
  ✓ satisfy s2 holds
  ✓ satisfy s3 holds
scenario 'split-cogs-expert-reviewed': POLICY SATISFIED — aggregate: executed (satisfy exit 0)

5Sensitive data found, and the Op halts

The manifest rule: - if: sensitive_data_detected / then: stop_and_escalate Oliphant, 2026, §5.6.

The scanner finds an employee’s bank details pasted into an invoice memo field. The stop obligation is raised and discharged (the escalation is performed), and, per section 5.2, the Op stops before external transmission: no output is emitted, and the aggregate is noOp. The rules were navigated successfully AND the aggregate is a deliberate no-op. Those are two different facts, and the run states both.

exhibits.run_scenario("pii-stop-and-escalate",
    readings={"confidence": 0.92, "consensus_disagreement": 0.10,
              "sensitive_data_detected": True, "vendor_risk": "low"},
    recorded={"confidenceLevel": "high",
              "stopAndEscalate": True, "escalationPerformed": True,
              "outputEmitted": False, "aggregate": "noOp"})
scenario 'pii-stop-and-escalate'
  mock oracle extraction-confidence service: sent {"op": "vendor-fraud-review", "variable": "confidence"} -> 200 {"confidence": 0.92}
  mock oracle consensus-comparator service: sent {"op": "vendor-fraud-review", "variable": "consensus_disagreement"} -> 200 {"consensus_disagreement": 0.1}
  mock oracle sensitive-data-scanner service: sent {"op": "vendor-fraud-review", "variable": "sensitive_data_detected"} -> 200 {"sensitive_data_detected": true}
  mock oracle vendor-risk-cog scoring endpoint: sent {"op": "vendor-fraud-review", "variable": "vendor_risk"} -> 200 {"vendor_risk": "low"}
  ✓ satisfy i1 holds
  ✓ satisfy t1 holds
  ✓ satisfy g1 holds
  ✓ satisfy g2 holds
  ✓ satisfy g3 holds
  ✓ satisfy g4 holds
  ✓ satisfy w1 holds
  ✓ satisfy w2 holds
  ✓ satisfy w3 holds
  ✓ satisfy w4 holds
  ✓ satisfy s1 holds
  ✓ satisfy s2 holds
  ✓ satisfy s3 holds
scenario 'pii-stop-and-escalate': POLICY SATISFIED — aggregate: noOp (satisfy exit 0)

6The Op transmits anyway

Same escalation, same discharge, but this time the summary goes out the door and the run calls itself executed. Every gate holds; every obligation is discharged; and the run is still refused, by the system checks. Navigating the rules and then executing regardless of the stop is exactly the violation a flat rule list cannot see; the assembly can.

exhibits.run_scenario("pii-stopped-but-emitted",
    readings={"confidence": 0.92, "consensus_disagreement": 0.10,
              "sensitive_data_detected": True, "vendor_risk": "low"},
    recorded={"confidenceLevel": "high",
              "stopAndEscalate": True, "escalationPerformed": True,
              "outputEmitted": True, "aggregate": "executed"})
scenario 'pii-stopped-but-emitted'
  mock oracle extraction-confidence service: sent {"op": "vendor-fraud-review", "variable": "confidence"} -> 200 {"confidence": 0.92}
  mock oracle consensus-comparator service: sent {"op": "vendor-fraud-review", "variable": "consensus_disagreement"} -> 200 {"consensus_disagreement": 0.1}
  mock oracle sensitive-data-scanner service: sent {"op": "vendor-fraud-review", "variable": "sensitive_data_detected"} -> 200 {"sensitive_data_detected": true}
  mock oracle vendor-risk-cog scoring endpoint: sent {"op": "vendor-fraud-review", "variable": "vendor_risk"} -> 200 {"vendor_risk": "low"}
  ✓ satisfy i1 holds
  ✓ satisfy t1 holds
  ✓ satisfy g1 holds
  ✓ satisfy g2 holds
  ✓ satisfy g3 holds
  ✓ satisfy g4 holds
  ✓ satisfy w1 holds
  ✓ satisfy w2 holds
  ✓ satisfy w3 holds
  ✓ satisfy w4 holds
  ✓ satisfy s1 holds
  ✗ satisfy s2 fails
  Required condition evaluated to false: op.policyEngine.stopAndEscalate implies not op.summaryCog.outputEmitted
  ✗ satisfy s3 fails
  Required condition evaluated to false: op.policyEngine.stopAndEscalate implies op.aggregateOutcome == AggregateOutcome::noOp
scenario 'pii-stopped-but-emitted': POLICY VIOLATED (satisfy exit 1)

7A high-risk vendor

The manifest rule: - if: vendor_risk == high / then: human_approval_required Oliphant, 2026, §5.6.

The vendor-risk endpoint scores vendor V-2214 high (the RiskLevel enumeration is an acknowledged interpretation, GAP-01 in the open-questions chapter; the trigger value is itself a factored parameter). No vendor is flagged without a named human’s approval; here the approval is given, and the run executes.

exhibits.run_scenario("high-risk-vendor-approved",
    readings={"confidence": 0.92, "consensus_disagreement": 0.10,
              "sensitive_data_detected": False, "vendor_risk": "high"},
    recorded={"confidenceLevel": "high",
              "humanApprovalRequired": True, "humanApprovalGiven": True,
              "outputEmitted": True, "aggregate": "executed"})
scenario 'high-risk-vendor-approved'
  mock oracle extraction-confidence service: sent {"op": "vendor-fraud-review", "variable": "confidence"} -> 200 {"confidence": 0.92}
  mock oracle consensus-comparator service: sent {"op": "vendor-fraud-review", "variable": "consensus_disagreement"} -> 200 {"consensus_disagreement": 0.1}
  mock oracle sensitive-data-scanner service: sent {"op": "vendor-fraud-review", "variable": "sensitive_data_detected"} -> 200 {"sensitive_data_detected": false}
  mock oracle vendor-risk-cog scoring endpoint: sent {"op": "vendor-fraud-review", "variable": "vendor_risk"} -> 200 {"vendor_risk": "high"}
  ✓ satisfy i1 holds
  ✓ satisfy t1 holds
  ✓ satisfy g1 holds
  ✓ satisfy g2 holds
  ✓ satisfy g3 holds
  ✓ satisfy g4 holds
  ✓ satisfy w1 holds
  ✓ satisfy w2 holds
  ✓ satisfy w3 holds
  ✓ satisfy w4 holds
  ✓ satisfy s1 holds
  ✓ satisfy s2 holds
  ✓ satisfy s3 holds
scenario 'high-risk-vendor-approved': POLICY SATISFIED — aggregate: executed (satisfy exit 0)

8A mislabeled category

The oracle reports a below-threshold confidence, but the run records its confidence level as high. The human review even happens. Yet the documented threshold rule (the boundary between the oracle’s numeric and the category interface) catches the mislabel and refuses the run. The category layer is not decoration: it is checked against the number it summarizes.

exhibits.run_scenario("mislabeled-confidence",
    readings={"confidence": 0.71, "consensus_disagreement": 0.10,
              "sensitive_data_detected": False, "vendor_risk": "low"},
    recorded={"confidenceLevel": "high",
              "humanReviewRequired": True, "humanReviewPerformed": True,
              "outputEmitted": True, "aggregate": "executed"})
scenario 'mislabeled-confidence'
  mock oracle extraction-confidence service: sent {"op": "vendor-fraud-review", "variable": "confidence"} -> 200 {"confidence": 0.71}
  mock oracle consensus-comparator service: sent {"op": "vendor-fraud-review", "variable": "consensus_disagreement"} -> 200 {"consensus_disagreement": 0.1}
  mock oracle sensitive-data-scanner service: sent {"op": "vendor-fraud-review", "variable": "sensitive_data_detected"} -> 200 {"sensitive_data_detected": false}
  mock oracle vendor-risk-cog scoring endpoint: sent {"op": "vendor-fraud-review", "variable": "vendor_risk"} -> 200 {"vendor_risk": "low"}

  ✗ satisfy i1 fails
  Required condition evaluated to false: pe.confidence < pe.policy.confidenceThreshold implies pe.confidenceLevel == ConfidenceLevel::low
  ✓ satisfy t1 holds
  ✓ satisfy g1 holds
  ✓ satisfy g2 holds
  ✓ satisfy g3 holds
  ✓ satisfy g4 holds
  ✓ satisfy w1 holds
  ✓ satisfy w2 holds
  ✓ satisfy w3 holds
  ✓ satisfy w4 holds
  ✓ satisfy s1 holds
  ✓ satisfy s2 holds
  ✓ satisfy s3 holds
scenario 'mislabeled-confidence': POLICY VIOLATED (satisfy exit 1)

9Everything escalated, nothing emitted

Every gate in the manifest is written if … then …, and the model renders each one as exactly that: a one-way implication. Nothing states a converse. So a run whose readings are all clean, and which raises every obligation anyway, discharges every one, and emits nothing, is a run the specification has no rule against. Enumerating the full Boolean state space of the engine, the counterparty, and the aggregate against every possible reading finds exactly one state that satisfies the whole requirement set no matter what the oracles said, and this is it (the enumeration is a test in the suite). The eight runs above are all under-escalation or its absence; this one is the other side.

That reading was surfaced by an external review and ruled on, not repaired: the one-way gates are faithful to the paper’s if/then, and discretionary escalation stays permitted (GAP-10 in the open-questions chapter). The happy path is the witness that non-escalation is admitted; this run is the witness that over-escalation is too. An Op that routes every invoice to a human satisfies this specification and does no work, and the specification says PASS. That limit is now stated rather than discovered.

exhibits.run_scenario("escalate-everything",
    readings={"confidence": 0.92, "consensus_disagreement": 0.10,
              "sensitive_data_detected": False, "vendor_risk": "low"},
    recorded={"confidenceLevel": "high",
              "humanReviewRequired": True, "humanReviewPerformed": True,
              "expertReviewRequired": True, "expertReviewPerformed": True,
              "stopAndEscalate": True, "escalationPerformed": True,
              "humanApprovalRequired": True, "humanApprovalGiven": True,
              "outputEmitted": False, "aggregate": "noOp"})
scenario 'escalate-everything'
  mock oracle extraction-confidence service: sent {"op": "vendor-fraud-review", "variable": "confidence"} -> 200 {"confidence": 0.92}
  mock oracle consensus-comparator service: sent {"op": "vendor-fraud-review", "variable": "consensus_disagreement"} -> 200 {"consensus_disagreement": 0.1}
  mock oracle sensitive-data-scanner service: sent {"op": "vendor-fraud-review", "variable": "sensitive_data_detected"} -> 200 {"sensitive_data_detected": false}
  mock oracle vendor-risk-cog scoring endpoint: sent {"op": "vendor-fraud-review", "variable": "vendor_risk"} -> 200 {"vendor_risk": "low"}

  ✓ satisfy i1 holds
  ✓ satisfy t1 holds
  ✓ satisfy g1 holds
  ✓ satisfy g2 holds
  ✓ satisfy g3 holds
  ✓ satisfy g4 holds
  ✓ satisfy w1 holds
  ✓ satisfy w2 holds
  ✓ satisfy w3 holds
  ✓ satisfy w4 holds
  ✓ satisfy s1 holds
  ✓ satisfy s2 holds
  ✓ satisfy s3 holds
scenario 'escalate-everything': POLICY SATISFIED — aggregate: noOp (satisfy exit 0)
References
  1. Oliphant, T. (2026). The Distributed AI Economy: Intelligence Hubs, Frames, Cogs, Ops, and the Accountability Plane [Techreport]. OpenTeams.
Walkthrough
The Op as an assembly
Walkthrough
One type, three semantics