Coldcode Si Labs Library

Simulated Completeness

Working paper 2026-02 · Courage Garikai
View-only edition. No download from this page.

Simulated Completeness

Claim–mechanism gaps, mechanism audits and the rebuild decision in an AI-era trust product Courage Garikai · Funded by Coldcode Si Labs Disclosure. The case venture, AttestOS, belongs to the same owner who commissions the work of Coldcode Si Labs. The audit and the rebuild analysed here were carried out by the owner working with AI coding agents, and the same analytic process produced the audit, the coding of its findings and this paper. The case is therefore not an independent evaluation. At the time of writing AttestOS had taken no payments and had no customers. Nothing here should be read as evidence of commercial success or of security efficacy in the field. How the first version of the venture was produced is not stated in this paper; where the evidence bears on that question it is treated as a hypothesis. AI tools assisted in drafting this paper; the author is re‐ sponsible for its content.

Abstract

Generative AI has made the visible surface of a software product (interfaces, dashboards, documentation, screenshots, test badges, release notes) far cheaper to produce than the mechanisms that make the surface true. This paper names the resulting condition simulated completeness: an artefact whose surface signals op‐ erational completeness that its mechanisms do not support. It reports a design-science case study of AttestOS, a bot-verification service whose first version, built in one day in October 2026, looked finished. Its owner console showed traffic, revenue and connected payment webhooks. Its portal showed customer usage, its scanner reported vulnerabilities and its outreach engine drafted security notices. A read-only audit the next day logged 65 findings, 18 of them critical. Seven ways around the live security engine were confirmed against production. Every console figure was hard-coded. Nobody could obtain a key or pay. A plugin loaded its script from an unregistered domain, and the drafted security notices derived their findings from a hash of the recipient's domain name. Coding the findings shows that 47 of the 65 (72%) were claim–mechanism gaps: a surface asserting some‐ thing the mechanism did not do. Every critical and every high-severity finding fell into this class; none of the ordinary engineering defects did. The gaps sort into eight types, and each type is exposed by a different probe, not by the venture's own tests. The venture was rebuilt rather than repaired. The rebuild bound claims to mechanisms through single sources of truth and tests, and replaced synthetic outreach with outreach built only from observed facts. Even so, six of the eleven issues found in the following day's checks were small claim–mechanism gaps in documentation and status labels. The paper contributes: a definition of simulated completeness that separates it from prototypes, technical debt and fraud; a taxonomy of claim–mechanism gaps; • •

Coldcode Si Labs Library WORKING PAPER 2026-02

Coldcode Si Labs Library Working Paper 2026-02 · First edition · 2026 a six-probe mechanism audit; a rebuild-or-repair test; seven propositions about how simulated completeness arises, where it does most harm and how it can be governed. The case is a single, pre-revenue venture analysed by an interested party. The propositions are offered for testing. Keywords: simulated completeness; claim–mechanism gap; AI-assisted software development; verification; trust products; credence goods; decoupling; bot detection; proof-of-work; design science

1. Introduction

For most of the history of software, a product that looked finished was usually close to finished. Interfaces were expensive to build, so they tended to be built last, on top of mechanisms that already worked. A dash‐ board full of numbers implied a database full of records, because nobody would pay to draw the first without the second. Buyers, investors and managers relied on this correlation, mostly without noticing. The demo was a reasonable proxy for the product. Generative AI weakens that correlation. A capable model can produce a polished landing page, an owner console with charts, a customer portal, user manuals, product covers, a test suite and a confident release note in hours. It can produce these whether or not anything behind them works. Field and laboratory studies report large productivity gains from AI assistance on bounded tasks [1], [2], [3]. The same literature reports outputs that are less secure than they appear [4], users who become more confident as their code becomes less secure [5], and experienced developers who believed AI made them 20% faster when it made them 19% slower [6]. Models also exhibit what has been called "potemkin understanding": correct-looking answers that are irre‐ concilable with any coherent grasp of the concept being applied [7]. If surfaces are cheap and mechanisms are not, the gap between what a product appears to do and what it does becomes a first-order management problem. This paper studies that gap in one venture where it was unusually large and unusually consequential. At‐ testOS is a bot-verification service: a small proof-of-work check runs in a visitor's browser, and the website's server verifies the resulting token before accepting a sign-up, login or contact form. Trust is the product. A bot-verification service that does not verify is worse than none, because its customers stop looking for the problem it claims to solve. On 26 October 2026 the venture's first version was committed to a public repository in eleven commits over two hours. It presented a landing page with live demonstrations, an owner "command centre", a paying-cus‐ tomer portal, product covers, manuals, downloadable integrations, a vulnerability scanner and an "autonom‐ ous scout" for outreach. On 27 October a read-only audit found that almost none of it worked as presented. Some of it was harmful to anyone who installed it. The venture was rebuilt the same day, and checked again live on 28 October. • • •

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

The case allows three questions to be asked with unusual precision, because every claim the first version made, every defect the audit found and every change the rebuild made is recorded in a public commit history and an itemised audit: RQ1. What forms does the gap between a software product's surface and its mechanisms take, and which of those forms carry the most risk? RQ2. How can such gaps be detected, and why do conventional controls such as test suites and demon‐ strations fail to detect them? RQ3. What design and governance choices close the gaps, and do the gaps stay closed? The paper answers these questions through a design-science case study [8], [9], [10]. Its contributions are as follows. First, it defines simulated completeness as a property of an artefact, not of its maker's intentions. It distin‐ guishes simulated completeness from three neighbouring ideas that it is often confused with: prototypes and "Wizard of Oz" simulations, which are disclosed or bounded by purpose [11]; technical debt, which is a known shortcut with a plan to repay it [12], [13]; fraud, which requires intent. Because the definition is intent-agnostic, it can be applied without knowing how an artefact was produced. That matters here, because this paper does not state how the first version of AttestOS was produced. Second, it offers a taxonomy of claim–mechanism gaps derived from coding all 65 audit findings: unen‐ forced controls, missing mechanisms, dangling references, fabricated state, theatrical actions, synthetic evid‐ ence, false or unsupported claims, and test theatre. In the case, these gaps account for every critical and highseverity finding. Third, it proposes a six-probe mechanism audit (Trace, Trigger, Vary, Resolve, Attack, Source). Each probe targets one or more gap types, and each is cheap relative to building. It also proposes a rebuild-or-repair test for deciding what to do once the gaps are known. Fourth, it reports what happened after the rebuild. Gaps recurred, at a smaller scale and in documentation and status labels, until claims were bound to mechanisms by single sources of truth and tests. This finding quali‐ fies any suggestion that a rebuild, or a single audit, settles the matter. Fifth, it states seven propositions connecting these observations to theories of signalling and credence goods [14], [15], [16], organisational decoupling [17], [18], symbolic management in new ventures [19], [20], and the economics of information security [21], [22]. The paper is the second in a series. The first studied a sister venture built under hard resource constraints and introduced the idea of verification debt: the accumulating gap between what a system was built to do and what anyone has checked that it does [23]. The present case shows what verification debt looks like when it is total, and what it takes to pay it down. • • • • • •

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

The remainder of the paper is organised as follows. Section 2 reviews the relevant literatures and identifies the gap this paper addresses. Section 3 describes the method, data and case. Section 4 reports findings on the first version, the audit, the rebuild and its aftermath. Section 5 develops the frameworks and propositions. Section 6 discusses implications, limitations and directions for research, and concludes.

2. Literature review

Simulated completeness sits where five literatures meet: the economics of quality uncertainty; institutional theory on the decoupling of appearance from activity; entrepreneurship research on symbolic management; research on AI-assisted software development; research on software verification and on human oversight of automation. The case domain, bot verification, has its own literature, which is reviewed last.

2.1 Quality uncertainty, credence goods and security markets

Akerlof's analysis of the market for "lemons" showed that when buyers cannot observe quality, sellers of low-quality goods can pass them off as high-quality. Prices then fall to reflect average quality, and good sellers may leave the market [14]. Spence showed how costly signals can separate types when the cost of the signal is correlated with the quality it signals [24]. The crucial condition is the correlation. A signal that is equally cheap for every type separates nothing. Darby and Karni distinguished goods by when their quality becomes observable. Search goods can be judged before purchase and experience goods after use. Credence goods cannot be judged even after use without ex‐ pert help, and these invite fraud [15]. Dulleck and Kerschbamer's review of credence-goods markets sets out the forms that fraud takes: overtreatment, undertreatment and overcharging. It also shows that liability, verifi‐ ability and reputation determine whether markets discipline it [16]. Security products are a canonical credence good. A buyer who installs a bot defence and sees no attacks can‐ not easily tell whether the defence worked or whether nobody attacked. Anderson argued that information se‐ curity fails as often for economic reasons as technical ones: misaligned incentives, and markets in which buyers cannot tell good security from bad, so that vendors compete on features and assurances rather than on protection [21]. Anderson and Moore extended the argument into a research programme on the economics of security [22]. Schneier's phrase "security theater" describes measures that provide the feeling of security without the reality [25]. Each of these accounts takes as given that a product either contains a mechanism or does not. None asks what happens when the cost of producing the appearance of a mechanism falls towards zero. In signalling terms, that is precisely the condition under which a signal stops separating types.

2.2 Decoupling: formal structure as ceremony

Meyer and Rowan argued that organisations adopt formal structures (job titles, procedures, certifications) be‐ cause those structures confer legitimacy, not because they make work more efficient. Where the formal struc‐ ture conflicts with efficient activity, organisations decouple the two. They maintain the structure ceremoni‐ • • • • •

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

ally while the real work proceeds differently, protected by a "logic of confidence and good faith" that dis‐ courages close inspection [17]. Bromley and Powell distinguished policy–practice decoupling, where adop‐ ted policies are not implemented, from means–ends decoupling, where policies are implemented but have little connection to the outcomes they are meant to achieve [18]. Simulated completeness is decoupling at the level of an artefact rather than an organisation. A dashboard is formal structure; the queries that should feed it are activity. A product with a polished console and no data behind it is policy–practice decoupling in software. The institutional literature also predicts the conditions under which decoupling persists. These are ceremonial inspection, high costs of verification and an audience inclined to good faith. All three describe a buyer, investor or manager looking at a demonstration. Power's account of the "audit society" adds a further warning. Audits can become "rituals of verification" that certify the auditability of a process rather than the quality of its outcomes [26]. A test suite that tests a mock is such a ritual. So is a release note that announces a "production release".

2.3 Symbolic management and the legitimacy of new ventures

New ventures lack the track record on which stakeholders normally rely, so they must acquire legitimacy by other means [27], [28]. Zott and Huy showed that entrepreneurs who used symbolic actions acquired re‐ sources more readily than those who did not. Such actions included conveying personal credibility, profes‐ sional organising, organisational achievement and the quality of stakeholder relationships [19]. Lounsbury and Glynn described how entrepreneurial stories create the legitimacy that precedes resources [20]. This lit‐ erature treats symbolic management as normal and often functional, and does not regard every demonstration that runs ahead of the product as deception. Product-development practice also legitimises simulation. In Kelley's "Wizard of Oz" method, a human be‐ hind the curtain simulates a system that does not yet exist, so that designers can learn from users before building it [11]. The lean-startup tradition recommends minimum viable products and tests of demand before construction [29]. What separates these practices from deception is not that they simulate. Three things sep‐ arate them: purpose: they simulate in order to learn; boundedness: they simulate a known, limited set of behaviours; disclosure or containment: the simulation is disclosed or is confined to research settings where no one relies on it for protection or payment. A Wizard of Oz study does not take money for a lock that does not lock. The boundary is important to this paper's argument. Simulated completeness is defined in Section 5 so as to exclude legitimate prototypes. The extreme cases are excluded as well, where symbolic management became the substance of the enterprise and investors and patients were harmed, of which the history of Theranos is the best documented [30]. Those cases are exactly what the legitimacy literature warns about when symbols run too far ahead of capability.

2.4 AI-assisted software development

Evidence on AI-assisted programming is mixed in a way that matters here. In a controlled experiment, de‐ velopers with an AI pair programmer completed a bounded task 55.8% faster [1]. Experiments on profession‐ al writing tasks [2] and field evidence from customer support [3] report substantial gains, concentrated • • •

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

among less experienced workers. A field experiment with consultants found a "jagged frontier": AI improved performance on tasks inside its capability frontier and degraded it on tasks outside, and users could not easily tell which side they were on [31]. The evidence on quality and judgement is less reassuring: About 40% of programs completed by an AI assistant in security-relevant scenarios contained vulnerabil‐ ities [4]. Users with an AI assistant wrote less secure code than users without one and were more likely to believe their code was secure [5]. In a randomised trial with experienced open-source developers, allowing AI tools increased task comple‐ tion time by 19%. Before the study, the same developers had forecast a 24% saving, and afterwards they estimated a 20% saving [6]. The gap between perceived and actual performance is itself a finding about how people judge AI-produced work. A second strand documents specific failure modes that produce surfaces without substance: Hallucination, the generation of fluent but unfounded content, is pervasive in language generation [32]. Package hallucination: code-generating models recommend software packages that do not exist. Across 16 models and 576,000 code samples, at least 5.2% of packages suggested by commercial models and 21.7% of those suggested by open-source models were hallucinated, with 205,474 unique invented names [33]. An attacker who registers an invented name receives the code execution intended for the real one. This links model behaviour directly to the software supply chain [34], [35]. Sycophancy: models tend to tell users what they appear to want to hear, and human preference data re‐ ward this [36]. Deception: a survey of AI deception documents systems that induce false beliefs in pursuit of other goals [37]. Reward hacking: in agentic coding environments, frontier reasoning models have been observed ex‐ ploiting flaws in their objectives, for example by making tests pass without solving the task. Optimising directly against a monitor can teach them to hide it [38]. Reward hacking is a specific case of the spe‐ cification problems catalogued in early AI-safety work [39]. It also illustrates Goodhart's law in its sever‐ al variants: a measure that becomes a target ceases to be a good measure [40]. Potemkin understanding: the finding that models produce benchmark-passing answers irreconcilable with coherent understanding [7] offers a vocabulary for the product-level analogue studied here. These literatures study models, tasks and code fragments. What is missing is the level of the product. That is, what happens when an entire product, with its interfaces, documentation, tests and outreach, is assembled un‐ der these conditions and then presented to people who must decide whether to rely on it. • • • • • • • • •

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

2.5 Verification, testing and oversight

Software engineering has long recognised that a test is only as good as its oracle, the means of deciding whether an output is correct. Barr and colleagues' survey of the "oracle problem" sets out why oracles are hard to obtain and how weak oracles produce false assurance [41]. Mocking is a common response to de‐ pendencies that are hard to test. Spadini and colleagues found that developers mock heavily and use mocks for many purposes [42]. A test suite composed of mocks verifies the mocks. Technical debt describes shortcuts taken knowingly, with interest paid until they are repaid [12], [13]. The first paper in this series proposed verification debt to describe a different quantity: the gap between what a system does and what anyone has checked that it does, which grows when building outpaces checking [23]. Technology readiness levels offer a staged vocabulary for how far a technology has been demonstrated, from concept to operational use [43]. They make the point that "demonstrated" must always be qualified by "in what environment". Human-factors research explains why demonstrations persuade. People misuse and over-trust automation [44]. Complacency and automation bias are robust effects, strongest when the automation is usually right and the human is busy [45]. Bainbridge's "ironies of automation" observe that the more a system is automated, the less practice its human supervisors have at catching its failures [46]. Reason's model of accidents as aligned holes in successive defensive layers emphasises that each layer of a defence may have its own latent failures [47]. Campbell's law states that the more a quantitative indicator is used for decisions, the more it be‐ comes subject to corruption pressures [48]. That law applies with special force when the indicator is gener‐ ated by the same process it is meant to evaluate.

2.6 Bot verification: CAPTCHAs, proof-of-work and private tokens

The case domain has a mature literature. CAPTCHAs were proposed as tests that humans pass and current programs fail, grounded in hard AI problems [49]. Two decades of evidence complicate the promise: Humans often find them hard [50]. Human labour markets solve them on demand at retail prices as low as $1 per thousand. CAPTCHA se‐ curity is therefore best understood in economic terms: the market price of a solution against the value of what it protects [51]. A study of modern deployed CAPTCHAs, with 1,400 participants solving 14,000 challenges, found that solving time and user perception were not always correlated, that experimental context mattered, and that CAPTCHAs cause users to abandon tasks [52]. Industry measurement attributes 51% of web traffic in 2024 to automation and 37% to "bad" bots [53]. Proof-of-work offers a different bargain: make each request cost the requester computation. Dwork and Naor proposed "pricing via processing" to combat junk mail [54]. Juels and Brainard proposed client puzzles against connection-depletion attacks [55], and Back's Hashcash popularised the idea [56]. Laurie and Clayton showed that a puzzle hard enough to deter spammers, who can steal computation from compromised ma‐ chines, would stop significant numbers of legitimate senders [57]. Liu and Camp replied that proof-of-work can work if the required work is weighted by reputation [58]. The literature therefore supports a precise, modest claim for proof-of-work: it imposes a cost, and it is useful alongside other signals, but it does not dis‐ tinguish humans from machines. • • • •

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

Privacy Pass introduced anonymous tokens that let a client who has passed a challenge once prove it later without being tracked [59]. It has since been standardised as an architecture, an HTTP authentication scheme and issuance protocols [60], [61], [62]. Web authentication with public-key credentials ("passkeys") is stand‐ ardised by the W3C [63]. The governance of automated readers, as opposed to attackers, has its own instru‐ ments: The Robots Exclusion Protocol, standardised in 2022 [64]. An audit of 14,000 web domains showing a rapid rise in restrictions on AI crawlers in 2023–2024 [65]. Commercial experiments that answer crawlers with HTTP 402 "Payment Required" [66]. Vulnerability disclosure has a standard contact file, security.txt [67].

2.7 The gap

Three observations follow from this review. First, the economics of signalling and credence goods predicts that when the cost of a signal no longer correlates with quality, signals stop separating types. Generative AI appears to be decorrelating the cost of software surfaces from the quality of software mechanisms, but no study has examined that decorrelation at the level of a whole product. Second, institutional theory offers a vocabulary of decoupling, and entrepreneurship research a vocabulary of symbolic management, but both were developed for organisations and people, not for artefacts that can produce their own ceremonial struc‐ ture. Third, research on AI-assisted development documents specific failure modes in models and code frag‐ ments, including hallucinated packages, insecure code, overconfidence and reward hacking, but not how they compose in a shipped product. It also says little about how leaders should detect and govern them. This paper addresses that gap with a case in which the decoupling of surface from mechanism was nearly total and the consequences of reliance were serious, and in which every claim and defect is documented.

3. Method and case

3.1 Research design

The study follows design-science research, which builds and evaluates artefacts to solve identified problems and draws prescriptive knowledge from the process [8]. It uses the process model of Peffers and colleagues [9]: Problem identification: the audit. Objectives: the rebuild specification. Design and development: the second version. Demonstration: deployment. Evaluation: live checks on the following day. Communication: this paper. In the terms of Gregor and Hevner, the contribution is an improvement, a new solution to a known problem, together with nascent design theory expressed as frameworks and propositions [10]. • • • • 1. 2. 3. 4. 5. 6.

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

The empirical vehicle is a single case. Single cases are appropriate when a case is revelatory, giving access to a phenomenon previously inaccessible to study, or when it is extreme [68]. Siggelkow argued that a single striking case can be persuasive precisely because it shows that something can happen and allows its mechan‐ isms to be inspected [69]. The AttestOS case is both. It is revelatory because every claim and every change is preserved in a public commit history and an itemised audit. It is extreme because the decoupling of surface from mechanism in its first version was close to complete. The case is used to build constructs and proposi‐ tions, not to estimate frequencies.

3.2 Data

Five sources of data were used. All are contemporaneous records, not recollections. The public repository. All 20 commits from 26 to 28 October 2026, their messages and their diffs, to‐ gether with the full tree at the end of the first version (commit 23d3f00) and at the end of the observation window (commit 963c469). The audit report. A read-only audit carried out on 27 October 2026, recording 65 findings with severity ratings (critical, high, medium, low), evidence and the live responses to proof-of-concept requests. The audit read every source file and confirmed that the deployed script was byte-identical to the repository's worker file. It probed every live route, sent proof-of-concept requests to production, and checked domain registrations, package registries, payment-platform links, provider analytics and git history. The rebuild plan and progress log. A working document recording the owner's decisions about the re‐ build (depth, outreach policy, deployment), its architecture and its progress. Operating data. The second version's production database on 28 October 2026: counts of accounts, sites, leads, prospects, messages, redeemed tokens and stored passkeys; per-site verification statistics; and the venture's own daily meter of database reads, writes and requests. The second version's source and tests. These include the tests that bind marketing facts to code, for ex‐ ample the test that the published price list equals the plan definitions.

3.3 Analysis

Analysis proceeded in four steps. Coding the findings. Each of the 65 findings was coded by asking one question: does the finding concern a surface that asserts something the mechanism does not do? A surface means an interface, document, test, message or claim. Findings that did were classed as claim–mechanism gaps and assigned to one of eight types, developed inductively and refined until every gap finding fitted exactly one type. Findings that did not were classed as engineering hygiene, for example tracked files that the ignore-list names, missing security headers, or an outdated compatibility date. Severity ratings were taken from the audit as recorded and were not revised. Mapping detection. For each gap type, the evidence that established the finding was examined to identify how it had been detected: by reading code, by sending live requests, by resolving an external reference or by checking a claim against its source. The detection procedures were then abstracted into probes. 1. 2. 3. 4. 5.

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

Tracing the rebuild. The rebuild commit and the commits that followed were read against the audit to estab‐ lish which findings each change closed, and how. The issues fixed in the follow-up commits of 28 October were coded with the same scheme as the audit findings. Measuring operation. Operating data were extracted directly from the production database and the venture's meter. Because one analytic process produced the audit, the coding and this paper, the coding is open to the bias that the coder's categories reflect the auditor's framing. Two safeguards were applied. The coding scheme and the full assignment of findings to categories are reported (Table 2), so that readers can recode. And the central quantitative claim, that severity concentrates in claim–mechanism gaps, is reported alongside the rule that generated it, so that its sensitivity can be judged. Section 6 returns to this limitation.

3.4 The case

The product. AttestOS sells bot verification for websites. A small script on a sign-up, login or contact form asks the AttestOS service for a signed challenge and solves a proof-of-work puzzle in the visitor's browser. It then trades the solution for a short-lived pass token. The website's server sends that token, with its secret key, to a verification endpoint whose request and response have the same shape as the industry-standard re‐ CAPTCHA "siteverify" call. It accepts the form only if the token is valid. The service is designed to be invis‐ ible to most visitors. Paid modules add: automation signals and a risk score; Private Access Tokens, which let some devices pass without solving [61], [62]; passkeys [63]; "AgentPass", a per-crawler policy that allows, charges (with HTTP 402) or blocks automated readers, and checks a crawler's claimed identity against the IP ranges its operator publishes. The owner's constraints. The venture shares an owner and constraints with the sister venture studied in the first paper of this series [23]: a zero cash budget; checkout through a third-party merchant platform; email through a free mailbox; a free serverless plan; a self-imposed ceiling of 20% of any free quota. For the rebuild the owner added four rules: honest claims only, no fabricated numbers, outreach based only on real scans, and the full product, free and paid tiers both real. The first version (26 October 2026). Eleven commits over two hours created the repository. The first was titled "initial production release v2.0.0 with Cloudflare edge worker". Later commits added: an owner "command centre", a landing page and a paying-customer portal; product covers, "live screenshots" and user guides; a design system and footer borrowed from the sister venture; • • • • • • • • • • • •

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

an "Autonomous Scout & First-Touch Inbound Mesh with RFC 9116 security inbox routing"; "Customer-in-the-Loop" enforcement; a "VIP Enterprise tenant" with "1-click credentials". At the end of the day the repository held 68 files and 21.2 MB, about 18 MB of it in unused images. The de‐ ployed service was a separate 590-line file of 838,648 bytes, 77.7% of which was base64-encoded images. A larger application in another directory (52 files: tenants, a payment gate, an "auto-tuner", hardware attesta‐ tion, the scout) was never deployed. Its 30 tests exercised that undeployed code. The audit (27 October 2026). The owner asked for a deep-dive audit of the kind previously applied to a sis‐ ter venture. The audit was read-only. Its findings are analysed in Section 4. The rebuild (27–28 October 2026). On the audit's evidence, the owner chose a full rebuild covering both tiers, outreach based on real scans in place of the scout, and live deployment after local testing. The rebuild was committed at 23:07 UTC on 27 October as a single service on one serverless worker, with a database, static pages and secrets held by the platform. The commit changed 156 files, adding 10,070 lines and remov‐ ing 9,342. A distribution engine for social channels, "armed" but dormant until accounts exist, followed at 23:23 UTC. On 28 October, live checks in a browser and the owner's own use produced six further commits.

Figure 1. Timeline of the case, 26–28 October 2026 (from the commit history and audit report)

Date (UTC) Event 26 Oct, 14:21– 16:21 First version: eleven commits; landing page, owner console, customer portal, covers, manuals, integrations, scanner, scout 27 Oct Read-only audit: 65 findings (18 critical, 21 high, 17 medium, 9 low); seven bypasses confirmed against production 27 Oct, 23:07 Rebuild committed: one worker, real free and paid tiers, real scanner, outreach from real scans, 47 tests 27 Oct, 23:11– 23:23 Bootstrapping of crawler ranges and token keys; distribution engine added, armed but dormant 28 Oct, 10:22– 11:23 Six follow-up commits fixing eleven issues found by live checks and the owner

State at the end of the window. On 28 October the production database held: one account (the owner's demonstration account) and one site; one lead (the owner's own sign-up), whose verification email had been sent; ten prospects found by the daily discovery job, all showing a visible reCAPTCHA v2 challenge and pub‐ lishing a role contact address, none yet contacted; four redeemed verification tokens. There were no payments, no stored passkeys and no agent keys. Outbound email was held by design until a postal address required by anti-spam law was configured. The venture was operational, with no customers and nothing sold. • • • • • • •

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

4. Findings

4.1 What the first version presented

The first version presented every signal a buyer, investor or manager would normally take as evidence of a working product. Its owner console reported 1,489,204 requests inspected, 142,380 blocked, 312,450 hard‐ ware attestations, 84.6 hours of attacker CPU "burned", $482.16 of revenue from AI crawlers, growth of 12.4%, and "Whop Webhooks: Connected (2 Plans Active)". Its customer portal reported 124,180 of 500,000 monthly verifications used, 86,410 hardware passes and "$42.80 AI micropayments earned". A policy change reported "Propagated… in <0.03ms". The landing page offered a five-second scan, live demonstrations, four product modules with covers, interactive return-on-investment calculators and downloadable integrations. The repository carried a "27/27 tests passing" badge. The commit history spoke of a "production release", "enforcement" and "provisioning". None of these signals was connected to a mechanism that would make it true. The following subsections de‐ scribe what lay behind them.

4.2 What lay behind the surface

The security engine admitted almost anything. Seven bypasses were confirmed against production: The signing secret was hard-coded in a public repository, so anyone could forge a session cookie that the service accepted. Any value in a "hardware token" field was accepted as "Secure Enclave Verified". A request could exempt itself by declaring that its path began with /public. Sending the user-agent string of an AI search crawler was enough to be admitted. Any value in an agent-key header "cleared" the paid crawler product. The client could choose the proof-of-work difficulty. At difficulty 1 a solution takes about sixteen hashes. The same solution verified repeatedly, despite documentation that promised replay protection. The owner console had no authentication and was pre-filled with a "master key" that also appeared in its HTML source. Customers could not become customers. There was no way to obtain a site key: the page that promised one, and the endpoint behind it, both returned "not found". All checkout links used placeholder plan identifi‐ ers that existed on no payment platform. There was no server-side verification endpoint, so even a site that installed the widget had no way to check a token on its server. A session cookie described as cutting verifica‐ tion costs by more than 90% could never be stored by a browser on a customer's site. It was set by a thirdparty domain on a response to a cross-origin request. The lead magnet invented its results. The scanner posted to an endpoint that did not exist. It then displayed the same findings for any address entered ("100% Solvable", "VULNERABLE", "Human users fail this test 18% of the time"), including sites with no CAPTCHA at all. 1. 2. 3. 4. 5. 6. 7.

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

The integrations harmed the people who installed them. The WordPress plugin loaded its script from a subdomain of an unregistered domain. The script therefore never loaded, the plugin's login check found no token, and with default settings it rejected every login. A site owner who installed it would be locked out of their own site. Anyone who registered the domain could serve JavaScript to the login pages of every site that had installed the plugin. The portal told customers to install a server package that did not exist on the public package registry, so whoever first published that name would obtain code execution on customers' servers. The outreach engine fabricated security findings. The "autonomous scout" drafted messages headed "[Se‐ curity Notice]" to security@, appsec@ and engineering@ addresses at target companies. Each message re‐ ported a vulnerability score, a bypass latency and a monthly loss figure. These numbers were computed from an MD5 hash of the recipient's domain name, in a function named generateSyntheticAudit. The messages claimed that "our engine recorded an environment integrity gap" and offered "a private, reproducible audit trace". No scan took place. The engine was not wired to send. The console's "Dispatch" button nevertheless printed "✓ Dispatched … via Cloudflare Edge". Authority was borrowed or wrong. The site cited "RFC 9505" as the standard for Private Access Tokens. RFC 9505 is A Survey of Worldwide Censorship Techniques [70]; Private Access Tokens are specified in RFCs 9576–9578. The site also cited an "RFC robots.ai" that does not exist. The proof-of-work labels were wrong by orders of magnitude. Difficulty was counted in hexadecimal digits, so each step multiplied the work by 16. A setting labelled "Heavy, ~1.2 s" required about 16.7 million hashes, and an "emergency lockdown" setting required 2⁶⁴, which no browser could complete. A claimed "256×– 4096× bot CPU tax" ignored that a native program computes hashes far faster than a browser. Prices contra‐ dicted each other: the landing page gave $49, $149, $99 and $299, while the configuration gave $39 and perrequest micro-prices that could not be charged through the chosen payment platform, whose fees include a fixed charge per transaction. The tests tested the mock. The 30 tests exercised the undeployed application. Some asserted that magic val‐ ues passed: a licence string beginning whop_demo_ unlocked paid tiers, and a hardware token signed with the literal text valid_hardware_enclave_signature_token was "verified".

4.3 The anatomy of the gaps

Table 1 summarises the coding of the 65 findings. Table 2 gives the assignment of every finding.

Table 1. Audit findings by type and severity

Type What the surface asserts Example in the case Findings Critical High Medium Low Unenforced control "This check protects you" Any hardware token accepted Missing mechanism "This capability exists" No server-side verification at all Dangling reference "This dependency exists" Script loaded from an unregistered domain Fabricated state "These numbers are measured" Console traffic and revenue figures Theatrical action "This button did something" "Provision tenant" adds a table row only Courage Garikai Coldcode Si Labs Library Working Paper 2026-02
Type What the surface asserts Example in the case Findings Critical High Medium Low Synthetic evidence "We observed this about you" Scanner and scout findings from fixed text or a hash False or unsupported claim "This is true of the product or the world" "RFC 9505"; "<80 ms"; contradictory prices Test theatre "This is tested" Tests exercise the undeployed mock All claim– mechanism gaps Engineering hygiene (no assertion) Missing security headers; tracked files the ignore-list names Total

Table 2. Assignment of audit findings (identifiers as recorded in the audit)

Type Findings Unenforced control C1–C8, H3, M15 Missing mechanism C12, H1, H2, H5, H7, H8, H10, H12, H14, H15, H16, M14 Dangling reference C13, C15, C16, C17, L6 Fabricated state C9, C11, H6 Theatrical action C10, H4, H13, L7 Synthetic evidence C14, C18 False or unsupported claim H11, H17–H21, M1, M3, M17, L9 Test theatre H9 Engineering hygiene M2, M4–M13, M16, L1–L5, L8

Three features of the distribution stand out. First, claim–mechanism gaps dominate: 47 of 65 findings (72%). The first version did not mainly fail in the way software usually fails, through bugs in mechanisms that exist. It failed because the mechanisms its sur‐ faces implied were absent, unenforced or fictional. Second, severity concentrates entirely in the gaps. All 18 critical and all 21 high-severity findings were claim–mechanism gaps. None of the 18 hygiene findings was rated above medium. In the case, the question "does the surface assert something the mechanism does not do?" separated every serious finding from every ordinary one. Because severity was rated by the auditor before the coding scheme existed, this is not a con‐ sequence of the coding rule. It is open to the objection that the auditor and the coder shared a frame (Section 6.3). Third, the gap types are distinct in kind, not just in degree. An unenforced control is a mechanism that ex‐ ists but does not discriminate. A missing mechanism is a capability that is described but has no implementa‐ tion path. A dangling reference points to something that does not exist outside the product. Fabricated state,

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

theatrical actions and synthetic evidence differ in what is faked: a measurement, an effect and an observation about a third party, respectively. The distinction matters because, as Section 4.4 shows, each type yields to a different detection method.

4.4 How the gaps were found

The venture's own controls did not find any of the gaps. Its test suite passed and its demonstrations ran. Its commit messages described each feature as delivered. The audit found them by six distinct procedures, sum‐ marised in Table 3.

Table 3. Detection probes and the gap types they expose

Probe Procedure Gap types exposed Case example Trace Follow a displayed value back to the query or record that produces it Fabricated state Console totals were literals in the page source Trigger Perform an action, then look for its side effect in storage, logs or the outside world Theatrical action "Save & sync policy" wrote only to the browser; the server echoed the request Vary Change an input that should change the output, and check whether it does Synthetic evidence The scanner gave identical findings for any address; scout scores were a function of the domain name only Resolve Check that every external name exists and is controlled: domains, packages, routes, plan identifiers Dangling reference; missing mechanism Script domain unregistered; package absent from the registry; key-issuing route returned "not found" Attack Attempt to pass a control with input that should fail Unenforced control Arbitrary token, spoofed user-agent and client-chosen difficulty were all admitted Source Check a claim against its authority or against arithmetic False claim; test theatre RFC 9505's title; 16⁶ hashes for "~1.2 s"; tests import the undeployed module

The probes share three properties. Each is cheap: the whole audit was completed within a day, for a product committed in a day. Each is external to the artefact, in that it consults something the artefact does not control, such as the database, the network, the domain registry, the standards body or the arithmetic. And none of them is a test in the ordinary sense. A test checks that the code does what the code's author expected. A probe checks that the product does what its surface says. Where the author's expectations are themselves part of the surface, as with tests that assert magic values, a test suite can only confirm the surface. This is the oracle problem in an acute form [41].

4.5 What an outsider could have seen

A striking feature of the case is how little privileged access the probes required. Of the 18 critical findings, 16 could have been established by someone with no access to the venture's systems, using only four things: the public website, the public verification interface its widget called, a domain-name lookup and a search of the public package registry. Domain lookup and registry search (Resolve): the unregistered domain behind the plugin's script, the takeover risk that followed from it, and the absent server package. Following the public buttons (Resolve): the "not found" response behind the key-issuing page and the placeholder checkout links. Two scans of different addresses (Vary): the scanner's invented findings. • • •

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

The public interface (Attack): the seven bypasses. Visiting the public paths (Trace and Trigger): the owner console and the portal. Their figures did not change between visits, and their actions did not survive a reload. Only the public signing secret and the scout's synthetic findings required reading source code. Because the repository was public, even those were visible to anyone who looked. The implication is uncomfortable for every party that might have relied on the first version. A buyer evaluat‐ ing it, an investor shown it or a journalist writing about it could have found most of its critical defects in un‐ der an hour, with tools available to everyone. That they would probably not have done so is the point of the institutional literature on decoupling. Inspection of formal structure is ceremonial by default [17], and fluent surfaces invite the trust that makes inspection seem unnecessary [45]. The case also shows the converse. An honest product can make these outsider probes easy and treat their res‐ ults as marketing. Since the rebuild, a prospective customer can do three things: send a forged or reused token to the verification interface and watch it fail; look up every domain the integrations reference and find only the live service; read the price list from the same interface that enforces it.

4.6 The rebuild decision

The audit offered the owner three depths of remedy: Make the product safe and honest. Make the free tier real. Make the paid tier real. The owner chose the third and decided to rebuild rather than repair. The record supports three reasons: Scale and kind of gap. With 47 claim–mechanism gaps, repair would have meant replacing most mech‐ anisms while preserving surfaces that were themselves false. Unsuitable foundation. The deployed service used the hosting platform's legacy script format. The audit concluded that fixing the most critical finding, the public signing secret, required rewriting the service in the platform's current module format, with secrets and a database bound to it. Tangled structure. Surface and mechanism were not separable. The deployed file was more than threequarters embedded images, and the application that held most of the logic was never deployed. The rebuild took four structural decisions, each aimed at a gap type (Table 4).

Table 4. How the rebuild addressed each gap type

Gap type Rebuild mechanism Unenforced control Challenges and tokens signed with keys derived from a platform-held secret; difficulty chosen only by the server; tokens bound to site and domain; single use enforced by a database insert; each property covered by a named test

• • • • • 1. 2. 3. • • •

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02
Gap type Rebuild mechanism Missing mechanism Self-service accounts and site keys; a server-side verification endpoint; payment-platform webhooks verified to the Standard Webhooks scheme and mapped to entitlements; server-verified passkeys and private tokens Dangling reference Every reference to the unregistered domain removed; integrations call only the live service; no instruction to install unpublished packages Fabricated state Console and portal read only from the database; empty states shown as empty Theatrical action Every control calls an endpoint that writes or sends; outreach that cannot be sent lawfully is shown as held Synthetic evidence The scanner fetches the page, its robots.txt and its security.txt, and reports only what it found; outreach is built only from those facts False or unsupported claim Corrected standards citations; honest limits stated in the public README ("Proof-of-work makes every automated submission cost CPU time. It doesn't prove identity") and the documentation; one plan file drives the API, portal and pricing page Test theatre Tests run against the deployed code paths with an in-memory database; tests bind published facts to code

Two of these decisions recur across types and are worth singling out. The first is the single source of truth for claims. Plan names, prices and limits live in one file, which the pricing page reads through the public API. A test fails the build if the distribution engine's copy of those facts differs from the plan file, and the en‐ gine refuses to publish a post containing a price shape or claim not in its facts file. The second is honest lim‐ its as content. The second version's README and documentation state what proof-of-work cannot do, con‐ sistent with the literature [57], [58]. Crawler identity is reported as "verified: null" when an operator pub‐ lishes no IP ranges, rather than guessed. Honest limits in numbers. The second version's default difficulty is 16 bits, which requires about 65,000 hash computations on average. The product's documentation estimates that a phone computes roughly 0.5–2 million hashes a second. A development test of the widget's solver, run outside a browser, measured about 1.7 million. The expected cost for a visitor is therefore roughly 0.05–0.3 seconds, done in the background. The first version had claimed "<80 ms" for the same amount of work, a figure the audit estimated at 0.3–1 second for the method that version used. The same arithmetic shows why the documentation must not claim more. An attacker running native code on rented machines pays a cost that is small in absolute terms. Human CAPTCHA-solving labour has long been available at around a dollar per thousand solutions [51], and the work needed to deter a determined, well-resourced sender would burden legitimate users [57]. Proof-of-work in this design is one input among several: it raises the cost of bulk submissions, while risk signals, rate limits, passkeys and private tokens do the discriminating [58]. The first version's claim of a "256×–4096× bot CPU tax" and the second version's statement that proof-of-work "doesn't prove identity" describe the same family of mechanism. Only the second is true.

4.7 Truthful outreach

The rebuild replaced the scout with an outreach system whose messages can contain only observed facts: A daily job looks for sites that show a visible CAPTCHA, identified by one of thirteen vendor patterns in the page, and that publish a role contact address. Government, military and academic domains are excluded. A message names the CAPTCHA it saw and offers the free scanner. • • •

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

A follow-up is sent after four days if there is no reply. Unsubscribe requests are honoured through a suppression list. Nothing is sent until the operator has configured the postal address that anti-spam law requires in com‐ mercial email. At the end of the window, ten prospects had been found, all using reCAPTCHA v2, and none had been con‐ tacted because the postal address was not yet set. The design trades reach for verifiability: every claim in an outreach message can be checked by its recipient by looking at their own page.

4.8 The gaps returned, smaller

The rebuild did not end the story. On 28 October, live checks in a browser and the owner's own use of the console led to six commits fixing eleven issues (Table 5).

Table 5. Issues fixed after the rebuild, 28 October 2026

Issue How found Type Documentation described replay protection inaccurately Review against code False claim Documentation listed widget attributes the widget did not read Review against code False claim Documentation's error table listed codes the verifier never returned Review against code False claim Documentation described the difficulty surge rule incorrectly Review against code False claim Site header said "Log in" to signed-in users on public pages Live browser check Status gap Owner console showed distribution channels as "active" when no account existed The owner Status gap Layout overflow in a grid Live browser check Hygiene Automation runs held because a bulk export of secrets was flagged Platform Hygiene Empty notice in the portal's catalogue Review Missing mechanism (minor) Console tab empty after signing in via a deep link Live browser check Hygiene robots.txt hid pages whose "noindex" tags crawlers therefore never saw Review Hygiene

Six of the eleven were claim–mechanism gaps, and all six were small. Four were documentation that had drifted from code written hours earlier. Two were status labels that overstated state. The second of these was noticed by the owner, not by any test or check: distribution channels with no account were shown as "active". The fix replaced a two-state vocabulary with three states: active; "on (waiting for account)"; and parked. The pattern is important. The rebuild eliminated fabricated state, theatrical actions, synthetic evidence, dangling references and unenforced controls as classes. But it did not eliminate the tendency for surfaces to run ahead of mechanisms. That tendency reappeared wherever a surface was written separately from the mechanism it described and nothing bound the two. The surfaces at issue were documentation, status vocabu‐ lary and the signed-in state of a header. Where a binding existed, as for prices, plan limits and the secrets list, no drift was found. • • •

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

Preparing this paper surfaced one more gap of the same kind, which had not been fixed at the time of writing. The public documentation states that a pass token "verifies once", and that this is "enforced in the database". The code honours this except in one condition: when the service's self-imposed daily budget of 20,000 data‐ base writes is used up. In that condition it verifies tokens without recording them, so a token could be used more than once, and it marks the response stateless: true. A comment in the code calls this a "docu‐ mented degradation", but no public document describes it. The design choice is defensible: it keeps custom‐ ers' forms working instead of failing them. On the last day of observation the service used 110 of its 20,000 daily writes, so the condition had not arisen. But the surface had again run ahead of the mechanism, in the one place where nothing bound the two. Even the comment asserting that a disclosure existed was itself an unchecked claim.

4.9 Operating footprint

The second version ran well inside its constraints. On its busiest observed day it recorded 1,065 database reads, 110 writes and 70 requests by its own meter, against self-imposed budgets of 1,000,000 reads and 20,000 writes. That is 0.11% of the provider's free daily write allowance. Its repository is public, so its auto‐ mation minutes are free. The verification statistics for the demonstration site on 27 October recorded four successful and three failed verifications, one allowed and three blocked crawler checks, and two requests flagged as risky. These figures demonstrate that the mechanisms run. They say nothing about efficacy against real attackers, of which there were none to observe.

5. Frameworks and propositions

5.1 Simulated completeness defined

Definition. An artefact exhibits simulated completeness when its surfaces signal operational capabilities that its mechanisms do not provide. Surfaces include interfaces, displayed values, documentation, test reports, status labels, release notes and outbound messages. The signal must also be addressed to parties who may rely on it. The definition has four elements, each of which does work: A surface and a mechanism. The concept applies wherever an artefact makes visible claims about its own behaviour. It does not apply to an artefact that makes no claims. A signal of operational capability. The surface must imply that something works, not merely that it is planned. A roadmap is not simulated completeness; a dashboard reporting traffic that does not exist is. The absence of the mechanism. The gap must be real, established by a probe (Section 5.3), not suspec‐ ted. Parties who may rely on it. These include buyers, users, investors, managers, recipients of messages and the operator. A sandbox demonstration that no one relies on does not qualify. The definition is silent on intent and on provenance. This is deliberate. Simulated completeness is a prop‐ erty of the artefact that can be established by inspection, whereas intent and provenance usually cannot be. A product can exhibit simulated completeness because a person built a demonstration and forgot to label it, be‐ 1. 2. 3. 4.

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

cause a team shipped under deadline, or because an AI system generated surfaces faster than anyone checked them. It can also happen because someone set out to deceive. The managerial response begins the same way in every case: find the gaps. Table 6 places the concept among its neighbours.

Table 6. Simulated completeness and neighbouring concepts

Concept Is the gap known to its maker? Is it disclosed to those who rely on it? Purpose Plan to close it Prototype or Wizard-ofOz simulation [11] Yes Yes, or confined to research Learning Build if learning supports it Minimum viable product [29] Yes Scope is honest; features are few but real Learning from real use Iterate Technical debt [12] Yes Usually internal Speed Repay with interest Verification debt [23] Partly: what is unchecked is known, not what is wrong Usually internal Speed Check Simulated completeness Not necessarily No: the surface asserts completeness Not defined Not defined Fraud Yes Concealed Gain from false belief None

Two consequences follow. First, simulated completeness can coexist with good faith. That is what makes it a governance problem rather than only an ethical one: good faith does not prevent it. Second, it is the absence of a plan to close the gap that distinguishes simulated completeness from debt. A technical-debt register is a surface that tells the truth about a gap.

5.2 A taxonomy of claim–mechanism gaps

The eight gap types of Table 1 can be arranged by what the surface asserts: Assertions about protection: unenforced controls. Assertions about capability: missing mechanisms and dangling references. Assertions about the present state: fabricated state and status gaps. The latter were observed in the second version, and are fabricated state's mild form. Assertions about effects: theatrical actions. Assertions about third parties: synthetic evidence. Assertions about the world: false or unsupported claims. Assertions about assurance: test theatre. The taxonomy is offered as a checklist for review. Its value lies less in classification than in coverage: a re‐ view that has asked all seven kinds of question has not left a whole class of gap unexamined. Assertions about third parties deserve particular attention because they carry legal and reputational exposure for the maker. The first version's scout would have asserted things about the security of named companies to their security teams. • • • • • • •

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

5.3 The six-probe mechanism audit

The probes of Table 3 can be used as a review protocol. For every surface element, apply the probe that matches what it asserts: Trace every displayed number to a query. Trigger every action and look for its effect outside the browser. Vary every input that is claimed to matter. Resolve every external name. Attack every control with input that should fail. Source every claim of standard, statistic or performance. The protocol has three design rules. Probe from outside. Each probe consults something the artefact does not control. Reading the artefact's own tests or documentation is not a probe, because those are surfaces. Prefer cheap probes early. Resolve and Source take minutes and require no access to code. An outsider such as a buyer, investor or journalist can apply them to any product. Trace and Trigger require access to the running system. Attack requires care and permission. Record negative results. A probe that finds nothing is evidence, and the record of probes applied is itself a surface that must tell the truth.

5.4 The rebuild-or-repair test

Once gaps are known, the operator must choose between repairing the artefact and rebuilding it. The case suggests four questions: Kind. Are most serious findings claim–mechanism gaps or hygiene? Hygiene is repaired; widespread gaps imply that the surfaces themselves must be rewritten. Foundation. Can the most critical gap be closed on the current foundation? In the case it could not: the service format could not hold secrets. Separability. Can surface be separated from mechanism? If the surface is entangled with mechanism (embedded assets, logic in pages, an undeployed second application), repair preserves the entanglement. Truth of the remaining surface. After the repairs, what proportion of the surface would still be true? If little, the surface is a liability and should be rewritten from mechanisms outward. When most answers point to rebuild, the case suggests rebuilding from the mechanism outward. Build the mechanism first, then the tests that bind the mechanism to its claims, then the surfaces that read from the mechanism. This reverses the order in which surfaces are cheapest to produce. • • • • • • 1. 2. 3. 4.

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

5.5 Binding claims to mechanisms

The follow-up issues of Section 4.8 suggest a design principle: every claim should have exactly one source, and that source should be either the mechanism itself or a file the mechanism reads. Where a claim must be written separately (documentation, marketing copy, status labels), a test or gate should fail when claim and mechanism diverge. The second version contains three instances of this: the price list read by the pricing page from the plan definitions; a test that the distribution engine's facts file equals the plan definitions; a guard test that every secret named in the setup guide is exported by the workflow. It contained none for its documentation, and that is where drift appeared. A status vocabulary is a special case. A binary label (active or inactive) cannot represent a capability that is switched on but waiting for an external condition, and so forces a false claim one way or the other. The fix adopted in the case, "active / on (waiting for account) / parked", generalises. A status vocabulary should have a state for every condition the mechanism can be in that matters to the person reading it.

5.6 Propositions

The following propositions connect the case to the literatures reviewed in Section 2. They are stated so that they can be tested in samples of ventures or products. P1 (surface–mechanism cost asymmetry). Generative AI reduces the cost of producing software surfaces more than it reduces the cost of producing the mechanisms those surfaces describe. Other things equal, products built with heavy AI assistance will therefore show larger surface-to-mechanism ratios unless mech‐ anism-level checks are imposed. Grounding: signalling theory predicts that signals stop separating types when their cost no longer correlates with quality [24]. The first version's deployed service was more than three-quarters embedded images, and its logic ran to a few hundred lines. Test: compare ratios of interface, documentation and asset volume to verified mechanism (endpoints with side effects, enforced controls) across products built with and without heavy AI assistance. P2 (demonstration selects for simulation). Where acceptance of software is based on demonstrations, screenshots or the builder's report, simulated completeness will be accepted at higher rates than where ac‐ ceptance is based on mechanism probes. The selection effect will be stronger the more fluent and complete the surface. Grounding: the logic of confidence and ceremonial inspection [17]; automation bias [45]; per‐ ceived versus actual productivity [6]; symbolic management [19]. P3 (severity concentrates in claim–mechanism gaps). In audits of products exhibiting simulated complete‐ ness, claim–mechanism gaps will account for a disproportionate share of high-severity findings relative to or‐ dinary engineering defects. Grounding: in the case, all 39 critical and high findings were gaps. Test: recode independent audit corpora with the scheme of Table 2. P4 (credence goods amplify harm). The harm caused by simulated completeness increases with the degree to which the product is a credence good. Security, compliance, health and financial-protection products will show the largest harms per gap, because buyers cannot observe failure until it is exploited. Grounding: cre‐ dence-goods theory [15], [16]; security economics [21]. In the case, a bot defence that admitted any token would have reported success indefinitely. • • •

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

P5 (dangling references become supply-chain exposure). References to external names that do not exist or are not controlled will convert into security exposure for downstream users at a rate that depends on how cheaply those names can be claimed. Examples are domains, packages, repositories, routes and plan identifi‐ ers. Products assembled with generative AI will contain more such references. Grounding: package hallucin‐ ation [33]; dangling DNS records [71]; supply-chain attacks [34], [35]. P6 (probes find what tests miss). External mechanism probes will detect claim–mechanism gaps at materi‐ ally higher rates than the product's own test suite. The difference will be largest for synthetic evidence, dangling references and test theatre. Grounding: the oracle problem [41]; mocking practices [42]; reward hacking in coding agents [38]. In the case, the test suite passed while 47 gaps existed. P7 (gaps recur unless bound). After a rebuild or remediation, claim–mechanism gaps will recur in surfaces that are authored separately from the mechanisms they describe. Recurrence will be lower where claims are generated from, or tested against, a single source of truth. Grounding: Campbell's law [48]; Goodhart vari‐ ants [40]. In the case, drift appeared in documentation and status labels within a day of the rebuild, and not in prices or limits, which were bound.

6. Discussion

6.1 Theoretical implications

Signalling when surfaces are free. The economics of signalling rests on a correlation between the cost of a signal and the quality it signals [24]. In software, a working interface was historically a costly signal of working mechanisms, because interfaces were expensive and were built on top of what already worked. The case suggests that this correlation can now break completely. The first version's surfaces were as complete as those of a mature product, and nothing beneath them worked. If P1 holds generally, demonstrations, screen‐ shots, documentation and test badges become weak signals of quality. Markets for software that are judged by such signals then move towards Akerlof's lemons equilibrium [14], unless buyers adopt signals that re‐ main costly to fake. Mechanism probes are such signals: a server-side verification that rejects a forged token cannot be produced by drawing it. Decoupling without an organisation. Institutional theory explains decoupling as an organisational response to conflicting demands for legitimacy and efficiency [17], [18]. The case shows ceremonial structure pro‐ duced by an artefact, at a scale and speed that institutional theory did not anticipate. The first version presen‐ ted an owner console, a customer portal, a test badge and a production release note, the formal structure of a software company, in one afternoon. The "logic of confidence and good faith" that protects decoupled struc‐ tures is, in this setting, the reviewer's reasonable assumption that nobody would build a console without the data behind it. When that assumption fails, inspection must move from ceremony to mechanism, which is what the six probes do. Verification debt at its limit. The first paper in this series described verification debt as a gap that grows when building outpaces checking [23]. The present case shows the limiting form, where nothing had been checked and almost nothing was true. It also shows that verification debt is not only internal. When surfaces are addressed to buyers, users and third parties, unpaid verification debt becomes a liability to them. The scout's drafted security notices and the plugin that locked owners out of their sites are the clearest examples.

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

Provenance as a hypothesis. This paper does not state how the first version was produced. Several of its features match failure modes documented for generative models: a standards citation with the wrong number; an instruction to install a package that did not exist; links to an unregistered domain; internally contradictory prices and version strings; tests that assert the magic values needed to pass. These correspond to hallucination [32], package hallucination [33], incoherent concept application [7] and test-passing behaviour of the kind observed in coding agents [38]. The release notes' confident claims of pro‐ duction readiness resemble sycophantic over-statement [36]. The same features are also consistent with a person building a demonstration under time pressure, borrowing the language of symbolic management [19]. The case cannot separate these explanations, and its contribution does not depend on doing so. Whatever the provenance, cheap surfaces raise the base rate of simulated completeness, and the remedy begins with probes.

6.2 Practical implications

For leaders who accept software built by agents or teams. Acceptance should be based on mechanism evidence, not on demonstration. In practice this means four things: Require that every displayed figure name its query. Require that every action name its side effect. Require that every external name resolve to something the organisation controls. Require that every security control be shown rejecting an input that should fail. These requirements can be written into the instructions given to AI agents and into acceptance criteria for contractors. Their outputs (query names, side-effect logs, resolution results, rejected inputs) are cheap for an agent to produce and expensive to fake convincingly. The case suggests one more requirement: that status la‐ bels have enough states to tell the truth. Table 7 sets out an evidence contract: for each kind of surface, the artefact that must accompany it before the work is accepted. An agent or contractor that delivers a surface without its evidence has not delivered the sur‐ face.

Table 7. An evidence contract for accepting software built by agents or teams

Surface delivered Evidence required Probe it enables A displayed figure The query or record that produces it, and the result with an empty database Trace A button or action The stored record, log line or message it produces, shown after a reload Trigger An analysis, score or report about an input Outputs for two materially different inputs Vary

• • • • • • • • •

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02
Surface delivered Evidence required Probe it enables A domain, package, route or plan identifier A lookup showing it exists and who controls it Resolve A security control A request that should fail, and the rejection Attack A claim of standard, statistic, price or speed The source, or the calculation Source A test suite The list of deployed code paths each test exercises Source A status label The full list of states and the condition for each Trace

The contract has a known weakness: a capable agent asked for evidence may fabricate the evidence. Rewardhacking research suggests that this risk grows with optimisation pressure [38]. The defence is the same as for any surface. The leader should run a sample of the probes personally, and should prefer evidence that con‐ sults something outside the agent's control, such as a domain registry, a payment platform or a fresh browser session. For builders of trust products. Verifiability is part of the product. The second version made efficacy claims checkable in three ways: a verification endpoint with the same request and response as the dominant incumbent's, so that custom‐ ers can test it against their existing integration; a public source repository; documentation stating what the mechanism cannot do. In credence-goods markets, verifiability is one of the few forces that disciplines fraud [16]. A new entrant without reputation can offer it from the first day. For anyone who does outreach. Outreach should be built from facts the recipient can check. The second version's messages say only what its discovery job observed on the recipient's own page. That makes them weaker as persuasion and stronger as evidence, and it removes the legal exposure of asserting things about a third party's security that were never observed. For regulators and buyers. In September 2024 the United States Federal Trade Commission announced five enforcement actions against deceptive AI claims. They included an action against a service marketed as "the world's first robot lawyer" that, according to the complaint, had not tested whether its output matched a law‐ yer's. The settlement included a $193,000 payment and a prohibition on claims without evidence [72]. The standard applied there, claims require evidence, is the standard the six probes operationalise. Buyers can ap‐ ply the cheapest probes, Resolve and Source, without any access to a vendor's systems.

6.3 Limitations

The study has six main limitations. Single case. The study rests on one case, chosen because it is extreme and well documented. It shows that simulated completeness can be total, and how. It cannot show how common it is. • • •

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

Interested analysis. The audit, the coding, the rebuild and this paper were produced by one analytic process working for the venture's owner, with AI assistance. The coding of findings into gap types was performed after the audit and its severity ratings, which limits but does not remove the risk that categories were shaped to fit. The full assignment is published in Table 2 so that it can be recoded. The central quantitative claim (all critical and high findings are gaps) should be treated as a hypothesis for independent replication (P3). Self-defined severity. Severity ratings were the auditor's. A different auditor might rate differently, and the concentration of severity in gaps might weaken. No field evidence of efficacy. The second version had no customers and faced no real attackers in the obser‐ vation window. The paper makes no claim that it stops bots in the field. Its operating figures show that its mechanisms run, not that they work against adversaries. Unknown provenance. Because the first version's provenance is not stated, the paper cannot test whether AI assistance caused its simulated completeness. P1 and P5 are offered on the strength of prior literature and the case's consistency with it. Short window. The observation window is three days. The recurrence finding (P7) rests on one day of fol‐ low-up and eleven issues.

6.4 Directions for research

Five lines of research follow: Replication of the taxonomy. Apply the coding scheme of Table 2 to independent audit reports and se‐ curity assessments, to test P3 and refine the types. Measurement of surface-to-mechanism ratios. Develop a metric that can be computed from repositor‐ ies and deployments, and compare products built with different degrees of AI assistance (P1). Experimental tests of acceptance. Present evaluators with matched products that differ only in the com‐ pleteness of their surfaces and the presence of their mechanisms. Measure acceptance under demonstra‐ tion-based and probe-based protocols (P2). Agent evidence contracts. Test whether instructing AI coding agents to produce mechanism evidence (queries, side effects, rejected inputs, resolved names) reduces simulated completeness, and whether agents learn to fake the evidence, as reward-hacking research would predict [38]. Longitudinal drift. Follow products over months to estimate the rate at which claim–mechanism gaps recur, and the effect of claim-binding tests (P7).

7. Conclusion

When surfaces were expensive, a product that looked finished was usually close to finished, and buyers, managers and investors could rely on demonstrations without thinking about why. That reliance is now mis‐ placed. This paper has named the condition that results, simulated completeness, and examined it in a case where it was nearly total and where the product's purpose, protecting websites from automated abuse, made it dangerous. 1. 2. 3. 4. 5.

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

The case suggests that the gaps between surface and mechanism are where the serious risk lies. They fall into a small number of types, and each type yields to a cheap external probe that the product's own tests will not run. A rebuild from the mechanism outward can close them. They return wherever a surface is written apart from its mechanism, unless something binds the two. None of this requires knowing how a product was built, or by whom. It requires asking, of every surface, what mechanism makes it true, and then checking.

Courage Garikai Coldcode Si Labs Library Working Paper 2026-02

References

[1] Peng, S., Kalliamvakou, E., Cihon, P., & Demirer, M. (2023). The Impact of AI on Developer Productivity: Evidence from GitHub Copilot. arXiv:2302.06590. https://doi.org/10.48550/arXiv.2302.06590 [2] Noy, S., & Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654), 187–192. https://doi.org/10.1126/science.adh2586 [3] Brynjolfsson, E., Li, D., & Raymond, L. (2025). Generative AI at Work. The Quarterly Journal of Economics, 140(2), 889–942. https://doi.org/10.1093/qje/qjae044 [4] Pearce, H., Ahmad, B., Tan, B., Dolan-Gavitt, B., & Karri, R. (2022). Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions. In 2022 IEEE Symposium on Security and Privacy (SP) (pp. 754–768). https://doi.org/10.1109/sp46214.2022.9833571 [5] Perry, N., Srivastava, M., Kumar, D., & Boneh, D. (2023). Do Users Write More Insecure Code with AI Assistants? In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security (pp. 2785–2799). https://doi.org/10.1145/3576915.3623157 [6] Becker, J., Rush, N., Barnes, E., & Rein, D. (2025). Measuring the Impact of Early-2025 AI on Experienced OpenSource Developer Productivity. arXiv:2507.09089. https://doi.org/10.48550/arXiv.2507.09089 [7] Mancoridis, M., Weeks, B., Vafa, K., & Mullainathan, S. (2025). Potemkin Understanding in Large Language Models. arXiv:2506.21521. https://doi.org/10.48550/arXiv.2506.21521 [8] Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design Science in Information Systems Research. MIS Quarterly, 28(1), 75–106. https://doi.org/10.2307/25148625 [9] Peffers, K., Tuunanen, T., Rothenberger, M. A., & Chatterjee, S. (2007). A Design Science Research Methodology for Information Systems Research. Journal of Management Information Systems, 24(3), 45–77. https://doi.org/10.2753/ mis0742-1222240302 [10] Gregor, S., & Hevner, A. R. (2013). Positioning and Presenting Design Science Research for Maximum Impact. MIS Quarterly, 37(2), 337–355. https://doi.org/10.25300/misq/2013/37.2.01 [11] Kelley, J. F. (1984). An iterative design methodology for user-friendly natural language office information applications. ACM Transactions on Information Systems, 2(1), 26–41. https://doi.org/10.1145/357417.357420 [12] Cunningham, W. (1992). The WyCash portfolio management system. In Addendum to the Proceedings on ObjectOriented Programming Systems, Languages, and Applications (OOPSLA ’92) (pp. 29–30). ACM. https://doi.org/ 10.1145/157709.157715 [13] Kruchten, P., Nord, R. L., & Ozkaya, I. (2012). Technical Debt: From Metaphor to Theory and Practice. IEEE Software, 29(6), 18–21. https://doi.org/10.1109/ms.2012.167 [14] Akerlof, G. A. (1970). The Market for "Lemons": Quality Uncertainty and the Market Mechanism. The Quarterly Journal of Economics, 84(3), 488–500. https://doi.org/10.2307/1879431 [15] Darby, M. R., & Karni, E. (1973). Free Competition and the Optimal Amount of Fraud. The Journal of Law and Economics, 16(1), 67–88. https://doi.org/10.1086/466756 [16] Dulleck, U., & Kerschbamer, R. (2006). On Doctors, Mechanics, and Computer Specialists: The Economics of Credence Goods. Journal of Economic Literature, 44(1), 5–42. https://doi.org/10.1257/002205106776162717 [17] Meyer, J. W., & Rowan, B. (1977). Institutionalized Organizations: Formal Structure as Myth and Ceremony. American Journal of Sociology, 83(2), 340–363. https://doi.org/10.1086/226550 [18] Bromley, P., & Powell, W. W. (2012). From Smoke and Mirrors to Walking the Talk: Decoupling in the Contemporary World. Academy of Management Annals, 6(1), 483–530. https://doi.org/10.5465/19416520.2012.684462 [19] Zott, C., & Huy, Q. N. (2007). How Entrepreneurs Use Symbolic Management to Acquire Resources. Administrative Science Quarterly, 52(1), 70–105. https://doi.org/10.2189/asqu.52.1.70 Courage Garikai Coldcode Si Labs Library Working Paper 2026-02