Methods and detailed results for the Outrigger shopping-offer audit

This appendix accompanies the consumer report. It describes the achieved archive of 80 offers and 156 substantive answers, including unresolved evidence. The original planned 100-offer primary collection and separate stretch cohort were not completed.

The question and the buyer scenario

The audit asked whether ChatGPT and Gemini could accurately and sufficiently check a supplied promotional offer using retailer sources, while identifying facts the shopper still needed to confirm.

Each prompt supplied the exact product and variant, retailer URL, advertised price and intended use. The scenario was an ordinary nonmember buying one selected product with standard delivery to ZIP 90001 in the United States. Both assistants received the same frozen prompt for that offer.

The requested areas were price, discount conditions, included items and required extras, the known cost before sales tax including shipping where available, delivery conditions and return terms. Repetition of a price supplied in the prompt did not establish independent verification. The test did not measure independent deal discovery, lowest-price search or product-recommendation quality.

The prompt template was:

I am considering [PRODUCT_AND_VARIANT] from [RETAILER] at an advertised price of [PRICE] USD for [BUYER_USE], with standard delivery to ZIP 90001 in the United States. Here is the offer: [PRODUCT_URL].

Please check whether this offer suits that use. Tell me the current item price and any discount conditions, what is included and any required extras, the known cost before sales tax including shipping if available, the delivery conditions, and the return terms. Use current retailer sources and link them. Clearly separate what you can verify from anything I need to confirm with the retailer.

Selection and achieved sample

Selection followed prespecified rules for U.S.-facing physical goods in electronics, home appliances, home and furniture, apparel, and outdoor and sporting goods. Products were selected in recorded default order from official promotion pages, subject to exclusions and a maximum of four qualifying products per retailer in the primary frame.

Manifests and prompts were frozen in staged batches, with hashes and amendments preserved. The full achieved sample was not frozen before the first responses were collected. This was a purposive convenience sample of promotions, not a random sample of all shopping offers or natural consumer conversations. Several offers came from the same retailers.

The five unadministered offers remain in the inventory and are excluded from answer analyses. The 13 nonanswers comprise nine recorded technical errors or resets and four historical combined refusal-or-no-answer labels. The four combined labels do not establish four proven refusals or a particular refusal mechanism.

There were 80 ChatGPT primary answers, 70 Gemini primary answers and six Gemini retry answers. The primary failures and the permitted retries remain separate records. No answer was regenerated to obtain a preferred result.

Reference evidence and capture

Only answers from actual consumer ChatGPT and Gemini website conversations counted as observations. Research-worker output, prepared leads and API-generated responses did not count as consumer answers.

The archive retains exact prompts, full response text and available visible model or mode information, screenshots, citations and destination records. Capture gaps remain documented. Missing screenshot or citation evidence does not itself establish that an answer was false.

Reference evidence includes official product pages, exact selected variants, research carts and applicable shipping and return policies. Volatile facts such as price, availability and discount conditions require observations relevant to each answer's time. Later checks retain their own dates and cannot retroactively establish an unobserved historical fact. Source observation clocks are explicit records or documented attestations, not filesystem modification times.

A research-cart quote is not a completed purchase or proof that every shopper would qualify. A displayed comparison price is not evidence of a retailer's historical transaction price. General policy wording and its application to a specific item, seller or destination are distinct questions. Conflicting or missing applicable evidence remains unresolved.

Coding and review

Material claims were assessed as correct, incorrect, omitted, acknowledged unknown, reference unknown or not applicable. An incorrect label required a documented contradiction with applicable evidence. An absent source did not establish falsehood. Correct acknowledgment that checkout or retailer confirmation was necessary was distinct from an unsupported definitive statement.

Response classifications used the following precedence after a substantive answer existed:

1. Contains a verified material error: at least one supported material contradiction, even if other fields remain unresolved.
2. Unscorable: without a verified error, reference evidence prevents adjudication of a material definitive assertion.
3. Accurate but incomplete: no established material error, but at least one requested area is omitted or inadequately addressed.
4. Reliable and complete: applicable requested areas are accurately addressed, necessary uncertainty is acknowledged appropriately, and no material false or unsupported definitive assertion remains.

Two outcomes remain held rather than being forced into those categories. A reliable answer is also distinct from a fully verified offer: it can responsibly acknowledge a purchase-critical fact that remains unresolved.

The original fixed independent-review sample contained 32 substantive responses. Initial coding concealed provisional scores; original labels, order and disagreements are preserved. Expanded source checks and adjudications took place after unblinding. All 156 substantive answers now have full-response mapping and documented case-review closeouts, including the two with held outcomes.

New analytical and independent review used language models. Material-error witnesses, proposed reliable-and-complete cases, discrepancies and featured examples received additional checks, with the final source review documented. This was model-assisted review, not independent human verification. Multiple models can share errors. Closing a review means its claims and limits have been examined, not that every fact is now known.

Detailed response outcomes

The held outcomes concern one Gemini answer and one ChatGPT answer on separate offers. These counts concern answers, including retries, rather than distinct offers or retailers. The three complete answers must not be interpreted as the only answers containing correct or useful information.

The zero in the first row is an observed classification count, not proof that those offers or answers were error-free. Most answers in that small group were unscorable. Pooled findings describe promotional offers checked ahead of Black Friday; they do not constitute a Black Friday offer failure rate.

Paired comparison and exclusions

Seventy offers produced substantive answers from both primary attempts. Including substantive retries raises availability to 76. Availability alone does not establish that two answers can be compared fairly.

Timing qualification uses recorded completion times or the documented amended interval rule. Under the interval rule, the largest possible completion-time gap must be at most 60 minutes. Offer changes or mode anomalies can invalidate a pair despite a short time gap. All final qualified pairs used the amended interval rule rather than the original point-clock criterion.

The table shows the first applicable disposition for all 80 administered offers. A case can have additional limitations beyond the first reason listed.

The retry-assisted scope adds two qualified pairs. Four other substantive retries make previously unanswered pairs classifiable as unscorable. The ten source-indeterminate candidates are not proven offer changes or false answers.

Qualification means the required observed product, variant, price, promotion and destination facts were supported around the answers, with applicable stable policy references treated as an explicit observational inference. It does not establish continuous stock, universal checkout eligibility or identical account and backend conditions. No general platform ranking or paired reliability percentage is reported.

Preserved limitations and data provenance

The record preserves misrouted citation artifacts and their separate corrections, a non-observed recovered upper clock, a timing discrepancy and documented attestation, a within-attempt mode change, capture gaps, and case-specific source, mode and citation limits. No successful hash check erases those limitations.

Working records can repeat or split a claim, so they are not a denominator for measuring the accuracy of unique claims.

The report draws on recorded answer outcomes, reconciled offer and attempt counts, comparisons across the offers, and preserved source evidence. This appendix does not make private account information or the working archive publicly available.

The exploratory error-pattern analysis describes a later thematic grouping of the 36 answers already classified as containing a verified material error. That grouping is post hoc and does not change the original outcomes or estimate general error prevalence.