Marketplace Anti-Bot Engineering

Technical program report · Updated August 2026

Marketplace anti-bot engineering

Four synthetic prototypes examine the decision chain from server-side observation to controlled intervention: HTTP behavior analysis, adaptive evaluation, economic coordination investigation, and policy selection under explicit player-impact constraints.

Technical summary Repository map
01

Technical summary

The work evaluates a layered control system, not a standalone classifier.

Request-time evidence is incomplete.

A bounded HTTP policy identifies obvious automation. Longer-window behavior and entity correlation are required for activity that remains ordinary at the individual-request level.

Static evaluation overstates control strength.

A reference detector catches its scripted positive control but fails against bounded adaptive search. Captured failures are minimized, replayed, and retained as regressions.

Economic coordination can outlast weak identifiers.

In the authored scenario, accounts with distinct device and network identifiers remain individually ordinary while synchronized market behavior supplies case-level evidence.

Policy approval requires evidence and rollback.

An offline-supported intervention advances only to canary and is rolled back when shifted legitimate behavior breaches the declared friction budget.

Program objective Reduce measurable automated advantage while bounding adverse effects on legitimate users, marketplace function, review operations, and privacy.

02

Architecture and repository boundaries

Each repository addresses a distinct control boundary.

The repositories are independently executable. The Go scorer and Red Queen harness integrate through a loopback-only observable-event contract. Connections to the economic investigation and intervention labs are architectural and schema-level; an end-to-end production pipeline is not claimed.

01

Observe

HTTP evidence

Bounded request-time scoring, asynchronous behavior, replay, reason codes, and analyst review.

HTTP Bot Defense Lab
02

Falsify

Adaptive pressure

Equal-budget search, unseen holdouts, hard negatives, counterexample minimization, and regression promotion.

Red Queen Lab
03

Investigate

Economic harm

Event-sourced exchange, sequence evidence, coordination graph, peer context, and market guardrails.

Market Integrity Lab
04

Intervene

Guarded rollout

Offline policy evaluation, evidence-support gates, player-friction budgets, canary, and rollback.

Intervention Lab
InputPrivacy-minimized observable events
Decision unitEvidence-backed, versioned policy
Primary outcomePlayer and marketplace harm reduced
Safety mechanismProgressive action with rollback
Repository scope and current integration status
RepositoryControl boundaryImplementationStatus
http-bot-defense-labOnline HTTP evidence and progressive enforcementGoIntegrated with Red Queen through a loopback scorer contract
bot-defense-red-team-labAdaptive search, replay, and regression generationPythonAttacks the Go scorer and retains machine-readable evidence
market-integrity-labOffline economic and coordination investigationPythonIndependent event-sourced simulation; schema-level program link
marketplace-intervention-labOffline policy evaluation and guarded rolloutPythonIndependent logged-policy simulation; schema-level program link
03

Evaluation basis and metric definitions

All reported outcomes are fixture-specific engineering evidence.

The evaluations are deterministic or seeded synthetic scenarios authored alongside the systems. They test implementation behavior, failure handling, replayability, and evaluation discipline. They do not estimate production prevalence, precision, recall, financial loss, or causal policy impact.

Detection

A run is detected when the scorer emits a challenge or restriction. An observe decision is not counted as detection.

Legitimate friction

The relevant action- or episode-level denominator is stated beside each result. Aggregate rates must be read with persona-level intervention risk.

Retained harm

Synthetic realized profit under a detector divided by profit for the same strategy under a no-op detector. It is a scenario ratio, not a monetary estimate.

Ring precision and recall

Account-membership measures for constructed rings, not case-level production precision and recall.

Residual harm

An offline policy-value estimate over mature held-out episodes. Support, uncertainty, and guardrail gates are evaluated separately.

Latency

Reported detector or evidence timing is named according to implementation. The market case threshold is retrospective offline timing, not streaming alert latency.

04

Results and limitations

The evidence identifies both implemented capabilities and unresolved failures.

Each result is tied to a declared synthetic fixture and a public source. Interpretation and limitations are stated adjacent to the measurement.

Project 01 · Go

HTTP Bot Defense Lab

Request-time scoring covers a narrower evidence window than longer-running correlation.

Repository ↗

What happened. Request-time scoring caught all ten fixed-loop accounts but missed fourteen randomized or coordinated accounts. Longer-window analysis found those constructed profiles without adding a false positive at the default threshold.

Why it matters. The online path now bounds both samples and account cardinality, defines retry and late-event semantics, and places JSONL audit delivery behind an observable fixed-capacity queue. At a lower threshold, three legitimate household accounts are flagged, making the cost of sensitivity visible.

What this result does—and does not—establish

It establishes deterministic acceptance behavior, truth separation, threshold sensitivity, and a reviewable architecture. It does not establish real precision or recall. The profiles were authored around known hypotheses and omit real-world ambiguity and drift.

Source: README · Demonstrated result and case-study evidence.

Project 02 · Python

Red Queen Lab

Static positive-control performance does not measure robustness to adaptation.

Repository ↗
100%scripted control detected
0%adaptive holdout detected
424actions in minimized replay

What happened. Under equal 240-evaluation budgets, evolutionary search found a strategy with 23.8% more objective value than random search. The frozen strategy then avoided intervention on five unseen seeds and retained all of its synthetic harm.

Why it matters. A discovered weakness becomes an exact, 81.7%-smaller replay and a permanent regression fixture. The aim is not to claim that an optimizer beats a real service; it is to institutionalize falsification.

Guardrail and interpretation

Macro legitimate-action friction was 0.19%, yet the power-trader persona was touched on 60% of its five seeds. The action-level average and persona-level intervention risk therefore need to be read together. Failure to discover an evasion would not prove security.

Source: committed summary.json and result interpretation.

Cross-repository engineering experiment

Discover → mitigate → reject → reattack

The Red Queen harness now attacks the actual Go scorer, not a co-located reference implementation.

Experiment note ↗
94actions in frozen v1 failure
54later market actions blocked by v4
34actions in the new v4 frontier

Same-trace result. The v1 failure realized 2,819 synthetic credits with no score. V4 detected that exact trace, blocked 54 later market actions, and reduced realized profit to 1,200.

Operating result. A broad cadence fix was rejected after it detected every power-trader and delegated-household seed. A narrower coordination rule passed the three declared hard negatives, then fresh search found a profitable three-account evasion. The useful result is a working improvement loop, not a solved detector.

Evaluation boundary

Two search seeds, three disjoint holdout seeds, 32 candidates per search method, and an authored synthetic economy. The fresh v1/v3/v4 strategies differ, so their profit values are not paired causal estimates. Only the frozen-trace replay compares identical actions.

Source: experiment report, committed machine-readable evidence, and Go implementation note.

Project 03 · Python

Market Integrity Lab

Repeated economic coordination can remain observable when weak identifiers differ.

Repository ↗

What happened. All six ring accounts used unique device, network, and session identifiers and remained below the account-only threshold. A full-batch graph/economic layer formed one six-account case with a retrospectively verified evidence threshold at T+385 seconds.

Why it matters. The paired treatment world applied a marketplace-only rule after that offline evidence threshold and exposed both reduced ring profit and liquidity tradeoffs. T+385 is not a measured streaming-alert latency.

Guardrail and interpretation

A two-sided liquidity-provider pattern is an explicit hard negative. The graph is evidence for investigation, not proof of common ownership or malicious intent. A paired simulation is useful for mechanism testing but is not a causal estimate of a real intervention.

Source: documented synthetic scenario result and metric definitions.

Project 04 · Python

Marketplace Intervention Lab

Point estimates are insufficient without support, uncertainty, and guardrail checks.

Repository ↗
Reference 108.8 estimated residual harm
Approved for canary 27.2 supported harm-aware policy
Rejected 8.4 attractive, unsupported estimate
Declared friction budget 6.5%
Shifted canary after 300 episodes 11.0% · rollback

What happened. The harm-aware policy cleared overlap, effective-sample-size, maximum-weight, uncertainty, legitimate-friction, and review-load gates. The aggressive policy reported lower harm but was rejected for poor support and guardrail breaches.

Why it matters. Offline approval led only to a canary. When the synthetic population shifted toward more high-volume legitimate traders, observed friction crossed the budget and stopped the policy automatically.

Method and interpretation

The lab uses SNIPS and doubly robust estimates on a held-out period, with clustered bootstrap uncertainty. The gate now requires at least 90% global outcome maturity; the committed run has 2,697 of 2,852 holdout episodes mature (94.6%). The policy gate cannot read actor truth or unchosen outcomes; a hidden oracle audits the frozen decision afterward.

Source: committed summary.json and evaluation method.

05

Control and decision model

The design separates evidence, ground truth, policy, and enforcement authority.

01

Truth stays offline

Detection and investigator surfaces cannot read synthetic actor labels. Evaluators alone join predictions to truth.

02

Signals are not verdicts

Network, device, speed, coordination, profitability, and model scores are evidence—not standalone proof.

03

Policies are versioned

Reason codes, thresholds, assignments, disagreement, and outcomes are explicit enough to replay and audit.

04

Actions are proportional

Observe, challenge, cooldown, temporary hold, and market restriction precede stronger account action.

05

Players are a guardrail

Legitimate friction, fill quantity, liquidity, review load, and appeals belong beside abuse reduction.

06

Failure becomes evidence

Adaptive counterexamples and canary breaches are retained as regression tests, not hidden as bad demos.

Decision ownership across an integrity program
LayerQuestionOutputFailure posture
TelemetryWhat happened?Minimal observable eventMeasure loss and missingness
DetectionWhat deserves attention?Score, reason codes, evidenceFail open where appropriate; alert
InvestigationWhat harm and relationships are supported?Truth-blind casePreserve uncertainty
PolicyWhat reversible action is justified?Versioned recommendationReject weak support
RolloutDid the action help safely?Guardrails and outcomeRollback on breach
06

Assumptions and unknowns

The architecture rests on provisional external assumptions.

The central premise is that meaningful companion-marketplace actions ultimately reach server-controlled HTTP services. This is a design hypothesis, not a statement about PlayStation architecture. Internal discovery could materially change the telemetry model, system boundaries, or appropriate controls.

Reasonable prototype presumptions

  • Authenticated server-side events can be associated with an account and time.
  • Marketplace actions and outcomes can be observed or reconstructed.
  • Profit-motivated automation adapts timing, accounts, targets, and weak identifiers.
  • Fast, coordinated, and profitable legitimate users exist.
  • Progressive, reversible interventions are preferable under uncertainty.
  • Success includes marketplace health and player experience—not only detection.

Unknowns that would change the design

  • Actual protocols, service boundaries, schemas, and identity semantics
  • Traffic volume, seasonality, launch behavior, latency, and availability budgets
  • Abuse base rates, harm definitions, label delay, and reviewer disagreement
  • Available interventions, accessibility needs, appeal policy, and support cost
  • Privacy, regional regulation, retention, and authorized linkage constraints
  • Whether logged propensities or controlled exploration support offline evaluation

Claims explicitly excluded

Synthetic metrics transfer to production · scenario credits equal money · behavior proves shared ownership or intent · device or network identity is reliable proof · historical replay establishes causal effect · passing an offline gate authorizes deployment · failure to find an evasion proves security · any prototype reflects Sony or PlayStation architecture.

07

Production-validation plan

No production decision should precede discovery, measurement, and shadow evaluation.

Days 0–30

Establish reality

  • Map product surfaces, events, identities, decisions, and owners.
  • Define harm, legitimate-player costs, review outcomes, and label maturity.
  • Audit completeness, clock quality, retries, retention, and access.
  • Reconstruct a small set of known incidents and hard negatives.

Exit criterion A reviewed measurement contract and telemetry gap register.

Days 31–60

Evaluate in shadow

  • Backfill interpretable features and version candidate policies.
  • Use time/account/ring-separated holdouts at the observed base rate.
  • Segment false positives by client, region, tenure, network, and accessibility.
  • Run permanent regressions plus bounded adaptive search.

Exit criterion Supported candidates and documented failure modes—no user impact.

Days 61–90

Intervene carefully

  • Start with reversible, surface-specific actions.
  • Canary by risk tier with exposure logging and a last-known-good policy.
  • Monitor harm, friction, liquidity, queue load, appeals, and drift.
  • Rollback automatically when declared guardrails breach.

Exit criterion One measurable intervention with governance and safe rollback.

Required discovery questions

Inputs needed before selecting architecture or enforcement policy

  1. Which player or marketplace harms matter most, and how are they currently measured?
  2. Which HTTP and non-HTTP surfaces are in scope, and where are decisions enforced?
  3. What labels exist, who adjudicates them, and how long until outcomes mature?
  4. Which false-positive, latency, accessibility, review-capacity, and appeal constraints are non-negotiable?
  5. What is the smallest reversible action that changes attacker economics without broadly degrading player experience?
08

Risk and governance requirements

Incorrect intervention can harm users and the marketplace.

Anti-bot systems can restrict account access, alter market liquidity, create support and appeal burdens, and introduce privacy risk. Detection quality is therefore only one part of the acceptance standard.

False attribution

Fast or coordinated behavior can be legitimate. Shared households, accessibility tools, delegated use, and expert traders require explicit hard-negative coverage and review paths.

Disproportionate action

Evidence strength should determine action strength. Uncertain cases should favor observation, challenge, cooldown, or narrowly scoped restriction over irreversible account action.

Marketplace distortion

Controls can reduce liquidity, increase spreads, delay fills, or shift abuse elsewhere. Harm prevention and market-health guardrails must be evaluated together.

Privacy and linkage

Entity correlation can create sensitive relationship inferences. Collection, retention, regional use, and investigator access require explicit authorization and minimization.

Adaptive displacement

A mitigation changes attacker incentives and the observed population. Permanent regressions and renewed adaptive evaluation are required after every material policy change.

Audit and recourse

Versioned decisions, reason codes, evidence provenance, exposure logging, appeal handling, and rollback ownership are required before consequential enforcement.

Minimum acceptance standard

A candidate control should not advance beyond shadow operation without representative labels, explicit metric denominators, calibrated base-rate evaluation, hard-negative coverage, privacy review, operational capacity, reversible rollout, and named rollback and appeal owners.