Matched-information protocol for six-hour model trading evaluation

DataCedar Arena asks a narrow comparative question: conditional on a common information set and common execution constraints, how do different model decisions perform over the following six hours? The unit of observation is a model-round. The matched factors are the cutoff, candidate set, packet bytes, system instruction, output schema, notional, and settlement window. The treatment is the model endpoint. Primary descriptive outcomes are cumulative gross paper P&L and return on a fixed $10,000 account. Secondary outcomes include fill rate, hit rate, maximum drawdown, model cost, latency, and exploratory confidence scores.

This is an open observational benchmark, not a clinical-style confirmatory trial. No ex ante superiority hypothesis, statistical power threshold, or multiple-testing correction has been registered. Results therefore describe this protocol and sample; they do not establish persistent investment skill. Repeated inspection and model substitution can create selection effects characteristic of backtest overfitting.[10]

1.1 Research question

The protocol compares decision quality when access to DataCedar information is held constant. It does not compare web-search ability, retrieval latency, proprietary memory, or prompt length. Each model may select one eligible stock-linked candidate, direction, future entry delay, resting limit, confidence value, and short thesis—or abstain.

1.2 Estimands

Model-round gross P&L

Dollar change on standardized $1,000 notional from simulated maker fill price to the six-hour closing mark; zero if no valid fill.

Season cumulative P&L

Sum of model-round gross paper P&L; no compounding and no capital reallocation across models.

Relative account path

$10,000 plus cumulative model P&L, compared visually with SPY normalized to $10,000 from its first recorded comparison observation.

Decision calibration

Exploratory agreement between a submitted 0–100 confidence value and positive gross return among filled orders only.

2.1 Temporal eligibility

Each round begins at cutoff t₀, floored to a six-hour UTC boundary. DataCedar applies known_at ≤ t₀ to every queried history table before serialization. Economic time and publication time remain separate fields where the source supports them. This is the same substantive distinction made by SEC filing timestamps[3] and by the Federal Reserve’s real-time periods and vintages.[4]

The rule reduces look-ahead leakage; it does not prove complete coverage. A record may be publicly available yet absent from DataCedar because a source was not covered, an ingestion was late, rights prevented serving, or the record failed normalization. Coverage status is therefore data, not an inference from an empty result.

2.2 Packet contents

2.3 Context budget and deterministic compaction

The serialized packet is capped at 8,000 characters; model output is capped at 240 tokens. Candidate records and protocol fields are mandatory. If optional context exceeds the packet cap, arrays are shortened deterministically until the payload fits. The configured removal order is coverage status, social signals, documents/news, events, then macro observations. Consequently, the existence of a source in DataCedar does not guarantee its presence in every round packet.

3.1 Eligibility screen

The tradeable object is an active stock-linked contract mapped to a named underlying company. Candidate construction queries the prior 24 hours of one-minute observations available by t₀. A contract is excluded if its mapped reference is inactive, fewer than 60 bars are present, the latest bar is missing, or the latest observation is more than 90 minutes old.

3.2 Deterministic ranking

This is an intentionally active-market sample, not a random cross-section of listed equities. It concentrates the experiment in contracts with recent movement or range and therefore introduces a known selection effect. Results should not be generalized to quiet securities or to the full equity universe.

4.1 Matched inputs

Model calls are issued concurrently through OpenRouter. Every arm receives the same system instruction, exact packet serialization, strict JSON schema, maximum output allowance, low reasoning effort, and round identifier. Provider routing is required to support the requested parameters; the served model identifier, usage, cost, latency, validation state, and response rationale are retained. OpenRouter data collection is requested as deny.

Permitted action

long, short, or no_trade

Eligible symbol

Must be one of the supplied candidates; no_trade requires an empty symbol

Entry delay

Integer from 60 through 300 minutes after t₀

Limit distance

Positive, tick-aligned, at most 250 bps from the decision reference

Maker offset

Long limit at least 1 bp below reference; short limit at least 1 bp above reference

Thesis

Plain text, normalized whitespace, maximum 180 characters

Confidence

Integer from 0 through 100

4.2 Validation and failure handling

A deterministic validator rejects extra fields, out-of-range numbers, ineligible symbols, off-tick prices, non-finite values, and limits inconsistent with the maker constraint. Invalid responses are published as invalid and create no order. A completely failed round is retained rather than imputed. A missing model lane may be recovered only from the stored packet and only while the minimum future-entry window remains feasible.

4.3 Current registered model arms

1. 01openai/gpt-5.6-sol
2. 02anthropic/claude-fable-5
3. 03google/gemini-3.6-flash
4. 04x-ai/grok-4.5
5. 05deepseek/deepseek-v4-pro
6. 06z-ai/glm-5.2

5.1 Activation

A valid trade becomes planned at t₀ and activates at the model-selected delay. At activation, the first eligible one-minute bar open is the post-only reference. A long limit at or above that open, or a short limit at or below it, is labeled post_only_rejected. The simulator does not move or reprice the order.

5.2 Fill rule

If no qualifying bar occurs before the six-hour boundary, the order expires with zero P&L. If the required price series is absent, the order is marked data_missing. A touched order is treated as fully filled at the limit for $1,000 paper notional. This assumption is intentionally optimistic relative to a real queue.

5.3 Live marking and settlement

At fill, the first mark is the limit and gross return is zero. Each later ingested one-minute close becomes the next mark. At the round boundary, the final available close is the settlement mark. The worker waits a configured ten-minute grace period before finalizing due rounds to reduce false missing-data states.

6.1 Gross return and P&L

Maker-entry fees are set to 0 bps. Funding, financing, borrow availability, borrow fees, liquidation, taxes, corporate distributions, and market impact are not included. Reported results are therefore gross paper results, not net investable returns.

6.2 Entry-distance statistic

The interface currently labels the distance between the cutoff reference and fill limit as slippage. By convention, a negative displayed number means price improvement relative to the cutoff: for a long, −(P₀−L)/P₀; for a short, −(L−P₀)/P₀, multiplied by 10,000. It is not realized market-impact slippage because every paper fill occurs at the submitted limit.

6.3 Market comparison

SPY is used as a practical observable proxy for the S&P 500 comparison. State Street states that SPY seeks, before expenses, to correspond generally to the S&P 500’s price and yield performance,[9] while S&P describes the index as a float-adjusted, large-cap U.S. equity measure.[8] The public comparison normalizes the first recorded one-minute SPY close to $10,000 and applies subsequent price returns. It does not subtract SPY expenses or include distributions.

6.4 Descriptive statistics

Fill rate

Filled or open paper orders divided by submitted trade intents. Abstentions are not trade intents.

Hit rate

Filled positions with gross return above zero divided by filled positions.

Maximum drawdown

Largest peak-to-trough decline in the displayed $10,000 account path.

Sharpe statistic

Mean six-hour round P&L divided by sample standard deviation, annualized by √(4×365), with zero risk-free rate.

Brier score

Mean squared difference between submitted confidence/100 and the binary positive-return outcome, restricted to filled orders.

Log loss

Negative binary log likelihood after bounding submitted probability away from 0 and 1.

The Sharpe statistic is descriptive and unstable in small samples; the original formulation and discussion emphasize comparability of return differential and variability.[7] No confidence interval, heteroskedasticity/autocorrelation adjustment, or correction for selection across models is currently reported.

7.1 Record structure

Before model calls begin, the runner stores the round ID, season ID, cutoff, close time, serialized packet, packet digest, estimated maximum model cost, and demo flag. Decisions, validation errors, model usage, provider-served identifiers, orders, minute marks, comparison marks, and settlement states are stored separately and joined by deterministic IDs.

Packets receive a SHA-256 digest following the NIST Secure Hash Standard.[5] The current public page reports that the packet is hashed; it does not publish the actual digest because the web adapter has not yet completed end-to-end exposure of the stored round digest. The season-derived placeholder previously shown in the interface has been removed.

7.2 Publication policy

Official public snapshots exclude demo rounds. Missed starts, model failures, invalid decisions, rejected maker orders, expired orders, and missing price windows remain observable failure states; they are not silently converted to successful trades. The public dataset is available directly.[1]

7.3 Methodology-disclosure standard

This document uses IOSCO Principle 10 as a disclosure checklist: key definitions, input selection, minimum data, calculation rules, disruption handling, review, and limitations are stated explicitly.[2] DataCedar Arena is a research benchmark, is not a regulated financial benchmark, and has not been assessed for IOSCO compliance.

1. 8.1

   Paper fills are optimistic

   An OHLC touch does not establish available size, queue priority, or executable sequencing inside the minute. All touched orders fill completely at the limit.

2. 8.2

   Gross returns omit material costs

   Funding, borrow, financing, spread capture, market impact, taxes, distributions, and operational failures are excluded. Zero maker fee is a stated experimental assumption.

3. 8.3

   The candidate universe is selected

   The top-eight momentum/range screen deliberately favors active names. It is neither random nor representative of all equities.

4. 8.4

   Context coverage is conditional

   Point-in-time filtering prevents future records from entering; it cannot create records that DataCedar did not collect or cannot serve. Optional context may also be compacted to meet the prompt budget.

5. 8.5

   Confidence calibration is provisional

   Current confidence lacks a prospectively defined target event in the model instruction, and only filled trade intents are resolved. Abstentions and unfilled limits are excluded from calibration statistics.

6. 8.6

   Model identity is routed

   The requested and served model identifiers are recorded, but upstream providers may revise weights, inference stacks, safety layers, or routing without a DataCedar code change.

7. 8.7

   Small samples invite overinterpretation

   Six-hour observations are serially dependent and the same market regime affects all arms. Rank changes are not evidence of statistical significance.

8. 8.8

   SPY is a proxy, not the index itself

   The comparison uses observed SPY prices, normalized from the first available mark. Tracking error, expenses, distributions, and missing bars can cause divergence from the S&P 500 index.

A material change to information eligibility, candidate ranking, model instruction, decision schema, execution rule, notional, horizon, fee assumption, comparison instrument, or statistic requires a new protocol version. Results produced under materially different rules must not be merged into the same season leaderboard.

Patch

Editorial clarification or interface correction that does not change decisions, fills, or statistics.

Minor

New disclosed field or non-material operational control; backward comparability reviewed and documented.

Major

Any change capable of altering the eligible packet, model decision, fill state, P&L, rank, or interpretation; requires a new season.

Methodology changes are approved by DataCedar Research and Engineering. Independent methodological review has not yet been commissioned. Known defects affecting published values should be corrected with a dated notice, affected rounds identified, and old outputs retained where feasible.

1. [1]
   DataCedar (2026). Arena public results dataset, season datacedar-six-hour-stocks-v2. Live machine-readable observations, decisions, orders, price paths, comparison series, and aggregate results.
2. [2]
   International Organization of Securities Commissions (2013). Principles for Financial Benchmarks, Final Report, FR07/13. Principle 10 informs the document’s disclosure structure; DataCedar does not claim IOSCO compliance or regulated-benchmark status.
3. [3]
   U.S. Securities and Exchange Commission (2024). EDGAR Application Programming Interfaces. Primary documentation for disseminated submissions and extracted XBRL data.
4. [4]
   Federal Reserve Bank of St. Louis (2026). FRED API: Real-Time Periods and ALFRED vintages. Primary documentation for distinguishing current historical values from information known at a past date.
5. [5]
   National Institute of Standards and Technology (2015). FIPS PUB 180-4: Secure Hash Standard. Specification underlying the SHA-256 content digest assigned to each packet.
6. [6]
   Brier, G. W. (1950). Verification of Forecasts Expressed in Terms of Probability. Original probability-score reference. Monthly Weather Review, 78(1), 1–3.
7. [7]
   Sharpe, W. F. (1994). The Sharpe Ratio. Original author’s discussion of the reward-to-variability measure and its limitations.
8. [8]
   S&P Dow Jones Indices (2026). S&P U.S. Indices Methodology. Primary methodology for the large-cap U.S. equity index used as the market reference.
9. [9]
   State Street Investment Management (2026). State Street SPDR S&P 500 ETF Trust (SPY). Primary product description for the ETF used to observe the S&P 500 comparison path.
10. [10]
    Bailey, D. H., Borwein, J., López de Prado, M., and Zhu, Q. J. (2014). The Effects of Backtest Overfitting on Out-of-Sample Performance. Research basis for treating repeated strategy selection and short samples cautiously.

Cutoff (t₀)

The UTC timestamp after which no record may enter the model information set for that round.

Known-at time

The first time a record was usable by DataCedar, distinct from the event’s economic or publication time.

Candidate

An eligible stock-linked contract mapped to a named public company. It is not a purchase of the issuer’s common stock.

Maker limit

A resting limit required to remain non-marketable at activation; a crossing order is rejected by the paper simulator.

Round

One matched-information experiment beginning at t₀ and ending six hours later.

Fill

A deterministic paper event produced when a later one-minute OHLC range touches the resting limit.