Open System One
Follow
ConceptArchitectureRoadmapGitHub
Research project · Building in publicStatus: pre-alpha

Models that decide, not write.

Open System One is an open-source effort to build neural models that turn unstructured state into typed, calibrated probability distributions — without autoregressive text generation.

state + typed decisionprobability distribution
View on GitHubExplore the idea
Open System One · decision passIllustrative output
Stateunstructured

The application receives an arbitrary URL from the request and performs an HTTP request to fetch remote content before rendering the result.

OPEN SYSTEM ONE
Questionbinary

Is input attacker controlled?

  • YES97%
  • NO3%
Questionbinary

Is this security-sensitive?

  • YES94%
  • NO6%
Questionchoice

Likely vulnerability class?

  • SSRF91%
  • Open Redirect4%
  • Other3%
  • None2%
Illustrative output — static data, not a live model. Probabilities may drift slightly to visualize distributions.
01Concept

Why another model? Generation is powerful — and often the wrong interface for a decision.

Generative LLMs are extraordinarily capable general-purpose systems. But many production tasks ultimately require bounded decisions. For those tasks, arbitrary text generation may be unnecessary machinery.

Generative model

sequential decode
  1. 01input
  2. 02token"Yes"
  3. 03token", this"
  4. 04token" is a"
  5. 05JSON{ "decision": … }
  6. 06parse
  7. 07decision

Tokens are sampled one at a time, assembled into a structured value, then parsed into a decision.

System-One model

direct distribution
  1. 01input
  2. 02forward pass
  3. 03probability distributionYES 0.96 · NO 0.04
YES
96%
NO
4%

One forward pass produces the full typed distribution — no token loop, no parsing step.

Illustrative pipelines — not benchmark results.

The workload

Is this transaction fraudulent? Is this vulnerability real? Which category does this belong to? How severe is this issue? A normal LLM answers by generating tokens that represent an answer — then someone has to parse them.

  • yes / no
  • classification
  • ranking
  • scoring
  • routing
  • confidence estimation

The position

Open System One explores a different abstraction: state + typed decision → probability distribution. Not a replacement for general-purpose models — a different architecture for a different class of work.

Different architecture for a different class of work.
02Core primitives

Three ways to ask — each returns a distribution, never prose.

Decision queries are typed at the interface. The model does not choose a format; the caller does. Results below are illustrative.

Binary

01 · two-way
Binary(
    "Is this vulnerability exploitable?"
)
distributionillustrative
  • TRUE
    0.972
  • FALSE
    0.028

One question, one calibrated bit — with a probability attached.

Choice

02 · multi-class
Choice(
    "What class of vulnerability is this?",
    [
        "SQL Injection",
        "SSRF",
        "Path Traversal",
        "None"
    ]
)
distributionillustrative
  • SQL Injection
    .01
  • SSRF
    .94
  • Path Traversal
    .02
  • None
    .03

A closed label set, scored as a full distribution over classes.

Score

03 · ordinal
Score(
    "How severe is this issue?",
    [
        "none",
        "low",
        "medium",
        "high",
        "critical"
    ]
)
distributionillustrative
  • none
    .06
  • low
    .14
  • medium
    .31
  • high
    .36
  • critical
    .13

An ordinal distribution — not a point estimate like 8/10.

03Parallel decisions

Encode the state once.
Ask many questions.

The intended research direction: encode expensive context once, then evaluate many independent decisions against the shared representation — in parallel.

Shared state

The application receives an arbitrary URL from the request and performs an HTTP request…

encoded once → hstate
Question Aindependent head

Is input attacker controlled?

  • YES
    97%
  • NO
    3%
Question Bindependent head

Is this security-sensitive?

  • YES
    94%
  • NO
    6%
Question Cindependent head

Likely vulnerability class?

  • SSRF
    91%
  • Other
    9%
Question Dindependent head

Is the candidate exploitable?

  • TRUE
    78%
  • FALSE
    22%

Repeated prompting

With an autoregressive model, each new question typically means re-supplying the full context and decoding another text response — then parsing it.

Shared-state hypothesis

We want to test architectures where the state is encoded once and many independent queries attend to that representation. No performance numbers yet — that is what the research is for.

04Calibration

90% should mean 90%. Probability is a claim about the world.

A useful decision model should not merely output a winning class. Its probability should have meaning. Across many predictions made at approximately 0.90, approximately 90% should prove correct.

Reliability — stated vs observedIllustrative — not our results
stated confidenceobserved accuracy
  • 50%
    50%
    52%
  • 60%
    60%
    61%
  • 70%
    70%
    69%
  • 80%
    80%
    81%
  • 90%
    90%
    89%

Hypothetical data illustrating the goal. These are not Open System One benchmark results — no trained model exists yet.

Why it matters for automation

if probability > 0.995:
    automate()

elif probability > 0.80:
    escalate()

else:
    reject()

Downstream policy only works if the number above it is honest. Calibration is what turns a score into an operating threshold.

“The model estimates uncertainty. Software decides what to do with it.”

Working principle
05Example use cases

Where bounded decisions already run the workload.

Credible targets for a decision-native model: domains where the output is a typed judgment, and someone downstream acts on it.

Security

Vulnerability analysis

  • Does attacker-controlled data reach the sink?
  • Is the sink security-sensitive?
  • Is sanitization sufficient?
  • Is the candidate actually exploitable?
Agents

Agentic workflows

  • Did the agent satisfy the requirement?
  • Does the result need human review?
  • Which specialist should investigate next?
Quant

Trading & quantitative systems

  • Which market regime best describes the observed state?
  • Does this state satisfy strategy X's entry conditions?
  • How uncertain is the classification?

Framed as classification and uncertainty estimation — not predictions of returns or profitable trades.

Documents

Document processing

  • Is this invoice anomalous?
  • Which workflow should process it?
  • Does this require manual review?
06Open questions

What we’re trying to find out

No finished implementation exists yet. These are the research questions the project starts from — each one measurable, each one allowed to fail.

  1. RQ1

    Can decision models beat generation at decision workloads?

    Compare non-generative decision heads against generative LLM baselines on the tasks where the output is a bounded, typed decision.

    measureslatencythroughputVRAMaccuracycalibration
  2. RQ2

    Can state be encoded once?

    Investigate architectures where many independent questions operate against one shared state representation, instead of re-processing context per query.

  3. RQ3

    Can probabilities actually be trusted?

    Study proper scoring and calibration metrics rather than benchmark accuracy alone.

    measuresBrier scoreNLLECErisk / coverage
  4. RQ4

    How small can these models become?

    Investigate whether specialized models in the hundreds-of-millions to low-billions parameter range can outperform substantially larger generative models on bounded decision tasks.

  5. RQ5

    How should they be trained?

    Compare supervision signals and objectives end to end, with calibration treated as a first-class training concern.

    comparehard labelssoft labelsteacher distillationproper scoring rulescalibration-aware trainingRL-style decision objectives
07Planned architecture

Current research direction — not a settled design.

A provisional sketch of how a System-One model could be structured. The architecture itself is part of the research: it will be revised, compared against baselines, and possibly discarded.

Current research directionprovisional · subject to change
Decision queries
STATE

unstructured context

State encoder

read once

Cross attention

queries read state

binary head

TRUE / FALSE

choice head

label set

score head

ordinal scale

Probability distributions
Schematic only. This describes an intended research direction, not an implemented system.
  1. 01

    State encoder processes context — the unstructured input is read once into a representation.

  2. 02

    Decision queries encode question plus possible outputs — the typed decision is part of the input, not a generation target.

  3. 03

    Decision representations attend to state — each query reads what it needs from the shared encoding.

  4. 04

    Independent heads produce logits — binary, choice, and score heads share the backbone but not the output space.

  5. 05

    Logits become typed distributions — softmax or ordinal parameterizations, calibrated before use.

08Roadmap

An honest sequence — no dates, no finished milestones.

The project is at the specification stage. Everything below is either in progress or planned; nothing is claimed as shipped.

  1. IN PROGRESS

    Milestone 00: Specification

    • Define decision representation
    • Define dataset format
    • Define evaluation methodology
  2. PLANNED

    Milestone 01: Baseline

    • Pretrained encoder / reranker
    • Non-generative decision heads
    • Binary / Choice / Score API
  3. PLANNED

    Milestone 02: Open System One Base

    • Multi-domain training
    • Soft-label distillation
    • Calibration evaluation
  4. PLANNED

    Milestone 03: Shared-state architecture

    • Encode context once
    • Parallel independent decision queries
    • Benchmark scaling
  5. PLANNED

    Milestone 04: Specialized models

    • OpenSystem Security
    • OpenSystem Agent Judge

Milestones will be marked complete only when their artifacts are public. Dates are deliberately omitted.

09Open source

The interesting parts should be inspectable.

The project aims to release architecture, training code, evaluation tooling, and model weights openly as they become available. None of these exist as published artifacts yet — below is the commitment, not an inventory.

Planned for releasestatus: not yet published
  • Model architectureplanned
  • Training codeplanned
  • Dataset specificationplanned
  • Evaluation suiteplanned
  • Weightsplanned
  • Calibration resultsplanned
  • Experiment configsplanned

Checkboxes stay empty until the artifact is actually published. If it is not linked from the repository, it does not exist yet.

10Participation

We’re starting from zero.

There is no polished model behind this page yet. The goal is to build the first versions in public, measure them against strong generative baselines, publish what works, and discard what doesn’t.

Follow development on GitHubRead the research plan

No waitlist · no demo · no benchmark scores — because none of those things exist yet.