[ Experiments ]

Active investigations.

Architecture, long-context, multimodal, agents, and structured-output experiments.

Papers: 12
Latest: 2025-11-14
Area: Experiments

Focus areas

Architecture

MoE routing and efficient attention variants.

Agents

Plan–act–verify loops with conservative product defaults.

Publications

12 papers in this area

2025-11-14Experiments

Agentic Tool Loops — Planning, Verification, and Stop Conditions

Experimental agent patterns on NextGen: plan–act–verify loops, and why Quark keeps tools conservative in production.

Read →

2025-06-12Publications

Distilling NextGen into Origin — Quality without Latency Tax

Task-selective distillation that lifts Origin’s coding and MMLU scores while preserving Flash-tier serving.

Read →

2025-03-19Experiments

Structured Output Constraints — JSON Reliability for Tools

Constrained decoding, schema adherence metrics, and failure taxonomies for Origin and NextGen tool use.

Read →

2024-11-21Publications

Quark Routing Prototype — Dual-Model Assistant Control Plane

First Quark router design: features, policies, cascade mode, and cost–quality tradeoffs before public launch.

Read →

2024-09-09Experiments

Multimodal Alignment Pilot — Vision and Audio into the LM Space

Early encoder–LM alignment experiments that unlocked NextGen multimodal features.

Read →

2024-07-18Experiments

Mixture-of-Experts Routing Study for NextGen-Class Models

Expert count, top-k routing, and load-balancing ablations that shaped NextGen’s sparse FFN design.

Read →

2024-05-02Experiments

Long-Context Mid-Training — From 4K to 128K and Toward 1M

Curriculum and position-extension methods that enabled Origin 128K and paved the way for NextGen 1M.

Read →

2024-04-16Datasets

Dataset Mixture Ablations — Code, Math, and Prose Tradeoffs

Systematic mixture ablations showing how code and math density reshape Origin-class downstream profiles.

Read →

2023-07-09Alignment

Instruction Tuning Pilot — From Base LM to Helpful Assistant

First SFT mixtures and evaluation rubrics that turned early checkpoints into usable assistants—precursor to Quark behavior.

Read →

2023-05-18Infrastructure

Grouped-Query Attention for Interactive Serving

Why Origin adopted GQA: KV-cache memory, batchability, and quality tradeoffs measured on internal canaries.

Read →

2023-03-22Experiments

Tokenizer Design Notes for a Shared Origin–NextGen Vocabulary

How we chose a byte-level BPE vocabulary for code-heavy workloads and future multimodal text interfaces—fertility measurements, code round-trip tests, and the case for boring infrastructure done right.

Read →

2022-09-08Experiments

Early Scaling Observations on Small Dense Transformers

2022 internal study of loss-vs-compute on dense decoder-only models—what scaled predictably, what failed to transfer to Origin, and what design decisions these experiments permanently shaped.

Read →

[ Research ]

Explore the full archive.

Browse every area, or email the research team about collaborations, datasets, and eval access.

All research Email research