[ Datasets ]

Data for frontier AI.

Filtering pipelines, mixture ablations, and provenance for pretraining and SFT.

Papers: 3
Latest: 2024-09-09
Area: Datasets

Focus areas

Provenance

Documented sourcing, filtering, and deduplication.

Mixtures

Versioned code/math/prose weights with eval gates.

Publications

3 papers in this area

2024-09-09Experiments

Multimodal Alignment Pilot — Vision and Audio into the LM Space

Early encoder–LM alignment experiments that unlocked NextGen multimodal features.

Read →

2024-04-16Datasets

Dataset Mixture Ablations — Code, Math, and Prose Tradeoffs

Systematic mixture ablations showing how code and math density reshape Origin-class downstream profiles.

Read →

2023-02-14Datasets

Data Quality Filtering v1 — Provenance, Dedup, and Toxicity Screens

First production-oriented data pipeline for IdonAI pretraining: filter architecture, near-dedup strategy, toxicity screens, PII heuristics, and measured impact on training quality versus unfiltered baselines.

Read →

[ Research ]

Explore the full archive.

Browse every area, or email the research team about collaborations, datasets, and eval access.

All research Email research