Dataset

SENSUS SFT v1.1

The training set behind the LORM models on the SENSUS Bench v1.0 leaderboard. Each of the 1,015 synthetic conversations gives a problem, a person's reasoning about it and a reply that interprets that reasoning. Free to use under CC BY 4.0.

Conversations
1,015
Format
Chat JSONL: system, user, assistant
Scope
Fractions, decimals and place value
Origin
Synthetic
License
CC BY 4.0

What's in the Dataset

Training Guide

Reproduce the recorded supervised fine-tuning run, then compare your model with the leaderboard.

  1. Get the Files

    Download the dataset from Hugging Face into one folder and run the offline verifier. It confirms the 1,015 conversations and every file fingerprint.

    python3 verify.py
  2. Choose a Base Model

    The recorded runs use Qwen3.8-27B, Qwen3-14B and Qwen3-8B. Record the exact checkpoint you use; a different checkpoint is a new experiment.

  3. Upload to Fireworks

    Open supervised fine-tuning, upload train.jsonl as a chat dataset and validate it. Training runs in your own account and is billed there.

    Fireworks supervised fine-tuning documentation ↗

  4. Configure the Recorded Recipe

    SettingRecorded value
    Training examples1,015
    AdapterLoRA, rank 32
    Epochs8
    Learning rate0.0003
    Other settingsService defaults for the base model
    EvaluationTemperature 0, thinking disabled
  5. Evaluate

    Generate one reply per evaluation case in the same format and grade it with the two-grader checklist method. Keep both graders' marks and report incomplete cases as incomplete.

  6. Compare

    The final recorded 27B run scored 0.838; two runs of each recipe differed by up to 0.03. A new run should land near the leaderboard values, not exactly on them.

Compare your modelSENSUS Bench v1.0 leaderboard →
Close window

Contact Us

20 to 6,000 characters.

Close window Close window