01 / the Solstice family

Solstice-1-175M

A small English chat model.
Trained from scratch. Built to explore.

OpenSML · Apache-2.0 · Research preview

parameters, rounded
175M
pretraining tokens
42.001B
transformer layers
44
context tokens
2,048
01 / overview

small, by design/

Solstice-1-175M is the first model in a new OpenSML family. It combines 42.001 billion pretraining tokens with three stages of fine-tuning for short English conversations, reading comprehension and instruction following.

The model uses 44 narrow transformer layers and runs locally with MLX on Apple Silicon. Its package includes merged FP32 weights, the original tokenizer, inference code and the records behind the results.

Exact parameters
174,202,432
Hidden width
576
Attention
9 query / 3 KV heads
Vocabulary
32,000 tokens
Release weights
FP32 · 697 MB
Supported runtime
Native MLX

The 2,048-token context includes previous messages, chat boundaries and generated output. This is a separate weight lineage from OpenSML-150M and JSONLite; only the tokenizer is reused.

02 / recorded examples

see the answers/

Saved outputs from Solstice-1-175M, including a factual error. Choose a prompt to inspect its answer. This is a recorded demonstration, not live inference.

You / supplied prompt

Use only the information below. Give a short answer. Anika brought a teal notebook. Greta brought a black notebook. What color was Anika’s notebook?

Solstice-1-175M / unchanged output

teal

Correct · End of turn · reading-00

Greedy FP32 generation · no system message · no repetition penalty. View the example records.

03 / benchmarks

results, in view/

Knowledge and commonsense

Complete evaluation splits, scored with zero-shot answer likelihoods. Values are percentages: accuracy / character-normalized accuracy. Bold marks the highest reported value for each metric.

BenchmarkSolstice-1-175Macc / acc_normOpenSML-150Macc / acc_norm
ARC Easy62.58 / 62.3356.65 / 55.43
ARC Challenge29.35 / 30.8926.02 / 29.52
PIQA66.49 / 66.2164.53 / 64.09
HellaSwag32.85 / 39.4030.55 / 33.94

Instruction following

IFEval measures whether responses follow explicit instructions. It reports prompt and instruction accuracy; there is no normalized-accuracy measure.

IFEval accuracySolstice-1-175MOpenSML-150M
Strict prompts17.93%15.16%
Strict instructions30.10%25.30%
Loose prompts19.22%15.71%
Loose instructions32.49%25.78%

These are reported comparisons, not a controlled head-to-head rerun. The 150M release uses different chat formatting and a 1,280-token IFEval cap; Solstice uses 1,024. All 130 capped Solstice responses remain in its scores. Small differences have not been tested for statistical significance.

Short conversation checks

A separate authored panel tests 180 short answers and 12 open-ended responses. Solstice matches 59.44% of the checked answers. These automatic checks are narrower than human judgments of chat quality.

The three repetition flags include two clear loops and one milder repeated idea. Stopping normally does not establish correctness. Download the results · Full protocol and limitations.

04 / training

from tokens
to conversation/

Five Apple Silicon Macs train one model from random initialization. Pretraining begins with educational web text, DCLM and English Wikipedia. Python and SQL join the mixture at 36.801B tokens; the final base reaches 42.001B.

The last 2B tokens use a learning-rate cooldown from 7.94e-5 to 1e-5. Prose validation loss finishes at 2.8356 on the training-time BF16 evaluation path.

PROSE VALIDATION / FINAL 6B TOKENS2.8356
Recorded prose validation loss, 36B to 42B tokensThe final 2B-token learning-rate cooldown begins at 40.001B. Validation loss ends at 2.8356.2.842.882.922.963.00final cooldown36B38B40B42Bpretraining tokens processed
Recorded validation measurements. The shaded interval is the final cooldown. Full curve data ↓

Three stages, one selected lineage

Fine-tuning runs on one Mac, using attention adapters and chat-boundary token updates. Each stage adds new examples while replaying earlier material. Later stages also preserve the parent model’s output distribution.

PRETRAINING42.001Bprocessed tokens
random initialization
SFT 12,000updates
48,000 conversations
SFT 2256updates
editing and constraints
SFT 3128selected updates
reasoning and grounding

This is supervised fine-tuning with replay and parent-distribution preservation, not DPO or reinforcement learning. The release merges the selected updates into FP32 weights. Processed tokens are not a count of unique data. Read the complete training report.

05 / run locally

your Mac.
your model/

Use Python 3.11 on Apple Silicon. The native runner checks the weight and tokenizer identities, retains conversation history and rejects inputs that exceed the context budget.

The weights, tokenizer and source are public. No sign-in is required to download the model.

>_ local setup / bash
python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install huggingface_hub
hf download OpenSML/Solstice-1-175M --local-dir Solstice-1-175M
cd Solstice-1-175M
python -m pip install -r requirements.txt
python scripts/verify_package.py
python inference.py --interactive

Use /reset to clear the conversation and /quit to exit. After setup, inference is offline. Transformers AutoModel, MLX-LM auto-loading, CUDA and GGUF are not validated for this package.

The packaged runtime reproduced 27 saved outputs exactly. This checks packaging and numerical parity, not general answer quality. Complete run guide.

06 / limitations

an honest
starting point/

Fluent can still be wrong.

Facts and explanations can be incorrect. The model can invent details, including personal experience or professional credentials.

Instructions are inconsistent.

Strict IFEval prompt accuracy is 17.93%. Longer or combined instructions often fail or reach the output limit.

Context has a ceiling.

The entire conversation and answer share 2,048 tokens. Reliable long conversations, coding, SQL and tool use are not established.

Evaluation has limits.

Repetition and wording sensitivity remain. Authored tests share task families with training; no broad human or safety assessment is claimed.

07 / resources

inspect the work/

Model weights and author-controlled code are designated Apache-2.0. Third-party materials retain their own terms. The source archive documents the run; it is not a verified one-command retraining package.