01 / the Solstice family
Solstice-1-175M
A small English chat model.
Trained from scratch. Built to explore.
OpenSML · Apache-2.0 · Research preview
- parameters, rounded
- 175M
- pretraining tokens
- 42.001B
- transformer layers
- 44
- context tokens
- 2,048
small, by design/
Solstice-1-175M is the first model in a new OpenSML family. It combines 42.001 billion pretraining tokens with three stages of fine-tuning for short English conversations, reading comprehension and instruction following.
The model uses 44 narrow transformer layers and runs locally with MLX on Apple Silicon. Its package includes merged FP32 weights, the original tokenizer, inference code and the records behind the results.
- Exact parameters
- 174,202,432
- Hidden width
- 576
- Attention
- 9 query / 3 KV heads
- Vocabulary
- 32,000 tokens
- Release weights
- FP32 · 697 MB
- Supported runtime
- Native MLX
The 2,048-token context includes previous messages, chat boundaries and generated output. This is a separate weight lineage from OpenSML-150M and JSONLite; only the tokenizer is reused.
see the answers/
Saved outputs from Solstice-1-175M, including a factual error. Choose a prompt to inspect its answer. This is a recorded demonstration, not live inference.
Use only the information below. Give a short answer. Anika brought a teal notebook. Greta brought a black notebook. What color was Anika’s notebook?
teal
Correct · End of turn · reading-00
Greedy FP32 generation · no system message · no repetition penalty. View the example records.
results, in view/
Knowledge and commonsense
Complete evaluation splits, scored with zero-shot answer likelihoods. Values are percentages: accuracy / character-normalized accuracy. Bold marks the highest reported value for each metric.
| Benchmark | Solstice-1-175Macc / acc_norm | OpenSML-150Macc / acc_norm |
|---|---|---|
| ARC Easy | 62.58 / 62.33 | 56.65 / 55.43 |
| ARC Challenge | 29.35 / 30.89 | 26.02 / 29.52 |
| PIQA | 66.49 / 66.21 | 64.53 / 64.09 |
| HellaSwag | 32.85 / 39.40 | 30.55 / 33.94 |
Instruction following
IFEval measures whether responses follow explicit instructions. It reports prompt and instruction accuracy; there is no normalized-accuracy measure.
| IFEval accuracy | Solstice-1-175M | OpenSML-150M |
|---|---|---|
| Strict prompts | 17.93% | 15.16% |
| Strict instructions | 30.10% | 25.30% |
| Loose prompts | 19.22% | 15.71% |
| Loose instructions | 32.49% | 25.78% |
These are reported comparisons, not a controlled head-to-head rerun. The 150M release uses different chat formatting and a 1,280-token IFEval cap; Solstice uses 1,024. All 130 capped Solstice responses remain in its scores. Small differences have not been tested for statistical significance.
Short conversation checks
A separate authored panel tests 180 short answers and 12 open-ended responses. Solstice matches 59.44% of the checked answers. These automatic checks are narrower than human judgments of chat quality.
The three repetition flags include two clear loops and one milder repeated idea. Stopping normally does not establish correctness. Download the results · Full protocol and limitations.
from tokens
to conversation/
Five Apple Silicon Macs train one model from random initialization. Pretraining begins with educational web text, DCLM and English Wikipedia. Python and SQL join the mixture at 36.801B tokens; the final base reaches 42.001B.
The last 2B tokens use a learning-rate cooldown from 7.94e-5 to 1e-5. Prose validation loss finishes at 2.8356 on the training-time BF16 evaluation path.
Three stages, one selected lineage
Fine-tuning runs on one Mac, using attention adapters and chat-boundary token updates. Each stage adds new examples while replaying earlier material. Later stages also preserve the parent model’s output distribution.
random initialization
48,000 conversations
editing and constraints
reasoning and grounding
This is supervised fine-tuning with replay and parent-distribution preservation, not DPO or reinforcement learning. The release merges the selected updates into FP32 weights. Processed tokens are not a count of unique data. Read the complete training report.
your Mac.
your model/
Use Python 3.11 on Apple Silicon. The native runner checks the weight and tokenizer identities, retains conversation history and rejects inputs that exceed the context budget.
The weights, tokenizer and source are public. No sign-in is required to download the model.
python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install huggingface_hub
hf download OpenSML/Solstice-1-175M --local-dir Solstice-1-175M
cd Solstice-1-175M
python -m pip install -r requirements.txt
python scripts/verify_package.py
python inference.py --interactiveUse /reset to clear the conversation and /quit to exit. After setup, inference is offline. Transformers AutoModel, MLX-LM auto-loading, CUDA and GGUF are not validated for this package.
The packaged runtime reproduced 27 saved outputs exactly. This checks packaging and numerical parity, not general answer quality. Complete run guide.
an honest
starting point/
Fluent can still be wrong.
Facts and explanations can be incorrect. The model can invent details, including personal experience or professional credentials.
Instructions are inconsistent.
Strict IFEval prompt accuracy is 17.93%. Longer or combined instructions often fail or reach the output limit.
Context has a ceiling.
The entire conversation and answer share 2,048 tokens. Reliable long conversations, coding, SQL and tool use are not established.
Evaluation has limits.
Repetition and wording sensitivity remain. Authored tests share task families with training; no broad human or safety assessment is claimed.
inspect the work/
Model weights and author-controlled code are designated Apache-2.0. Third-party materials retain their own terms. The source archive documents the run; it is not a verified one-command retraining package.