introducing

Solstice-1-175M

the model is only
part of the story.

a vocabulary of its own.

Custom tokenizers and carefully prepared training data lay the groundwork for learning.

built on apple silicon.

We train models from scratch on Apple Silicon, exploring what accessible hardware can make possible.

results you can inspect.

We share evaluations, training records and limitations so others can inspect the results and build on the work.

distributed training

Our current cluster brings five Apple Silicon Macs together over a Thunderbolt RDMA ring. Explore the hardware, how the workload is split, and how every worker stays in sync.

Inside distributed training

training observatory

See how our models learned.

Follow Solstice-1-175M through 42B pretraining tokens and three fine-tuning stages. Replay the curves and inspect the inherited checkpoints.

Explore the observatory

training loss

2.4861
loss

throughput

26,186
tokens / second

learning rate

1.00e-5
optimizer rate

gradient norm

0.7830
L2 norm
Recorded steps · early steps expandedTrainingValidation

research

View all