Open Athena finishes 535B-parameter model trained live in the open
What's the deal? Open AthenaDealroom has a profile for this one. Try Dealroom → has completed the main pre-training phase of Marin "Mochi," a 535B-parameter model, and released all 57 checkpoints on Hugging Face. The Stanford-born group trained the mixture-of-experts model — with roughly 23B active parameters — on 18.2T tokens, streaming the entire run live since August.
Why now? The model has finished pre-training, though evaluations and post-training are still underway. Open Athena says a full technical report and additional blog posts on its data curation and architecture will follow.
What's the endgame? The group is building in the open, releasing both process and artifacts. It calls this the largest live-streamed pre-training run ever and frames the checkpoints as material for researchers to study how large models are trained.
What about the model's evaluations? During training, unexpected changes in evaluation loss — plus concerns raised by Stanford student Michael Ryan — led Open Athena to investigate benchmark contamination. It found substantial leakage from common benchmarks that its own decontamination procedure had missed, limiting which evaluations it can fairly report.
What could go wrong? Open Athena acknowledges releasing pre-training artifacts raises safety concerns. The checkpoints have had no instruction tuning or safety post-training and "should not be treated as ready-to-deploy assistants," it says. It argues the research benefits outweigh the risks, especially as independent safety researchers have little access to artifacts from closed frontier labs.
The signal: Open development offers a rare window into how frontier models are built, at a time when closed labs pour enormous compute into post-training that Mochi's intermediate checkpoints have not undergone. Ian Hogarth, chair of the AI Safety Institute, calls it one of the most interesting AI projects right now — a sign the "open frontier" is scaling up.
Read more: openathena.ai
Image credit: Open Athena