A GPT-4o-class model ran on a home PC! 'Strata' that could change the conventional wisdom of local AI (1/5)

What happened

On September 24, a tool called Strata for running AI models on your own computer (called local LLM) was released to the public. The writer ran a huge model on an RTX 4090 gaming PC with 64GB of normal memory. The model file is over 80GB, which normally could not run on this kind of PC.

Key numbers

• The model has 125 billion total parameters (125B), but only about 6 billion (6B) are active at one time.
• The compressed model file is over 80GB.
• The writer's PC has an RTX 4090 graphics card and 64GB of RAM.
• The writer measured about 80 tokens per second when writing a report on the 'singularity'.

Why it was hard before

A local AI model was limited by the video memory (VRAM) on the graphics card, or by shared memory on some Macs. A bigger model needed more of this fast memory. Strata splits the model's data across VRAM, normal memory (RAM), and the SSD (a storage drive). This lets a big model run on a smaller PC.

Why it matters

If this works widely, people could run smart AI at home without paying for cloud services. The writer says it could change how local AI is used. The article also says the model is roughly as capable as GPT-4o, an OpenAI model that was a main product about a year earlier.

What to watch

• Strata currently supports only Qwen3.8-Flash-Next.
• The writer used a separate PC as a server and connected from other PCs, because the setup uses almost all memory.
• Whether more models and more PCs can run Strata smoothly.