Running 80B Qwen on Mac with 4.3GB RAM, 35B on iPhone

A developer shared techniques for running the large Qwen language models on Apple hardware with minimal memory usage. The approach enables an 80-billion-parameter model on a Mac using 4.3GB of RAM, and a 35-billion-parameter variant on an iPhone. This could make powerful on-device AI more accessible, reducing the need for cloud computing.

Sources (1)