Apple Crams Gargantuan AI Models into iPhones with “LLM in a Flash”

Apple is prepping to pack huge AI smarts into iPhones and iPads with a new technique called “LLM in a Flash.” This allows running advanced large language models (LLMs) that exceed a device’s available memory.

LLMs like Ghat GPT contain billions of parameters, demanding beefy GPUs during training. Even efficient models strain mobile hardware, restricting on-device inference. But Apple’s novel memory optimization unlocks phone-powered LLMs with near-PC capabilities.

The key innovation lies in storing LLM parameters in energy-efficient flash storage rather than limited DRAM. Parameters are intelligently loaded to DRAM as needed during inference, cuts down on expensive data transfers while maximizing reuse in memory.

Additional tricks tailored to flash storage’s strengths, like reading parameters in optimally-sized chunks, squeeze extra performance gains. Together, these enable inference for LLMs 2x larger than the available iPhone memory while accelerating speed over naive loading by 4-5x.

This slimmed-down LLM runs laps around stripped-down mobile NLP models today in accuracy and versatility. Soon, you may enjoy iPhone features rivaling Google Assistant without an internet connection?

Don’t expect consumer deployment soon though. Apple must carefully optimize factors like latency, battery drain, and thermals when running monster AI locally. We likely won’t see complex conversational assistants until Apple can deliver a seamless experience.

But “LLM in a Flash” demonstrates Apple’s commitment to on-device intelligence using innovative methods. Rather than cloud reliance, Apple wants iPhones packed with private, smart AI handy in the pocket.

Just as Apple pioneered smartphone computing, its willingness to push boundaries with AI silicon, software optimization, and app integration could soon bring sci-fi-like experiences to iPhones, with all processing securely done on-device.

So while flashy AI concept demos grab headlines today, don’t sleep on Apple’s behind-the-scenes research? Their patient approach to productizing AI could make iPhones the most intelligent, private-by-design personal assistants yet.

Latest

Nothing Phone (2a) Will be Powered by MediaTek’s Dimensity 7200 Pro chipset

Y'all have to give it up to Carl Pei...

Bluetooth Broadcastin’ Bud: Samsung Amps Up Auracast Support

Samsung is taking their audio experiences up a notch...

Apple Shuffles Leadership for Audio Products: What Does It Mean for the Future of AirPods and Other Audio Devices?

Report from Mark Gurman, Apple has made significant changes...

Huawei Makes Waves with Fresh Hardware Splash, eKitStor Xtreme M.2 SSDs speeds up to 7400MB/s

Just when you thought the data wars were cooling...

Newsletter

Don't miss

Nothing Phone (2a) Will be Powered by MediaTek’s Dimensity 7200 Pro chipset

Y'all have to give it up to Carl Pei...

Bluetooth Broadcastin’ Bud: Samsung Amps Up Auracast Support

Samsung is taking their audio experiences up a notch...

Apple Shuffles Leadership for Audio Products: What Does It Mean for the Future of AirPods and Other Audio Devices?

Report from Mark Gurman, Apple has made significant changes...

Huawei Makes Waves with Fresh Hardware Splash, eKitStor Xtreme M.2 SSDs speeds up to 7400MB/s

Just when you thought the data wars were cooling...

Apple Vision Pro Headset: Apple Fans Hit Return on Their Pricey Mixed Reality Vision

Let's be crystal clear upfront - the Vision Pro...
Max Hyland
Max Hyland
Long form contributor Apple iPhone, iPad, watch reviews, opinion, editorial

Nothing Phone (2a) Will be Powered by MediaTek’s Dimensity 7200 Pro chipset

Y'all have to give it up to Carl Pei and his crew at Nothing – they really know how to stir up excitement, even...

Bluetooth Broadcastin’ Bud: Samsung Amps Up Auracast Support

Samsung is taking their audio experiences up a notch with expanded Auracast capabilities, Samsung announced that their upcoming Galaxy S23, Z Fold5, Z Flip5...

Apple Shuffles Leadership for Audio Products: What Does It Mean for the Future of AirPods and Other Audio Devices?

Report from Mark Gurman, Apple has made significant changes to the leadership team responsible for audio products such as AirPods, Macs, and other devices....