Topic

On-Device Large Model Deployment: Challenges and Optimization Practices in Storage Systems

Future phones will be AI phones — deploying large models on-device is the key capability that enables them. This talk analyzes and breaks down the challenges encountered during on-device large-model deployment from the perspective of storage systems, and presents vivo’s solutions. The content covers foundational system support in Linux kernel file systems, memory, block I/O, and storage media, as well as how higher-layer applications can better leverage system capabilities given model characteristics, development difficulty, and resource constraints. Finally, the talk offers perspectives on the future evolution of storage systems. Outline: 1. Why phones should run large models on the edge 2. Pain points faced when deploying large models on-device 3. System-level solutions addressing the above pain points 4. Outlook on the future development of storage systems