TaiwanTechnology and telecom2024 – ongoingMediaTek
Phi small language models in Dimensity 9400 chips
Chipmaker MediaTek reports more than 50% faster on-device AI using Microsoft's Phi models
Problem
Phone makers wanted on-device AI such as translation to run with low latency and high accuracy, without a steady internet connection.
Approach
MediaTek used Phi models from the Azure AI Foundry catalog and put Phi-3.5 into its Dimensity 9400 chipsets. It applied speculative decoding in Azure to speed up token processing. Data stays on the device.
Results
Takeaway
MediaTek reported that using small language models made on-device processing more than 50% faster than the earlier model, which ran at 12 tokens per second. It also claims 30% better power efficiency.
Categories
- Industry
- Technology and telecom
- Business function
- IT and R&D
- Innovation type
- Product and service innovation
- AI technology
- Generative AI
- Adoption
- Vendor solution
- Evidence
- Vendor's case study
Sources
Source summaries
Microsoft described how MediaTek built small language models into its phone chips. It reported speed and power-efficiency gains.
Full details are in the original sources.
