The arrival of smarter smartphones

Though AI on smartphones is not new, we've seen AI running on ISPs and NPUs on-device for nearly a decade, the emergence of LLMs has led us to re-think and re-define the definition of "AI-capable" smartphones powered by generative AI (GenAI). This blog explores this new definition and why smartphones can be essential for vendors to implement on-device GenAI, as well as what on-device AI applications on smartphones will look like in the future.

What is the definition of an "AI-capable smartphone"?

Unlike PCs, smartphones have embraced AI technology for much longer, making "AI-capable smartphones" a much broader concept to define. Chipset vendors such as Qualcomm, Mediatek, Samsung and Google have focused on improving NPU/TPU performance for years. Smartphone vendors including Apple, Huawei, vivo, and Xiaomi are implementing AIML algorithms locally to improve imaging quality, battery life, and typing experience. However, integrating LLM and other GenAI models on-device will require a different computing platform and software capabilities. Canalys believes an "AI-capable" smartphone should meet the following criteria:

Minimal requirements: A smartphone that meets all the following criteria can be defined as an "AI-capable smartphone".

• The SoC should include a dedicated unit (eg, Qualcomm Hexagon, MediaTek APU, Google TPU) to accelerate AI-related tasks.
• The smartphone should be able to run LLMs (eg, Google Gemini, Samsung Gauss) and other generative AI models (eg, Stable Diffusion) on-device.
• LLM inference performance on-device should be faster than the average adult's reading speed, equating to 10 tokens per second (based on LLaMA-2 7B, or equivalent).
• Image generation using on-device AI should be less than two seconds (Stable Diffusion v1.5, 20 steps, 512*512, or equivalent).

Optional requirements: An "AI-capable smartphone" may not necessarily meet all the following requirements. However, meeting the optional requirements can enhance the user experience.

• A LLM (eg, Google Gemini, Samsung Gauss) should be pre-installed out of the box.
• GenAI-related APIs and SDKs are available for third-party developers to integrate GenAI-powered features in the mobile applications that can be run on-device.
• First-party GenAI-powered features developed by the vendor are pre-installed on the device and can be run on-device efficiently.

Why smartphones are essential to on-device AI?

Prominent installed base: Smartphones have the most extensive installed base compared to edge computing devices such as PCs. Canalys expects the total installed base of global smartphones to reach five billion in 2023, much larger than the 1.4 billion installed base for laptops and desktops combined. The gap will become more prominent, especially in developing markets and among younger generations, who have jumped over the PC era and spend more time on content consumption and social media using mobile-native apps. AI applications will, therefore, reach a larger audience when integrated into the smartphone form factor.

Portable form factor: Smartphones are portable and can accelerate the penetration of AI app usage. A pocket-sized device enables use cases that more closely align with daily activities, ranging from communication to entertainment. These use cases can complement the productivity use cases of PCs, thus widening the reach of AI in digital life. The availability of different sensors in smartphones further enhances data collection for AI, paving the way for the development of highly personalized AI applications.

Robust app ecosystem: Smartphone's robust app ecosystem will enable a thriving developer community for vendors. Unlike smartwatches, which typically have fewer applications in the marketplace, and focus on specific use cases such as health and workout suggestions, smartphones offer a broader canvas for AI innovation. The strength of the app ecosystem facilitates seamless AI integration across apps and fosters the creation of diverse third-party AI applications under a strong developer community.

Hub in the smart device ecosystem: A smartphone's connectivity and versatility means it plays a more critical role than other edge devices. Major players in the industry, such as Samsung with its Galaxy Ecosystem and Xiaomi with its smartphone x AIoT strategy, put smartphones at the center of their vast device ecosystems. Integrating AI on-device will further enhance the interconnected experience, empowering smartphone AI to generate personalized actions to control other devices based on user behavior and contextual scenarios.

How will on-device AI revolutionize smartphones in the near future?

The following chart shows a smartphone's major use cases including Entertainment, Communication, and Information in a pyramid view. We also list some (early) potential AI-powered features for each use case. AI-powered features such as enhanced photo editing and auto-reply, are expected to be used by most users. These high-frequency, AI-powered features will help vendors boost the usage of AI on smartphones and gain more mindshare among consumers.

Smartphones are also a light productivity device for most users compared to PCs. So vendors still need to develop meaningful features in the "productivity" area to attract prosumers in the premium segment. It is worth noting that prosumers expect quality, accuracy, and reliability in Gen AI output. Therefore, establishing credibility and trust in the AI features and their on-device models is essential, providing opportunities for differentiation for vendors that can demonstrate their AI capabilities early on to capture the premium prosumer market.

Some examples of AI features we identified from early AI-capable smartphones which are on the horizon.

Background tasks：

• Hardware optimization: AI can help the operating system optimize battery life and camera performance based on the pattern of user behaviors or scenarios.
• Cross-device automation: Harnessing data from smartphone sensors, AI will understand user scenarios and provide cross-device automation, such as turning on lights or TV upon the user's arrival home. Traditionally set by users, AI will now intelligently facilitate automation in the background, transforming the user experience.

Entertainment:

• Image and video editing: In addition to erasing objects and intelligent cutouts, users can leverage AI-powered features including automated color correction, background replacement, and style transfers for artistic or creative enhancements for a better photo and video editing experience.
• Social media post creation: AI can generate posts based on videos or photos to help users increase their social network engagement.

Communication:

• Automated call answering: AI will generate more natural sounds based on user tones in their automatic call answering. A better understanding of the other party's intention powered by AI will enhance the quality of the conversation.
• Auto Reply: Leveraging learned user patterns, AI will automatically generate natural reply suggestions on messaging apps.

Information:

• Video search: AI can help understand the content of a video and then search for specific information within the video. For example, using dialogue to search for a particular scene.
• On-device AI-powered chatbot and search: AI will help better understand user's search queries in natural languages and improve the search results. The on-device chatbot should understand and record user data and historical conversations, with user privacy protection as the top priority.

Productivity:

• Content summarization: AI will distill lengthy content from various formats (eg, audio, video, email attachments) into concise summaries, making it more convenient for users to consume on their smartphones.
• Schedule assistant: AI will leverage various data sources across apps to automatically create and manage calendar schedules while ensuring user data is protected.

We must emphasize the importance of building a third-party developer community in smartphone vendors' overall AI strategy and, more importantly, broadening smartphone use cases and creating AI-native apps. TikTok and Uber, mobile-first apps that shape how we use smartphones today, are often made by third-party developers, which will bring out Gen AI-powered apps, leveraging vendors' foundation models. Smartphone vendors must work closely with the developer community to gain future competitive advantage through unique AI features running on their devices. Some vendors have already taken steps, and our previous blog discussed Samsung's roadmap to develop AI leadership by collaborating with developers. Chinese vendors, OPPO, vivo, and Xiaomi, are also proactively fortifying the developer community by rolling out open-source LLMs. Stay tuned for our next blog to dive deeper into the opportunities and challenges for the Chinese vendors in the AI race.