Utilizing Qualcomm NPUs for Mobile AI Development with LiteRT

As mobile AI models grow, LiteRT developers need to maximize performance. NPUs are increasingly crucial for on-device AI offering lower latency, higher throughput, and reduced power consumption compared to CPUs and GPUs. Leveraging Qualcomm® NPUs via the Qualcomm AI Engine Direct Delegate significantly improves mobile AI app performance on Snapdragon devices. In collaboration with Qualcomm, the Google AI Edge team will demonstrate integrating this delegate into Android apps, highlighting performance gains over traditional processors and showing how to get started.

Qualcomm AI Engine Direct Delegate

The Qualcomm AI Engine Direct Delegate enables users to run LiteRT models using the Qualcomm AI Stack. Using the Qualcomm AI Engine Direct Delegate is essential to running inference on the NPU for your LiteRT model on-device. Supported devices include:

  • Snapdragon 8 Gen 1 (SM8450)
  • Snapdragon 8 Gen 2 (SM8550)
  • Snapdragon 8 Gen 3 (SM8650)
  • Snapdragon 8 Elite (SM8750)
  • and more

Applications on these devices will benefit from the Qualcomm AI Stack by targeting the NPU which provides the best performance for AI models.

How to leverage the NPU using Qualcomm AI Engine Direct Delegate

First, download the Qualcomm AI Engine Direct Delegate available on Maven Central. To set up the delegate in an Android Java Application, the following dependencies are needed:

dependencies {
 implementation 'com.qualcomm.qti:qnn-runtime:2.34.0'
 implementation 'com.qualcomm.qti:qnn-litert-delegate:2.34.0' }

To use the delegate:

try {
  // Created default Options
  QnnDelegate.Options options =