Skip to main content

Enabling Qualcomm NPU Support in Kuwa v0.4.0

· 2 min read
Yung-Hsiang Hu

Kuwa v0.4.0 now includes support for Qualcomm's Neural Processing Unit (NPU), contributed by Thuniverse AI Inc. This integration delivers significantly faster generation speeds while minimizing power consumption. Because Kuwa can't reliably detect NPU availability on all systems, this feature is not enabled by default. Follow these steps to enable Qualcomm NPU support:

  1. Update Kuwa: Ensure you're running the latest version of Kuwa. Click the "Upgrade Kuwa" icon in the start menu.

  2. Access Maintenance Tools: Open the "Maintenance Tool" from the start menu to access Kuwa's internal utilities.

  3. Install QNN Dependencies: In the Maintenance Tool's console (type cmd to open it), execute the command build.bat qnn. This installs the necessary Qualcomm AI Engine Direct SDK dependencies.

  4. Launch Kuwa and Identify NPU Bots: Restart Kuwa. You'll now see several new chatbots with "@NPU" appended to their names, indicating NPU support.

  5. First Chat and Model Download: Initiate a conversation with one of these NPU-enabled bots. The model will download on the first interaction, resulting in a slightly longer initial response time.

  6. Verify NPU Usage: After the initial download, you'll observe that the model is running on the NPU, providing noticeably faster generation speeds.

info

For a pre-optimized experience with enhanced performance, consider the Turu for Qualcomm AI Hackathon distribution from Thuniverse AI.