Kuwa v0.4.0 now includes support for Qualcomm's Neural Processing Unit (NPU), contributed by Thuniverse AI Inc. This integration delivers significantly faster generation speeds while minimizing power consumption. Because Kuwa can't reliably detect NPU availability on all systems, this feature is not enabled by default. Follow these steps to enable Qualcomm NPU support:
-
Update Kuwa: Ensure you're running the latest version of Kuwa. Click the "Upgrade Kuwa" icon in the start menu.
-
Access Maintenance Tools: Open the "Maintenance Tool" from the start menu to access Kuwa's internal utilities.
-
Install QNN Dependencies: In the Maintenance Tool's console (type
cmdto open it), execute the commandbuild.bat qnn. This installs the necessary Qualcomm AI Engine Direct SDK dependencies.
-
Launch Kuwa and Identify NPU Bots: Restart Kuwa. You'll now see several new chatbots with "@NPU" appended to their names, indicating NPU support.
-
First Chat and Model Download: Initiate a conversation with one of these NPU-enabled bots. The model will download on the first interaction, resulting in a slightly longer initial response time.
-
Verify NPU Usage: After the initial download, you'll observe that the model is running on the NPU, providing noticeably faster generation speeds.
For a pre-optimized experience with enhanced performance, consider the Turu for Qualcomm AI Hackathon distribution from Thuniverse AI.