cross-posted from: https://lemmit.online/post/4242386
This is an automated archive made by the Lemmit Bot.
The original was posted on /r/pcmasterrace by /u/trander6face on 2024-10-24 11:11:47+00:00.
cross-posted from: https://lemmit.online/post/4242386
This is an automated archive made by the Lemmit Bot.
The original was posted on /r/pcmasterrace by /u/trander6face on 2024-10-24 11:11:47+00:00.
You need a model compiled for the architecture. I saw some for the RK35xx devices when shopping for hardware. I do not think there is software made to split up or run models in general on a NPU. The models must be configured for the physical hardware topology. The stuff that runs on most devices is very small, and these either need a ton of custom fine tuning or they are barely capable of simple tasks.
On the other hand, segmentation models are small, and that makes layers, object identification, and background removal stuff work. Looking at your CPU speed, and available memory, it is unlikely to make much difference. You are also memory constrained for running models, though you could use deepspeed to load from a disk drive too.