I got TinyLlama 1.1B and n8n working on my Samsung M32 which has 6GB RAM and MediaTek G80 processor. I set everything up using Debian through proot-distro in Termux without needing root access. Both applications are running locally and performing way better than I thought they would on this hardware. The setup process was smoother than expected but now I’m wondering if there’s a method to tap into the GPU for better performance. Has anyone figured out how to enable GPU acceleration in this kind of setup? I’m curious if the MediaTek G80’s GPU can be utilized somehow to speed things up even more.
mediatek chips r super locked down. even if u manage to get gpu access, it’ll kill ur battery fast. better off playing with cpu threads instead - on my old phone, limiting cores helped a lot with speed.
Your MediaTek G80 won’t handle GPU acceleration for LLM inference. The Mali-G52 doesn’t have the computational precision transformer models need, plus Android blocks direct GPU memory access anyway. Focus on memory bandwidth instead - adjust your model’s context window size and batch processing parameters. I’ve tested similar setups on budget devices and tweaking memory allocation in your proot environment gives way better performance gains than trying to force GPU acceleration. Since you’re already getting decent performance, your CPU’s doing the heavy lifting just fine.
GPU acceleration through proot-distro won’t work well with MediaTek processors. They don’t expose GPU compute capabilities like desktop GPUs do. Your G80’s Mali-G52 MP2 doesn’t have proper OpenCL drivers in most Linux environments, and Termux’s sandbox makes direct hardware access nearly impossible. I’ve tried similar setups on Helio processors - CPU optimization beats attempting GPU passthrough every time. You might get small improvements by enabling ARM NEON optimizations if TinyLlama supports them, but honestly? GPU acceleration gains on mobile SoCs are usually overhyped. Your current setup’s already performing way better than expected for that hardware.
This topic was automatically closed 4 days after the last reply. New replies are no longer allowed.