I finally decided to try making videos with AI locally after hearing so much buzz about the new models. Been having a blast with it this week and wanted to share what I learned.
Lots of folks with 12GB cards are having trouble running the bigger models. The key thing I figured out is using GGUF format for everything. Regular model files just won’t fit in our limited VRAM.
My setup uses GGUF versions of the main model and text encoder, plus a speed optimization addon. With some smart memory management, I can generate a 4-5 second clip in about 5 minutes at decent quality.
My hardware:
RTX 3060 12GB
32GB system RAM
AMD Ryzen 3600
What you need:
Main model files (high and low noise versions) - about 8GB each
Text encoder model - 4GB
Speed enhancement addons - 600MB each
Custom workflow files for different generation types
The whole thing runs at around 640p resolution with minimal steps but the results are pretty solid for what it is. Way better than waiting hours with the full-size models that barely fit.
I’ve got a similar setup with my RTX 3070 and memory fragmentation kills me after several generations. I restart every 10-12 clips now - stops those random OOM crashes even when VRAM looks fine. Also turned off Windows memory compression since it messes with how models load.
Your setup looks good, but watch actual VRAM allocation vs what’s reported. They don’t always match and you’ll hit walls out of nowhere. I do low precision for previews first, then full quality only on the good ones. Saves a ton of time.
Nice work getting that running! I’ve been doing similar setups across different GPUs at work and GGUF is definitely the way to go for 12GB cards.
Converting the models yourself instead of downloading pre-converted ones makes a huge difference. You get better control over quantization levels and can squeeze out more performance. I usually go with Q5_K_M for the main model and Q8_0 for the text encoder since it’s more sensitive to quality loss.
Bumping up to 720p is totally doable if you drop the steps even lower. I run mine at 8 steps with CFG around 1.5 and it still looks decent for most content.
If you want to dive deeper into conversion, this video breaks down exactly how to do it:
The guy walks through the whole quantization workflow which helped me understand why certain settings work better. Game changer for getting models that actually fit properly instead of barely squeezing in.
Manual setup works but gets tedious fast when you’re experimenting with different models or running batches. I wasted way too many weekends tweaking GGUF configs and babysitting downloads.
Automating the pipeline changed everything. I queue multiple generations, auto-switch model configs based on content, and handle conversions hands-off.
The real magic happens when you connect other tools. Mine pulls prompts from spreadsheets, generates videos, auto-upscales the winners, and sorts everything into folders. Cuts hours off the manual grind.
Your hardware’s perfect for this - 32GB RAM handles background processes while your GPU cranks out generations.
If you want to streamline without writing scripts, Latenode chains all these steps together easily: https://latenode.com
i’ve been testing this on my 3070 ti too. setting up a pagefile on an nvme drive made a huge difference - windows swaps way faster than regular storage. if you’re still maxing out memory, drop the text encoder precision. you won’t really notice the quality difference.
Having the same memory issues with my RTX 4060 Ti. Dropping batch size to 1 freed up almost 2GB VRAM - generation time stays the same but way more stable. Watch your temps too. These long runs will throttle your GPU after 10-15 minutes if your cooling sucks. I undervolted mine by 50mv and temps dropped 8C with zero performance hit. Huge difference on extended sessions. Also, put your GGUF files on your fastest drive. The model reloads every time, so I went from SATA SSD to NVMe and startup dropped from 45 seconds to 15 per cycle.
