Why are my video generations so random? Any advice?

My setup:

• 64 GB RAM
• 5070TI 16GB VRAM

I am using the smallest/turbo/pruned versions of the models, of course.

My problem:

No matter how much I try, writing the prompt myself, letting Maestro director or prompt enhancers make the heavy lifting or ask other services like ChatGPT to do that for me ... even short prompts, no matter if dialog is part of it or not, it feels very random what you'll get. When including dialog and references especially. I was hoping for patches to improve things, but even with the new references options where you can (nice btw!) define characters with image and audio reference a single time and re-use later.

Some of the problems:

1. generated storyboard seems very detailed, but taking a closer look even I am unable to understand what's getting asked there, eg giving countless instructions using references which all seem very similar or the same besides <Subject 1> and so on
2. scene description, mood definitions or anything else regularly bleeds into dialog, resulting in long sequences of gibberish, speech of descriptive text, ...
3. audio references do not stick to the person, generated audio has the voice profile of some other reference or a default voice
4. even if I define something simple, like 2 persons with a picture and audio each and 2 locations, results vary drastically, but you'll not get what you've asked for one single time
5. when giving let's say 4 locations and a timeline-like storyboard, but not too complicated, then generation seems sticking mainly to whatever gets used first. You may end up with none of the other locations at all or with some imaginary mashup of the first with whatever follows later.
6. and so on ...

When using references I'd use Minimax H3 Omni, but it's basically the same for LTX Video 2.3/2.5.

Is this an unavoidable limitation of the 16 GB VRAM, is it because of the heavily shrunken down models so they fit without constant swapping RAM <> VRAM and basically unusable speed, ... or what's going on here?

Thanks for hints, tips, advice!