A new AI video generation workflow is gaining traction. Instead of prompting a diffusion model directly:
- Use LLMs to generate 3D animation keyframes with Blender, three.js, etc.
- Once the scene looks right, feed its keyframe data or skeleton/depth video to a diffusion model like MiniMax H3 as reference, and prompt it to generate the final video.
Quite a few AI agent skills already do this, but they're just SKILL.md files, not actual apps. Or they need you to install 3rd party apps like Blender, making the process unintuitive.
CozyClay: AI-powered Previs
CozyClay is not a skill.md file, it is a full standalone web app you can run, which includes everything.
https://github.com/NomaDamas/CozyClay
You can talk to the sidebar agent to iterate on the 3D animation, like this:
Then once you're done, you can export the keyframe pack or the skeleton/depth video:
Then you can feed this to any AI video generation app to generate a real video.
No Local LLM Agents Yet
Local LLMs don't seem supported yet, but I'm curious how well open-source local agents would work.
If you have an OpenAI account, it's worth trying. "Sign in with ChatGPT" lets you log in with one click, no API keys needed (uses your Codex limits).
System Requirements
Since AI runs through remote APIs (OpenAI, Claude, Gemini, etc.), no powerful GPU or high VRAM is needed. Runs on any machine or OS.
Posted on X:
https://x.com/cocktailpeanut/status/2112931544390123985?s=20
A new AI video generation workflow is gaining traction. Instead of prompting a diffusion model directly:
Quite a few AI agent skills already do this, but they're just SKILL.md files, not actual apps. Or they need you to install 3rd party apps like Blender, making the process unintuitive.
CozyClay: AI-powered Previs
CozyClay is not a skill.md file, it is a full standalone web app you can run, which includes everything.
https://github.com/NomaDamas/CozyClay
You can talk to the sidebar agent to iterate on the 3D animation, like this:
Then once you're done, you can export the keyframe pack or the skeleton/depth video:
Then you can feed this to any AI video generation app to generate a real video.
No Local LLM Agents Yet
Local LLMs don't seem supported yet, but I'm curious how well open-source local agents would work.
If you have an OpenAI account, it's worth trying. "Sign in with ChatGPT" lets you log in with one click, no API keys needed (uses your Codex limits).
System Requirements
Since AI runs through remote APIs (OpenAI, Claude, Gemini, etc.), no powerful GPU or high VRAM is needed. Runs on any machine or OS.