- Pricing
- Free
- CogVideoX-2B
About CogVideoX
CogVideoX is an open-source text-to-video and image-to-video generation model, with CogVideoX-2B and the larger CogVideoX-5B as the main releases. It is aimed at researchers and developers who want to run video generation locally rather than through a hosted API, and originates from the QingYing commercial video generation service. The original CogVideo, released in 2022, was the first open-source large Transformer-based text-to-video generation model.
A user can run CogVideoX-5B on a desktop GPU such as an RTX 3060, or the smaller CogVideoX-2B on older GPUs like a GTX 1080TI, after installing dependencies and following the inference code in cli_demo. The model only supports English text input, and the project recommends first optimizing a prompt with a large model such as GLM-4 or GPT-4, since the model is trained on long, detailed prompts. For multi-GPU inference, the sequential CPU offload optimization must be disabled.
CogVideoX-5B-I2V can take a background image as input and generate a video combined with prompt words, so the series now supports text-to-video generation, video continuation, and image-to-video generation. CogVideoX1.5-5B supports 10-second videos at higher resolution, and CogVideoX1.5-5B-I2V supports video generation at any resolution. Quantization with TorchAO or Optimum-quanto can reduce memory requirements enough to run on a free Colab T4 GPU, and DDIM Inverse is supported in CogVideoX-5B and CogVideoX1.5-5B.
CogVideoX is released under the Apache 2.0 License for the 2B model's weights, with the code hosted on GitHub under the Apache-2.0 license. Running the model requires a Python version between 3.10 and 3.12 inclusive, and FP8 precision requires an NVIDIA H100 or newer GPU with CUDA 12.4. Disabling memory optimizations multiplies memory consumption roughly three times while increasing speed by about three to four times, and only the diffusers version of the model supports quantization.
Use cases
- Generating a video from a text prompt
- Turning a still image into a video with prompt guidance
Key features
- Generates video from text and image prompts
- Image-to-video generation from a background image
- Runs on consumer GPUs down to a GTX 1080TI
- 10-second, higher-resolution video generation in CogVideoX1.5
- Quantized inference to fit on free Colab GPUs
- DDIM Inverse support
- LoRA fine-tuning support
- Cost-effective single-GPU fine-tuning via cogvideox-factory
Pros and cons
Pros
Cons
Pricing
- Apache-2.0 license
Reviews
Alternatives to CogVideoX
More in AI Image Segmentation
PixExtender
PixExtender is an AI image extender that expands images to exact sizes and aspect ratios, online, with free credits to start.
AudioCraft
AudioCraft is a PyTorch library for deep learning research on audio generation, with models like MusicGen and AudioGen and an EnCodec-based codec.
Tancky AI
Tancky AI is an online AI image combiner and photo merger that merges up to 4 images into a single photorealistic scene with commercial rights.
Image2Layer
Image2Layer separates a flat image into editable layers (up to 10) and exports layered PSD, PNGs, or ZIP packages for Photoshop and other tools.
AudioLDM
AudioLDM is a latent diffusion text-to-audio model that generates sound effects, speech and music and edits audio by text with style transfer and inpainting.
Ultralytics
Ultralytics is a leading provider of AI and machine learning solutions, specializing in open-source software development. Their flagship product, Ultralytics YOLO, offers advanced capabilities for real-time object detection, empowering users to harness the power of AI with ease. With a strong commitment to data privacy, Ultralytics ensures that users retain full ownership of their data, securely hosted on trusted platforms like Google Cloud and Amazon AWS.