Lightricks

/

LTX-2.5-Diffusers

like 33

Follow

LTX.io 4.69k

This PR adds prompt_enhancer and processor components to the Lightricks/LTX-2.5-Diffusers repo based on the google/gemma-4-E2B-it repo, following the diffusers docs. The changes can be tested by loading with the revision corresponding to this PR:

import torch
from diffusers import LTX2Pipeline

pipe = LTX2Pipeline.from_pretrained("Lightricks/LTX-2.5-Diffusers", revision="refs/pr/13", torch_dtype=torch.bfloat16)
...

or for the modular pipeline:

import torch
from diffusers import ComponentsManager, ModularPipeline

cm = ComponentsManager()
pipe = ModularPipeline.from_pretrained("Lightricks/LTX-2.5-Diffusers", components_manager=cm)
pipe.load_components(revision="refs/pr/13", dtype=torch.bfloat16)
...

Saving the modular pipeline with new prompt_enhancer and processor components using save_pretrained changes the modular_model_config.json config file with respect to main by adding library and class info for all components (not just the new ones). I don't think these changes should break anything; as far as I can tell, loading and performing inference on the revised checkpoint should work as expected.

Saving the modular pipeline with new prompt_enhancer and processor components using save_pretrained changes the modular_model_config.json config file with respect to main by adding library and class info for all components (not just the new ones). I don't think these changes should break anything; as far as I can tell, loading and performing inference on the revised checkpoint should work as expected.

Thx for the PR @dg845
For the prompt_enhancer do we need both the shards and the model.safetensors? It is a duplicate weights and extra 10GB of unnecessary download for the users?

I think not having the weights is possible for the LTX-2.5 modular pipeline, but not the standard pipeline. I have opened an alternative PR that allows the modular pipeline to load the prompt_enhancer and processor directly from google/gemma-4-E2B-it: #14.

I merged the #14 PR.
I assume that this PR is still relevant for the standard pipeline, right?
In my previous comment I meant the the prompt_enhancer folder contains duplicate weights. There is a prompt_enhancer/model.safetensors file and

prompt_enhancer/model-00001-of-00003.safetensors
prompt_enhancer/model-00002-of-00003.safetensors
prompt_enhancer/model-00003-of-00003.safetensors

If I am not mistaken, prompt_enhancer/model.safetensors is redundant

Thanks for clarifying! Since #14 has been merged, I have opened a new PR with only the prompt_enhancer model shards (following the other components in Lightricks/LTX-2.5-Diffusers) and only the standard pipeline changes at #15.