bi.AddlmZmZmZmZmZddlZddlmZm Z ddl m Z ddl m Z mZddlmZmZd d lmZd d lmZd d lmZd d lmZd dlmZdZGddeeZy))CallableDictListOptionalUnionN) CLIPTextModel CLIPTokenizer)DDPMWuerstchenScheduler) deprecatereplace_example_docstring)DeprecatedPipelineMixinDiffusionPipeline) PaellaVQModel)WuerstchenDiffNeXt)WuerstchenPrior)WuerstchenDecoderPipeline)WuerstchenPriorPipelineax Examples: ```py >>> from diffusions import WuerstchenCombinedPipeline >>> pipe = WuerstchenCombinedPipeline.from_pretrained("warp-ai/Wuerstchen", torch_dtype=torch.float16).to( ... "cuda" ... ) >>> prompt = "an image of a shiba inu, donning a spacesuit and helmet" >>> images = pipe(prompt=prompt) ``` c/eZdZdZdZdZdededede de d ed ed e d e ffd Z d4de efdZd5de edeej&effdZd5de edeej&effdZd5dZdZej2eedddddddddddddddddddgddgfde eeeefd ed!ed"ed#e eed$ed%ed&e eed'ed(e eeeefd)e ej<d*e ej<d+ed,e eej>eej>fde ej<d-e ed.e d/e eeee!gdfd0eed1e eeee!gdfd2eef*d3Z"xZ#S)6WuerstchenCombinedPipelinea Combined Pipeline for text-to-image generation using Wuerstchen This model inherits from [`DiffusionPipeline`]. Check the superclass documentation for the generic methods the library implements for all the pipelines (such as downloading or saving, running on a particular device, etc.) Args: tokenizer (`CLIPTokenizer`): The decoder tokenizer to be used for text inputs. text_encoder (`CLIPTextModel`): The decoder text encoder to be used for text inputs. decoder (`WuerstchenDiffNeXt`): The decoder model to be used for decoder image generation pipeline. scheduler (`DDPMWuerstchenScheduler`): The scheduler to be used for decoder image generation pipeline. vqgan (`PaellaVQModel`): The VQGAN model to be used for decoder image generation pipeline. prior_tokenizer (`CLIPTokenizer`): The prior tokenizer to be used for text inputs. prior_text_encoder (`CLIPTextModel`): The prior text encoder to be used for text inputs. prior_prior (`WuerstchenPrior`): The prior model to be used for prior pipeline. prior_scheduler (`DDPMWuerstchenScheduler`): The scheduler to be used for prior pipeline. z0.33.1T tokenizer text_encoderdecoder schedulervqganprior_tokenizerprior_text_encoder prior_priorprior_schedulerc t ||j|||||||||  t|||| |_t ||||||_y)N) rrrrrr rrr!)priorrrr)rrrrr)super__init__register_modulesr prior_piper decoder_pipe) selfrrrrrrrr r! __class__s v/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.pyr%z#WuerstchenCombinedPipeline.__init__Jsv  %#1++  2+%%   6%  N attention_opc:|jj|yN)r(*enable_xformers_memory_efficient_attention)r)r-s r+r0zEWuerstchenCombinedPipeline.enable_xformers_memory_efficient_attentionqs DD\Rr,gpu_iddevicecx|jj|||jj||y)a Offloads all models to CPU using accelerate, reducing memory usage with a low impact on performance. Compared to `enable_sequential_cpu_offload`, this method moves one whole model at a time to the GPU when its `forward` method is called, and the model remains in GPU until the next model runs. Memory savings are lower than with `enable_sequential_cpu_offload`, but performance is much better due to the iterative execution of the `unet`. r1r2N)r'enable_model_cpu_offloadr(r)r1r2s r+r5z3WuerstchenCombinedPipeline.enable_model_cpu_offloadts4 00v0N 22&2Pr,cx|jj|||jj||y)u Offloads all models (`unet`, `text_encoder`, `vae`, and `safety checker` state dicts) to CPU using 🤗 Accelerate, significantly reducing memory usage. Models are moved to a `torch.device('meta')` and loaded on a GPU only when their specific submodule's `forward` method is called. Offloading happens on a submodule basis. Memory savings are higher than using `enable_model_cpu_offload`, but performance is lower. r4N)r'enable_sequential_cpu_offloadr(r6s r+r8z8WuerstchenCombinedPipeline.enable_sequential_cpu_offload~s4 55VF5S 77vf7Ur,cx|jj|||jj||y)N)iterabletotal)r' progress_barr()r)r:r;s r+r<z'WuerstchenCombinedPipeline.progress_bars2 $$he$D &&&Fr,c t|jjdi||jjdi|y)N)r'set_progress_bar_configr()r)kwargss r+r?z2WuerstchenCombinedPipeline.set_progress_bar_configs2///9&9111;F;r,i<g@ grpillatentspromptheightwidthprior_num_inference_stepsprior_timestepsprior_guidance_scalenum_inference_stepsdecoder_timestepsdecoder_guidance_scalenegative_prompt prompt_embedsnegative_prompt_embedsnum_images_per_prompt generator output_type return_dictprior_callback_on_step_end(prior_callback_on_step_end_tensor_inputscallback_on_step_end"callback_on_step_end_tensor_inputsc i}|jdd!|jd|d<tddd|jdd!tddd|jd|d<|jdid | |ndd |d |d |d |d|d| | ndd| d| d| d|d|ddddd|d||}|d}|jd|||nd||| | |||||d |}|S)a% Function invoked when calling the pipeline for generation. Args: prompt (`str` or `List[str]`): The prompt or prompts to guide the image generation for the prior and decoder. negative_prompt (`str` or `List[str]`, *optional*): The prompt or prompts not to guide the image generation. Ignored when not using guidance (i.e., ignored if `guidance_scale` is less than `1`). prompt_embeds (`torch.Tensor`, *optional*): Pre-generated text embeddings for the prior. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, text embeddings will be generated from `prompt` input argument. negative_prompt_embeds (`torch.Tensor`, *optional*): Pre-generated negative text embeddings for the prior. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, negative_prompt_embeds will be generated from `negative_prompt` input argument. num_images_per_prompt (`int`, *optional*, defaults to 1): The number of images to generate per prompt. height (`int`, *optional*, defaults to 512): The height in pixels of the generated image. width (`int`, *optional*, defaults to 512): The width in pixels of the generated image. prior_guidance_scale (`float`, *optional*, defaults to 4.0): Guidance scale as defined in [Classifier-Free Diffusion Guidance](https://huggingface.co/papers/2207.12598). `prior_guidance_scale` is defined as `w` of equation 2. of [Imagen Paper](https://huggingface.co/papers/2205.11487). Guidance scale is enabled by setting `prior_guidance_scale > 1`. Higher guidance scale encourages to generate images that are closely linked to the text `prompt`, usually at the expense of lower image quality. prior_num_inference_steps (`Union[int, Dict[float, int]]`, *optional*, defaults to 60): The number of prior denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. For more specific timestep spacing, you can pass customized `prior_timesteps` num_inference_steps (`int`, *optional*, defaults to 12): The number of decoder denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. For more specific timestep spacing, you can pass customized `timesteps` prior_timesteps (`List[float]`, *optional*): Custom timesteps to use for the denoising process for the prior. If not defined, equal spaced `prior_num_inference_steps` timesteps are used. Must be in descending order. decoder_timesteps (`List[float]`, *optional*): Custom timesteps to use for the denoising process for the decoder. If not defined, equal spaced `num_inference_steps` timesteps are used. Must be in descending order. decoder_guidance_scale (`float`, *optional*, defaults to 0.0): Guidance scale as defined in [Classifier-Free Diffusion Guidance](https://huggingface.co/papers/2207.12598). `guidance_scale` is defined as `w` of equation 2. of [Imagen Paper](https://huggingface.co/papers/2205.11487). Guidance scale is enabled by setting `guidance_scale > 1`. Higher guidance scale encourages to generate images that are closely linked to the text `prompt`, usually at the expense of lower image quality. generator (`torch.Generator` or `List[torch.Generator]`, *optional*): One or a list of [torch generator(s)](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. latents (`torch.Tensor`, *optional*): Pre-generated noisy latents, sampled from a Gaussian distribution, to be used as inputs for image generation. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor will ge generated by sampling using the supplied random `generator`. output_type (`str`, *optional*, defaults to `"pil"`): The output format of the generate image. Choose between: `"pil"` (`PIL.Image.Image`), `"np"` (`np.array`) or `"pt"` (`torch.Tensor`). return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~pipelines.ImagePipelineOutput`] instead of a plain tuple. prior_callback_on_step_end (`Callable`, *optional*): A function that calls at the end of each denoising steps during the inference. The function is called with the following arguments: `prior_callback_on_step_end(self: DiffusionPipeline, step: int, timestep: int, callback_kwargs: Dict)`. prior_callback_on_step_end_tensor_inputs (`List`, *optional*): The list of tensor inputs for the `prior_callback_on_step_end` function. The tensors specified in the list will be passed as `callback_kwargs` argument. You will only be able to include variables listed in the `._callback_tensor_inputs` attribute of your pipeline class. callback_on_step_end (`Callable`, *optional*): A function that calls at the end of each denoising steps during the inference. The function is called with the following arguments: `callback_on_step_end(self: DiffusionPipeline, step: int, timestep: int, callback_kwargs: Dict)`. `callback_kwargs` will include a list of all tensors as specified by `callback_on_step_end_tensor_inputs`. callback_on_step_end_tensor_inputs (`List`, *optional*): The list of tensor inputs for the `callback_on_step_end` function. The tensors specified in the list will be passed as `callback_kwargs` argument. You will only be able to include variables listed in the `._callback_tensor_inputs` attribute of your pipeline class. Examples: Returns: [`~pipelines.ImagePipelineOutput`] or `tuple` [`~pipelines.ImagePipelineOutput`] if `return_dict` is True, otherwise a `tuple`. When returning a tuple, the first element is a list with the generated images. prior_callbackNcallbackz1.0.0ztPassing `prior_callback` as an input argument to `__call__` is deprecated, consider use `prior_callback_on_step_end`prior_callback_stepszzPassing `prior_callback_steps` as an input argument to `__call__` is deprecated, consider use `prior_callback_on_step_end`callback_stepsrErFrGrK timestepsguidance_scalerNrOrPrQrRrDrSptrTFrWrXr) image_embeddingsrErKr^r_rNrRrSrTrWrXr>)getpopr r'r()r)rErFrGrHrIrJrKrLrMrNrOrPrQrRrDrSrTrUrVrWrXr@ prior_kwargs prior_outputsrboutputss r+__call__z#WuerstchenCombinedPipeline.__call__s^ ::& - 9'-zz2B'CL $  G  ::,d 3 ? &M  .4ZZ8N-OL) *' *26   !:  &  0  0F/MOSW ( $: #8      "< 0X# &)+#$##  -#/6R 3'1+##!5/Q    r,r/)NN)$__name__ __module__ __qualname____doc___last_supported_version_load_connected_pipesr rrr rrr%rrr0intrtorchr2strr5r8r<r?no_gradr TEXT2IMAGE_EXAMPLE_DOC_STRINGrfloatTensor Generatorboolrrh __classcell__)r*s@r+rr+s6' %  % $% $ % + %  % '% *% %% 1% NSxPXGYSQx}QUSXS_S_adSdMeQVHSMVRWX]XdXdfiXiRjVG<U]]_<=37)+15&)#%37(+;?049=%&MQ*.%* QU?HkKO9B -asDI~./aa a $' a "$u+. a$a!a$DK0a!&a"%T#Y"78a  -a!) 6a #aE%//43H"HIJa %,,'!a"c]#a$%a&%-XsC6F6L-M$N'a(37s))a*'xc40@$0F'GH+a,-1I-a>ar,r)typingrrrrrrp transformersrr schedulersr utilsr r pipeline_utilsrrmodeling_paella_vq_modelrmodeling_wuerstchen_diffnextrmodeling_wuerstchen_priorrpipeline_wuerstchenrpipeline_wuerstchen_priorrrsrr>r,r+rsE98 519G3<6:> !H!8:KHr,