bi`QddlmZmZmZmZmZddlZddlZddl m Z m Z ddl m Z ddlmZmZmZmZddlmZdd lmZmZmZd d lmZd d lmZer ddlmcmZ d Z!ndZ!ejDe#Z$dZ%GddeeZ&y))CallableDictListOptionalUnionN) CLIPTextModel CLIPTokenizer)DDPMWuerstchenScheduler) deprecateis_torch_xla_availableloggingreplace_example_docstring) randn_tensor)DeprecatedPipelineMixinDiffusionPipelineImagePipelineOutput) PaellaVQModel)WuerstchenDiffNeXtTFa Examples: ```py >>> import torch >>> from diffusers import WuerstchenPriorPipeline, WuerstchenDecoderPipeline >>> prior_pipe = WuerstchenPriorPipeline.from_pretrained( ... "warp-ai/wuerstchen-prior", torch_dtype=torch.float16 ... ).to("cuda") >>> gen_pipe = WuerstchenDecoderPipeline.from_pretrain("warp-ai/wuerstchen", torch_dtype=torch.float16).to( ... "cuda" ... ) >>> prompt = "an image of a shiba inu, donning a spacesuit and helmet" >>> prior_output = pipe(prompt) >>> images = gen_pipe(prior_output.image_embeddings, prompt=prompt) ``` c eZdZdZdZgdZ d%dededede de d e d d ffd Z d Z d&dZedZedZedZej(eed dd dd dd d ddd dgf deej0eej0fdeeeefdedeee de deeeeefdedeeej:eej:fdeej0d eed!ed"eeeee gd fd#eefd$Z!xZ"S)'WuerstchenDecoderPipelineaR Pipeline for generating images from the Wuerstchen model. This model inherits from [`DiffusionPipeline`]. Check the superclass documentation for the generic methods the library implements for all the pipelines (such as downloading or saving, running on a particular device, etc.) Args: tokenizer (`CLIPTokenizer`): The CLIP tokenizer. text_encoder (`CLIPTextModel`): The CLIP text encoder. decoder ([`WuerstchenDiffNeXt`]): The WuerstchenDiffNeXt unet decoder. vqgan ([`PaellaVQModel`]): The VQGAN model. scheduler ([`DDPMWuerstchenScheduler`]): A scheduler to be used in combination with `prior` to generate image embedding. latent_dim_scale (float, `optional`, defaults to 10.67): Multiplier to determine the VQ latent space size from the image embeddings. If the image embeddings are height=24 and width=24, the VQ latent shape needs to be height=int(24*10.67)=256 and width=int(24*10.67)=256 in order to match the training conditions. ztext_encoder->decoder->vqgan)latentstext_encoder_hidden_statesnegative_prompt_embedsimage_embeddings tokenizer text_encoderdecoder schedulervqganlatent_dim_scalereturnNcrt||j||||||j|y)N)rrr r!r")r#)super__init__register_modulesregister_to_config)selfrrr r!r"r# __class__s m/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/diffusers/pipelines/wuerstchen/pipeline_wuerstchen.pyr'z"WuerstchenDecoderPipeline.__init__[sF  %   1ABc|t||||}n;|j|k7rtd|jd||j|}||jz}|S)N) generatordevicedtypezUnexpected latents shape, got z , expected )rshape ValueErrortoinit_noise_sigma)r*r2r1r0r/rr!s r,prepare_latentsz)WuerstchenDecoderPipeline.prepare_latentsosg ?"5IfTYZG}}% #A'--P[\a[b!cddjj(GI666r-c t|tr t|nd}|j|d|jjdd}|j }|j } |j|ddj } | jd|jdk\rtj|| s|jj| dd|jjdz df} tjd |jjd | |ddd|jjf}| ddd|jjf} |j|j|| j| } | j} | j!|d } d}|r;|dg|z}nt#|t#|ur$t%dt#|dt#|dt|t&r|g}n1|t|k7r!t)d|dt|d|d|d |}|j|d|jjdd}|j|j j||j j| }|j}|jd}|j+d|d}|j-||z|d}| |fS)Nr max_lengthTpt)paddingr8 truncationreturn_tensorslongest)r:r<z\The following part of your input was truncated because CLIP can only handle sequences up to z tokens: )attention_maskrdimz?`negative_prompt` should be the same type to `prompt`, but got z != .z`negative_prompt`: z has batch size z, but `prompt`: zT. Please make sure that passed `negative_prompt` matches the batch size of `prompt`.) isinstancelistlenrmodel_max_length input_idsr?r2torchequal batch_decodeloggerwarningrr4last_hidden_staterepeat_interleavetype TypeErrorstrr3repeatview)r*promptr0num_images_per_promptdo_classifier_free_guidancenegative_prompt batch_size text_inputstext_input_idsr?untruncated_ids removed_texttext_encoder_outputr!uncond_text_encoder_hidden_states uncond_tokens uncond_input*negative_prompt_embeds_text_encoder_outputseq_lens r, encode_promptz'WuerstchenDecoderPipeline.encode_promptzs-%/vt$>66q$..JiJilmJmprJrGr7stL NNNN334Il^M ,A/P1P1P/P,PQN+A/P1P1P/P,PQN"//0A0A&0IZhZkZklrZs/t%8%J%J"%?%Q%QRgmn%Q%o",0) &&!#z 1 fT/%::UVZ[jVkUlmV ~Q(OS1!0 1 s?33 )/)::J3K_J`ax/ |<33 !0 >>$>>::# *L:>9J9J&&))&1,B]B]B`B`agBh:K: 61[0l0l -8==a@G0Q0X0XYZ\qst0u -0Q0V0V22GR1 -*+LLLr-c|jSN_guidance_scaler*s r,guidance_scalez(WuerstchenDecoderPipeline.guidance_scales###r-c |jdkDS)Nrrgris r,rWz5WuerstchenDecoderPipeline.do_classifier_free_guidances##a''r-c|jSrf)_num_timestepsris r, num_timestepsz'WuerstchenDecoderPipeline.num_timestepss"""r- grpilTrrrUnum_inference_steps timestepsrjrXrVr/ output_type return_dictcallback_on_step_end"callback_on_step_end_tensor_inputsc < |jdd}|jdd}| tddd| tddd| Ltfd| Ds8tdjd | Dcgc]}|jvs|c}j }j j}|_t|ts,t|tr|g}ntd t|d jr>| 1`. Higher guidance scale encourages to generate images that are closely linked to the text `prompt`, usually at the expense of lower image quality. negative_prompt (`str` or `List[str]`, *optional*): The prompt or prompts not to guide the image generation. Ignored when not using guidance (i.e., ignored if `decoder_guidance_scale` is less than `1`). num_images_per_prompt (`int`, *optional*, defaults to 1): The number of images to generate per prompt. generator (`torch.Generator` or `List[torch.Generator]`, *optional*): One or a list of [torch generator(s)](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. latents (`torch.Tensor`, *optional*): Pre-generated noisy latents, sampled from a Gaussian distribution, to be used as inputs for image generation. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor will ge generated by sampling using the supplied random `generator`. output_type (`str`, *optional*, defaults to `"pil"`): The output format of the generate image. Choose between: `"pil"` (`PIL.Image.Image`), `"np"` (`np.array`) or `"pt"` (`torch.Tensor`). return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~pipelines.ImagePipelineOutput`] instead of a plain tuple. callback_on_step_end (`Callable`, *optional*): A function that calls at the end of each denoising steps during the inference. The function is called with the following arguments: `callback_on_step_end(self: DiffusionPipeline, step: int, timestep: int, callback_kwargs: Dict)`. `callback_kwargs` will include a list of all tensors as specified by `callback_on_step_end_tensor_inputs`. callback_on_step_end_tensor_inputs (`List`, *optional*): The list of tensor inputs for the `callback_on_step_end` function. The tensors specified in the list will be passed as `callback_kwargs` argument. You will only be able to include variables listed in the `._callback_tensor_inputs` attribute of your pipeline class. Examples: Returns: [`~pipelines.ImagePipelineOutput`] or `tuple` [`~pipelines.ImagePipelineOutput`] if `return_dict` is True, otherwise a `tuple`. When returning a tuple, the first element is a list with the generated image embeddings. callbackNcallback_stepsz1.0.0zhPassing `callback` as an input argument to `__call__` is deprecated, consider use `callback_on_step_end`znPassing `callback_steps` as an input argument to `__call__` is deprecated, consider use `callback_on_step_end`c3:K|]}|jvywrf)_callback_tensor_inputs).0kr*s r, z5WuerstchenDecoderPipeline.__call__..2s#F 23A-- -F sz2`callback_on_step_end_tensor_inputs` has to be in z , but found z2'prompt' must be of type 'list' or 'str', but got rCz;'negative_prompt' must be of type 'list' or 'str', but got rr@)r0)r1zI'image_embeddings' must be of type 'torch.Tensor' or 'np.array', but got z5'num_inference_steps' must be of type 'int', but got zo In Case you want to provide explicit timesteps, please use the 'timesteps' argument.rr )rrr0r>)reffnetclip) model_outputtimestepsampler/rrrorderr)r9nprplatentzSOnly the output types `pt`, `np`, `pil` and `latent` are supported not output_type=rrrp):popr allr3r{_execution_devicer r1rhrDrErRrQrPrWrIcatrndarrayTensorr4intrdsize zeros_likeconfigr#r! set_timestepsrrrFr6rm enumerate progress_barexpandchunklerprjstep prev_samplelocalsgetattr XLA_AVAILABLExm mark_stepr" scale_factordecoderclamppermutecpufloatnumpy numpy_to_pilmaybe_free_model_hooksr)%r*rrUrqrrrjrXrVr/rrsrtrurvkwargsrxryr}r0r1 prompt_embedsrrr latent_height latent_widthlatent_features_shapeitratiopredicted_latentspredicted_latents_textpredicted_latents_uncondcallback_kwargscallback_outputsstep_idximagess%` r,__call__z"WuerstchenDecoderPipeline.__call__sT::j$/$4d;   z   %  A  . 9#F 7YF C DTEaEaDbbn|^pHvwbckokGkGbGpqpHoIJ  '' ""-&$'&#& "TUYZ`UaTbbc dee  + +*:ot3Tos3'6&7O#UVZ[jVkUllmn & -$yy)9qA  & 3$||,jw # // II')9)9:J)KL M! ,11!4t{{7S7SST +003dkk6R6RRS !1!6!6q!9GZZ&&w/66<rsr98 51ZZ-\\3<))MM   H %(F+ 79JF+r-