biJddlmZmZmZmZmZmZmZddlZddl m Z m Z ddl m Z mZddlmZmZddlmZddlmZmZd d lmZmZmZer ddlmcmZd Znd Zd Z GddeeZ!y))AnyCallableDictListOptionalTupleUnionN)CLIPTextModelWithProjection CLIPTokenizer)PipelineImageInputVaeImageProcessor) UVit2DModelVQModel)AmusedScheduler)is_torch_xla_availablereplace_example_docstring)DeprecatedPipelineMixinDiffusionPipelineImagePipelineOutputTFa Examples: ```py >>> import torch >>> from diffusers import AmusedInpaintPipeline >>> from diffusers.utils import load_image >>> pipe = AmusedInpaintPipeline.from_pretrained( ... "amused/amused-512", variant="fp16", torch_dtype=torch.float16 ... ) >>> pipe = pipe.to("cuda") >>> prompt = "fall mountains" >>> input_image = ( ... load_image( ... "https://huggingface.co/datasets/diffusers/docs-images/resolve/main/open_muse/mountains_1.jpg" ... ) ... .resize((512, 512)) ... .convert("RGB") ... ) >>> mask = ( ... load_image( ... "https://huggingface.co/datasets/diffusers/docs-images/resolve/main/open_muse/mountains_1_mask.png" ... ) ... .resize((512, 512)) ... .convert("L") ... ) >>> pipe(prompt, input_image, mask).images[0].save("out.png") ``` c/VeZdZUdZeed<eed<eed<eed<e ed<e ed<dZ dgZ dededede de f fd Z ejee d d eeeeefd ed ededededeeeeefdeedeej2deej4deej4deej4deej4dedeeeeej4gd fdedeeeefdedeeefdeeeeefeeff(dZ xZ!S)!AmusedInpaintPipelinez0.33.1image_processorvqvae tokenizer text_encoder transformer schedulerz text_encoder->transformer->vqvaect||j|||||t|ddr/dt |j j jdz znd|_t|jd|_ t|jdddd |_ |jjd y) N)rrrrrrrF)vae_scale_factor do_normalizeT)r#r$ do_binarizedo_convert_grayscale do_resizelinear)masking_schedule)super__init__register_modulesgetattrlenrconfigblock_out_channelsr#rrmask_processorrregister_to_config)selfrrrrr __class__s m/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/diffusers/pipelines/amused/pipeline_amused_inpaint.pyr+zAmusedInpaintPipeline.__init__Ts  %#  ELDRY[_D`A#djj''::;a? @fg  1$BWBWfkl/!22!%   ))8)DNpromptimage mask_imagestrengthnum_inference_stepsguidance_scalenegative_promptnum_images_per_prompt generator prompt_embedsencoder_hidden_statesnegative_prompt_embedsnegative_encoder_hidden_states return_dictcallbackcallback_stepscross_attention_kwargs"micro_conditioning_aesthetic_scoremicro_conditioning_crop_coord temperaturec | | | | td| | | | td|| | | tdt|tr|g}| t|}n| jd}||z}| ||j |ddd|j j jj|j}|j|dd }|j} |jd } | j|d } | j|d d } |d kDr| |dgt|z}t|tr|g}|j |ddd|j j jj|j}|j|dd }|j} |jd } | j|d } | j|d d } tj| | g} tj| | g} |j j#|}|jd d\}}tj$|||d|d |g|j| j&}|j)d}|j+|d kDrd|zn|d}|j,j/|||jt1t|j,j2|z}t|j,j2|z }|j4j&tj6k(xr |j4j8j:}|r|j4j=|j4j?|j|j4j&|jj@}|j\}} }!}"|j4jC|ddjE||!|"}|jFj#|||jHz||jHz}#|#jE|#jd|!|"jKj|jL}#|j,j8jN||#<|#jQ|jSz }$|j|d d }|jU|5}%tW|t|j,j2D]?}&|j,j2|&}'|d kDrtjX|gdz}(n|}(|j[|(|| | |})|d kDr|)j]d\}*}+|*||+|*z zz})|j,j_|)|'|| |$j`}|&t|j,j2d z k(s|&d z|j,jbzdk(r>|%je|,|&|zdk(r$|&tg|j,dd z},||,|'|ths,tkjlB ddd|dk(r|}-n|j4jo|d|||jHz||jHz|j4j8jpfjrjudd }-|j jw|-|}-|r|j4jy|j{|s|-fSt}|-S#1swYxYw)a The call function to the pipeline for generation. Args: prompt (`str` or `List[str]`, *optional*): The prompt or prompts to guide image generation. If not defined, you need to pass `prompt_embeds`. image (`torch.Tensor`, `PIL.Image.Image`, `np.ndarray`, `List[torch.Tensor]`, `List[PIL.Image.Image]`, or `List[np.ndarray]`): `Image`, numpy array or tensor representing an image batch to be used as the starting point. For both numpy array and pytorch tensor, the expected value range is between `[0, 1]` If it's a tensor or a list or tensors, the expected shape should be `(B, C, H, W)` or `(C, H, W)`. If it is a numpy array or a list of arrays, the expected shape should be `(B, H, W, C)` or `(H, W, C)` It can also accept image latents as `image`, but if passing latents directly it is not encoded again. mask_image (`torch.Tensor`, `PIL.Image.Image`, `np.ndarray`, `List[torch.Tensor]`, `List[PIL.Image.Image]`, or `List[np.ndarray]`): `Image`, numpy array or tensor representing an image batch to mask `image`. White pixels in the mask are repainted while black pixels are preserved. If `mask_image` is a PIL image, it is converted to a single channel (luminance) before use. If it's a numpy array or pytorch tensor, it should contain one color channel (L) instead of 3, so the expected shape for pytorch tensor would be `(B, 1, H, W)`, `(B, H, W)`, `(1, H, W)`, `(H, W)`. And for numpy array would be for `(B, H, W, 1)`, `(B, H, W)`, `(H, W, 1)`, or `(H, W)`. strength (`float`, *optional*, defaults to 1.0): Indicates extent to transform the reference `image`. Must be between 0 and 1. `image` is used as a starting point and more noise is added the higher the `strength`. The number of denoising steps depends on the amount of noise initially added. When `strength` is 1, added noise is maximum and the denoising process runs for the full number of iterations specified in `num_inference_steps`. A value of 1 essentially ignores `image`. num_inference_steps (`int`, *optional*, defaults to 16): The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. guidance_scale (`float`, *optional*, defaults to 10.0): A higher guidance scale value encourages the model to generate images closely linked to the text `prompt` at the expense of lower image quality. Guidance scale is enabled when `guidance_scale > 1`. negative_prompt (`str` or `List[str]`, *optional*): The prompt or prompts to guide what to not include in image generation. If not defined, you need to pass `negative_prompt_embeds` instead. Ignored when not using guidance (`guidance_scale < 1`). num_images_per_prompt (`int`, *optional*, defaults to 1): The number of images to generate per prompt. generator (`torch.Generator`, *optional*): A [`torch.Generator`](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. prompt_embeds (`torch.Tensor`, *optional*): Pre-generated text embeddings. Can be used to easily tweak text inputs (prompt weighting). If not provided, text embeddings are generated from the `prompt` input argument. A single vector from the pooled and projected final hidden states. encoder_hidden_states (`torch.Tensor`, *optional*): Pre-generated penultimate hidden states from the text encoder providing additional text conditioning. negative_prompt_embeds (`torch.Tensor`, *optional*): Pre-generated negative text embeddings. Can be used to easily tweak text inputs (prompt weighting). If not provided, `negative_prompt_embeds` are generated from the `negative_prompt` input argument. negative_encoder_hidden_states (`torch.Tensor`, *optional*): Analogous to `encoder_hidden_states` for the positive prompt. output_type (`str`, *optional*, defaults to `"pil"`): The output format of the generated image. Choose between `PIL.Image` or `np.array`. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~pipelines.stable_diffusion.StableDiffusionPipelineOutput`] instead of a plain tuple. callback (`Callable`, *optional*): A function that calls every `callback_steps` steps during inference. The function is called with the following arguments: `callback(step: int, timestep: int, latents: torch.Tensor)`. callback_steps (`int`, *optional*, defaults to 1): The frequency at which the `callback` function is called. If not specified, the callback is called at every step. cross_attention_kwargs (`dict`, *optional*): A kwargs dictionary that if specified is passed along to the [`AttentionProcessor`] as defined in [`self.processor`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention_processor.py). micro_conditioning_aesthetic_score (`int`, *optional*, defaults to 6): The targeted aesthetic score according to the laion aesthetic classifier. See https://laion.ai/blog/laion-aesthetics/ and the micro-conditioning section of https://huggingface.co/papers/2307.01952. micro_conditioning_crop_coord (`Tuple[int]`, *optional*, defaults to (0, 0)): The targeted height, width crop coordinates. See the micro-conditioning section of https://huggingface.co/papers/2307.01952. temperature (`Union[int, Tuple[int, int], List[int]]`, *optional*, defaults to (2, 0)): Configures the temperature scheduler on `self.scheduler` see `AmusedScheduler#set_timesteps`. Examples: Returns: [`~pipelines.pipeline_utils.ImagePipelineOutput`] or `tuple`: If `return_dict` is `True`, [`~pipelines.pipeline_utils.ImagePipelineOutput`] is returned, otherwise a `tuple` is returned where the first element is a list with the generated images. NzGpass either both `prompt_embeds` and `encoder_hidden_states` or neitherzXpass either both `negatve_prompt_embeds` and `negative_encoder_hidden_states` or neitherz,pass only one of `prompt` or `prompt_embeds`rpt max_lengthT)return_tensorspadding truncationrM)rDoutput_hidden_statesr!?)devicedtyper)rVrU)total) micro_condspooled_text_embrArG) model_outputtimestepsampler?starting_mask_ratioorderlatent)force_not_quantizeshape)? ValueError isinstancestrr.rbrmodel_max_length input_idsto_execution_devicer text_embeds hidden_statesrepeattorchconcatr preprocesstensorrV unsqueezeexpandr set_timestepsint timestepsrfloat16r/ force_upcastfloatencodelatentsquantizereshaper1r#boolrU mask_token_idsumnumel progress_barrangecatrchunkstep prev_sampler_updater- XLA_AVAILABLExm mark_stepdecodelatent_channelsr]clip postprocesshalfmaybe_free_model_hooksr).r3r7r8r9r:r;r<r=r>r?r@rArBrC output_typerDrErFrGrHrIrJ batch_sizergoutputsheightwidthrYstart_timestep_idxneeds_upcastingrz latents_bszchannelslatents_height latents_widthmaskr^rir\ model_inputr[ uncond_logits cond_logitsstep_idxoutputs. r5__call__zAmusedInpaintPipeline.__call__rsX  %*?*G  !&;&Gfg g " .3Q3Y " */M/Yj  N}4&:LQ^QjKL L fc "XF  VJ&,,Q/J"77  #$>>:: ' i4112 '' tZ^'_G#//M$+$9$9"$= !%,,-BAF 5 < <=RTUWX Y C %-"*')dS[&8Oos3'6&7O NN##'(##~~>> + )BBt556 ++I4^b+c)0)<)<&181F1Fr1J.%;%B%BCXZ[%\ "-K-R-RShjkmn-o *!LL*@-)PQM$)LL2PRg1h$i !$$//6 BC( ll-a0-a02  ))'--  "++A. !((>C;OZU_acd  $$%8+tG]G]^!#dnn&>&>"?("JK !9!9:=PP****emm;^ @Q@Q@^@^  JJ   **##EHH4::3C3CDLbLbH$cdll?F}}< X~}**%%g.q1!4<<[.Zgh""-- $"7"77$BWBW9W ||DJJqM>=INNPSST[TbTbc--;; "hhj7==?:..!61=   %8  9$ #\-s4>>3K3K/LM# #>>33A6!C'"'))WIM":K")K#// +$1*?+A 0  "C'1=1C1CA1F.M;#0>[S`E`3a#aL..--!-%"'(; . + DNN44599q1uH\H\>\`a>a '')+N0Ba0G#$(K#K 8W= LLNG# #$ #L ( "FZZ&&#'d333T222JJ%%55 ' fTT!QZ ))55fkJF ! ##%9 "6**{$ #$ #s4E\> \>>])NNNrS g$@Nr!NNNNNpilTNr!N)rr)rr)"__name__ __module__ __qualname___last_supported_versionr__annotations__rr r rrmodel_cpu_offload_seq_exclude_from_cpu_offloadr+rmno_gradrEXAMPLE_DOC_STRINGrr rrer rxrt GeneratorTensorr}rrrrr __classcell__)r4s@r5rrDsZ&&& N--> ") EE!E2 E ! E # E<U]]_1237$()-#% $;?/0/3048<9=AE GK;?239?>D-V+tCy#~./V+"V+' V+  V+ ! V+V+"%T#Y"78V+ (}V+EOO,V+  -V+ ( 5V+!) 6V+)1(>V+ !V+"8S#u||$rs\ EDD CD*)F\\))MM@F+35FF+r6