bi ddlZddlmZmZmZmZmZmZddlZddl m Z ddl m Z m Z mZmZddlmZddlmZmZddlmZmZmZmZdd lmZmZmZdd lmZdd l m!Z!dd l"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)dd l*m+Z+ddl,m-Z-m.Z.ddl/m0Z0ddl1m2Z2ddl3m4Z4e%r ddl5m6cm7Z8dZ9ndZ9e&jte;Z d dee?deee@ejfdeee?deeeBfdZCGdde-e.eeeee4 ZDy)!N)AnyCallableDictListOptionalUnion)version)CLIPImageProcessor CLIPTextModel CLIPTokenizerCLIPVisionModelWithProjection) FrozenDict)PipelineImageInputVaeImageProcessor)FromSingleFileMixinIPAdapterMixinStableDiffusionLoraLoaderMixinTextualInversionLoaderMixin) AutoencoderKLImageProjectionUNet2DConditionModel)adjust_lora_scale_text_encoder)KarrasDiffusionSchedulers)USE_PEFT_BACKEND deprecateis_torch_xla_availableloggingreplace_example_docstringscale_lora_layersunscale_lora_layers) randn_tensor)DiffusionPipelineStableDiffusionMixin)StableDiffusionPipelineOutput)StableDiffusionSafetyChecker)PAGMixinTFa Examples: ```py >>> import torch >>> from diffusers import AutoPipelineForText2Image >>> pipe = AutoPipelineForText2Image.from_pretrained( ... "runwayml/stable-diffusion-v1-5", torch_dtype=torch.float16, enable_pag=True ... ) >>> pipe = pipe.to("cuda") >>> prompt = "a photo of an astronaut riding a horse on mars" >>> image = pipe(prompt, pag_scale=0.3).images[0] ``` c|jttd|jd}|jttd|jd}|||z z}||zd|z |zz}|S)a Rescales `noise_cfg` tensor based on `guidance_rescale` to improve image quality and fix overexposure. Based on Section 3.4 from [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://huggingface.co/papers/2305.08891). Args: noise_cfg (`torch.Tensor`): The predicted noise tensor for the guided diffusion process. noise_pred_text (`torch.Tensor`): The predicted noise tensor for the text-guided diffusion process. guidance_rescale (`float`, *optional*, defaults to 0.0): A rescale factor applied to the noise predictions. Returns: noise_cfg (`torch.Tensor`): The rescaled noise prediction tensor. r(T)dimkeepdim)stdlistrangendim) noise_cfgnoise_pred_textguidance_rescalestd_textstd_cfgnoise_pred_rescaleds b/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/diffusers/pipelines/pag/pipeline_pag_sd.pyrescale_noise_cfgr8Gs"""tE!_5I5I,J'KUY"ZHmmU1inn%= >mMG#x''9: #66!>N:NR[9[[I num_inference_stepsdevice timestepssigmasc | | td|dttj|jj j v}|std|jd|jd ||d||j}t|}||fS|dttj|jj j v}|std|jd|jd ||d||j}t|}||fS|j|fd |i||j}||fS) a Calls the scheduler's `set_timesteps` method and retrieves timesteps from the scheduler after the call. Handles custom timesteps. Any kwargs will be supplied to `scheduler.set_timesteps`. Args: scheduler (`SchedulerMixin`): The scheduler to get timesteps from. num_inference_steps (`int`): The number of diffusion steps used when generating samples with a pre-trained model. If used, `timesteps` must be `None`. device (`str` or `torch.device`, *optional*): The device to which the timesteps should be moved to. If `None`, the timesteps are not moved. timesteps (`List[int]`, *optional*): Custom timesteps used to override the timestep spacing strategy of the scheduler. If `timesteps` is passed, `num_inference_steps` and `sigmas` must be `None`. sigmas (`List[float]`, *optional*): Custom sigmas used to override the timestep spacing strategy of the scheduler. If `sigmas` is passed, `num_inference_steps` and `timesteps` must be `None`. Returns: `Tuple[torch.Tensor, int]`: A tuple where the first element is the timestep schedule from the scheduler and the second element is the number of inference steps. zYOnly one of `timesteps` or `sigmas` can be passed. Please choose one to set custom valuesr<zThe current scheduler class zx's `set_timesteps` does not support custom timestep schedules. Please check whether you are using the correct scheduler.)r<r;r=zv's `set_timesteps` does not support custom sigmas schedules. Please check whether you are using the correct scheduler.)r=r;r;) ValueErrorsetinspect signature set_timesteps parameterskeys __class__r<len) schedulerr:r;r<r=kwargsaccepts_timesteps accept_sigmass r7retrieve_timestepsrMbs>!3tuu'3w/@/@AXAX/Y/d/d/i/i/k+ll .y/B/B.CDab  M)FMfM'' !)n ) ))   C(9(9):Q:Q(R(](](b(b(d$ee .y/B/B.CD_`  GvfGG'' !)n ) ))  3MFMfM'' ) ))r9c5eZdZdZdZgdZdgZgdZ dGded e d e d e d e de d edededeeeefffd Z dHdeej.deej.deedeefdZdIdZdZdZdZ dJdZdIdZ dejBfdej.ded ejDd!ej.fd"Z#e$d#Z%e$d$Z&e$d%Z'e$d&Z(e$d'Z)e$d(Z*e$d)Z+ejXe-e.dddd*ddd+dd,d-ddddddd.ddd-ddd/gd0d-fd1eeeefd2eed3eed4ed5eed6eed7ed8eeeeefd9eed:ed;eeej^eej^fd/eej.deej.deej.deed?ed@ee1ee2fdAedeedBee3eee1gdfdCeedDedEef2dFZ4xZ5S)KStableDiffusionPAGPipelinea? Pipeline for text-to-image generation using Stable Diffusion. This model inherits from [`DiffusionPipeline`]. Check the superclass documentation for the generic methods implemented for all pipelines (downloading, saving, running on a particular device, etc.). The pipeline also inherits the following loading methods: - [`~loaders.TextualInversionLoaderMixin.load_textual_inversion`] for loading textual inversion embeddings - [`~loaders.StableDiffusionLoraLoaderMixin.load_lora_weights`] for loading LoRA weights - [`~loaders.StableDiffusionLoraLoaderMixin.save_lora_weights`] for saving LoRA weights - [`~loaders.FromSingleFileMixin.from_single_file`] for loading `.ckpt` files - [`~loaders.IPAdapterMixin.load_ip_adapter`] for loading IP Adapters Args: vae ([`AutoencoderKL`]): Variational Auto-Encoder (VAE) model to encode and decode images to and from latent representations. text_encoder ([`~transformers.CLIPTextModel`]): Frozen text-encoder ([clip-vit-large-patch14](https://huggingface.co/openai/clip-vit-large-patch14)). tokenizer ([`~transformers.CLIPTokenizer`]): A `CLIPTokenizer` to tokenize text. unet ([`UNet2DConditionModel`]): A `UNet2DConditionModel` to denoise the encoded image latents. scheduler ([`SchedulerMixin`]): A scheduler to be used in combination with `unet` to denoise the encoded image latents. Can be one of [`DDIMScheduler`], [`LMSDiscreteScheduler`], or [`PNDMScheduler`]. safety_checker ([`StableDiffusionSafetyChecker`]): Classification module that estimates whether generated images could be considered offensive or harmful. Please refer to the [model card](https://huggingface.co/runwayml/stable-diffusion-v1-5) for more details about a model's potential harms. feature_extractor ([`~transformers.CLIPImageProcessor`]): A `CLIPImageProcessor` to extract features from generated images; used as inputs to the `safety_checker`. z&text_encoder->image_encoder->unet->vae)safety_checkerfeature_extractor image_encoderrP)latents prompt_embedsnegative_prompt_embedsNTvae text_encoder tokenizerunetrIrQrRrequires_safety_checkerpag_applied_layersc :t||pt|jdddk7rVd|d|jjd} t dd| d t |j} d| d<t| |_|Xt|jd dd ur?d|d } t d d| d t |j} d| d <t| |_|%| r#tjd|jd| | td|duxrtt|jdxr\tjtj|jj j"tjdk} |duxr1t|jdxr|jj$dk}| r=|r;d} t dd| d t |j} d| d<t| |_|j'||||||||t|ddr/dt)|j*jj,dz znd|_t1|j.|_|j5| |j7| y)N steps_offsetr(z*The configuration file of this scheduler: z; is outdated. `steps_offset` should be set to 1 instead of a(. Please make sure to update the config accordingly as leaving `steps_offset` might led to incorrect results in future versions. If you have downloaded this checkpoint from the Hugging Face Hub, it would be very nice if you could open a Pull request for the `scheduler/scheduler_config.json` filezsteps_offset!=1z1.0.0F) standard_warn clip_sampleTa has not set the configuration `clip_sample`. `clip_sample` should be set to False in the configuration file. Please make sure to update the config accordingly as not setting `clip_sample` in the config might lead to incorrect results in future versions. If you have downloaded this checkpoint from the Hugging Face Hub, it would be very nice if you could open a Pull request for the `scheduler/scheduler_config.json` filezclip_sample not setz)You have disabled the safety checker for a by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .zMake sure to define a feature extractor when loading {self.__class__} if you want to use the safety checker. If you do not want to use the safety checker, you can pass `'safety_checker=None'` instead._diffusers_versionz 0.9.0.dev0 sample_size@aThe configuration file of the unet has set the default `sample_size` to smaller than 64 which seems highly unlikely. If your checkpoint is a fine-tuned version of any of the following: - CompVis/stable-diffusion-v1-4 - CompVis/stable-diffusion-v1-3 - CompVis/stable-diffusion-v1-2 - CompVis/stable-diffusion-v1-1 - runwayml/stable-diffusion-v1-5 - runwayml/stable-diffusion-inpainting you should change 'sample_size' to 64 in the configuration file. Please make sure to update the config accordingly as leaving `sample_size=32` in the config might lead to incorrect results in future versions. If you have downloaded this checkpoint from the Hugging Face Hub, it would be very nice if you could open a Pull request for the `unet/config.json` filezsample_size<64)rVrWrXrYrIrPrQrRrVr#)vae_scale_factor)rZ)super__init__getattrconfigr]rdictr_internal_dictloggerwarningrGr@hasattrr parser` base_versionraregister_modulesrHrVblock_out_channelsrdrimage_processorregister_to_configset_pag_applied_layers)selfrVrWrXrYrIrPrQrRrZr[deprecation_message new_configis_unet_version_less_0_9_0is_unet_sample_size_less_64rGs r7rfz#StableDiffusionPAGPipeline.__init__s   WY-=-=~q%QUV%V