bi=ddlmZddlmZmZmZddlZddlmZm Z ddl m Z ddl m Z dd lmZd d lmZmZmZeGd d e ZGddeeZy)) dataclass)OptionalTupleUnionN) ConfigMixinregister_to_config) BaseOutput)apply_forward_hook) ModelMixin) DecoderOutput DecoderTiny EncoderTinyc0eZdZUdZej ed<y)AutoencoderTinyOutputz Output of AutoencoderTiny encoding method. Args: latents (`torch.Tensor`): Encoded outputs of the `Encoder`. latentsN)__name__ __module__ __qualname____doc__torchTensor__annotations__i/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/diffusers/models/autoencoders/autoencoder_tiny.pyrrs\\rrc eZdZdZdZe d&dededeedfdeedfded ed ed ed eedfd eedfdede de de de ffd Z de jde jfdZde jde jfdZd'dZd'dZd(de ddfdZd'dZde jde jfdZde jde jfdZed(de jd e deeee jffd!Ze d)de jd"ee j6d e deeee jffd#Z d(d$e jd e deeee jffd%ZxZS)*AutoencoderTinya A tiny distilled VAE model for encoding images into latents and decoding latent representations into images. [`AutoencoderTiny`] is a wrapper around the original implementation of `TAESD`. This model inherits from [`ModelMixin`]. Check the superclass documentation for its generic methods implemented for all models (such as downloading or saving). Parameters: in_channels (`int`, *optional*, defaults to 3): Number of channels in the input image. out_channels (`int`, *optional*, defaults to 3): Number of channels in the output. encoder_block_out_channels (`Tuple[int]`, *optional*, defaults to `(64, 64, 64, 64)`): Tuple of integers representing the number of output channels for each encoder block. The length of the tuple should be equal to the number of encoder blocks. decoder_block_out_channels (`Tuple[int]`, *optional*, defaults to `(64, 64, 64, 64)`): Tuple of integers representing the number of output channels for each decoder block. The length of the tuple should be equal to the number of decoder blocks. act_fn (`str`, *optional*, defaults to `"relu"`): Activation function to be used throughout the model. latent_channels (`int`, *optional*, defaults to 4): Number of channels in the latent representation. The latent space acts as a compressed representation of the input image. upsampling_scaling_factor (`int`, *optional*, defaults to 2): Scaling factor for upsampling in the decoder. It determines the size of the output image during the upsampling process. num_encoder_blocks (`Tuple[int]`, *optional*, defaults to `(1, 3, 3, 3)`): Tuple of integers representing the number of encoder blocks at each stage of the encoding process. The length of the tuple should be equal to the number of stages in the encoder. Each stage has a different number of encoder blocks. num_decoder_blocks (`Tuple[int]`, *optional*, defaults to `(3, 3, 3, 1)`): Tuple of integers representing the number of decoder blocks at each stage of the decoding process. The length of the tuple should be equal to the number of stages in the decoder. Each stage has a different number of decoder blocks. latent_magnitude (`float`, *optional*, defaults to 3.0): Magnitude of the latent representation. This parameter scales the latent representation values to control the extent of information preservation. latent_shift (float, *optional*, defaults to 0.5): Shift applied to the latent representation. This parameter controls the center of the latent space. scaling_factor (`float`, *optional*, defaults to 1.0): The component-wise standard deviation of the trained latent space computed using the first batch of the training set. This is used to scale the latent space to have unit variance when training the diffusion model. The latents are scaled with the formula `z = z * scaling_factor` before being passed to the diffusion model. When decoding, the latents are scaled back to the original scale with the formula: `z = 1 / scaling_factor * z`. For more details, refer to sections 4.3.2 and D.1 of the [High-Resolution Image Synthesis with Latent Diffusion Models](https://huggingface.co/papers/2112.10752) paper. For this Autoencoder, however, no such scaling factor was used, hence the value of 1.0 as the default. force_upcast (`bool`, *optional*, default to `False`): If enabled it will force the VAE to run in float32 for high image resolution pipelines, such as SD-XL. VAE can be fine-tuned / trained to a lower range without losing too much precision, in which case `force_upcast` can be set to `False` (see this fp16-friendly [AutoEncoder](https://huggingface.co/madebyollin/sdxl-vae-fp16-fix)). T in_channels out_channelsencoder_block_out_channels.decoder_block_out_channelsact_fn upsample_fnlatent_channelsupsampling_scaling_factornum_encoder_blocksnum_decoder_blockslatent_magnitude latent_shift force_upcastscaling_factor shift_factorc t|t|t| k7r tdt|t| k7r tdt ||| |||_t ||| |||||_| |_| |_ ||_ d|_ d|_ d|z|_ d|_d|_|j|jz|_|j#| |j#d y) NzQ`encoder_block_out_channels` should have the same length as `num_encoder_blocks`.zQ`decoder_block_out_channels` should have the same length as `num_decoder_blocks`.)r!r" num_blocksblock_out_channelsr%)r!r"r1r2r(r%r&Fr g?i)r2)r-)super__init__len ValueErrorrencoderrdecoderr+r,r. use_slicing use_tilingspatial_scale_factortile_overlap_factortile_sample_min_sizetile_latent_min_sizer )selfr!r"r#r$r%r&r'r(r)r*r+r,r-r.r/ __class__s rr4zAutoencoderTiny.__init__as&  ) *c2D.E Epq q ) *c2D.E Epq q"#()9   #'%)9&?#  !1(, %&|O!#( $'!$($=$=AZAZ$Z! 3MN U3rxreturnc|jd|jzj|jj ddS)zraw latents -> [0, 1]r rr)divr+addr,clampr?rAs r scale_latentszAutoencoderTiny.scale_latentss;uuQ.../33D4E4EFLLQPQRRrcp|j|jjd|jzS)z[0, 1] -> raw latentsr )subr,mulr+rGs runscale_latentszAutoencoderTiny.unscale_latentss-uuT&&'++A0E0E,EFFrNcd|_y)z Enable sliced VAE decoding. When this option is enabled, the VAE will split the input tensor in slices to compute decoding in several steps. This is useful to save some memory and allow larger batch sizes. TNr9r?s renable_slicingzAutoencoderTiny.enable_slicings  rcd|_y)z Disable sliced VAE decoding. If `enable_slicing` was previously enabled, this method will go back to computing decoding in one step. FNrNrOs rdisable_slicingzAutoencoderTiny.disable_slicings !rr:c||_y)a Enable tiled VAE decoding. When this option is enabled, the VAE will split the input tensor into tiles to compute decoding and encoding in several steps. This is useful for saving a large amount of memory and to allow processing larger images. N)r:)r?r:s r enable_tilingzAutoencoderTiny.enable_tilings %rc&|jdy)z Disable tiled VAE decoding. If `enable_tiling` was previously enabled, this method will go back to computing decoding in one step. FN)rTrOs rdisable_tilingzAutoencoderTiny.disable_tilings 5!rc@|j}|j}t||jz}||z }t d|j d|}t d|j d|}t jt jt j||z ||z dz z gdzd}|jddj|j}t j|j dd|j d|z|j d|z|j } |D]} |D]} |d | | |z| | |zf} | d | |z| |z|z| |z| |z|zf} |j| }|j d|j d}}| dk(rt j|dn|d}| dk(rt j|dn|d}||z}|d d |d |f|d d |d |f}}| j!||zd|z | zz| S) Encode a batch of images using a tiled encoder. When this option is enabled, the VAE will split the input tensor into tiles to compute encoding in several steps. This is useful to keep memory use constant regardless of image size. To avoid tiling artifacts, the tiles overlap and are blended together to form a smooth output. Args: x (`torch.Tensor`): Input batch of images. Returns: `torch.Tensor`: Encoded batch of images. rrr ijindexingdevice.N)r;r=intr<rangeshaperstackmeshgridarangerFtor`zerosr7 ones_likecopy_r?rAsf tile_size blend_size traverse_sizetitj blend_masksoutijtile_intile_outtilehw blend_mask_i blend_mask_j blend_masks r _tiled_encodezAutoencoderTiny._tiled_encodesR & &-- T%=%==> !J. 1aggbk= 1 1aggbk= 1kk NNELLR8JOakZ[n BCq&u{1~>kZ[n )L8 #'RaR! #4jbqb"1"6MjzD0A Nh3NNO P P rc$|j}|j}t||jz}||z }t d|j d|}t d|j d|}t jt jt j||z||zdz z gdzd}|jddj|j}t j|j dd|j d|z|j d|z|j } |D]} |D]} |d | | |z| | |zf} | d | |z| |z|z| |z| |z|zf} |j| }|j d|j d}}| dk(rt j|dn|d}| dk(rt j|dn|d}||zd d |d |f}| j!||zd|z | zz| S) rXrrYrZrr r[r\rr_.N)r;r>rar<rbrcrrdrerfrFrgr`rhr8rirjrks r _tiled_decodezAutoencoderTiny._tiled_decodes4 & &-- T%=%==> !J. 1aggbk= 1 1aggbk= 1kk NNELLR8JOakZ[n BCq&u{1~>kZ[n *\93BQB;G zD0A Nh3NNO P P r return_dictc|jrr|jddkDr`|jdDcgc]0}|jr|j |n|j |2}}t j|}n.|jr|j |n|j |}|s|fSt|Scc}w)Nrr)r) r9rcsplitr:r~r7rcatr)r?rArx_sliceoutputs rencodezAutoencoderTiny.encode!s    Qijipipqris^et""7+DLLQXDYYFYYv&F.2ooT''*4< generatorc|jrr|jddkDr`|jdDcgc]0}|jr|j |n|j |2}}t j|}n.|jr|j |n|j |}|s|fSt|Scc}w)Nrrsample) r9rcrr:rr8rrr)r?rArrrrs rdecodezAutoencoderTiny.decode0s    Qijipipqris^et""7+DLLQXDYYFYYv&F.2ooT''*4<? kk,'..6MC((r)rr@rrrrrelunearestr^r )rrrr)rrrrrg?Fg?g)rBN)T)NT) rrrr _supports_gradient_checkpointingr rarstrfloatboolr4rrrHrLrPrRrTrVr~rr rrrr Generatorrrr __classcell__)r@s@rr r )sv3j(,$6F6F$ )*.:.: !!" #!!949494%*#s(O 94 %*#s(O 94  949494$'94"#s(O94"#s(O9494949494 !9494vSu||S SGG%,,G !%%%".u||. .`-u||- -^ 5 54 55I^`efkfrfr`sIsCt 5 5`d,,*25??*C,Y], }eELL11 2,,&!) )) }eELL11 2 )rr ) dataclassesrtypingrrrrconfiguration_utilsrr utilsr utils.accelerate_utilsr modeling_utilsr vaerrrrr rrrrsP ")) B8'88  J   q)j+q)r