biLddlmZmZmZmZddlZddlmZddlmcm Z ddl Zddl m Z mZddlmZddlmZddlmZdd lmZdd lmZdd lmZd d lmZmZej<eZ dZ!GddejDZ#GddejDZ$GddejJZ&GddejDZ'GddejPZ)GddejDZ*GddejDZ+GddejDZ,GddejDZ-Gd d!ejDZ.Gd"d#ejDZ/Gd$d%ejDZ0Gd&d'ejDZ1Gd(d)ejDZ2d*Z3d+Z4Gd,d-ee eZ5y).)ListOptionalTupleUnionN) ConfigMixinregister_to_config)FromOriginalModelMixin)logging)apply_forward_hook)get_activation)AutoencoderKLOutput) ModelMixin) DecoderOutputDiagonalGaussianDistributioncZeZdZ dfd Zdej dej fdZxZS) AvgDown3Dc t|||_||_||_||_|j|j z|j z|_||j z|zdk(sJ||j z|z|_yNr)super__init__ in_channels out_channelsfactor_tfactor_sfactor group_sizeselfrrrr __class__s k/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/diffusers/models/autoencoders/autoencoder_kl_wan.pyrzAvgDown3D.__init__&s| &(    mmdmm3dmmC T[[(<71<<<% 3|Cxreturnc 0|j|jd|jzz |jz}dddd|df}tj||}|j\}}}}}|j ||||jz|j||j z|j ||j z|j }|j ddddddddj}|j |||jz||jz||j z||j z}|j ||j|j||jz||j z||j z}|jd }|S) Nr rrrdim) rshapeFpadviewrpermute contiguousrrrmean) r!r%pad_tr0BCTHWs r#forwardzAvgDown3D.forward7sodmm!;;t}}L!Q5!$ EE!SM 1aA FF   MM   MM   MM  IIaAq!Q1 - 8 8 : FF  O         FF    OO         FFqFMr$r)__name__ __module__ __qualname__rtorchTensorr; __classcell__r"s@r#rr%s)  D"  %,, r$rcfeZdZ ddedeffd Zddej dej fdZxZS) DupUp3Drrc t|||_||_||_||_|j|j z|j z|_||j z|zdk(sJ||j z|z|_yr)rrrrrrrrepeatsr s r#rzDupUp3D.__init__[s| &(    mmdmm3dmmC dkk)K71<<<#dkk1[@ r$r%r&c |j|jd}|j|jd|j|j |j |j |jd|jd|jd}|jddddddd dj}|j|jd|j|jd|j z|jd|j z|jd|j z}|r!|dddd|j dz dddddf}|S) Nrr,rr rr*r(r+r)) repeat_interleaverGr1sizerrrr2r3)r!r% first_chunks r#r;zDupUp3D.forwardms#   !  4 FF FF1I    MM MM MM FF1I FF1I FF1I  IIaAq!Q1 - 8 8 : FF FF1I    FF1I % FF1I % FF1I %   !Q )+Q12Ar$r<)F) r=r>r?intrr@rAr;rBrCs@r#rErEZs=  AAA$U\\r$rEceZdZdZ d dededeeeeeeffdeeeeeeffdeeeeeeffddf fd Zd fd ZxZ S) WanCausalConv3dau A custom 3D causal convolution layer with feature caching support. This layer extends the standard Conv3D layer by ensuring causality in the time dimension and handling feature caching for efficient inference. Args: in_channels (int): Number of channels in the input image out_channels (int): Number of channels produced by the convolution kernel_size (int or tuple): Size of the convolving kernel stride (int or tuple, optional): Stride of the convolution. Default: 1 padding (int or tuple, optional): Zero-padding added to all three sides of the input. Default: 0 rr kernel_sizestridepaddingr&Nct|||||||jd|jd|jd|jdd|jdzdf|_d|_y)N)rrrOrPrQr rrrrr)rrrQ_padding)r!rrrOrPrQr"s r#rzWanCausalConv3d.__init__sx #%#  a$,,q/4<<?DLLYZO]^aeamamnoap]prst   r$c<t|j}|`|jddkDrN|j|j}t j ||gd}|dxx|j dzcc<tj||}t|)|S)Nr*rr r,) listrTtodevicer@catr.r/r0rr;)r!r%cache_xrQr"s r#r;zWanCausalConv3d.forwardst}}%  4==#3a#7jj*G 7A,A.A AJ'--* *J EE!W wq!!r$)rrN) r=r>r?__doc__rLrrrr;rBrCs@r#rNrNs &4545 !!!3c3m 445 ! c5c3//0 ! sE#sC-001 ! !(""r$rNc @eZdZdZd dededededdf fd Zd ZxZS) WanRMS_norma A custom RMS normalization layer. Args: dim (int): The number of dimensions to normalize over. channel_first (bool, optional): Whether the input tensor has channels as the first dimension. Default is True. images (bool, optional): Whether the input represents image data. Default is True. bias (bool, optional): Whether to include a learnable bias term. Default is False. r- channel_firstimagesbiasr&Nc.t||sdnd}|r|g|n|f}||_|dz|_t j t j||_|r.t j t j||_ yd|_ y)N)rrr)rrg?) rrr_scalenn Parameterr@onesgammazerosra)r!r-r_r`rabroadcastable_dimsr.r"s r#rzWanRMS_norm.__init__sy .4Y&.;*)*#*#X \\%**U"34 8{{1(:(:1DtzzQTXT^T^^aeajajjjr$TTF) r=r>r?r\rLboolrr;rBrCs@r#r^r^s> FCFFTFX\FimFkr$r^c"eZdZdZfdZxZS) WanUpsamplea Perform upsampling while ensuring the output tensor has the same data type as the input. Args: x (torch.Tensor): Input tensor to be upsampled. Returns: torch.Tensor: Upsampled tensor with the same data type as the input. c\t||jj|Sr[)rr;floattype_as)r!r%r"s r#r;zWanUpsample.forwards#wqwwy)11!44r$)r=r>r?r\r;rBrCs@r#rqrqs55r$rqc DeZdZdZd dedededdffd Zddgfd ZxZS) WanResampleax A custom resampling module for 2D and 3D data. Args: dim (int): The number of input/output channels. mode (str): The resampling mode. Must be one of: - 'none': No resampling (identity operation). - 'upsample2d': 2D upsampling with nearest-exact interpolation and convolution. - 'upsample3d': 3D upsampling with nearest-exact interpolation, convolution, and causal 3D convolution. - 'downsample2d': 2D downsampling with zero-padding and convolution. - 'downsample3d': 3D downsampling with zero-padding, convolution, and causal 3D convolution. Nr-modeupsample_out_dimr&c t|||_||_||dz}|dk(r>t j t ddt j||dd|_y|d k(rUt j t ddt j||dd|_t||dzd d |_ y|d k(rFt j t jd t j||dd|_y|dk(r[t j t jd t j||dd|_t||d dd|_ yt j|_y)Nr upsample2d)@r{z nearest-exact) scale_factorrwrrrQ upsample3d)rrr)rrr downsample2d)rrrr)r r )rP downsample3d)r rrrS)rPrQ) rrr-rwre SequentialrqConv2dresamplerN time_conv ZeroPad2dIdentity)r!r-rwrxr"s r#rzWanResample.__init__sD    #"ax  < MM/J #/A>DM\ !MM/J #/A>DM-S#'9iXDN ^ #MM",,|*DbiiPSUXZ[djFklDM ^ #MM",,|*DbiiPSUXZ[djFklDM,S#y\efDNKKMDMr$rc |j\}}}}}|jdk(r||d} || d|| <|dxxdz cc<n|ddddt dddddfj} | jddkra|| \|| dk7rTt j || dddddddddfjdj| j| gd} | jddkrR|| M|| dk(rEt j t j| j| j| gd} || dk(r|j|}n|j||| }| || <|dxxdz cc<|j|d||||}t j|dddddddddddf|dddddddddddffd}|j|||dz||}|jd}|jddddd j||z|||}|j|}|j!|||jd|jd|jdjddddd }|jd k(r||d} || "|j|| <|dxxdz cc<|S|ddddddddddfj} |jt j || ddddddddddf|gd}| || <|dxxdz cc<|S) Nr~rReprr rlr,rr*r)rJrwCACHE_Tcloner.r@rY unsqueezerWrX zeros_likerreshapestackr2rr1) r!r% feat_cachefeat_idxbcthwidxrZs r#r;zWanResample.forward s5 1aA 99 $%qkc?*&+JsOQK1$K1whiA 56<<>G}}Q'!+ 30KPZ[^P_chPh"'))'_Q2q!^<FFqILLW^^\^eflm#}}Q'!+ 30KPZ[^P_chPh"'))U-=-=g-F-I-I'..-Y[b,cij"k!#%/ NN1- NN1jo>&-JsOQK1$K !Q1a3A Qq!Q1a'7%8!Aq!Q1r?r\rLstrrr;rBrCs@r#rvrvs7 *C*s*c*T*>%)A3+r$rvc LeZdZdZ d dededededdf fd Zdd gfd ZxZ S) WanResidualBlockaE A custom residual block module. Args: in_dim (int): Number of input channels. out_dim (int): Number of output channels. dropout (float, optional): Dropout rate for the dropout layer. Default is 0.0. non_linearity (str, optional): Type of non-linearity to use. Default is "silu". in_dimout_dimdropout non_linearityr&Nct|||_||_t ||_t |d|_t||dd|_ t |d|_ tj||_ t||dd|_||k7rt||d|_ytj|_y)NFr`rrr})rrrrr nonlinearityr^norm1rNconv1norm2reDropoutrconv2r conv_shortcut)r!rrrrr"s r#rzWanResidualBlock.__init__Bs   *=9!6 $VWaC  7 zz'* $Wgq!D DJgDU_VWa@[][f[f[hr$rc (|j|}|j|}|j|}||d}|ddddt dddddfj }|j ddkrY||Tt j||dddddddddfjdj|j|gd}|j|||}|||<|dxxdz cc<n|j|}|j|}|j|}|j|}||d}|ddddt dddddfj }|j ddkrY||Tt j||dddddddddfjdj|j|gd}|j|||}|||<|dxxdz cc<||zS|j|}||zSNrr rlr,r)rrrrrr.r@rYrrWrXrrrr)r!r%rrrrrZs r#r;zWanResidualBlock.forwardVs    q ! JJqM   a   !1+C1whiA-.446G}}Q!# 3(C))Z_Q2q!^%D%N%Nq%Q%T%TU\UcUc%dfm$ntuv 1jo.A%JsO QK1 K 1 A JJqM   a  LLO  !1+C1whiA-.446G}}Q!# 3(C))Z_Q2q!^%D%N%Nq%Q%T%TU\UcUc%dfm$ntuv 1jo.A%JsO QK1 K 1u  1 A1u r$)rcsilu r=r>r?r\rLrsrrr;rBrCs@r#rr7sY# iii i  i  i(%)A3(r$rc(eZdZdZfdZdZxZS)WanAttentionBlockz} Causal self-attention with a single head. Args: dim (int): The number of channels in the input tensor. ct|||_t||_t j ||dzd|_t j ||d|_y)Nrr) rrr-r^normrerto_qkvproj)r!r-r"s r#rzWanAttentionBlock.__init__sO  $ iiS1Wa0 IIc3* r$c|}|j\}}}}}|jdddddj||z|||}|j|}|j |}|j||zd|dzd}|jddddj }|j dd\} } } tj| | | }|jdjdddj||z|||}|j|}|j|||||}|jddddd}||zS)Nrr rrr*rlr,) rJr2rrrr3chunkr/scaled_dot_product_attentionsqueezerr1) r!r%identity batch_sizechannelstimeheightwidthqkvqkvs r#r;zWanAttentionBlock.forwardsH45FFH1 HdFE IIaAq! $ , ,Z$->&RW X IIaLkk!nkk*t+Q1 bAkk!Q1%002))A2)&1a  * *1a 3 IIaL Aq ) 1 1*t2CXvW\ ] IIaL FF:tXvu = IIaAq! $8|r$)r=r>r?r\rr;rBrCs@r#rrs+r$rc DeZdZdZd dedededeffd Zddgfd ZxZ S) WanMidBlockz Middle block for WanVAE encoder and decoder. Args: dim (int): Number of input/output channels. dropout (float): Dropout rate. non_linearity (str): Type of non-linearity to use. r-rr num_layersc Vt|||_t||||g}g}t |D]9}|j t ||j t||||;tj||_ tj||_ d|_ y)NF) rrr-rrangeappendrre ModuleList attentionsresnetsgradient_checkpointing) r!r-rrrrr_r"s r#rzWanMidBlock.__init__s $Cg}EF z" OA   /4 5 NN+Cg}M N O-- 3}}W- &+#r$Nrc|jd|||}t|j|jddD]\}}|||}||||}|S)Nrr)rzipr)r!r%rrattnresnets r#r;zWanMidBlock.forwardsi DLLOAz8 4 ab1AB 0LD&Gq*h/A  0 r$)rcrrrrCs@r#rrs8 ,C ,% ,c ,`c ,%)A3 r$rc.eZdZdfd ZddgfdZxZS)WanResidualDownBlockc2t |t|||rdnd|rdnd|_g}t |D] }|j t ||||}"tj||_ |r|rdnd} t|| |_ yd|_ y)Nr rrrrrrw) rrr avg_shortcutrrrrerrrv downsampler) r!rrrnum_res_blockstemperal_downsample down_flagrrrwr"s r#rzWanResidualDownBlock.__init__s &  -Q1#Q  ~& A NN+FGWE FF }}W-  %8>nD*7>D #D r$Nrc|j}|jD] }||||}|j|j|||}||j|zSr[)rrrr)r!r%rrx_copyrs r#r;zWanResidualDownBlock.forwardsdll 0Fq*h/A 0    '  J9A4$$V,,,r$)FF)r=r>r?rr;rBrCs@r#rrs$2%)A3-r$rc \eZdZdZdddgddggddd d f d ed ed effd ZddgfdZxZ S) WanEncoder3dao A 3D encoder module. Args: dim (int): The base number of channels in the first layer. z_dim (int): The dimensionality of the latent space. dim_mult (list of int): Multipliers for the number of channels in each block. num_res_blocks (int): Number of residual blocks in each block. attn_scales (list of float): Scales at which to apply attention mechanisms. temperal_downsample (list of bool): Whether to downsample temporally in each block. dropout (float): Dropout rate for the dropout layers. non_linearity (str): Type of non-linearity to use. rr*rr r*r*r rnrcrFrr is_residualc t|||_||_||_||_||_||_t| |_ dg|zD cgc]} || z } } d} t|| ddd|_ tjg|_tt!| dd| ddD]\}\}}| rP|jj#t%|||||t'|dz k7r||nd|t'|dz k7\t)|D]R}|jj#t+|||| |vr$|jj#t-||}T|t'|dz k7s||rd nd }|jj#t/|| | d z} t1|| d |_t5|d|_t||dd|_d|_ycc} w)Nr?rrr}rlF)rrrrrr{rr)rrr-z_dimdim_multr attn_scalesrrrrNconv_inrer down_blocks enumeraterrrlenrrrrvr mid_blockr^norm_outconv_outr)r!rr-rrrrrrrrudimsrdirrrrwr"s r#rzWanEncoder3d.__init__s    ,&#6 *=9#$x0Aa00'{DGQJ ==,$-c$s)T!"X.F$G ! A   ''(&FG3x=[\K\F\,?,Bbg"#s8}q'8"8  ~.%A$$++,D$$++Kd,KLSLE1 !6%Wg}QRS$GE: '1E &+#S1s G5Nrc ||d}|ddddt dddddfj}|jddkrY||Ttj||dddddddddfj dj |j|gd}|j|||}|||<|dxxdz cc<n|j|}|jD]}| ||||}||}|j|||}|j|}|j|}||d}|ddddt dddddfj}|jddkrY||Ttj||dddddddddfj dj |j|gd}|j|||}|||<|dxxdz cc<|S|j|}|Sr)rrr.r@rYrrWrXrrrrrr)r!r%rrrrZlayers r#r;zWanEncoder3d.forwardJs   !1+C1whiA-.446G}}Q!# 3(C))Z_Q2q!^%D%N%Nq%Q%T%TU\UcUc%dfm$ntuv Q 30A%JsO QK1 K QA%% E%!Z2!H   NN1j( 3 MM!    a   !1+C1whiA-.446G}}Q!# 3(C))Z_Q2q!^%D%N%Nq%Q%T%TU\UcUc%dfm$ntuv aC1A%JsO QK1 K a Ar$) r=r>r?r\rLrrorr;rBrCs@r#rrs\   /#!@,@,@,@,D%)A3%r$rcZeZdZdZ ddedededededed effd Zd d gdfd Z xZ S)WanResidualUpBlocka A block that handles upsampling for the WanVAE decoder. Args: in_dim (int): Input dimension out_dim (int): Output dimension num_res_blocks (int): Number of residual blocks dropout (float): Dropout rate temperal_upsample (bool): Whether to upsample on temporal dimension up_flag (bool): Whether to upsample or not non_linearity (str): Type of non-linearity to use Frrrrtemperal_upsampleup_flagrc t |||_||_|rt |||rdndd|_nd|_g}|} t |dzD]!} |jt| ||||} #tj||_ |r!|rdnd} t|| ||_ d|_yd|_ d|_y)Nr rrr~rz)rwrxF)rrrrrErrrrrerrrv upsamplerr) r!rrrrrrrr current_dimr upsample_moder"s r#rzWanResidualUpBlock.__init__s    '/Q !D !%D  ~)* "A NN+K'=Y Z!K "}}W-  ,=L> %%NN1j(;NN1%    (D%%f+%FFAr$)rcFFr) r=r>r?r\rLrsrorrr;rBrCs@r#rrrsp $"'#(,(,(, (,  (,  (,(,(,T%)A3Er$rc ZeZdZdZ d dededededeedef fd Zdd gdfd Z xZ S) WanUpBlocka A block that handles upsampling for the WanVAE decoder. Args: in_dim (int): Input dimension out_dim (int): Output dimension num_res_blocks (int): Number of residual blocks dropout (float): Dropout rate upsample_mode (str, optional): Mode for upsampling ('upsample2d' or 'upsample3d') non_linearity (str): Type of non-linearity to use Nrrrrrrc Jt |||_||_g}|}t |dzD]!} |j t |||||}#tj||_ d|_ |&tjt||g|_ d|_ y)NrrF) rrrrrrrrerr upsamplersrvr) r!rrrrrrrrrr"s r#rzWanUpBlock.__init__s    ~)* "A NN+K'=Y Z!K "}}W-   $ mm[}-U,VWDO&+#r$rc|jD]}| ||||}||}|j0||jd|||}|S|jd|}|S)rr)rr)r!r%rrrKrs r#r;zWanUpBlock.forwardsll F%1j(31I   ?? &%&DOOA&q*h?'DOOA&q)r$)rcNr) r=r>r?r\rLrsrrrr;rBrCs@r#rrsf "'+#,,, ,  ,  } ,,<%)A3Dr$rc ^eZdZdZddgddggdddd d f d ed ed effd Zddgd fdZxZ S) WanDecoder3dak A 3D decoder module. Args: dim (int): The base number of channels in the first layer. z_dim (int): The dimensionality of the latent space. dim_mult (list of int): Multipliers for the number of channels in each block. num_res_blocks (int): Number of residual blocks in each block. attn_scales (list of float): Scales at which to apply attention mechanisms. temperal_upsample (list of bool): Whether to upsample temporally in each block. dropout (float): Dropout rate for the dropout layers. non_linearity (str): Type of non-linearity to use. rr*rr FTTrcrrFrrrc t|||_||_||_||_||_||_t||_ |dg|dddzD cgc]} || z } } t|| ddd|_ t| d||d|_ tjg|_t#t%| dd| ddD]}\} \}}| dkDr| s|dz}| t'|dz k7}d}|r|| rd}n|rd }| rt)|||||r|| nd || }nt+|||||| }|j j-|t/d |_t|| dd|_d |_ycc} w)Nrlrrrr}rr r~rzF)rrrrrrr)rrrrrrr)rrr-rrrrrrrrNrrrrer up_blocksrrrrrrr^rrr)r!r-rrrrrrrrrrrrrrrrup_blockr"s r#rzWanDecoder3d.__init__s    ,&!2*=9#+2,(4R4.!@AAaAA'ud1gq!D %T!Wg}QRSr*$-c$s)T!"X.F$G" , A 1u[13x=1,,G M,Q/ ,  , -!##1#>E&7&:5#"/&!##1#"/"/  NN ! !( +E" ,J$GE: 'q!L &+#eBs E:Nrc ||d}|ddddt dddddfj}|jddkrY||Ttj||dddddddddfj dj |j|gd}|j|||}|||<|dxxdz cc<n|j|}|j|||}|jD]}|||||}|j|}|j|}||d}|ddddt dddddfj}|jddkrY||Ttj||dddddddddfj dj |j|gd}|j|||}|||<|dxxdz cc<|S|j|}|S)Nrr rlr,rr)rrr.r@rYrrWrXrrrrrr)r!r%rrrKrrZrs r#r;zWanDecoder3d.forwardjs  !1+C1whiA-.446G}}Q!# 3(C))Z_Q2q!^%D%N%Nq%Q%T%TU\UcUc%dfm$ntuv Q 30A%JsO QK1 K QA NN1j( 3 KHJkJA K MM!    a   !1+C1whiA-.446G}}Q!# 3(C))Z_Q2q!^%D%N%Nq%Q%T%TU\UcUc%dfm$ntuv aC1A%JsO QK1 K a Ar$) r=r>r?r\rrLrorr;rBrCs@r#rrs`  -#!J,J,J,J,X%)A3E#r$rc |dk(r|S|jdk7rtd|j|j\}}}}}||zdk7s||zdk7rtd|d|d|d|j|||||z|||z|}|j ddd d d d dj }|j|||z|z|||z||z}|S) Nrr(Invalid input shape: rzHeight (z ) and width (z#) must be divisible by patch_size ()r+r*r rr- ValueErrorr.r1r2r3)r% patch_sizerrframesrrs r#patchifyrsQuuw!|0 :;;23''/J&&% a5:#5#:8F8=?bcmbnnopqq z8VVz-A:uXbObdnoA !Q1aA&113A z8j0:=vvQ[G[]bfp]pqA Hr$c L|dk(r|S|jdk7rtd|j|j\}}}}}|||zz}|j|||||||}|j dddddddj }|j|||||z||z}|S) Nrr(r rr*rr+r r )r%r r c_patchesrrrrs r# unpatchifyrsQuuw!|0 :;;34770J 665Z*45H z8ZVVUSA !Q1aA&113A z8VVj-@%*BTUA Hr$c$eZdZdZdZedddgddggdd gd gd dd d dd dfdedeededeedede e de e de de e de e de dededeedeedeeddf"fd Z dd*ej&d,e deeeeffd-Zd>d.ej&d,e fd/Zed>d.ej&d,e deeej&ffd0Zd1ej&d2ej&d3edej&fd4Zd1ej&d2ej&d3edej&fd5Zd*ej&defd6Zd>d.ej&d,e deeej&ffd7Z d?d8ej&d9e d,e d:eejBdeeej&ff d;Z"xZ#S)@AutoencoderKLWana; A VAE model with KL loss for encoding videos into latents and decoding latent representations into videos. Introduced in [Wan 2.1]. This model inherits from [`ModelMixin`]. Check the superclass documentation for it's generic methods implemented for all models (such as downloading or saving). F`Nrr rrc)gy):gM Og^)gQ?gtVƿgZӼ?gBfjÿgU0*?gL F%u?gMg&†?gz6>׿g F%uȿg[ AcgMJ?gW2ıҿ)g_L@gNё\C?gQ@g?@g9#J{?g|a2U?gHPs@g0*@gJ{/L& @gJY8@g]C@g(?gK46?gS:?go_Ι@g-?rr*base_dimdecoder_base_dimrrrrrr latents_mean latents_stdrrrr scale_factor_temporalscale_factor_spatialr&c t|||_||_|ddd|_||}t | ||dz||||||  |_t|dz|dzd|_t||d|_ t||||||j|| |  |_ dt|jz|_ d|_d|_d|_d|_d|_d|_|j*t)d |jj+Dnd |j *t)d |j j+Dnd d |_y) Nrlr ) rr-rrrrrrrr) r-rrrrrrrrFc3<K|]}t|tywr[ isinstancerN.0ms r# z,AutoencoderKLWan.__init__..4Za:a9Zrc3<K|]}t|tywr[r"r$s r#r'z,AutoencoderKLWan.__init__..7r(r))decoderencoder)rrrrrrr,rN quant_convpost_quant_convrr+rspatial_compression_ratio use_slicing use_tilingtile_sample_min_heighttile_sample_min_widthtile_sample_stride_heighttile_sample_stride_widthsummodules_cached_conv_counts)r!rrrrrrrrrrrrrr rrr"s r#rzAutoencoderKLWan.__init__sql  #6 !4TrT!:  #' ##!))# 3#   *%!)UQYB.ueQ?# )#"44%#   *+c$2J2J.K)K&!  '*#%("*-&(+% ||'Z4<X>X")B)ddFdFd&(@(aDDaDa%r$cd|_y)z Disable tiled VAE decoding. If `enable_tiling` was previously enabled, this method will go back to computing decoding in one step. FN)r1r!s r#disable_tilingzAutoencoderKLWan.disable_tilingZs  r$cd|_y)z Enable sliced VAE decoding. When this option is enabled, the VAE will split the input tensor in slices to compute decoding in several steps. This is useful to save some memory and allow larger batch sizes. TNr0r<s r#enable_slicingzAutoencoderKLWan.enable_slicingas  r$cd|_y)z Disable sliced VAE decoding. If `enable_slicing` was previously enabled, this method will go back to computing decoding in one step. FNr?r<s r#disable_slicingz AutoencoderKLWan.disable_slicinghs !r$c|jd|_dg|_dg|jz|_|jd|_dg|_dg|jz|_y)Nr+rr,)r8 _conv_num _conv_idx _feat_map _enc_conv_num _enc_conv_idx _enc_feat_mapr<s r# clear_cachezAutoencoderKLWan.clear_cacheosa11)<$..0!55i@S"Vd&8&88r$r%c |j\}}}}}|jr/||jkDs||jkDr|j |S|j |j j!t||j j}d|dz dzz}t|D]}dg|_ |dk(r;|j|ddddddddddf|j|j}K|j|dddddd|dz zzdd|zzddddf|j|j} tj| gd}|j} |j | S)Nr rr*rrrr )r.r1r3r2 tiled_encoderJconfigr rrrHr,rIr@rYr-) r!r%r num_framerriter_routout_encs r#_encodezAutoencoderKLWan._encodeysm)*&1i ??(B(B BftOjOjFj$$Q' '  ;; ! ! -t{{'='=>AY]q((u 0A"#D Avll1Q2A2q!^#4ASAS^b^p^plq||aAQU Oa!a%i7A=>#11!//$ iid Q/ 0ooc"  r$ return_dictc(|jrU|jddkDrC|jdDcgc]}|j|}}t j |}n|j|}t |}|s|fSt|Scc}w)a Encode a batch of images into latents. Args: x (`torch.Tensor`): Input batch of images. return_dict (`bool`, *optional*, defaults to `True`): Whether to return a [`~models.autoencoder_kl.AutoencoderKLOutput`] instead of a plain tuple. Returns: The latent representations of the encoded videos. If `return_dict` is True, a [`~models.autoencoder_kl.AutoencoderKLOutput`] is returned, otherwise a plain `tuple` is returned. rr) latent_dist)r0r.splitrUr@rYrr)r!r%rVx_sliceencoded_slicesr posteriors r#encodezAutoencoderKLWan.encodes    QCD771:Ndll73NNN .)A QA03 < "y99OsBzc `|j\}}}}}|j|jz}|j|jz}|jr||kDs||kDr|j ||S|j |j|} t|D]} dg|_ | dk(r?|j| dddd| | dzddddf|j|jd} O|j| dddd| | dzddddf|j|j} tj | gd} |jj!t! |jj} tj" d d } |j |s| fSt%| S) NrVrrT)rrrKrMr rLgr)minmaxsample)r.r2r/r3r1 tiled_decoderJr.rrEr+rFr@rYrOr rclampr) r!r^rVrrPrrtile_latent_min_heighttile_latent_min_widthr%rrRrSs r#_decodezAutoencoderKLWan._decodes)*&1i!%!\>\ \ ??(= =J`A`$$QK$@ @    #y! 0ASDNAvllaAAIq!+,RVR`R`nr#||AaAAIq!&;$<bfbpbp|qiid Q/ 0 ;; ! ! -ST[[-C-CDCkk#4S1 6MC((r$c:|jr_|jddkDrM|jdDcgc]}|j|j}}t j |}n|j|j}|s|fSt|Scc}w)a Decode a batch of images. Args: z (`torch.Tensor`): Input batch of latent vectors. return_dict (`bool`, *optional*, defaults to `True`): Whether to return a [`~models.vae.DecoderOutput`] instead of a plain tuple. Returns: [`~models.vae.DecoderOutput`] or `tuple`: If return_dict is True, a [`~models.vae.DecoderOutput`] is returned, otherwise a plain `tuple` is returned. rrrc)r0r.rYrirdr@rYr)r!r^rVz_slicedecoded_slicesdecodeds r#decodezAutoencoderKLWan.decodes    QJK''RS*Uwdll73::UNUii/Gll1o,,G: G,,Vs"Bar blend_extentc t|jd|jd|}t|D]J}|dddddd| |zddfd||z z z|dddddd|ddf||z zz|dddddd|ddf<L|S)Nrrar.r)r!rorrpys r#blend_vzAutoencoderKLWan.blend_vs1772; \B |$ A Aq<-!*;Q!>?1q!>?1q >> !%!\>\ \$($B$BdFdFd$d!#'#@#@DDbDb#b -0II +.FF q&$"@"@A AC1eT%B%BC 3  ":>a"77 {+&A*+D&Av Arr1q43N3N/N+NPQTUX\XrXrTrPr!rs QU Oa!a%i7D$?$? ??D$>$> >> @  <<9K9KVZVhVh<iD??40DKK%& 599Tq12' 3( KK - .  o >FAsJ$S> h4q5<<QU AlKDq5<<AE D+FD!!$q!Q0J1J0JLeMeLe'e"fg h   uyy< = >ii +Aq!^m^]l],RS r$c |j\}}}}}||jz}||jz}|j|jz} |j|jz} |j|jz} |j |jz} |j|jz } |j|j z }g}t d|| D]}g}t d|| D]}|jg}t |D]q}dg|_|dddd||dz||| z||| zf}|j|}|j||j|j}|j|s|jtj|d|j||jg}t|D]\}}g}t|D]{\}}|dkDr|j!||dz ||| }|dkDr|j#||dz ||}|j|ddddddd|jd|j f}|jtj|dtj|dddddddd|d|f}|s|fSt%| S) a Decode a batch of images using a tiled decoder. Args: z (`torch.Tensor`): Input batch of latent vectors. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~models.vae.DecoderOutput`] instead of a plain tuple. Returns: [`~models.vae.DecoderOutput`] or `tuple`: If return_dict is True, a [`~models.vae.DecoderOutput`] is returned, otherwise a plain `tuple` is returned. rNrrMr r,rlrrc)r.r/r2r3r4r5rrJrEr.r+rFrr@rYrrurwr)r!r^rVrryrr sample_height sample_widthrgrhr|r}r~rrrrrrrrrmrrdecs r#rezAutoencoderKLWan.tiled_decode8s+,'''1j&%!?!?? t=== !%!\>\ \$($B$BdFdFd$d!#'#@#@DDbDb#b 22T5S5SS 0043P3PP q&";< AC1e%=> 3  "z*)A&'SDNQ1q1u9a!6L2L.LaRSVkRkNkklD//5D"ll4DNNUYUcUcldGKK( )  599Tq12 3 KK    o >FAsJ$S> t4q5<<QU AlKDq5<<AE D+FD!!$q!Q0P$2P2P0PRqTXTqTqRq'q"rs t   uyy< = >ii +Aq!^m^]l],RS6MC((r$rdsample_posterior generatorc|}|j|j}|r|j|}n|j}|j ||}|S)z Args: sample (`torch.Tensor`): Input sample. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`DecoderOutput`] instead of a plain tuple. )rr`)r]rXrdrwrn) r!rdrrVrr%r\r^rs r#r;zAutoencoderKLWan.forwardwsU KKN..   9 5A Akk!k5 r$)NNNN)r&N)T)FTN)$r=r>r?r\ _supports_gradient_checkpointingr rLrrrrsrorr:r=r@rBrJr@rArUr rrrr]rirrnrurwrNre Generatorr;rBrCs@r#rrs`(-$*.+#%*=% $$ $"$(/0./gs s #3-s  s * s  s %[s "$Zs s 5ks 8%[9s \]s ^_s `as bSMcs d (}es f'smgs h is s n15/35948 b ( b (}b$,E? b #+5/ b  b<  !9437::,0: "E*F$GG H::4))D)>- -4-5X]XdXdIdCe--0%,,cell%,,cell@ell@/B@D=)ell=)=)}^c^j^jOjIk=)D"' /3    EOO,  }ell* + r$r)6typingrrrrr@torch.nnretorch.nn.functional functionalr/torch.utils.checkpointconfiguration_utilsrr loadersr utilsr utils.accelerate_utilsr activationsrmodeling_outputsrmodeling_utilsrvaerr get_loggerr=loggerrModulerrEConv3drNr^Upsamplerqrvrrrrrrrrrrrr$r#rsd0/ B-8(2'<   H % 2 2j)bii)X*"bii*"Zk"))k4 5"++ 5X"))XvGryyGT* *Z$"))$N!-299!-Hv299vrUUpBBJ~299~B . (Pz;0FPr$