bi^ddlmZmZmZddlZddlmZddlmZm Z ddl m Z ddl m Z ddlmZdd lmZdd lmZdd lmZdd lmZd dlmZmZGddej6ZGddej6ZGddej6ZGddej6ZGddej6Z Gddej6Z!Gddej6Z"Gddej6Z#Gdd ej6Z$Gd!d"ej6Z%Gd#d$eee Z&y)%)OptionalTupleUnionN) ConfigMixinregister_to_config)FromOriginalModelMixin)apply_forward_hook)get_activation))PixArtAlphaCombinedTimestepSizeEmbeddings)AutoencoderKLOutput) ModelMixin)RMSNorm) DecoderOutputDiagonalGaussianDistributionceZdZ d dededeeeeeeffdeeeeeeffdeeeeeeffdededeffd Zd e jd e jfd Z xZ S)LTXVideoCausalConv3d in_channels out_channels kernel_sizestridedilationgroups padding_mode is_causalc t |||_||_||_t |t r|n|||f|_t |t r|n|ddf}t |t r|n|||f}|jddz} |jddz} d| | f} tj|||j|||| ||_ y)Nrr r)rrrpaddingr) super__init__rrr isinstancetuplernnConv3dconv) selfrrrrrrrr height_pad width_padr __class__s k/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/diffusers/models/autoencoders/autoencoder_kl_ltx.pyr!zLTXVideoCausalConv3d.__init__!s &("*4[%*H;{\gitNu)(E:81a@P%fe4666:R%%a(A- $$Q'1, j),II     %   hidden_statesreturnc|jd}|jrE|ddddddddddfjdd|dz ddf}tj||gd}nv|ddddddddddfjdd|dz dzddf}|ddddddddddfjdd|dz dzddf}tj|||gd}|j |}|S)Nrrr dim)rrrepeattorch concatenater&)r'r-time_kernel_sizepad_left pad_rights r+forwardzLTXVideoCausalConv3d.forwardDs++A. >>$Q2A2q!^4;;QCSVWCWYZ\]<^_H!--x.GQOM$Q2A2q!^4;;QDTWXDX]^C^`acdghI!--x .RXYZM -0 r,)rrrrzerosT) __name__ __module__ __qualname__intrrstrboolr!r4Tensorr9 __classcell__r*s@r+rr s 9:3456#! ! ! 3c3m 445 ! c5c3//0 ! U3S=112 ! ! ! ! F U\\ ell r,rceZdZdZ ddedeededededed ed ed ed dffd Z dde jdee jdee jd e jfdZ xZS)LTXVideoResnetBlock3da A 3D ResNet block used in the LTXVideo model. Args: in_channels (`int`): Number of input channels. out_channels (`int`, *optional*): Number of output channels. If None, defaults to `in_channels`. dropout (`float`, defaults to `0.0`): Dropout rate. eps (`float`, defaults to `1e-6`): Epsilon value for normalization layers. elementwise_affine (`bool`, defaults to `False`): Whether to enable elementwise affinity in the normalization layers. non_linearity (`str`, defaults to `"swish"`): Activation function to use. conv_shortcut (bool, defaults to `False`): Whether or not to use a convolution shortcut. Nrrdropoutepselementwise_affine non_linearityr inject_noisetimestep_conditioningr.c t ||xs|}t||_t |d||_t ||d||_t |d||_tj||_ t ||d||_ d|_ d|_||k7r3tj||dd|_ t ||dd||_d|_d|_|r^tj$t'j(|dd|_tj$t'j(|dd|_d|_| r5tj$t'j,d ||d zz |_yy) N:0yE>rGrHr)rrrrT)rGrHbiasrrrrrr?)r r!r nonlinearityrnorm1rconv1norm2r$DropoutrFconv2norm3 conv_shortcut LayerNormper_channel_scale1per_channel_scale2 Parameterr4r:scale_shift_tablerandn) r'rrrFrGrHrIrrJrKr*s r+r!zLTXVideoResnetBlock3d.__init__hs[ #2{ *=9[dGYZ )#,AYb  \tHZ[ zz'* )$>!W-0F0FGb0ccD151C .GWgw)Q[9GCM))-8  =1  " " .)//4M!KK=;O;OWdWjWjM*]T=T=T-TVZ\]_cehVh,iiM =#8#8B#?@HHQO  ! ! -)Q[9GCM))-8  ]3  =1  " " .)//4M!KK=;O;OWdWjWjM*]T=T=T-TVZ\]_cehVh,iiM :: !ZZq" 56>>r1EF    )''/F%. r,)Nư>FswishTFFNN)r;r<r=__doc__r>rfloatr@r?r!r4rA Generatorr9rBrCs@r+rErESs.'+#($"&+-b-bsm-b -b  -b ! -b-b-b-b $-b -b`qu-ll-*25<<*@-T\]b]l]lTm- -r,rEc eZdZ d dededeeeeeeffdededdf fd Zd e jde jfd Z xZ S) LTXVideoDownsampler3drrrrrr.Nc"t|t|tr|n|||f|_||dz|dz|dz|z|_||jd|jdz|jdzz}t ||dd|||_yNrrr r)rrrrrr)r r!r"r#r group_sizerr&)r'rrrrrr*s r+r!zLTXVideoDownsampler3d.__init__s  *65 9fPV?W &2VAY>J|[# AQ(G$++VW.(XY (#%%   r,r-c tj|ddddd|jddz f|gd}|jdd|jdfjdd|jdfjdd|jdf}|j ddddd ddd j dd}|jdd|j f}|jd}|j|}|jdd|jdfjdd|jdfjdd|jdf}|j ddddd ddd j dd}||z}|S) Nrrr r0rQr2r) r4catrrjpermuteflattenr~meanr&)r'r-residuals r+r9zLTXVideoDownsampler3d.forwards =A7KQ!9K7K1K#Lm"\bcd   # #ADKKN'; < Yq2t{{1~. / Yq2t{{1~. /  ##Aq!Q1a;CCAqI%%a"doo)>?==Q=' -0  # #ADKKN'; < Yq2t{{1~. / Yq2t{{1~. /  &--aAq!Q1EMMaQRS %0 r,)rTr: r;r<r=r>rrr@r?r!r4rAr9rBrCs@r+r{r{s} 45#    c5c3//0        0U\\ellr,r{ceZdZ d dedeeeeeeffdededededdffd Zd e jde jfd Z xZ S) LTXVideoUpsampler3drrrrupscale_factorrr.Nct|t|tr|n|||f|_||_||_||dz|dz|dz|z}t||dd|||_yr}) r r!r"r#rrrrr&) r'rrrrrrrr*s r+r!zLTXVideoUpsampler3d.__init__s|  *65 9fPV?W   ,#fQi/&);fQiGNZ (#%%   r,r-c |j\}}}}}|jr|j|d|jd|jd|jd|||}|j dddddddd j ddj d dj dd}|jd|jdz|jdz|j z}|jd|ddd}|dddd|jddz df}|j|}|j|d|jd|jd|jd|||}|j dddddddd j ddj d dj dd}|dddd|jddz df}|jr|z}|S) Nr2rrr rrrrrQ) rlrreshaperrrrr3r&) r'r- batch_size num_channels num_framesheightwidthrrepeatss r+r9zLTXVideoUpsampler3d.forwards>K>Q>Q; L*fe ==$,,B A A APZ\bdiH ''1aAq!Q?GG1MUUVWYZ[ccdeghiH{{1~ A6QGDL_L__Gq'1a;H1dkk!nq&8&: :;H -0 %-- DKKNDKKNDKKNJX^`e &--aAq!Q1EMMaQRS[[\]_`aiijkmno %aDKKNQ,>,@&@A ==)H4Mr,)rTFrr:rrCs@r+rrs45#  c5c3//0          4U\\ellr,rceZdZdZdZ ddedeededededed e d e ffd Z dd e jd ee jdee jde jfdZxZS)LTXVideoDownBlock3D Down block used in the LTXVideo model. Args: in_channels (`int`): Number of input channels. out_channels (`int`, *optional*): Number of output channels. If None, defaults to `in_channels`. num_layers (`int`, defaults to `1`): Number of resnet layers. dropout (`float`, defaults to `0.0`): Dropout rate. resnet_eps (`float`, defaults to `1e-6`): Epsilon value for normalization layers. resnet_act_fn (`str`, defaults to `"swish"`): Activation function to use. spatio_temporal_scale (`bool`, defaults to `True`): Whether or not to use a downsampling layer. If not used, output dimension would be same as input dimension. Whether or not to downsample across temporal dimension. is_causal (`bool`, defaults to `True`): Whether this layer behaves causally (future frames depend only on past frames) or not. Trr num_layersrF resnet_eps resnet_act_fnspatio_temporal_scalerc |t ||xs|}g} t|D]"} | jt ||||||$t j | |_d|_|r)t j t||dd|g|_d|_ ||k7rt |||||||_ d|_ y)NrrrFrGrIrrr r r rPF) r r!rangeappendrEr$ ModuleListresnets downsamplersrconv_outgradient_checkpointing) r'rrrrFrrrrr_r*s r+r!zLTXVideoDownBlock3D.__init__Fs #2{ z" A NN% +!,#""/'   }}W-   " ($/%0$%("+  !D  , &1')+# DM',#r,r-rbrcr.cFt|jD]D\}}tjr!|jr|j ||||};||||}F|j |j D] }||} |j|j|||}|Sz-Forward method of the `LTXDownBlock3D` class.) enumeraterr4is_grad_enabledr_gradient_checkpointing_funcrrr'r-rbrciresnet downsamplers r+r9zLTXVideoDownBlock3D.forward~s#4<<0 GIAv$$&4+F+F $ A A&-Y]_h i &}dI F  G    (#00 ; +M :  ; == $ MM-yIMr,)NrrsrtruTTrvr;r<r=rw _supports_gradient_checkpointingr>rrxr?r@r!r4rAryr9rBrCs@r+rr,s.(,$ '+ $&*6,6,sm6, 6,  6,  6,6, $6,6,v(,/3 ||u||$EOO,   r,rceZdZdZdZ ddedeededededed e d e d effd Z dd e jdee jdee jde jfdZxZS)LTXVideo095DownBlock3DrTrrrrFrrrrdownsample_typec t ||xs|}g} t|D]"} | jt ||||||$t j | |_d|_|rt j |_| dk(r1|jjt||dd|d |_ y| dk(r0|jjt||d|d |_ y| d k(r0|jjt||d |d |_ y| d k(r(|jjt||d|d |_ y) Nrr&rrrPspatial)rr r )rrrrtemporal)r rrspatiotemporalF) r r!rrrEr$rrrrr{r) r'rrrrFrrrrrrrr*s r+r!zLTXVideo095DownBlock3D.__init__ss #2{ z" A NN% +!,#""/'   }}W-   " D &(!!((($/%0$%("+ 8',#'!I-!!(()$/lS\hq$',#!J.!!(()$/lS\hq',#!$44!!(()$/lS\hq ',#r,r-rbrcr.ct|jD]D\}}tjr!|jr|j ||||};||||}F|j |j D] }||} |Sr)rrr4rrrrrs r+r9zLTXVideo095DownBlock3D.forwards#4<<0 GIAv$$&4+F+F $ A A&-Y]_h i &}dI F  G    (#00 ; +M :  ;r,)NrrsrtruTTr&rvrrCs@r+rrs.(,$ '+ $&*%?,?,sm?, ?,  ?,  ?,?, $?,?,?,H(,/3 ||u||$EOO,   r,rceZdZdZdZ ddedededededed ed ed d ffd Z dde jde e jde e jd e jfdZxZS)LTXVideoMidBlock3das A middle block used in the LTXVideo model. Args: in_channels (`int`): Number of input channels. num_layers (`int`, defaults to `1`): Number of resnet layers. dropout (`float`, defaults to `0.0`): Dropout rate. resnet_eps (`float`, defaults to `1e-6`): Epsilon value for normalization layers. resnet_act_fn (`str`, defaults to `"swish"`): Activation function to use. is_causal (`bool`, defaults to `True`): Whether this layer behaves causally (future frames depend only on past frames) or not. TrrrFrrrrJrKr.Nc t |d|_|rt|dzd|_g} t |D]$} | j t ||||||||&tj| |_ d|_ y)NrQrrrrFrGrIrrJrKF) r r! time_embedderr rrrEr$rrr) r'rrrFrrrrJrKrrr*s r+r!zLTXVideoMidBlock3d.__init__s ! !J;YZ?\]!^D z" A NN% +!,#""/'!-*?   }}W- &+#r,r-rbrcc|ja|j|jdd|jd|j}|j |jddddd}t |j D]D\}}tjr!|jr|j||||};||||}F|S)z,Forward method of the `LTXMidBlock3D` class.Nrtimestep resolution aspect_ratior hidden_dtyper2r) rrsizerhviewrrr4rrr)r'r-rbrcrrs r+r9zLTXVideoMidBlock3d.forward?s    )%%!(--a0*00 &D99]//2B1a@D"4<<0 GIAv$$&4+F+F $ A A&-Y]_h i &}dI F  G r,)rrsrtruTFFrv)r;r<r=rwrr>rxr?r@r!r4rArryr9rBrCs@r+rrs$(,$  $"&+!,!,!, !,  !,  !,!,!, $!, !,L(,/3 ||u||$EOO,   r,rceZdZdZdZ ddedeededededed e d e d e d e d e deffd Z dde jdee jdee jde jfdZxZS)LTXVideoUpBlock3da Up block used in the LTXVideo model. Args: in_channels (`int`): Number of input channels. out_channels (`int`, *optional*): Number of output channels. If None, defaults to `in_channels`. num_layers (`int`, defaults to `1`): Number of resnet layers. dropout (`float`, defaults to `0.0`): Dropout rate. resnet_eps (`float`, defaults to `1e-6`): Epsilon value for normalization layers. resnet_act_fn (`str`, defaults to `"swish"`): Activation function to use. spatio_temporal_scale (`bool`, defaults to `True`): Whether or not to use a downsampling layer. If not used, output dimension would be same as input dimension. Whether or not to downsample across temporal dimension. is_causal (`bool`, defaults to `True`): Whether this layer behaves causally (future frames depend only on past frames) or not. TrrrrFrrrrrJrKupsample_residualrc t||xs|}d|_| rt|dzd|_d|_||k7rt ||||||| | |_d|_|r,tjt|| zd|| | g|_g} t|D]$}| jt ||||||| | &tj| |_ d|_ y)NrQrrr)rrrrF)r r!rr conv_inrE upsamplersr$rrrrrr)r'rrrrFrrrrrJrKrrrrr*s r+r!zLTXVideoUpBlock3d.__init__ts  #2{ ! !J;YZ?\]!^D  , &0')+#)&; DL  mm'$~5("+!2'5  DOz" A NN% ,!-#""/'!-*?   }}W- &+#r,r-rbrcr.c |j|j|||}|ja|j|jdd|jd|j}|j |jddddd}|j |j D] }||} t|jD]D\}}tjr!|jr|j||||};||||}F|S)Nrrr2r) rrrrrhrrrrr4rrr)r'r-rbrc upsamplerrrs r+r9zLTXVideoUpBlock3d.forwards << # LLiHM    )%%!(--a0*00 &D99]//2B1a@D ?? &!__ 9 )- 8  9#4<<0 GIAv$$&4+F+F $ A A&-Y]_h i &}dI F  G r,) NrrsrtruTTFFFrrvrrCs@r+rrZs .(,$ '+ $&*"&+"'B,B,smB, B,  B,  B,B, $B,B,B, $B, B,B,N(,/3 ||u||$EOO,   r,rceZdZdZ ddededeedfdeedfdeedfdeedfd eedfd ed ed ed effd Z de jde jfdZ xZ S)LTXVideoEncoder3da The `LTXVideoEncoder3d` layer of a variational autoencoder that encodes input video samples to its latent representation. Args: in_channels (`int`, defaults to 3): Number of input channels. out_channels (`int`, defaults to 128): Number of latent channels. block_out_channels (`Tuple[int, ...]`, defaults to `(128, 256, 512, 512)`): The number of output channels for each block. spatio_temporal_scaling (`Tuple[bool, ...], defaults to `(True, True, True, False)`: Whether a block should contain spatio-temporal downscaling layers or not. layers_per_block (`Tuple[int, ...]`, defaults to `(4, 3, 3, 3, 4)`): The number of layers per block. patch_size (`int`, defaults to `4`): The size of spatial patches. patch_size_t (`int`, defaults to `1`): The size of temporal patches. resnet_norm_eps (`float`, defaults to `1e-6`): Epsilon value for ResNet normalization layers. is_causal (`bool`, defaults to `True`): Whether this layer behaves causally (future frames depend only on past frames) or not. rrblock_out_channels.down_block_typesspatio_temporal_scalinglayers_per_blockr patch_size patch_size_tresnet_norm_epsrc t|||_| |_||dzz|_|d} t |j| dd| |_|ddk(} t|| rdndz }tjg|_ t|D]}| }| s|dz|kr||dzn||} n||dz} ||dk(rt|| ||| ||| }n5||dk(rt|| ||| ||| || }ntd |||jj|t!| |d| | |_t%|d d|_tj(|_t | |dzdd| |_d|_y)Nr rrrrPr2rr)rrrrrr)rrrrrrrzUnknown down block type: )rrrrrMFrN)r r!rrrrrlenr$r down_blocksrrr ValueErrorrr mid_blockrnorm_outSiLUconv_actrr)r'rrrrrrrrrrroutput_channel is_ltx_095num_block_out_channelsr input_channel down_blockr*s r+r!zLTXVideoEncoder3d.__init__s& $(&Q6+A.+(('   &b)-EE !$%7!8AQR!S==,-. 0A*M>?!eF\>\!3AE!:btuvbw!3AE!:"&;;0 -!//2.*A!*D'  "!$(@@3 -!//2.*A!*D'$3A$6 !#<=Ma=PK>Q>Q; L*fe *c 1"aK A:%--  &;SBSUVXhjk &--aAq!Q1EMMaQRS  ]3  "t'B'B".. ] $ A A*m \  ]!==dnnm\M".. : *= 9  :!NN=9M m&;&;Ar&BCKKBPQR  m4  m4 $QV, #**1m.@.@.Ca.GAqQ  =,"?QG r,) rrrrrrrTTTFrQrrrrQr&r&r&r&rQrrtT)r;r<r=rwr>rr?r@rxr!r4rAr9rBrCs@r+rrs6.B- 5N,;+K!%#U,U,U,"#s(O U,  S/ U,"'tSy!1U, S/U,sCxU,U,U, !U,"#U,n%U\\%ell%r,rc eZdZdZ ddededeedfdeedfdeedfded ed ed ed eedfd edeedfdeedfddffd Zdde jde e jde jfdZ xZ S)LTXVideoDecoder3da The `LTXVideoDecoder3d` layer of a variational autoencoder that decodes its latent representation into an output sample. Args: in_channels (`int`, defaults to 128): Number of latent channels. out_channels (`int`, defaults to 3): Number of output channels. block_out_channels (`Tuple[int, ...]`, defaults to `(128, 256, 512, 512)`): The number of output channels for each block. spatio_temporal_scaling (`Tuple[bool, ...], defaults to `(True, True, True, False)`: Whether a block should contain spatio-temporal upscaling layers or not. layers_per_block (`Tuple[int, ...]`, defaults to `(4, 3, 3, 3, 4)`): The number of layers per block. patch_size (`int`, defaults to `4`): The size of spatial patches. patch_size_t (`int`, defaults to `1`): The size of temporal patches. resnet_norm_eps (`float`, defaults to `1e-6`): Epsilon value for ResNet normalization layers. is_causal (`bool`, defaults to `False`): Whether this layer behaves causally (future frames depend only on past frames) or not. timestep_conditioning (`bool`, defaults to `False`): Whether to condition the model on timesteps. rrr.rrrrrrrJrKrupsample_factorr.Nct|||_||_||dzz|_t t |}t t |}t t |}t t | } t t | } t t | } |d}t||dd| |_t||d|| | d| |_ t|}tjg|_t|D]Z}|| |z}||| |z}t!||||dz|||| | |dz| | || | }|jj#|\t%|dd |_tj(|_t||jdd| |_d|_d|_d|_| rtj4t7j8d t6j: |_t=|dzd|_tj4t7j>d||d zz |_d |_ y)Nr rrrrP)rrrrrJrK) rrrrrrrJrKrrrMFrNg@@)rhrR)!r r!rrrr#reversedrrrrrr$r up_blocksrrrrrrrrrr_timestep_scale_multiplierr^r4tensorfloat32r r`r)r'rrrrrrrrrrJrKrrrrrrup_blockr*s r+r!zLTXVideoDecoder3d.__init__sV $((:q=8"8,>#?@"'1H(I"J *:!;<Xl34 !(+<"=> 9:+A.+#.aXYen  ,&'*&%a"7  "%%7!8r*-. ,A*oa.@@M/2oa6HHN()++AE2*&=a&@#)!a%0&;"3A"6.q1 H NN ! !( +# ,( $5Q   ,&T5F5FTU^_kt  "!%)-& -/\\%,,vUZUbUb:c-dD *!J>\]K]_`!aD %'\\%++a2PSacfSf2f%gD "&+#r,r-rbc |j|}|j||jz}tjrN|jrB|j |j ||}|jD]}|j |||}n,|j ||}|jD] }|||} |j|jddjdd}|j|j|jdd|jd|j}|j|jdddddjdd}||j dz}|j#d\}}|d|zz|z}|j%|}|j'|}|j(}|j*}|j,\}} } } } |j/|d|||| | | }|j1dddd d d d d jd d jd djd d }|S)Nrr2rr)r r2rer0rr rrQrr)rrr4rrrrrrrirrrrhrrjr_rkrrrrrlrr) r'r-rbrshiftscalerrrrrrrs r+r9zLTXVideoDecoder3d.forwardsI ]3  ) ) 5$888D  "t'B'B ==dnnm]abM NN a $ A A(M[_ `  a!NN=$?M NN > ( =  > m&;&;Ar&BCKKBPQR    )%%!(--a0*00 &D99]//2B1a@JJ1gVD$001LMMD;;1;-LE5)QY7%?M m4  m4 OO>K>Q>Q; L*fe%--j"c1aU[]bc %--aAq!Q1EMMaQRS[[\]_`aiijkmno r,) rrrrrrQrrtFFFFFFrrrrrN)r;r<r=rwr>rr@rxr!r4rArr9rBrCs@r+rrqs2:.B4M,;!%)E&+.J,8Q,Q,Q,"#s(O Q, "'tSy!1 Q,  S/ Q,Q,Q,Q,Q,D#I&Q, $Q,!s+Q,tSy)Q, Q,f*U\\*%,,9O*[`[g[g*r,rc2eZdZdZdZe d>dedededeedfd eedfd eedfd eedfd eedfd ee dfdee dfdee dfdeedfdee dfdeedfde dedede de de de dededdf0fd Z d?de ede ed e ed!e e d"e e d#e e ddfd$Z d@d%Zd@d&Zd@d'Zd(ej$dej$fd)Ze dAd(ej$d*e deeeeffd+Z dBd,ej$d-e ej$d*e deeej$ffd.Ze dBd,ej$d-e ej$d*e deeej$ffd/Zd0ej$d1ej$d2edej$fd3Zd0ej$d1ej$d2edej$fd4Zd0ej$d1ej$d2edej$fd5Zd(ej$dej$fd6Z dAd,ej$d-e ej$d*e deeej$ffd7Z d(ej$defd8Z! dAd,ej$d-e ej$d*e deeej$ffd9Z" dCd:ej$d-e ej$d;e d*e dX>X"*D*gHgHg')B)ddFdFd&(@(aDDaDa%-J-pdNpNp*r,cd|_y)z Disable tiled VAE decoding. If `enable_tiling` was previously enabled, this method will go back to computing decoding in one step. FN)rr's r+disable_tilingz$AutoencoderKLLTXVideo.disable_tilings  r,cd|_y)z Enable sliced VAE decoding. When this option is enabled, the VAE will split the input tensor in slices to compute decoding in several steps. This is useful to save some memory and allow larger batch sizes. TNrr+s r+enable_slicingz$AutoencoderKLLTXVideo.enable_slicings  r,cd|_y)z Disable sliced VAE decoding. If `enable_slicing` was previously enabled, this method will go back to computing decoding in one step. FNr.r+s r+disable_slicingz%AutoencoderKLLTXVideo.disable_slicings !r,xc|j\}}}}}|jr ||jkDr|j|S|jr/||j kDs||j kDr|j|S|j|}|Sr) rlrr$_temporal_tiled_encoderr#r" tiled_encoder)r'r2rrrrrencs r+_encodezAutoencoderKLLTXVideo._encodes>?gg; L*fe  & &:8W8W+W..q1 1 ??(B(B BftOjOjFj$$Q' 'll1o r, return_dictc(|jrU|jddkDrC|jdDcgc]}|j|}}t j |}n|j|}t |}|s|fSt|Scc}w)a Encode a batch of images into latents. Args: x (`torch.Tensor`): Input batch of images. return_dict (`bool`, *optional*, defaults to `True`): Whether to return a [`~models.autoencoder_kl.AutoencoderKLOutput`] instead of a plain tuple. Returns: The latent representations of the encoded videos. If `return_dict` is True, a [`~models.autoencoder_kl.AutoencoderKLOutput`] is returned, otherwise a plain `tuple` is returned. rr) latent_dist)rrlsplitr7r4rrr)r'r2r8x_sliceencoded_slicesh posteriors r+encodezAutoencoderKLLTXVideo.encodes    QCD771:Ndll73NNN .)A QA03 < "y99OsBzrbc|j\}}}}}|j|jz} |j|jz} |j|j z} |j r|| kDr|j|||S|jr|| kDs|| kDr|j|||S|j||} |s| fSt| S)Nr8sample) rlr"rr#r$rr_temporal_tiled_decoder tiled_decoderr) r'rArbr8rrrrrtile_latent_min_heighttile_latent_min_widthtile_latent_min_num_framesdecs r+_decodezAutoencoderKLLTXVideo._decodes?@gg; L*fe!%!\>\ \%)%D%DHgHg%g"  & &:8R+R..q$K.P P ??(= =J`A`$$Q+$F Fll1d#6MC((r,c|jr|jddkDr|O|jd|jdfDcgc]!\}}|j||j#}}}n7|jdDcgc]}|j|j}}t j |}n|j||j}|s|fSt|Scc}}wcc}w)a Decode a batch of images. Args: z (`torch.Tensor`): Input batch of latent vectors. return_dict (`bool`, *optional*, defaults to `True`): Whether to return a [`~models.vae.DecoderOutput`] instead of a plain tuple. Returns: [`~models.vae.DecoderOutput`] or `tuple`: If return_dict is True, a [`~models.vae.DecoderOutput`] is returned, otherwise a plain `tuple` is returned. rrrD)rrlr;rLrEr4rr)r'rArbr8z_slicet_slicedecoded_slicesdecodeds r+decodezAutoencoderKLLTXVideo.decodes"    QSTSZSZ[\S]_c_i_ijk_lRm">NgwDLL'299""OPggVWj!Y7$,,w"7">">!Y!Yii/Gll1d+22G: G,,""Zs &C*"C0ab blend_extentc t|jd|jd|}t|D]J}|dddddd| |zddfd||z z z|dddddd|ddf||z zz|dddddd|ddf<L|S)Nrrminrlr)r'rSrTrUys r+blend_vzAutoencoderKLLTXVideo.blend_v9s1771:qwwqz<@ |$ A Aq<-!*;Q!>?1q!>?1q?1q >> !%!\>\ \$($B$BdFdFd$d!#'#@#@DDbDb#b -0II +.FF q&$"@"@A AC1eT%B%BC !||aAq1t'B'B#BBADLfLfHfDffg 4  ! KK   o =FAsJ$S> h4q5<<QU AlKDq5<<AE D+FD!!$q!Q0J1J0JLeMeLe'e"fg h   uyy; < =ii +Aq!^m^]l],RS r,c v|j\}}}}}||jz} ||jz} |j|jz} |j|jz} |j|jz} |j |jz}|j|jz }|j|j z }g}t d|| D]b}g}t d||D]=}|j|dddddd||| z||| zf|}|j|?|j|dg}t|D]\}}g}t|D]{\}}|dkDr|j||dz |||}|dkDr|j||dz ||}|j|ddddddd|jd|j f}|jtj|dtj|dddddddd| d| f}|s|fSt|S)a Decode a batch of images using a tiled decoder. Args: z (`torch.Tensor`): Input batch of latent vectors. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~models.vae.DecoderOutput`] instead of a plain tuple. Returns: [`~models.vae.DecoderOutput`] or `tuple`: If return_dict is True, a [`~models.vae.DecoderOutput`] is returned, otherwise a plain `tuple` is returned. rNrrQr0rrD)rlrr"r#r%r&rrrrrZr]r4rr)r'rArbr8rrrrr sample_height sample_widthrHrIrdrerfrgrhrrirjrkrlrmrnrKs r+rGz"AutoencoderKLLTXVideo.tiled_decodesf"?@gg; L*fe!?!?? t=== !%!\>\ \$($B$BdFdFd$d!#'#@#@DDbDb#b 22T5S5SS 0043P3PP q&";< AC1e%=> !||AaAq17M3M/MqSTWlSlOl&l$most 4  ! KK    o =FAsJ$S> t4q5<<QU AlKDq5<<AE D+FD!!$q!Q0P$2P2P0PRqTXTqTqRq'q"rs t   uyy; < =ii +Aq!^m^]l],RS6MC((r,c t|j\}}}}}|dz |jzdz}|j|jz}|j|jz} || z } g} t d||jD]} |dddd| | |jzdzddddf} |j r0||j kDs||jkDr|j| } n|j| } | dkDr| ddddddddddf} | j| g}t| D]m\} } | dkDr=|j| | dz | | } |j| ddddd| ddddfH|j| ddddd| dzddddfotj|dddddd|f}|S)Nrrr r0)rlrr$r'rrr"r#r5rrrr`r4r)r'r2rrrrrlatent_num_framesrJtile_latent_stride_num_framesblend_num_framesrirrnrmr6s r+r4z,AutoencoderKLLTXVideo._temporal_tiled_encodes>?gg; L*fe'!^0O0OORSS%)%D%DHgHg%g"(,(J(JdNmNm(m%58UUq*d&H&HI AQ1q4#B#BBQFF1LMDFT-H-H$HETXTnTnLn((.||D)1uAq!"aN+ JJt    ~ YGAt1u||CAJ6FG!!$q!-K.K-KQPQ'Q"RS!!$q!-P/Lq/P-PRSUV'V"WX  Yii *1a1C2C1C+CD r,c J|j\}}}}}|dz |jzdz} |j|jz} |j|jz} |j |jz} |j |jz} |j |j z }g}td|| D]}|dddd||| zdzddddf}|jrC|jd| kDs|jd| kDr|j||dj}n|j||}|dkDr|ddddddddddf}|j|g}t|D]\}}|dkDrI|j||dz ||}|ddddd|j ddddf}|j|T|j|ddddd|j dzddddftj |dddddd| f}|s|fSt#| S) Nrrr2rfTrCr r0rD)rlrr"rr#r$r'rrrGrErrrr`r4rr)r'rArbr8rrrrrnum_sample_framesrHrIrJrtrurirrnrQrmrKs r+rFz,AutoencoderKLLTXVideo._temporal_tiled_decodesG?@gg; L*fe'!^t/N/NNQRR!%!\>\ \%)%D%DHgHg%g"(,(J(JdNmNm(m%::T=_=__q*&CD AQ1q#==AA1aGHDDJJrN5J$JdjjY[n_uNu++D$D+IPP,,tT21u!!QQ/2 JJw    ~ ^GAt1u||CAJ6FGAq"FD$F$F"F1LM!!$'!!$q!-Ut/Q/QTU/U-UWXZ['["\]  ^ii *1a1C2C1C+CD6MC((r,rEsample_posteriorrcc|}|j|j}|r|j|}n|j}|j ||} |s | jfS| S)N)rc)r@r:rEmoderR) r'rErbrxr8rcr2r?rArKs r+r9zAutoencoderKLLTXVideo.forwardsb KKN..   9 5A Akk!T"JJ=  r,)rrrrrrrrrr)FFFFFrrrFrQrrtg?TFNN)NNNNNN)r.N)T)NT)NFTN)&r;r<r=rwrrr>rr?r@rxr!rr)r,r/r1r4rAr7r rrrr@rrLrRrZr]r`r5rGr4rFryr9rBrCs@r+rr s%N(,$".B- 7K,;4C4MRV$)$)%-ell%;$)JN$) }ell* +$)R(,!& /3  u||$    EOO,  u||U\\) *r,r)'typingrrrr4torch.nnr$configuration_utilsrrloadersr utils.accelerate_utilsr activationsr embeddingsr modeling_outputsrmodeling_utilsr normalizationrvaerrModulerrEr{rrrrrrrrr,r+rs *) B-8(B2'#<02990fqBIIqh.BII.b1"))1hg"))gTmRYYmbPPf{ {|V VrY YxHJ 5KHr,