biXddlZddlmZmZmZmZddlZddlZddl m Z m Z ddl m Z mZddlmZmZmZerddlZ d dZGd d ee Zy) N)ListOptionalTupleUnion) ConfigMixinregister_to_config) deprecateis_scipy_available)KarrasDiffusionSchedulersSchedulerMixinSchedulerOutputc $|dk(rd}n|dk(rd}ntd|g}t|D]<}||z }|dz|z }|jtd||||z z |>t j |tj S)a Create a beta schedule that discretizes the given alpha_t_bar function, which defines the cumulative product of (1-beta) over time from t = [0,1]. Contains a function alpha_bar that takes an argument t and transforms it to the cumulative product of (1-beta) up to that part of the diffusion process. Args: num_diffusion_timesteps (`int`): the number of betas to produce. max_beta (`float`): the maximum beta to use; use values lower than 1 to prevent singularities. alpha_transform_type (`str`, *optional*, default to `cosine`): the type of noise schedule for alpha_bar. Choose from `cosine` or `exp` Returns: betas (`np.ndarray`): the betas used by the scheduler to step the model outputs cosinecftj|dzdz tjzdz dzS)NgMb?gT㥛 ?r)mathcospits i/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/diffusers/schedulers/scheduling_deis_multistep.py alpha_bar_fnz)betas_for_alpha_bar..alpha_bar_fn;s-88QY%/$''9A=>!C Cexpc2tj|dzS)Ng()rrrs rrz)betas_for_alpha_bar..alpha_bar_fn@s88AI& &rz"Unsupported alpha_transform_type: r dtype) ValueErrorrangeappendmintorchtensorfloat32)num_diffusion_timestepsmax_betaalpha_transform_typerbetasit1t2s rbetas_for_alpha_barr-"s.x' D  & '=>R=STUU E * +M ( (!e. . S\"- R0@@@(KLM <<U]] 33rc.eZdZdZeDcgc]}|j c}}ZdZe d@de de de de de e jd e d e d ed e d e de de dede ede ede ede ede e de de dede f,dZedZedZdAde fdZ dBde d ee ej.fd!e e fd"Zd#ej2d$ej2fd%Zd&Zd'Zd(ej2d$ej2fd)Zd(ej2de d$ej2fd*Z dCd(ej2de d+e d,e d$ej2f d-Zdd.d/ej2d#ej2d$ej2fd0Z dd.d/ej2d#ej2d$ej2fd1Z!dd.d2e"ej2d#ej2d$ej2fd3Z#dd.d2e"ej2d#ej2d$ej2fd4Z$dDd5Z%d6Z& dEd/ej2d7ee ej2fd#ej2d8ed$ee'e(ff d9Z)d#ej2d$ej2fd:Z*d;ej2dZ,d?Z-ycc}}w)FDEISMultistepScheduleru `DEISMultistepScheduler` is a fast high order solver for diffusion ordinary differential equations (ODEs). This model inherits from [`SchedulerMixin`] and [`ConfigMixin`]. Check the superclass documentation for the generic methods the library implements for all schedulers such as loading and saving. Args: num_train_timesteps (`int`, defaults to 1000): The number of diffusion steps to train the model. beta_start (`float`, defaults to 0.0001): The starting `beta` value of inference. beta_end (`float`, defaults to 0.02): The final `beta` value. beta_schedule (`str`, defaults to `"linear"`): The beta schedule, a mapping from a beta range to a sequence of betas for stepping the model. Choose from `linear`, `scaled_linear`, or `squaredcos_cap_v2`. trained_betas (`np.ndarray`, *optional*): Pass an array of betas directly to the constructor to bypass `beta_start` and `beta_end`. solver_order (`int`, defaults to 2): The DEIS order which can be `1` or `2` or `3`. It is recommended to use `solver_order=2` for guided sampling, and `solver_order=3` for unconditional sampling. prediction_type (`str`, defaults to `epsilon`): Prediction type of the scheduler function; can be `epsilon` (predicts the noise of the diffusion process), `sample` (directly predicts the noisy sample`) or `v_prediction` (see section 2.4 of [Imagen Video](https://imagen.research.google/video/paper.pdf) paper). thresholding (`bool`, defaults to `False`): Whether to use the "dynamic thresholding" method. This is unsuitable for latent-space diffusion models such as Stable Diffusion. dynamic_thresholding_ratio (`float`, defaults to 0.995): The ratio for the dynamic thresholding method. Valid only when `thresholding=True`. sample_max_value (`float`, defaults to 1.0): The threshold value for dynamic thresholding. Valid only when `thresholding=True`. algorithm_type (`str`, defaults to `deis`): The algorithm type for the solver. lower_order_final (`bool`, defaults to `True`): Whether to use lower-order solvers in the final steps. Only valid for < 15 inference steps. use_karras_sigmas (`bool`, *optional*, defaults to `False`): Whether to use Karras sigmas for step sizes in the noise schedule during the sampling process. If `True`, the sigmas are determined according to a sequence of noise levels {σi}. use_exponential_sigmas (`bool`, *optional*, defaults to `False`): Whether to use exponential sigmas for step sizes in the noise schedule during the sampling process. use_beta_sigmas (`bool`, *optional*, defaults to `False`): Whether to use beta sigmas for step sizes in the noise schedule during the sampling process. Refer to [Beta Sampling is All You Need](https://huggingface.co/papers/2407.12173) for more information. timestep_spacing (`str`, defaults to `"linspace"`): The way the timesteps should be scaled. Refer to Table 2 of the [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://huggingface.co/papers/2305.08891) for more information. steps_offset (`int`, defaults to 0): An offset added to the inference steps, as required by some model families. r Nnum_train_timesteps beta_startbeta_end beta_schedule trained_betas solver_orderprediction_type thresholdingdynamic_thresholding_ratiosample_max_valuealgorithm_type solver_typelower_order_finaluse_karras_sigmasuse_exponential_sigmasuse_beta_sigmasuse_flow_sigmas flow_shifttimestep_spacing steps_offsetuse_dynamic_shiftingtime_shift_typec|jjrts tdt |jj|jj |jj gdkDr td|+tj|tj|_ n|dk(r-tj|||tj|_ nk|dk(r6tj|dz|dz|tjdz|_ n0|d k(rt||_ nt|d |jd |jz |_tj"|j d |_tj&|j$|_tj&d|j$z |_tj,|j(tj,|j*z |_d|j$z |j$z dz|_d |_| dvr1| dvr|j5dnt| d |j| dvr2| dvr|j5dntd| d |jd|_t9jd |dz |t8jdddj;}tj<||_dg|z|_ d |_!d|_"d|_#|j0jId|_y)Nz:Make sure to install scipy if you want to use beta sigmas.r znOnly one of `config.use_beta_sigmas`, `config.use_exponential_sigmas`, `config.use_karras_sigmas` can be used.rlinear scaled_linear?rsquaredcos_cap_v2z is not implemented for ?rdim)deis) dpmsolverz dpmsolver++rN)r:)logrho)midpointheunbh1bh2rP)r;z solver type cpu)%configr?r ImportErrorsumr>r=rr#r$r%r)linspacer-NotImplementedError __class__alphascumprodalphas_cumprodsqrtalpha_tsigma_tloglambda_tsigmasinit_noise_sigmar num_inference_stepsnpcopy from_numpy timesteps model_outputslower_order_nums _step_index _begin_indexto)selfr0r1r2r3r4r5r6r7r8r9r:r;r<r=r>r?r@rArBrCrDrErks r__init__zDEISMultistepScheduler.__init__s4 ;; & &/A/CZ[ [  ++T[[-O-OQUQ\Q\QnQno pst tA   $m5==IDJ h & H>QY^YfYfgDJ o - C3H[chcpcpquvvDJ 1 1,-@ADJ%7OPTP^P^O_&`a aDJJ& #mmDKKQ?zz$"5"56 zz!d&9&9"9:  $,,/%))DLL2II D///43F3FF3N !$  )!==''v'>)^,<_convert_to_exponentialr?_convert_to_betar@interplenr#rjrerprkrgr5rlrmrnro) rqrgr{r|rk step_ratiore log_sigmassigmar] sigma_lasts r set_timestepsz$DEISMultistepScheduler.set_timestepss >;;33 8S8SWd8d dd%'VVBZDKK " ;; ' ': 5 At{{>>BDWZ[D[\2"$!  [[ ) )Y 688=PST=TUJ1&9A&=>KRRTUYWYUYZ[^\^_ddfmmnpnvnvwI 11 1I [[ ) )Z 788;NNJ $++"A"A1zkRXXZ__ahhikiqiqrI NI;;//01JK A 3 33t7J7JJsRSVVF^ ;; ( (WWV_))+F,,vSf,gFSY!Z%$"2"25*"E!Z[aacI^^VVBC[$9:AA"**MF [[ / /WWV_))+F11FXk1lFSY!Z%$"2"25*"E!Z[I^^VVBC[$9:AA"**MF [[ ( (WWV_))+F**VQd*eFSY!Z%$"2"25*"E!Z[I^^VVBC[$9:AA"**MF [[ ( ([[A (G(G$GI\_`I`aF6\FWWT[[33f<T[[E[E[^_E_ciDi@ijklomopuuwF$++"A"AAGGII^^VVBC[$9:AA"**MFYYy"))As6{*CVLFt221559L9LQ9OOTWWJ^^Vj\$:;BB2::NF&&v. )))477vU[[7Y#&y>   KK $ $%!"  kknnU+ I"[ "[ "[s8\>\"6\'samplereturncb|j}|j^}}}|tjtjfvr|j }|j ||tj|z}|j}tj||jjd}tj|d|jj}|jd}tj|| ||z }|j ||g|}|j!|}|S)a{ "Dynamic thresholding: At each sampling step we set s to a certain percentile absolute pixel value in xt0 (the prediction of x_0 at timestep t), and if s > 1, then we threshold xt0 to the range [-s, s] and then divide by s. Dynamic thresholding pushes saturated pixels (those near -1 and 1) inwards, thereby actively preventing pixels from saturation at each step. We find that dynamic thresholding results in significantly better photorealism as well as better image-text alignment, especially when using very large guidance weights." https://huggingface.co/papers/2205.11487 r rL)r"max)rshaper#r%float64floatreshaperhprodabsquantilerWr8clampr9 unsqueezerp)rqrr batch_sizechannelsremaining_dims abs_sampless r_threshold_samplez(DEISMultistepScheduler._threshold_sampleAs 06 - H~  6 6\\^F Hrww~7N,NOZZ\ NN:t{{'M'MST U KK 1$++66  KKNVaR+a/ HF~F5! rctjtj|d}||ddtjfz }tj|dk\dj dj |jddz }|dz}||}||}||z ||z z } tj | dd} d| z |z| |zz} | j|j} | S)Ng|=r)axisr)rr ) rhrcmaximumnewaxiscumsumargmaxcliprr) rqrr log_sigmadistslow_idxhigh_idxlowhighwrs rrz"DEISMultistepScheduler._sigma_to_tcsFF2::eU34 Jq"**}55))UaZq188a8@EE*JZJZ[\J]`aJaEbQ;!(#9_t , GGAq! Ug H , IIekk "rcr|jjr d|z }|}||fSd|dzdzdzz }||z}||fS)Nr rrI)rWr@)rqrrarbs r_sigma_to_alpha_sigma_tz.DEISMultistepScheduler._sigma_to_alpha_sigma_t{sW ;; & &%iGG E1HqLS01GgoGrrct|jdr|jj}nd}t|jdr|jj}nd}||n|dj }||n|dj }d}t j dd|}|d|z z}|d|z z}||||z zz|z} | S)z6Constructs the noise schedule of Karras et al. (2022). sigma_minN sigma_maxrUrg@r )hasattrrWrritemrhrZ) rqrrgrrrhoramp min_inv_rho max_inv_rhores rrz)DEISMultistepScheduler._convert_to_karrass 4;; , --II 4;; , --II!*!6IIbM4y1}a !RSS   Z   DOO,77> ;; & &) 3, 66'AG [[ ( (H 4"G [[ ( (N :&<)??G [[ ( (,= =kk$//2Gw55G+DKK,G,G+HIWW  ;; # #,,W5G ;; % % /Ww..'9 9%&OP Prct|dkDr|dn|jdd}t|dkDr|dn|jdd}|t|dkDr|d}n td| tdd d | tdd d |j|j dz|j|j }}|j |\} }|j |\} }tj| tj|z } tj| tj|z } | | z } |jjd k(r)| | z |z|tj| d z z|zz }|Std)au One step for the first-order DEIS (equivalent to DDIM). Args: model_output (`torch.Tensor`): The direct output from the learned diffusion model. timestep (`int`): The current discrete timestep in the diffusion chain. prev_timestep (`int`): The previous discrete timestep in the diffusion chain. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. Returns: `torch.Tensor`: The sample tensor at the previous timestep. rrNr prev_timesteprrrkrrPassing `prev_timestep` is deprecated and has no effect as model output conversion is now handled via an internal counter `self.step_index`rNrKr) rrrr rerurr#rcrWr:rr[)rqrrrrrrrbsigma_sraalpha_srdlambda_shx_ts rdeis_first_order_updatez.DEISMultistepScheduler.deis_first_order_updates0"$i!m47J1M#&t9q=QfjjRV6W >4y1}a !RSS   Z   $ ^   ;;t':;T[[=Y77@77@99W% '(::99W% '(:: x  ;; % % /W$.'UYYq\C=O2PT`1``C &&OP Prmodel_output_listc.t|dkDr|dn|jdd}t|dkDr|dn|jdd}|t|dkDr|d}n td| tddd | tddd |j|j dz|j|j |j|j dz } }}|j |\} }|j |\} }|j | \} } |d |d }} || z || z | | z }}}|jjd k(rCd}||||||||z }||||||||z }| || z || zz||zzz}|Std)a One step for the second-order multistep DEIS. Args: model_output_list (`List[torch.Tensor]`): The direct outputs from learned diffusion model at current and latter timesteps. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. Returns: `torch.Tensor`: The sample tensor at the previous timestep. r timestep_listNr rrrrPassing `timestep_list` is deprecated and has no effect as model output conversion is now handled via an internal counter `self.step_index`rrUrNc|tj| tj|zdz ztj|tj|z z S)Nr rhrc)rbcs rind_fnzIDEISMultistepScheduler.multistep_deis_second_order_update..ind_fnsCRVVAYJ2Q67266!9rvvay;PQQrr rrrr rerurrWr:r[)rqrrrrrrrbsigma_s0sigma_s1raalpha_s0alpha_s1m0m1rho_trho_s0rho_s1rcoef1coef2rs r"multistep_deis_second_order_updatez9DEISMultistepScheduler.multistep_deis_second_order_updateKs($'t9q=QfjjRV6W #&t9q=QfjjRV6W >4y1}a !RSS  $ ^   $ ^  KK!+ , KK ( KK!+ ,$  77@!99(C(!99(C("2&(9"(=B '' 18h3FS[H[vv ;; % % / R5&&1F6664RRE5&&1F6664RREVh.;ebjHICJ%&OP Prct|dkDr|dn|jdd}t|dkDr|dn|jdd}|t|dkDr|d}n td| tddd | tddd |j|j dz|j|j |j|j dz |j|j dz f\}}} } |j |\} }|j |\} }|j | \} } |j | \}} |d |d |d }}}|| z || z | | z | |z f\}}}}|jjdk(rdd}||||||||||z }||||||||||z }||||||||||z }| || z ||zz||zz||zzz}|Std)a One step for the third-order multistep DEIS. Args: model_output_list (`List[torch.Tensor]`): The direct outputs from learned diffusion model at current and latter timesteps. sample (`torch.Tensor`): A current instance of a sample created by diffusion process. Returns: `torch.Tensor`: The sample tensor at the previous timestep. rrNr rrrrrrrUrrNc6|tj|tj|tj|z dzztj|tj|zz tj|ztj|dzzdtj|zz dzz}tj|tj|z tj|tj|z z}||z S)Nr rr)rrrd numerator denominators rrzHDEISMultistepScheduler.multistep_deis_third_order_update..ind_fnsFF1IRVVAY!6!:;ffQi"&&)+,ffQi ffQi1n%"&&)m $    "vvay266!94RVVAY9NO  ;..rrr)rqrrrrrrrbrrsigma_s2rarralpha_s2rrm2rrrrho_s2rrrcoef3rs r!multistep_deis_third_order_updatez8DEISMultistepScheduler.multistep_deis_third_order_updatesf*$'t9q=QfjjRV6W #&t9q=QfjjRV6W >4y1}a !RSS  $ ^   $ ^  KK!+ , KK ( KK!+ , KK!+ , 1 -8X 77@!99(C(!99(C(!99(C(&r*,=b,ACTUWCXB g  x  x  x  ) %vvv ;; % % / /5&&&9F66SY[aA$($6$6q1u$=D  q ! >!-2 ;; # #q (D,A,AA,EIZ66|F6SK [[ % % *d.C.Ca.GK]AA$BTBT]cAdK@@ASAS\b@cK  4;;#;#; ;  ! !Q & ! A> !;77rc|S)a? Ensures interchangeability with schedulers that need to scale the denoising model input depending on the current timestep. Args: sample (`torch.Tensor`): The input sample. Returns: `torch.Tensor`: A scaled input sample. )rqrrrs rscale_model_inputz(DEISMultistepScheduler.scale_model_inputJs  roriginal_samplesnoiserkc*|jj|j|j}|jjdk(rvt j |ra|jj|jt j}|j|jt j}n@|jj|j}|j|j}|j |Dcgc]}|j||}}nG|j|jg|jdz}n|jg|jdz}||j}t|jt|jkr=|jd}t|jt|jkr=|j!|\} } | |z| |zz} | Scc}w)NrmpsrrrU)rerpr{rtyper#is_floating_pointrkr%rxrrurflattenrrr) rqrrrkrerr step_indicesrrarb noisy_sampless r add_noisez DEISMultistepScheduler.add_noiseZs'7'>'>FVF\F\]  " " ' '5 0U5L5LY5W!%!2!23C3J3JRWR_R_!2!` ! %5%<%"22Wu_D _sHc.|jjSN)rWr0rts r__len__zDEISMultistepScheduler.__len__|s{{...r)ig-C6?g{Gz?rGNrrFgףp= ?rKrNrPTFFFFrKrZrFr~)r)NN)333333?r#r!)T).__name__ __module__ __qualname____doc__r name _compatiblesorderr intrstrrrhndarrayboolrrpropertyrurxrzrr#r{rr rrrrrrrrrrrrr rrrr IntTensorrr").0es00rr/r/NsR1f%>>qAFF>L E$("%.2(",1"%$#"&,116*/*/&) *%*,/L, L,L, L,  L,   + L,L,L,L,%*L, L,L,L, L,$D>L, !)!L,""$#L,$"$%L,&UO'L,()L,*+L,,#-L,./L,L,\  !!(3(hlQ,#&Q,05c5<<6G0HQ,U]^cUdQ,h D0 ELLRWR^R^4TW\a\h\h.dgr9sK$ // A1XX !)4Xo /^[o /r