biLddlZddlmZmZmZmZmZddlZddl Z ddl m Z m Z ddl mZmZddlmZddlmZmZmZerddlZ d d ZGd d ee Zy) N)CallableListOptionalTupleUnion) ConfigMixinregister_to_config) deprecateis_scipy_available) randn_tensor)KarrasDiffusionSchedulersSchedulerMixinSchedulerOutputc $|dk(rd}n|dk(rd}ntd|g}t|D]<}||z }|dz|z }|jtd||||z z |>t j |tj S)a Create a beta schedule that discretizes the given alpha_t_bar function, which defines the cumulative product of (1-beta) over time from t = [0,1]. Contains a function alpha_bar that takes an argument t and transforms it to the cumulative product of (1-beta) up to that part of the diffusion process. Args: num_diffusion_timesteps (`int`): the number of betas to produce. max_beta (`float`): the maximum beta to use; use values lower than 1 to prevent singularities. alpha_transform_type (`str`, *optional*, default to `cosine`): the type of noise schedule for alpha_bar. Choose from `cosine` or `exp` Returns: betas (`np.ndarray`): the betas used by the scheduler to step the model outputs cosinecftj|dzdz tjzdz dzS)NgMb?gT㥛 ?r)mathcospits c/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/diffusers/schedulers/scheduling_sasolver.py alpha_bar_fnz)betas_for_alpha_bar..alpha_bar_fn<s-88QY%/$''9A=>!C Cexpc2tj|dzS)Ng()rrrs rrz)betas_for_alpha_bar..alpha_bar_fnAs88AI& &rz"Unsupported alpha_transform_type: rdtype) ValueErrorrangeappendmintorchtensorfloat32)num_diffusion_timestepsmax_betaalpha_transform_typerbetasit1t2s rbetas_for_alpha_barr/#s.x' D  & '=>R=STUU E * +M ( (!e. . S\"- R0@@@(KLM <<U]] 33rc0LeZdZdZeDcgc]}|j c}}ZdZedddddddd dd d d d dd d d d d e d dddfde de de de de e ejee fde de de de edede de de ded e ed!e ed"e ed#e ed$e e d%e d&e e d'e d(e f.d)Zed*Zed+ZdVd,e fd-ZdWd.e d/e e ej2ffd0Zd1ej6d2ej6fd3Zd4Zd5Zd6ej6d2ej6fd7Zd6ej6d.e d2ej6fd8Z dXd6ej6d.e d9e d:e d2ej6f d;Z!dd<d=ej6d1ej6d2ej6fd>Z"d?Z#d@Z$dAZ%dBZ&d=ej6d1ej6dCej6dDe dEej6d2ej6f dFZ'dGej6dHej6dIej6dJej6dDe dEej6d2ej6fdKZ(dYdLZ)dMZ* dZd=ej6dNe d1ej6dOed2e e+e,ff dPZ-d1ej6d2ej6fdQZ.dRej6dCej6dSej^d2ej6fdTZ0dUZ1ycc}}w)[SASolverScheduleru `SASolverScheduler` is a fast dedicated high-order solver for diffusion SDEs. This model inherits from [`SchedulerMixin`] and [`ConfigMixin`]. Check the superclass documentation for the generic methods the library implements for all schedulers such as loading and saving. Args: num_train_timesteps (`int`, defaults to 1000): The number of diffusion steps to train the model. beta_start (`float`, defaults to 0.0001): The starting `beta` value of inference. beta_end (`float`, defaults to 0.02): The final `beta` value. beta_schedule (`str`, defaults to `"linear"`): The beta schedule, a mapping from a beta range to a sequence of betas for stepping the model. Choose from `linear`, `scaled_linear`, or `squaredcos_cap_v2`. trained_betas (`np.ndarray`, *optional*): Pass an array of betas directly to the constructor to bypass `beta_start` and `beta_end`. predictor_order (`int`, defaults to 2): The predictor order which can be `1` or `2` or `3` or '4'. It is recommended to use `predictor_order=2` for guided sampling, and `predictor_order=3` for unconditional sampling. corrector_order (`int`, defaults to 2): The corrector order which can be `1` or `2` or `3` or '4'. It is recommended to use `corrector_order=2` for guided sampling, and `corrector_order=3` for unconditional sampling. prediction_type (`str`, defaults to `epsilon`, *optional*): Prediction type of the scheduler function; can be `epsilon` (predicts the noise of the diffusion process), `sample` (directly predicts the noisy sample`) or `v_prediction` (see section 2.4 of [Imagen Video](https://imagen.research.google/video/paper.pdf) paper). tau_func (`Callable`, *optional*): Stochasticity during the sampling. Default in init is `lambda t: 1 if t >= 200 and t <= 800 else 0`. SA-Solver will sample from vanilla diffusion ODE if tau_func is set to `lambda t: 0`. SA-Solver will sample from vanilla diffusion SDE if tau_func is set to `lambda t: 1`. For more details, please check https://huggingface.co/papers/2309.05019 thresholding (`bool`, defaults to `False`): Whether to use the "dynamic thresholding" method. This is unsuitable for latent-space diffusion models such as Stable Diffusion. dynamic_thresholding_ratio (`float`, defaults to 0.995): The ratio for the dynamic thresholding method. Valid only when `thresholding=True`. sample_max_value (`float`, defaults to 1.0): The threshold value for dynamic thresholding. Valid only when `thresholding=True` and `algorithm_type="dpmsolver++"`. algorithm_type (`str`, defaults to `data_prediction`): Algorithm type for the solver; can be `data_prediction` or `noise_prediction`. It is recommended to use `data_prediction` with `solver_order=2` for guided sampling like in Stable Diffusion. lower_order_final (`bool`, defaults to `True`): Whether to use lower-order solvers in the final steps. Default = True. use_karras_sigmas (`bool`, *optional*, defaults to `False`): Whether to use Karras sigmas for step sizes in the noise schedule during the sampling process. If `True`, the sigmas are determined according to a sequence of noise levels {σi}. use_exponential_sigmas (`bool`, *optional*, defaults to `False`): Whether to use exponential sigmas for step sizes in the noise schedule during the sampling process. use_beta_sigmas (`bool`, *optional*, defaults to `False`): Whether to use beta sigmas for step sizes in the noise schedule during the sampling process. Refer to [Beta Sampling is All You Need](https://huggingface.co/papers/2407.12173) for more information. lambda_min_clipped (`float`, defaults to `-inf`): Clipping threshold for the minimum value of `lambda(t)` for numerical stability. This is critical for the cosine (`squaredcos_cap_v2`) noise schedule. variance_type (`str`, *optional*): Set to "learned" or "learned_range" for diffusion models that predict variance. If set, the model's output contains the predicted Gaussian variance. timestep_spacing (`str`, defaults to `"linspace"`): The way the timesteps should be scaled. Refer to Table 2 of the [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://huggingface.co/papers/2305.08891) for more information. steps_offset (`int`, defaults to 0): An offset added to the inference steps, as required by some model families. rig-C6?g{Gz?linearNrepsilonFgףp= ??data_predictionTinflinspacernum_train_timesteps beta_startbeta_end beta_schedule trained_betaspredictor_ordercorrector_orderprediction_typetau_func thresholdingdynamic_thresholding_ratiosample_max_valuealgorithm_typelower_order_finaluse_karras_sigmasuse_exponential_sigmasuse_beta_sigmasuse_flow_sigmas flow_shiftlambda_min_clipped variance_typetimestep_spacing steps_offsetc|jjrts tdt |jj|jj |jj gdkDr td|+tj|tj|_ n|dk(r-tj|||tj|_ nk|dk(r6tj|dz|dz|tjdz|_ n0|d k(rt||_ nt|d |jd |jz |_tj"|j d |_tj&|j$|_tj&d|j$z |_tj,|j(tj,|j*z |_d|j$z |j$z dz|_d |_| dvrt| d |jd|_t7jd |dz |t6jdddj9}tj:||_dgt?||dz z|_ dgt?||dz z|_!| d|_"n| |_"| dk(|_#d |_$d|_%d|_&d|_'|j0jQd|_y)Nz:Make sure to install scipy if you want to use beta sigmas.rznOnly one of `config.use_beta_sigmas`, `config.use_exponential_sigmas`, `config.use_karras_sigmas` can be used.rr2 scaled_linear?rsquaredcos_cap_v2z is not implemented for r4rdim)r5noise_predictionc|dk\r|dkrdSdS)Ni rrrs rz,SASolverScheduler.__init__..s18Saarr5cpu))configrHr ImportErrorsumrGrFr!r%r&r'r+r7r/NotImplementedError __class__alphascumprodalphas_cumprodsqrtalpha_tsigma_tloglambda_tsigmasinit_noise_sigmanum_inference_stepsnpcopy from_numpy timestepsmax timestep_list model_outputsr@ predict_x0lower_order_nums last_sample _step_index _begin_indexto)selfr8r9r:r;r<r=r>r?r@rArBrCrDrErFrGrHrIrJrKrLrMrNros r__init__zSASolverScheduler.__init__s6 ;; & &/A/CZ[ [  ++T[[-O-OQUQ\Q\QnQno pst tA   $m5==IDJ h & H>QY^YfYfgDJ o -OcM'--    J1 1,-@ADJ%7OPTP^P^O_&`a aDJJ& #mmDKKQ?zz$"5"56 zz!d&9&9"9:  $,,/%))DLL2II D///43F3FF3N !$ !H H%(88PQUQ_Q_P`&ab b$( KK#6#:rtjdd|jj z |dz} d | z }tj|jj@|zd|jj@dz |zzz ddj}||jj zj}tj2||ddgjtj4}ntjB|tj dtE||}d|j(dz |j(dz d z} tj2|| ggjtj4}tjF||_$tjF|jK|tj |_&tE||_'dgtQ|jjR|jjTdz z|_+d|_,d|_-d|_.d|_/|jHjKd |_$ycc} wcc} wcc} w)a Sets the discrete timesteps used for the diffusion chain (to be run before inference). Args: num_inference_steps (`int`): The number of diffusion steps used when generating samples with a pre-trained model. device (`str` or `torch.device`, *optional*): The device to which the timesteps should be moved to. If `None`, the timesteps are not moved. rr7rNrVleadingtrailingzY is not supported. Please make sure to choose one of 'linspace', 'leading' or 'trailing'.rQ) in_sigmasrkr4)rr r[)0r% searchsortedfliprhr\rKr8numpyitemrMrlr7roundrmastypeint64arangerNr!arrayrcrgrF_convert_to_karras _sigma_to_t concatenater'rG_convert_to_exponentialrH_convert_to_betarIrJinterplenrnrirxrorkrpr=r>rrrtrurvrw) ryrkr clipped_idx last_timestepro step_ratiori log_sigmassigmara sigma_lasts r set_timestepszSASolverScheduler.set_timestepss((DMMA3)GIgIgh ++99KGNNPVVX  ;; ' ': 5 A}q02E2IJPPRSWUWSWXY\Z\]bbdkklnltltu [[ ) )Y 6&+>+BCJ1&9A&=>KRRTUYWYUYZ[^\^_ddfmmnpnvnvwI 11 1I [[ ) )Z 788;NNJ -ZK@FFHMMOVVWYW_W_`I NI;;//01JK A 3 33t7J7JJsRSVVF^ ;; ( (WWV_))+F,,vSf,gFSY!Z%$"2"25*"E!Z[aacI^^VVBC[$9:AA"**MF [[ / /WWV_))+F11FXk1lFSY!Z%$"2"25*"E!Z[I^^VVBC[$9:AA"**MF [[ ( (WWV_))+F**VQd*eFSY!Z%$"2"25*"E!Z[I^^VVBC[$9:AA"**MF [[ ( ([[A (G(G$GI\_`I`aF6\FWWT[[33f<T[[E[E[^_E_ciDi@ijklomopuuwF$++"A"AAGGII^^VVBC[$9:AA"**MFYYy"))As6{*CVLFt221559L9LQ9OOTWWJ^^Vj\$:;BB2::NF&&v. )))477vU[[7Y#&y>   ++T[[-H-H1-L MN!"  kknnU+ I"[ "[ "[s#\0)\5!\:samplereturncb|j}|j^}}}|tjtjfvr|j }|j ||tj|z}|j}tj||jjd}tj|d|jj}|jd}tj|| ||z }|j ||g|}|j!|}|S)a{ "Dynamic thresholding: At each sampling step we set s to a certain percentile absolute pixel value in xt0 (the prediction of x_0 at timestep t), and if s > 1, then we threshold xt0 to the range [-s, s] and then divide by s. Dynamic thresholding pushes saturated pixels (those near -1 and 1) inwards, thereby actively preventing pixels from saturation at each step. We find that dynamic thresholding results in significantly better photorealism as well as better image-text alignment, especially when using very large guidance weights." https://huggingface.co/papers/2205.11487 rrS)r$rp)r shaper%r'float64floatreshaperlprodabsquantiler\rBclamprC unsqueezerx)ryrr batch_sizechannelsremaining_dims abs_sampless r_threshold_samplez#SASolverScheduler._threshold_sampleWs 06 - H~  6 6\\^F Hrww~7N,NOZZ\ NN:t{{'M'MST U KK 1$++66  KKNVaR+a/ HF~F5! rctjtj|d}||ddtjfz }tj|dk\dj dj |jddz }|dz}||}||}||z ||z z } tj | dd} d| z |z| |zz} | j|j} | S)Ng|=r)axisr)rpr) rlrgmaximumnewaxiscumsumargmaxcliprr) ryrr log_sigmadistslow_idxhigh_idxlowhighwrs rrzSASolverScheduler._sigma_to_tysFF2::eU34 Jq"**}55))UaZq188a8@EE*JZJZ[\J]`aJaEbQ;!(#9_t , GGAq! Ug H , IIekk "rcr|jjr d|z }|}||fSd|dzdzdzz }||z}||fS)NrrrQ)r\rI)ryrrerfs r_sigma_to_alpha_sigma_tz)SASolverScheduler._sigma_to_alpha_sigma_tsW ;; & &%iGG E1HqLS01GgoGrrct|jdr|jj}nd}t|jdr|jj}nd}||n|dj }||n|dj }d}t j dd|}|d|z z}|d|z z}||||z zz|z} | S)z6Constructs the noise schedule of Karras et al. (2022). sigma_minN sigma_maxrVrg@r)hasattrr\rrrrlr7) ryrrkrrrhoramp min_inv_rho max_inv_rhoris rrz$SASolverScheduler._convert_to_karrass 4;; , --II 4;; , --II!*!6IIbM The algorithm and model type are decoupled. You can use either data_prediction or noise_prediction for both noise prediction and data prediction models. Args: model_output (`torch.Tensor`): The direct output from the learned diffusion model. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. Returns: `torch.Tensor`: The converted model output. rrNr/missing `sample` as a required keyword argumentro1.0.0zPassing `timesteps` is deprecated and has no effect as model output conversion is now handled via an internal counter `self.step_index`)r5r3)learned learned_ranger v_predictionflow_predictionzprediction_type given as zd must be one of `epsilon`, `sample`, `v_prediction`, or `flow_prediction` for the SASolverScheduler.)rUzQ must be one of `epsilon`, `sample`, or `v_prediction` for the SASolverScheduler.)rpopr!r rir}rr\rDr?rLrArrerf) ryrrargskwargsrrrerfx0_predr3s rconvert_model_outputz&SASolverScheduler.convert_model_outputs:"$i!m47J1M >4y1}a !RSS   Z   DOO,77> ;; % %)< <{{**i7;;,,0LL#/2A2#6L!Gl$::gE,,8&,,>!F*W|-CC,,0AA++doo6 7\#99 / 0K0K/LMVV {{''009N[[ ' '+? ?{{**i7;;,,0LL*1bqb51G*G,,8!Gl$::gE,,>!L07V3CC / 0K0K/LMAA {{''#'<<#94<<;Q!Gg$55@009!Gg$55@N/@rct|dvsJd|dk(r2tj| tj||z dz zS|dk(r;tj| |dztj||z z|dzz zS|dk(rMtj| |dzd|zzdztj||z z|dzd|zzdzz zS|dk(r_tj| |dzd|dzzzd|zzdztj||z z|dzd|dzzzd|zzdzz zSy) zd Calculate the integral of exp(-x) * x^order dx from interval_start to interval_end rrrr)order is only supported for 0, 1, 2 and 3rrrrNr%r)ryorderinterval_start interval_ends r%get_coefficients_exponential_negativez7SASolverScheduler.get_coefficients_exponential_negativeJs $Q&QQ$ A:99l]+uyy9V/WZ[/[\ \ aZ99l]+!#uyy1N'OOS_bcScd aZ99l]+"Q%77!;uyyXfIf?gg?Q%559; aZ99l]+"Q):%::Q=OORSS))L>9:;?Qq%881|;KKaOQ rc|dvsJdd|dzz|z}d|dzz|z}|dk(r;tj|dtj||z z zd|dzzz S|dk(rGtj||dz |dz tj||z zz zd|dzzdzz S|dk(rYtj||dzd|zz dz|dzd|zz dztj||z zz zd|dzzdzz S|dk(rktj||dzd|dzzz d|zzdz |dzd|dzzz d|zzdz tj||z zz zd|dzzdzz Sy ) zl Calculate the integral of exp(x(1+tau^2)) * x^order dx from interval_start to interval_end rrrrrrrNr)ryrrrtauinterval_end_covinterval_start_covs r%get_coefficients_exponential_positivez7SASolverScheduler.get_coefficients_exponential_positivebs& $Q&QQ$QJ,6#q&jN: A: *+q599?ORd?d=e3f/fgklortuoukuv aZ *+%))A-=MPb=b;c1dde QJ1$ & aZ *+%q(1/?+??!C)1,q3E/EEIii"25G"G HIJJ QJ1$ & aZ *+%q(1/?/B+BBQIYEYY\]])1,q3Eq3H/HH1OaKaadeeii"25G"G HIJJ QJ1$ & rc |dvsJ|t|dz k(sJ|dk(rdggS|dk(r@d|d|dz z |d |d|dz z gd|d|dz z |d |d|dz z ggS|dk(r|d|dz |d|dz z}|d|dz |d|dz z}|d|dz |d|dz z}d|z |d |dz |z |d|dz|z gd|z |d |dz |z |d|dz|z gd|z |d |dz |z |d|dz|z ggS|dk(r|d|dz |d|dz z|d|dz z}|d|dz |d|dz z|d|dz z}|d|dz |d|dz z|d|dz z}|d|dz |d|dz z|d|dz z}d|z |d |dz |dz |z |d|dz|d|dzz|d|dzz|z |d |dz|dz|z gd|z |d |dz |dz |z |d|dz|d|dzz|d|dzz|z |d |dz|dz|z gd|z |d |dz |dz |z |d|dz|d|dzz|d|dzz|z |d |dz|dz|z gd|z |d |dz |dz |z |d|dz|d|dzz|d|dzz|z |d |dz|dz|z ggSy)zB Calculate the coefficient of lagrange polynomial rrrrrN)r)ryr lambda_list denominator1 denominator2 denominator3 denominator4s rlagrange_polynomial_coefficientz1SASolverScheduler.lagrange_polynomial_coefficientsa  $$$K(1,,,, A:C5L aZQ+a.89 ^O{1~ A'FG Q+a.89 ^O{1~ A'FG  aZ'N[^; AQ\]^Q_@_`L'N[^; AQ\]^Q_@_`L'N[^; AQ\]^Q_@_`L $!!n_{1~5EN[^3lB  $!!n_{1~5EN[^3lB  $!!n_{1~5EN[^3lB "aZQ+a.0q>KN24q>KN24  Q+a.0q>KN24q>KN24  Q+a.0q>KN24q>KN24  Q+a.0q>KN24q>KN24  $!!n_{1~5 AF,V#AQ7%a.;q>9:%a.;q>9:# # "!n_{1~5 AF,V  $!!n_{1~5 AF,V#AQ7%a.;q>9:%a.;q>9:# # "!n_{1~5 AF,V  $!!n_{1~5 AF,V#AQ7%a.;q>9:%a.;q>9:# # "!n_{1~5 AF,V  $!!n_{1~5 AF,V#AQ7%a.;q>9:%a.;q>9:# # "!n_{1~5 AF,V E-- +rc |dvsJ|t|k(sJdg}|j|dz |}t|D]}}d} t|D]Z} |jr'| ||| |j |dz | z |||zz } 6| ||| |j |dz | z ||zz } \|j | t||k(sJd|S)N)rrrrz4the length of lambda list must be equal to the orderrrz3the length of coefficients does not match the order)rrr"rsrrr#) ryrrrrr coefficientslagrange_coefficientr, coefficientjs rget_coefficients_fnz%SASolverScheduler.get_coefficients_fns $$$K((`*``( #CCEAI{[u -AK5\ ??#7#:1#=@j@j A ~|SA$K #7#:1#=@j@j A ~|A$K      , -< E)`+``)rnoiserrc  t|dkDr|dn|jdd}|t|dkDr|d}n td|t|dkDr|d}n td|t|dkDr|d}n td |t|d kDr|d }n td | tdd d |j} |j |j dz|j |j } } |j| \} } |j| \} } tj| tj| z }tj| tj| z }tj|}||z }g}t|D]n}|j |z }|j|j |\}}tj|tj|z }|j|p|j|||||}|}|jrM|dk(rG|j |j dz }|j|\}}tj|tj|z }|dxxdtjd|dzz|zz|dzdz |d|dzzzdz tjd|dzz| zzd|dzzdzz z z||z z z cc<|dxxdtjd|dzz|zz|dzdz |d|dzzzdz tjd|dzz| zzd|dzzdzz z z||z z zcc<t|D]i}|jr<|d|dzz| ztj|dz |zz||z| |dz zz }K|d|dzz | z||z| |dz zz }k|jr;| tj dtjd|dzz|zz z|z}n7|| ztj tjd|zdz z|z}|jr,tj|dz |z| | z z|z|z|z}n| | z |z|z|z}|j#|j$}|S)ag One step for the SA-Predictor. Args: model_output (`torch.Tensor`): The direct output from the learned diffusion model at the current timestep. prev_timestep (`int`): The previous discrete timestep in the diffusion chain. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. order (`int`): The order of SA-Predictor at this timestep. Returns: `torch.Tensor`: The sample tensor at the previous timestep. r prev_timestepNrrrz.missing `noise` as a required keyword argumentr.missing `order` as a required keyword argumentr,missing `tau` as a required keyword argumentrzPassing `prev_timestep` is deprecated and has no effect as model output conversion is now handled via an internal counter `self.step_index`r4rrr!r rrrir}rr%rg zeros_liker"r#rrsrrdrxr ) ryrrrrrrrrmodel_output_listrfsigma_s0realpha_s0rh lambda_s0 gradient_parthrr,sialpha_sisigma_si lambda_sigradient_coefficientsx temp_sigma temp_alpha_s temp_sigma_s temp_lambda_s noise_partx_ts r!stochastic_adams_bashforth_updatez3SASolverScheduler.stochastic_adams_bashforth_updates;6$'t9q=QfjjRV6W >4y1}a !RSS =4y1}Q !QRR =4y1}Q !QRR ;4y1}1g !OPP  $ ^  !.. KK!+ , KK ( 77@!99(C(99W% '(::IIh'%))H*== ((0 y  u *A1$B!%!=!=dkk"o!N Hh (+eii.AAI   y )  * !% 8 8 8U`be f  ?? "[[1)<= -1-I-I*-U* l % , 7%))L:Q Q %a(iiS!Vx 789!tax1CF #3a#7%))QaZUVTVDW:X#X^_beghbh^hmn]n"ooq!=02( &a(iiS!Vx 789!tax1CF #3a#7%))QaZUVTVDW:X#X^_beghbh^hmn]n"ooq!=02(u rAaZii#q& H 456,A./(!a%1 2 1sAv:!8;PQR;S!SVgjknojohpVq!qq  r ?? 5::a%))BaK!O2L.L#MMPUUJwEIIa!e4Dq4H)IIEQJ ??))c1fIM*g.@AAE UXbbCX%*]:ZGCffQWWo rthis_model_outputru last_noise this_samplec . t|dkDr|dn|jdd} |t|dkDr|d}n td|t|dkDr|d}n td|t|dkDr|d}n td |t|d kDr|d }n td |t|d kDr|d }n td | tddd|j} |j |j |j |j dz } } |j| \} } |j| \}} tj| tj| z }tj|tj| z }tj|}||z }g}t|D]n}|j |z }|j|j |\}}tj|tj|z }|j|p| |gz}|j|||||}|}|jr|dk(r|dxxdtjd|dzz|zz|dz |d|dzzzdz tjd|dzz| zzd|dzzdz|zz z zz cc<|dxxdtjd|dzz|zz|dz |d|dzzzdz tjd|dzz| zzd|dzzdz|zz z zzcc<t|D]i}|jr<|d|dzz| ztj|dz |zz||z||dz zz }K|d|dzz | z||z||dz zz }k|jr;| tj dtjd|dzz|zz z|z}n7|| ztj tjd|zdz z|z}|jr,tj|dz |z| | z z|z|z|z}n| |z |z|z|z}|j#|j$}|S)a One step for the SA-Corrector. Args: this_model_output (`torch.Tensor`): The model outputs at `x_t`. this_timestep (`int`): The current timestep `t`. last_sample (`torch.Tensor`): The generated sample before the last predictor `x_{t-1}`. this_sample (`torch.Tensor`): The generated sample after the last predictor `x_{t}`. order (`int`): The order of SA-Corrector at this step. Returns: `torch.Tensor`: The corrected sample tensor at the current timestep. r this_timestepNrz4missing `last_sample` as a required keyword argumentrz3missing `last_noise` as a required keyword argumentrz4missing `this_sample` as a required keyword argumentrrrrzPassing `this_timestep` is deprecated and has no effect as model output conversion is now handled via an internal counter `self.step_index`r4rr)ryrrurrrrrrrrrfrrerrhrrr rr,r r r r model_prev_listrrrrs rstochastic_adams_moulton_updatez1SASolverScheduler.stochastic_adams_moulton_updates>$'t9q=QfjjRV6W  4y1}"1g  !WXX  4y1}!!W  !VWW  4y1}"1g  !WXX =4y1}Q !QRR ;4y1}1g !OPP  $ ^  !.. KK ( KK!+ , 77@!99(C(99W% '(::IIh'%))H*== ((5 y  u *A1$B!%!=!=dkk"o!N Hh (+eii.AAI   y )  * ,/@.AA $ 8 8 8U`be f  ?? &a(iiS!Vx 7891uQaZ 01 4uyy!c1f*RSQSAT7U U[\_bde_e[ejkZknoZoppr( &a(iiS!Vx 7891uQaZ 01 4uyy!c1f*RSQSAT7U U[\_bde_e[ejkZknoZoppr( u pAaZii#q& H 456,A./&Ah/ 0 1sAv:!8;PQR;S!SVehilmhmfnVo!oo  p ?? 5::a%))BaK!O2L.L#MMPZZJwEIIa!e4Dq4H)IIJVJ ??))c1fIM*g.@AAE UXbbCX%*]:ZGCffQWWo rc| |j}||k(j}t|dk(rt|jdz }|St|dkDr|dj}|S|dj}|S)Nrr)rononzerorr)ryrschedule_timestepsindex_candidatesr}s rindex_for_timestepz$SASolverScheduler.index_for_timesteps  %!% .(:CCE  A %T^^,q0J ! "Q &)!,113J*!,113Jrc|jVt|tjr%|j |j j }|j||_y|j|_y)zF Initialize the step_index counter for the scheduler. N) r isinstancer%Tensorrxrorr#rvrw)ryrs r_init_step_indexz"SASolverScheduler._init_step_index(sW    #(ELL1#;;t~~'<'<=#66x@D #00D rr return_dictc|j td|j|j||jdkDxr|jdu}|j ||}|rS|j |jd}|j||j|j||j|}tt|jj|jjdz dz D]@} |j | dz|j | <|j| dz|j| <B||j d<||jd<t#|j$||j&|j(} |jj*rt-|jjt/|j0|jz } t-|jjt/|j0|jz dz} n,|jj} |jj} t-| |j2dz|_t-| |j2d z|_ |j4dkDsJ|jdkDsJ||_| |_ |j |jd}|j7||| |j4| } |j2t|jj|jjdz kr|xj2dz c_|xj8dz c_|s| fSt;| S) a Predict the sample from the previous timestep by reversing the SDE. This function propagates the sample with the SA-Solver. Args: model_output (`torch.Tensor`): The direct output from learned diffusion model. timestep (`int`): The current discrete timestep in the diffusion chain. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. generator (`torch.Generator`, *optional*): A random number generator. return_dict (`bool`): Whether or not to return a [`~schedulers.scheduling_utils.SchedulerOutput`] or `tuple`. Returns: [`~schedulers.scheduling_utils.SchedulerOutput`] or `tuple`: If return_dict is `True`, [`~schedulers.scheduling_utils.SchedulerOutput`] is returned, otherwise a tuple is returned where the first element is the sample tensor. NzaNumber of inference steps is 'None', you need to run 'set_timesteps' after creating the schedulerrrrV)rrurrrrr) generatorrr r)rrrrr) prev_sample)rkr!r}r'rurr@rqrrthis_corrector_orderr"rpr\r=r>rrr rrr rEr$rrortthis_predictor_orderrrvr)ryrrrr*r( use_correctormodel_output_convert current_taur,rr-r,r+s rstepzSASolverScheduler.step4s0<  # # +s  ?? "  ! !( +!+L0@0@0L #88f8U --(:(:2(>?K99"6 ,,??"// :Fs4;;66 8S8SVW8WX[\\] >A$($6$6q1u$=D  q !$($6$6q1u$=D  q ! >"62!)2   &&$$   ;; ( (#&t{{'B'BCDWZ^ZiZiDi#j #&t{{'B'BCDWZ^ZiZiDilmDm#n #';;#>#> #';;#>#> $'(S>SVW>W$X!$'(S>SVW>W$X!((1,,,((1,,,!mmD$6$6r$:; <<-++ =   3t{{'B'BDKKD_D_bcDc#d d  ! !Q & ! A> !;77rc|S)a? Ensures interchangeability with schedulers that need to scale the denoising model input depending on the current timestep. Args: sample (`torch.Tensor`): The input sample. Returns: `torch.Tensor`: A scaled input sample. rY)ryrrrs rscale_model_inputz#SASolverScheduler.scale_model_inputs  roriginal_samplesroc|jj|j|_|jj|j}|j|j}||dz}|j }t |j t |j kr=|jd}t |j t |j kr=d||z dz}|j }t |j t |j kr=|jd}t |j t |j kr=||z||zz}|S)N)rrrQrVr)rcrxrr flattenrrr)ryr4rrorcsqrt_alpha_prodsqrt_one_minus_alpha_prod noisy_sampless r add_noisezSASolverScheduler.add_noisesa#1144r<)NT)2__name__ __module__ __qualname____doc__rname _compatiblesrr rintstrrrrlndarrayrrboolrzpropertyr}rrr%rrr&rrrrrrrrrrrrrr#r'rrr1r3 IntTensorr:r=).0es00rr1r1OsAF%>>qAFF>L E$("%BF  ('+",1"%/"&,116*/*/&)%*5\M'+ *1S, S,S, S,  S,  bjj$u+&= >? S,S,S,S,8$S,S,%*S, S,S, S, $D>!S,"!)#S,$"$%S,&"$'S,(UO)S,*"+S,, }-S,./S,01S,S,j  !!(3(O,O,U3PUP\P\K\E]O,d D0 ELLRWR^R^4TW\a\h\h.dgrSsN$ 99 A1,XX !)4Xv/ v/r