bi]ddlZddlmZddlmZmZmZmZddlZ ddl Z ddl m Z m Z ddlmZmZmZddlmZdd lmZerddlZej0eZeGd d eZGd d ee Zy)N) dataclass)ListOptionalTupleUnion) ConfigMixinregister_to_config) BaseOutputis_scipy_availablelogging) randn_tensor)SchedulerMixinc0eZdZUdZej ed<y)FlowMatchLCMSchedulerOutputaJ Output class for the scheduler's `step` function output. Args: prev_sample (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)` for images): Computed sample `(x_{t-1})` of previous timestep. `prev_sample` should be used as next model input in the denoising loop. prev_sampleN)__name__ __module__ __qualname____doc__torch FloatTensor__annotations__i/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/diffusers/schedulers/scheduling_flow_match_lcm.pyrr"s"""rrc!eZdZdZgZdZe d:dedede de ede ed e ed e ed e d e ed e e de e de e de de e ede e fdZ edZedZedZd;defdZdefdZdefdZ dd-ej.deeej.fdej.d.e ejDd/e dee#e$ff d0Z%d1ej6dej6fd2Z&d1ej6d&edej6fd3Z' d?d1ej6d&ed4ed5edej6f d6Z(d7Z)d8Z*d9Z+y)@FlowMatchLCMSchedulera LCM scheduler for Flow Matching. This model inherits from [`SchedulerMixin`] and [`ConfigMixin`]. Check the superclass documentation for the generic methods the library implements for all schedulers such as loading and saving. Args: num_train_timesteps (`int`, defaults to 1000): The number of diffusion steps to train the model. shift (`float`, defaults to 1.0): The shift value for the timestep schedule. use_dynamic_shifting (`bool`, defaults to False): Whether to apply timestep shifting on-the-fly based on the image resolution. base_shift (`float`, defaults to 0.5): Value to stabilize image generation. Increasing `base_shift` reduces variation and image is more consistent with desired output. max_shift (`float`, defaults to 1.15): Value change allowed to latent vectors. Increasing `max_shift` encourages more variation and image may be more exaggerated or stylized. base_image_seq_len (`int`, defaults to 256): The base image sequence length. max_image_seq_len (`int`, defaults to 4096): The maximum image sequence length. invert_sigmas (`bool`, defaults to False): Whether to invert the sigmas. shift_terminal (`float`, defaults to None): The end value of the shifted timestep schedule. use_karras_sigmas (`bool`, defaults to False): Whether to use Karras sigmas for step sizes in the noise schedule during sampling. use_exponential_sigmas (`bool`, defaults to False): Whether to use exponential sigmas for step sizes in the noise schedule during sampling. use_beta_sigmas (`bool`, defaults to False): Whether to use beta sigmas for step sizes in the noise schedule during sampling. time_shift_type (`str`, defaults to "exponential"): The type of dynamic resolution-dependent timestep shifting to apply. Either "exponential" or "linear". scale_factors ('list', defaults to None) It defines how to scale the latents at which predictions are made. upscale_mode ('str', defaults to 'bicubic') Upscaling method, applied if scale-wise generation is considered rNnum_train_timestepsshiftuse_dynamic_shifting base_shift max_shiftbase_image_seq_lenmax_image_seq_len invert_sigmasshift_terminaluse_karras_sigmasuse_exponential_sigmasuse_beta_sigmastime_shift_type scale_factors upscale_modecP|jjrts tdt |jj|jj |jj gdkDr td| dvr tdtjd||tjdddj}tj|jtj}||z }|s||zd|dz |zzz }||z|_d|_d|_||_d|_||_||_|jd|_|j,dj/|_|j,d j/|_y) Nz:Make sure to install scipy if you want to use beta sigmas.rznOnly one of `config.use_beta_sigmas`, `config.use_exponential_sigmas`, `config.use_karras_sigmas` can be used.>linear exponentialz;`time_shift_type` must either be 'exponential' or 'linear'.dtypecpur)configr+r ImportErrorsumr*r) ValueErrornplinspacefloat32copyr from_numpyto timesteps _step_index _begin_index_shift _init_size_scale_factors _upscale_modesigmasitem sigma_min sigma_max)selfr r!r"r#r$r%r&r'r(r)r*r+r,r-r.r@rGs r__init__zFlowMatchLCMScheduler.__init__]s|& ;; & &/A/CZ[ [  ++T[[-O-OQUQ\Q\QnQno pst tA  "; ;Z[ [KK#68KSUS]S]^_cac_cdiik $$Y/222G 00#V^qEAI+?'?@F"55  +)ii& R--/Q,,.rc|jS)z. The value used for shifting. rCrKs rr!zFlowMatchLCMScheduler.shifts {{rc|jS)zg The index counter for current timestep. It will increase 1 after each scheduler step. )rArOs r step_indexz FlowMatchLCMScheduler.step_indexs rc|jS)zq The index for the first timestep. It should be set from pipeline with `set_begin_index` method. rBrOs r begin_indexz!FlowMatchLCMScheduler.begin_indexs    rrTc||_y)z Sets the begin index for the scheduler. This function should be run from pipeline before the inference. Args: begin_index (`int`): The begin index for the scheduler. NrS)rKrTs rset_begin_indexz%FlowMatchLCMScheduler.set_begin_indexs (rc||_yNrN)rKr!s r set_shiftzFlowMatchLCMScheduler.set_shifts  rc ||_||_y)z Sets scale factors for a scale-wise generation regime. Args: scale_factors (`list`): The scale factors for each step upscale_mode (`str`): Upscaling method N)rErF)rKr-r.s rset_scale_factorsz'FlowMatchLCMScheduler.set_scale_factorss,)rsampletimestepnoisereturnc|jj|j|j}|jjdk(rvt j |ra|jj|jt j}|j|jt j}n@|jj|j}|j|j}|j |Dcgc]}|j||}}nG|j|jg|jdz}n|jg|jdz}||j}t|jt|jkr=|jd}t|jt|jkr=||zd|z |zz}|Scc}w)aN Forward process in flow-matching Args: sample (`torch.FloatTensor`): The input sample. timestep (`int`, *optional*): The current timestep in the diffusion chain. Returns: `torch.FloatTensor`: A scaled input sample. )devicer3mpsr2rr4?)rGr?rar3typeris_floating_pointr@r<rTindex_for_timesteprQshapeflattenlen unsqueeze) rKr\r]r^rGschedule_timestepst step_indicessigmas r scale_noisez!FlowMatchLCMScheduler.scale_noises(v}}FLLI ==   &5+B+B8+L!%!2!26== !2!V {{6== {FH!%!2!26==!A {{6==1H    #T\]qD33A7IJ]L] __ ( OO,x~~a/@@L!,,-q0AAL|$,,.%++V\\!22OOB'E%++V\\!22#+!77 ^sG?c4||jjzSrXr6r )rKrns r _sigma_to_tz!FlowMatchLCMScheduler._sigma_to_tst{{6666rmurnrlc|jjdk(r|j|||S|jjdk(r|j|||Sy)Nr1r0)r6r,_time_shift_exponential_time_shift_linearrKrsrnrls r time_shiftz FlowMatchLCMScheduler.time_shiftsU ;; & &- 7//E1= = [[ ( (H 4**2ua8 85rc^d|z }|dd|jjz z }d||z z }|S)a, Stretches and shifts the timestep schedule to ensure it terminates at the configured `shift_terminal` config value. Reference: https://github.com/Lightricks/LTX-Video/blob/a01a171f8fe3d99dce2728d60a73fecf4d4238ae/ltx_video/schedulers/rf.py#L51 Args: t (`torch.Tensor`): A tensor of timesteps to be stretched and shifted. Returns: `torch.Tensor`: A tensor of adjusted timesteps such that the final value equals `self.config.shift_terminal`. rr4)r6r()rKrl one_minus_z scale_factor stretched_ts rstretch_shift_to_terminalz/FlowMatchLCMScheduler.stretch_shift_to_terminals? !e "2!dkk.H.H*HI ;56 rnum_inference_stepsrarGr@cN|jjr | td|$|"t|t|k7r td|+|t||k7s|1t||k7r#td| t|n t|}||_|du}|r2t j |jt j}|e|It j|j|j|j|j|}||jjz }n=t j |jt j}t|}|jjr|j|d|}n%|j|zd|jdz |zzz }|jj r|j#|}|jj$r|j'||}nS|jj(r|j+||}n)|jj,r|j/||}t1j2|j5t0j|}|s||jjz}n4t1j2|j5t0j|}|jj6rUd|z }||jjz}t1j8|t1j:d|j< g}n6t1j8|t1j>d|j< g}||_ ||_!d|_"d|_#y) a Sets the discrete timesteps used for the diffusion chain (to be run before inference). Args: num_inference_steps (`int`, *optional*): The number of diffusion steps used when generating samples with a pre-trained model. device (`str` or `torch.device`, *optional*): The device to which the timesteps should be moved to. If `None`, the timesteps are not moved. sigmas (`List[float]`, *optional*): Custom values for sigmas to be used for each diffusion step. If `None`, the sigmas are computed automatically. mu (`float`, *optional*): Determines the amount of shifting applied to sigmas when performing resolution-dependent timestep shifting. timesteps (`List[float]`, *optional*): Custom values for timesteps to be used for each diffusion step. If `None`, the timesteps are computed automatically. NzC`mu` must be passed when `use_dynamic_shifting` is set to be `True`z4`sigmas` and `timesteps` should have the same lengthzq`sigmas` and `timesteps` should have the same length as num_inference_steps, if `num_inference_steps` is providedrcr) in_sigmasr~)r3ra)ra)$r6r"r9rir~r:arrayastyper<r;rrrJrIr rxr!r(r}r)_convert_to_karrasr*_convert_to_exponentialr+_convert_to_betarr>r?r'catonesrazerosr@rGrArB)rKr~rarGrsr@is_timesteps_provideds r set_timestepsz#FlowMatchLCMScheduler.set_timestepss 4 ;; + + bc c  )"76{c)n, !WXX  *"s6{6I'I%#i.I > KK$$T^^4d6F6Ft~~6VXk !@!@@FXXf%,,RZZ8F"%f+  ;; + +__Rf5FZZ&(Aa60I,IJF ;; % %33F;F ;; ( (,,vSf,gF [[ / /11FXk1lF [[ ( (**VQd*eF!!&),,5==,P$!@!@@I((366U]]SY6ZI ;; $ $6\F!@!@@IYY 1V]](KLMFYY Afmm(LMNF"  rc| |j}||k(j}t|dkDrdnd}||jS)Nrr)r@nonzerorirH)rKr]rkindicesposs rrfz(FlowMatchLCMScheduler.index_for_timesteptsL  %!% %1::< w !*{CCis,I3rct|jdr|jj}nd}t|jdr|jj}nd}||n|dj }||n|dj }d}t j dd|}|d|z z}|d|z z}||||z zz|z} | S)z6Constructs the noise schedule of Karras et al. (2022).rINrJr4rg@r)hasattrr6rIrJrHr:r;) rKrr~rIrJrhoramp min_inv_rho max_inv_rhorGs rrz(FlowMatchLCMScheduler._convert_to_karrass 4;; , --II 4;; , --II!*!6IIbM0/!)0/"$0/0/ U ,0/ sm!0/0/d   !!(3(u *t *".2 .!!.u0001.))* .    .`79U959U\\9 5<<ELL..2+/(,"+/ d!%c]d!c5<<'(d!e% d! UO d! DK( d!L #104 ND''NDu0001ND!! ND EOO, ND  ND *E1 2NDbELLRWR^R^4TW\a\h\h.dgD0/rr)r dataclassesrtypingrrrrnumpyr:rconfiguration_utilsr r utilsr r r utils.torch_utilsrscheduling_utilsr scipy.statsr get_loggerrloggerrrrrrrsp !// A;;,,   H %  #* #  #A/NKA/r