biLddlZddlmZmZddlZddlZddlmZddl m Z ddl m Z m Z ddlmZmZmZmZmZmZmZer ddlmcmZdd ed ed efd Zdd ee deej:fdZddeee e fdeej:fdZ!y)N)OptionalUnion)AcceleratorState)CUDA_DISTRIBUTED_TYPES)DistributedTypeRNGType)is_hpu_availableis_mlu_availableis_musa_availableis_npu_availableis_sdaa_availableis_torch_xla_availableis_xpu_availableseeddevice_specific deterministiccl|r|tjz }tj|tjj|t j |tr t jj|ntr t jj|ntr t jj|ntr t jj|nst!r t j"j|nIt%r t j&j|nt j(j|t+rt-j.||rt j0dyy)a Helper function for reproducible behavior to set the seed in `random`, `numpy`, `torch`. Args: seed (`int`): The seed to set. device_specific (`bool`, *optional*, defaults to `False`): Whether to differ the seed on each device slightly with `self.process_index`. deterministic (`bool`, *optional*, defaults to `False`): Whether to use deterministic algorithms where available. Can slow down training. TN)r process_indexrandomrnptorch manual_seedrxpumanual_seed_allrnpur mlursdaar musar hpucudarxm set_rng_stateuse_deterministic_algorithms)rrrs R/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/accelerate/utils/random.pyset_seedr''s  "000 KKIINN4 d !!$'   !!$'   !!$'   ""4(   ""4(   !!$' ""4(  **40rng_type generatorcH |tjk(rtj}n@|tjk(r tj j}n |tj k(r:tsJdtjtj}n|tjk(r1tsJdtjj}n||tjk(r1tsJdtjj}n8|tj k(r0t#sJdtj$j}n|tj&k(r0t)sJdtj*j}n|tj,k(r0t/sJdtj0j}no|tj2k(r0t5sJdtj6j}n,|tj8k(r|Jd|j;}t=}|j>t@j k(r_jCtjD}tjF|gtjH|jK}nI|j>tLvs|j>t@jNk(s|j>t@jPk(st|j>t@jRk(sW|j>t@jTk(s:|j>t@jVk(s|j>t@jXk(rLjC|jZ}tj\j_|d |jK}n=|j>t@j`k(r tj\j_d |tjk(rtjby|tjk(r tj jcy|tjk(r tjjcy|tjk(r tjjcy|tj k(r tj$jcy|tj&k(r tj*jcy|tj,k(r tj0jcytj2k(r tj6jc|y|tj k(r$tjb|jey|tj8k(r|jg|yy) Nz8Can't synchronize XLA seeds as torch_xla is unavailable.z;Can't synchronize NPU seeds on an environment without NPUs.z;Can't synchronize MLU seeds on an environment without MLUs.z=Can't synchronize SDAA seeds on an environment without SDAAs.z=Can't synchronize MUSA seeds on an environment without MUSAs.z;Can't synchronize XPU seeds on an environment without XPUs.z;Can't synchronize HPU seeds on an environment without HPUs.z)Need a generator to synchronize its seed.r)4r TORCHr get_rng_stateCUDAr"XLArtensorr#NPUrrMLUr rSDAArrMUSAr r XPUrrHPUr r! GENERATOR get_staterdistributed_typer to xla_devicecollective_broadcast mark_stepcpur MULTI_MLU MULTI_SDAA MULTI_MUSA MULTI_NPU MULTI_XPU MULTI_HPUdevice distributed broadcast MULTI_CPUr$item set_state)r)r* rng_statestates r&synchronize_rng_staterMNs'7== '') W\\ !JJ,,. W[[ %'c)cc'LL!1!1!34 W[[ !`#``!II++- W[[ !`#``!II++- W\\ ! "c$cc"JJ,,. W\\ ! "c$cc"JJ,,. W[[ !`#``!II++- W[[ !`#``!II++- W&& &$Q&QQ$'')   E !4!44LL1   , MMO  "88  ! !_%>%> >  ! !_%?%? ?  ! !_%?%? ?  ! !_%>%> >  ! !_%>%> >  ! !_%>%> >LL.  ##Iq1MMO   ?#<#< < ##Iq17==  I& W\\ !   + W[[   * W[[   * W\\ !   + W\\ !   + W[[   * gkk !  * W[[  )* W&& &I& 'r( rng_typesc>|D]}tt||y)N)r*)rMr )rNr*r)s r&synchronize_rng_statesrPs"Fgh/9EFr()FF)NN)N)"rtypingrrnumpyrrrLr constantsr dataclassesr r importsr r r rrrrtorch_xla.core.xla_modelcore xla_modelr#intboolr' GeneratorrMliststrrPr(r&r_s" $-1))$13$1$1d$1NI'HW$5I'RWRaRaIbI'XFd5g+>&?FHUZUdUdLeFr(