bi-dZddlZddlZddlZddlZddlZddlZddlmZddl m Z m Z m Z m Z mZmZmZmZmZddlmZddZdZd ed efd Z dd ed edefdZdefdZy)z A collection of utilities for ensuring that training can always occur. Heavily influenced by the [toma](https://github.com/BlackHC/toma) library. N)version) is_cuda_availableis_hpu_availableis_ipex_availableis_mlu_availableis_mps_availableis_musa_availableis_npu_availableis_sdaa_availableis_xpu_available)compare_versionsc|rtjtrtjj yt rtjj ytrtjj ytrtjj ytrtjj ytdrtjj yt!rtj"j yt%ryy)z Clears the device cache by calling `torch.{backend}.empty_cache`. Can also run `gc.collect()`, but do note that this is a *considerable* slowdown and should be used sparingly. z2.0) min_versionN)gccollectr torchxpu empty_cachermlur sdaar musar npur mpsrcudargarbage_collections R/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/accelerate/utils/memory.pyclear_device_cacher+s                e ,        ct|ts t|}tt|D]}d||< t d|S)aN Releases memory from `objects` by setting them to `None` and calls `gc.collect()` and `torch.cuda.empty_cache()`. Returned objects should be reassigned to the same variables. Args: objects (`Iterable`): An iterable of objects Returns: A list of `None` objects to replace `objects` Example: ```python >>> import torch >>> from accelerate.utils import release_memory >>> a = torch.ones(1000, 1000).cuda() >>> b = torch.ones(1000, 1000).cuda() >>> a, b = release_memory(a, b) ``` NTr) isinstancelistrangelenr)objectsis rrelease_memoryr(FsE, gt $w- 3w<  $/ Nr exceptionreturncgd}ttr,tjdk(rt fd|DSy)z Checks if `exception` relates to CUDA out-of-memory, XPU out-of-memory, CUDNN not supported, or CPU out-of-memory Args: exception (`Exception`): An exception )z out of memory.z(cuDNN error: CUDNN_STATUS_NOT_SUPPORTED.z*DefaultCPUAllocator: can't allocate memoryz1FATAL ERROR :: MODULE:PT_DEVMEM Allocation failedrc3@K|]}|jdvyw)rN)args).0errr)s r z+should_reduce_batch_size..ssC3)..++CsF)r" RuntimeErrorr%r-any)r) _statementss` rshould_reduce_batch_sizer4ds:K )\*s9>>/Ba/GC{CCC r functionstarting_batch_sizereduce_batch_size_fncftjt|S|fdfd}|S)a A basic decorator that will try to execute `function`. If it fails from exceptions related to out-of-memory or CUDNN, the batch size is multiplied by 0.9 and passed to `function` `function` must take in a `batch_size` parameter as its first argument. Args: function (`callable`, *optional*): A function to wrap starting_batch_size (`int`, *optional*): The batch size to try and fit into memory Example: ```python >>> from accelerate.utils import find_executable_batch_size >>> @find_executable_batch_size(starting_batch_size=128) ... def train(batch_size, model, optimizer): ... ... >>> train(model, optimizer) ``` )r6c$tdzS)Ng?)int) batch_sizesrr7z8find_executable_batch_size..reduce_batch_size_fnsZ#-.J r c 6tdttjjj }t |t |dzkrbdjt|dd|ddDcgc] \}}|d|c}}}tdjdjd|d  d k(r td  g|i|Scc}}w#t$r*}t|rtd nYd}~nd}~wwxYwZ) NTrrz, =zBatch size was passed into `zS` as the first argument when called.Remove this as the decorator already does so: `(z)`rz-No executable batch size found, reached zero.)rr#inspect signature parameterskeysr%joinzip TypeError__name__r1 Exceptionr4) r-kwargsparamsargvaluearg_strer;r5r7s r decoratorz-find_executable_batch_size..decorators1d3g''1<<AACD v;#d)a- (iiCqr TXYZY[T\D] ^jc5C5%!1 ^_G.x/@/@.ABBBJBSBSATTUV]U^^`b Q"#RSS  C  C$$ D- DD) functoolspartialfind_executable_batch_size)r5r6r7rNr;s` ` @rrQrQws=:  !;Qdee$J#  . r device_indexctjtjjtjdjk\r# tj j |dStrPtjtjjd}t|ddrddl m}||dStjdtj j|S#t$rY?wxYw)Nz2.6rintel_extension_for_pytorchz>=z2.5) mem_get_infozThe XPU `mem_get_info` API is available in IPEX version >=2.5 or PyTorch >=2.6. The current returned available memory is incorrect. Please consider upgrading your IPEX or PyTorch version.)rparser __version__releaserrUrGr importlibmetadatarintel_extension_for_pytorch.xpuwarningswarnmax_memory_allocated)rR ipex_versionrUs rget_xpu_available_memoryr`s}}U&&'//7==3G3O3OO 99)),7: :  }}Y%7%7%?%?@]%^_ L$ 6 D -a0 0 MM F 99 ) ), 77   s!C?? D  D )F)NN)__doc__rOrrYr?r\r packagingrimportsrrrrr r r r r versionsrrr(rGboolr4callabler:rQr`r rris      ' 6< d(ae??47?U]?D838r