bitddlZddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl m Z ddl m Z ddlmZddlmZddl mZddlZddlZddlmZdd lmZmZmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:m;Z;mZ>m?Z?m@Z@mAZAd ZBeB\ZCZDZEd eFfd ZGeGeDd ZHdcdZIeIddZJdZKdZLdZMdZNdZOdZPdZQdZRdZSdZTdZUdZVdZWd ZXd!ZYd"ZZd#Z[d$Z\d%Z]d&Z^d'Z_d(Z`d)Zad*Zbd+Zcd,Zdd-Zed.Zfd/Zgd0Zhd1Zid2Zjd3Zkd4Zld5Zmd6Znddd7Zod8Zpd9Zqd:Zrd;Zsd<Ztd=Zud>Zvd?Zwd@ZxdAZydBZzdCZ{dDZ|dEZ}dFZ~ee?e4e;e3gxre ZdGZdHZdIZGdJdKe jZGdLdMe jZGdNdOe jZdPZGdQdRZdSZded efdTZdfdUeFd efdVZdWZdXZGdYdZeZdgd[eFefd\Zd]ed efd^Ze dhd_ed`ed efdaZdbZy)iN)contextmanager)partial)Path)Union)mock)AcceleratorState)*check_cuda_fp8_capabilitycompare_versionsgatheris_aim_availableis_bnb_availableis_clearml_availableis_comet_ml_availableis_cuda_availableis_datasets_availableis_deepspeed_availableis_dvclive_availableis_fp8_availableis_fp16_availableis_habana_gaudi1is_hpu_availableis_import_timer_availableis_matplotlib_availableis_mlflow_availableis_mlu_availableis_mps_availableis_musa_availableis_npu_availableis_pandas_availableis_pippy_availableis_pytest_availableis_schedulefree_availableis_sdaa_availableis_swanlab_availableis_tensorboard_availableis_timm_availableis_torch_versionis_torch_xla_availableis_torchao_available*is_torchdata_stateful_dataloader_availableis_torchvision_availableis_trackio_availableis_transformer_engine_availableis_transformers_availableis_triton_availableis_wandb_availableis_xpu_available str_to_boolctr9dtjjtjjfSt r9dtjjtjjfSt drddtjjfSt rdddfStr9dtjjtjjfStr9d tjjtjjfStr9d tjjtjjfStr9d tj jtj jfSt#r9d tj$jtj$jfSt'r9d tj(jtj(jfSdddfS)Nxlacudaz2.0) min_versionmpscyNrr<X/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/accelerate/test_utils/testing.pyzget_backend..Zr=mlusdaamusanpuxpuhpucpucyr;r<r<r=r>r?zget_backend..hr@r=)r)torchr6 device_countmemory_allocatedrrr8current_allocated_memoryrrAr$rBrrCrrDr2rErrFr<r=r> get_backendrMRsejj--/1L1LLL  uzz..0%**2M2MMM e ,a;;;;  a""  eii,,. 0J0JJJ  uzz..0%**2M2MMM  uzz..0%**2M2MMM  eii,,. 0J0JJJ  eii,,. 0J0JJJ  eii,,. 0J0JJJa""r=returnc ddg}|jD]F\}}t|tr|r|jd|-|0|jd|d|H|S)z Wraps around `kwargs` to help simplify launching from `subprocess`. Example: ```python # returns ['accelerate', 'launch', '--num_processes=2', '--device_count=2'] get_launch_command(num_processes=2, device_count=2) ``` acceleratelaunchz--=)items isinstanceboolappend)kwargscommandkvs r>get_launch_commandr[nsiX&G )1 a 1 NNRs8 $ ] NNRs!A3< ( ) Nr=g?) num_processesmonitor_intervalFc tj|} t|}|S#t$rtd|dwxYw#t$r|}Y|SwxYw)NzIf set, z must be yes or no.)osenvironr3 ValueErrorKeyError)keydefaultvalue_values r>parse_flag_from_envrgsp B 3  B 'F M Bxu,?@A A B  Ms? #< A ARUN_SLOW)rdc8tjd|S)z+Decorator that skips a test unconditionallyzTest was skipped)unittestskip test_cases r>rkrks ,8==+ ,Y 77r=cBtjtd|S)z Decorator marking a test as slow. Slow tests are skipped by default. Set the RUN_SLOW environment variable to a truthy value to run them. z test is slow)rj skipUnless_run_slow_testsrls r>slowrqs @8   ? JJr=cHtjtdk(d|S)zu Decorator marking a test that must be only ran on the CPU. These tests are skipped when a GPU is available. rGztest requires only a CPUrjro torch_devicerls r> require_cpurus& R8  |u46P QR[ \\r=cHtjtdk7d|S)z Decorator marking a test that requires a hardware accelerator backend. These tests are skipped when there are no hardware accelerator available. rGtest requires a GPUrsrls r>require_non_cpurxs% M8  |u46K LY WWr=cdtjtxr t d|S)z Decorator marking a test that requires CUDA. These tests are skipped when there are no GPU available or when TorchXLA is available. rw)rjrorr)rls r> require_cudarzs1 l8  02S;Q;S7SUj klu vvr=c|tjtxr t xs t d|S)z Decorator marking a test that requires CUDA or HPU. These tests are skipped when there are no GPU available or when TorchXLA is available. ztest requires a GPU or HPU)rjrorr)rrls r>require_cuda_or_hpur|s?  8     =%;%=!=TBRBTVr  r=cJtjtd|S)zn Decorator marking a test that requires XPU. These tests are skipped when there are no XPU available. test requires a XPU)rjror2rls r> require_xpurs$ J8  /13H I) TTr=ctxr t }t}tj|xs|d|S)z Decorator marking a test that requires CUDA or XPU. These tests are skipped when there are no GPU available or when TorchXLA is available. ztest requires a CUDA GPU or XPU)rr)r2rjro)rmcuda_condition xpu_conditions r>require_cuda_or_xpursA '(I1G1I-IN$&M b8  ~>@a bcl mmr=cHtjtdk7d|S)zB Decorator marking a test that should be skipped for XPU. rEztest requires a non-XPUrsrls r>require_non_xpur& Q8  |u46O PQZ [[r=cHtjtdk7d|S)zB Decorator marking a test that should be skipped for HPU. rFztest requires a non-HPUrsrls r>require_non_hpurrr=cJtjtd|S)zj Decorator marking a test that requires FP16. These tests are skipped when FP16 is not supported. ztest requires FP16 support)rjrorrls r> require_fp16rs% R8  024P QR[ \\r=ct}tjjr t sd}t r t rd}tj|d|S)zh Decorator marking a test that requires FP8. These tests are skipped when FP8 is not supported. Fztest requires FP8 support) rrIr6 is_availabler rrrjro)rmfp8_is_availables r> require_fp8rsW() zz )B)D .0  M8  /1L Mi XXr=cNtjtddd|S)N>=z2.5.0z$test requires FSDP2 (torch >= 2.5.0))rjror(rls r> require_fsdp2rs& g8  /g>@f ghq rrr=cJtjtd|S)zn Decorator marking a test that requires MLU. These tests are skipped when there are no MLU available. ztest require a MLU)rjrorrls r> require_mlur$ I8  /13G H SSr=cJtjtd|S)zp Decorator marking a test that requires SDAA. These tests are skipped when there are no SDAA available. ztest require a SDAA)rjror$rls r> require_sdaar $ K8  024I J9 UUr=cJtjtd|S)zp Decorator marking a test that requires MUSA. These tests are skipped when there are no MUSA available. ztest require a MUSA)rjrorrls r> require_musarrr=cJtjtd|S)zn Decorator marking a test that requires NPU. These tests are skipped when there are no NPU available. ztest require a NPU)rjrorrls r> require_npurrr=cJtjtd|S)z Decorator marking a test that requires MPS backend. These tests are skipped when torch doesn't support `mps` backend. z0test requires a `mps` backend support in `torch`)rjrorrls r> require_mpsrs% g8  /13e fgp qqr=cbtjtxr td|S)zv Decorator marking a test that requires transformers and datasets. These tests are skipped when they are not. z$test requires the Hugging Face suite)rjror/rrls r>require_huggingface_suiter&s5 8  !#?(=(?.  r=cJtjtd|S)zi Decorator marking a test that requires transformers. These tests are skipped when they are not. z&test requires the transformers library)rjror/rls r>require_transformersr0% f8  8: require_timmr7s% V8  024T UV_ ``r=cJtjtd|S)zh Decorator marking a test that requires torchvision. These tests are skipped when they are not. z%test requires the torchvision library)rjror,rls r>require_torchvisionr>s% d8  79;b cdm nnr=cJtjtd|S)zc Decorator marking a test that requires triton. These tests are skipped when they are not. z test requires the triton library)rjror0rls r>require_tritonrEs% Z8  246X YZc ddr=cJtjtd|S)zi Decorator marking a test that requires schedulefree. These tests are skipped when they are not. z&test requires the schedulefree library)rjror#rls r>require_schedulefreerLrr=cJtjtd|S)zi Decorator marking a test that requires bitsandbytes. These tests are skipped when they are not. z&test requires the bitsandbytes library)rjrorrls r> require_bnbrSs% ]8  /13[ \]f ggr=cNtjtdd|S)zp Decorator marking a test that requires TPUs. These tests are skipped when there are no TPUs available. T) check_is_tpuztest requires TPUrjror)rls r> require_tpurZs' _8  54HJ] ^_h iir=cLtjt d|S)z Decorator marking a test as requiring an environment without TorchXLA. These tests are skipped when TorchXLA is available. z%test requires an env without TorchXLArrls r>require_non_torch_xlaras( f8  #9#;;=d efo ppr=c^tjtdk7xr tdk(d|S)z Decorator marking a test that requires a single device. These tests are skipped when there is no hardware accelerator available or number of devices is more than one. rGr9z)test requires a single device accelerator)rjrortrJrls r>require_single_deviceris6  8  3,!"35`  r=cxtjtjj dk(d|S)z Decorator marking a test that requires CUDA on a single GPU. These tests are skipped when there are no GPU available or number of GPUs is more than one. r9rwrjrorIr6rJrls r>require_single_gpurss3 V8  uzz668A=?T UV_ ``r=cxtjtjj dk(d|S)z Decorator marking a test that requires CUDA on a single XPU. These tests are skipped when there are no XPU available or number of xPUs is more than one. r9r~rjrorIrErJrls r>require_single_xpur{s3 U8  uyy5571<>S TU^ __r=cHtjtdkDd|S)z Decorator marking a test that requires a multi-device setup. These tests are skipped on a machine without multiple devices. r9z,test requires multiple hardware accelerators)rjrorJrls r>require_multi_devicers& a8  |a/1_ `aj kkr=cxtjtjj dkDd|S)z Decorator marking a test that requires a multi-GPU setup. These tests are skipped on a machine without multiple GPUs. r9ztest requires multiple GPUsrrls r>require_multi_gpurs3 ]8  uzz6681<>[ \]f ggr=cxtjtjj dkDd|S)z Decorator marking a test that requires a multi-XPU setup. These tests are skipped on a machine without multiple XPUs. r9ztest requires multiple XPUsrrls r>require_multi_xpurs3 \8  uyy557!;=Z [\e ffr=cxtjtxs txr tdkDd|S)z Decorator marking a test that requires a multi-GPU setup. These tests are skipped on a machine without multiple GPUs or XPUs. r9z#test requires multiple GPUs or XPUs)rjrorr2rJrls r>require_multi_gpu_or_xpurs=  8     2 0 2H q8HJo  r=cJtjtd|S)z| Decorator marking a test that requires DeepSpeed installed. These tests are skipped when DeepSpeed isn't installed ztest requires DeepSpeed)rjrorrls r>require_deepspeedrs% T8  579R ST] ^^r=cltjtddxr tdddd|S)zn Decorator marking a test that requires TP installed. These tests are skipped when TP isn't installed rz2.3.0 transformersz4.52.0zGtest requires torch version >= 2.3.0 and transformers version >= 4.52.0)rjror(r rls r> require_tprs@ 8  w'\,<^TS[,\Q  r=cz|tt|Stjt d|d||S)z Decorator marking that a test requires a particular torch version to be tested. These tests are skipped when an installed torch version is less than the required one. )versionrztest requires torch version >= )rrequire_torch_min_versionrjror()rmrs r>rrsE 0'BB l8  /g>Babiaj@k lmv wwr=cJtjtd|S)z Decorator marking a test that requires tensorboard installed. These tests are skipped when tensorboard isn't installed ztest requires Tensorboard)rjror&rls r>require_tensorboardrs% X8  79;V WXa bbr=cJtjtd|S)zt Decorator marking a test that requires wandb installed. These tests are skipped when wandb isn't installed ztest requires wandb)rjror1rls r> require_wandbrs$ L8  135J KI VVr=cJtjtd|S)zx Decorator marking a test that requires trackio installed. These tests are skipped when trackio isn't installed ztest requires trackio)rjror-rls r>require_trackior% P8  357N OPY ZZr=cJtjtd|S)zz Decorator marking a test that requires comet_ml installed. These tests are skipped when comet_ml isn't installed ztest requires comet_ml)rjrorrls r>require_comet_mlrs% R8  468P QR[ \\r=cJtjtd|S)zp Decorator marking a test that requires aim installed. These tests are skipped when aim isn't installed ztest requires aim)rjror rls r> require_aimrs$ H8  /13F G RRr=cJtjtd|S)zx Decorator marking a test that requires clearml installed. These tests are skipped when clearml isn't installed ztest requires clearml)rjrorrls r>require_clearmlrrr=cJtjtd|S)zx Decorator marking a test that requires dvclive installed. These tests are skipped when dvclive isn't installed ztest requires dvclive)rjrorrls r>require_dvcliverrr=cJtjtd|S)zx Decorator marking a test that requires swanlab installed. These tests are skipped when swanlab isn't installed ztest requires swanlab)rjror%rls r>require_swanlabrrr=cJtjtd|S)zv Decorator marking a test that requires pandas installed. These tests are skipped when pandas isn't installed ztest requires pandas)rjror rls r>require_pandasr$ N8  246L Mi XXr=cJtjtd|S)zv Decorator marking a test that requires mlflow installed. These tests are skipped when mlflow isn't installed ztest requires mlflow)rjrorrls r>require_mlflowrrr=cdtjtxr t d|S)z Decorator marking a test that requires pippy installed. These tests are skipped when pippy isn't installed It is also checked if the test is running on a Gaudi1 device which doesn't support pippy. ztest requires pippy)rjror!rrls r> require_pippyrs1 g8  13Nrequire_import_timerrs% ^8  8:<\ ]^g hhr=cJtjtd|S)z Decorator marking a test that requires transformers engine installed. These tests are skipped when transformers engine isn't installed z!test requires transformers engine)rjror.rls r>require_transformer_enginers% g8  >@Be fgp qqr=cJtjtd|S)zx Decorator marking a test that requires torchao installed. These tests are skipped when torchao isn't installed ztest requires torchao)rjror*rls r>require_torchaorrr=cJtjtd|S)z Decorator marking a test that requires matplotlib installed. These tests are skipped when matplotlib isn't installed ztest requires matplotlib)rjrorrls r>require_matplotlibr&s% V8  68:T UV_ ``r=cBtjtd|S)z Decorator marking that a test requires at least one tracking library installed. These tests are skipped when none are installed zYtest requires at least one tracker to be available and for `comet_ml` to not be installed)rjro_atleast_one_tracker_availablerls r>require_trackersr4s)  8  &c  r=cJtjtd|S)z Decorator marking a test that requires torchdata.stateful_dataloader. These tests are skipped when torchdata with stateful_dataloader module isn't installed. z+test requires torchdata.stateful_dataloader)rjror+rls r>%require_torchdata_stateful_dataloaderr?s* 8  246c  r=cdtr%ddl}|jjd|S|S)a Decorator marking a test with order(1). When pytest-order plugin is installed, tests marked with this decorator are garanteed to run first. This is especially useful in some test settings like on a Gaudi instance where a Gaudi device can only be used by a single process at a time. So we make sure all tests that run in a subprocess are launched first, to avoid device allocation conflicts. If pytest is not installed, test will be returned as is. rNr9)r"pytestmarkorder)rmrs r> run_firstrKs/#v{{  #I.. r=c:eZdZdZdZedZedZdZy)TempDirTestCasea A TestCase class that keeps a single `tempfile.TemporaryDirectory` open for the duration of the class, wipes its data at the start of a test, and then destroyes it at the end of the TestCase. Useful for when a class or API requires a single constant folder throughout it's use, such as Weights and Biases The temporary directory location will be stored in `self.tmpdir` TcHttj|_y)zECreates a `tempfile.TemporaryDirectory` and stores it in `cls.tmpdir`N)rtempfilemkdtemptmpdirclss r> setUpClasszTempDirTestCase.setUpClassjs(**,- r=ctjj|jr t j |jyy)z1Remove `cls.tmpdir` after test suite has finishedN)r_pathexistsrshutilrmtreers r> tearDownClasszTempDirTestCase.tearDownClassos- 77>>#** % MM#** % &r=c|jrh|jjdD]I}|jr|j $|j s5t j|Kyy)zsetUpzTempDirTestCase.setUpusU    ((0 (<<>KKM[[]MM$'  ( r=N) __name__ __module__ __qualname____doc__r classmethodrrr r<r=r>rr^s9N..&& (r=rc"eZdZdZfdZxZS)AccelerateTestCasez A TestCase class that will reset the accelerator state at the end of every test. Every test that checks or utilizes the `AcceleratorState` class should inherit from this to avoid silent failures due to state being shared between tests. cLt|tjdy)NT)supertearDownr _reset_state)r  __class__s r>rzAccelerateTestCase.tearDowns %%d+r=)r r rrr __classcell__)rs@r>rrs ,,r=rcTeZdZdZdeej eej ffdZy)MockingTestCasea A TestCase class designed to dynamically add various mockers that should be used in every test, mimicking the behavior of a class-wide mock when defining one normally will not do. Useful when a mock requires specific information available only initialized after `TestCase.setUpClass`, such as setting an environment variable with that information. The `add_mocks` function should be ran at the end of a `TestCase`'s `setUp` function, after a call to `super().setUp()` such as: ```python def setUp(self): super().setUp() mocks = mock.patch.dict(os.environ, {"SOME_ENV_VAR", "SOME_VALUE"}) self.add_mocks(mocks) ``` mocksct|ttfr|n|g|_|jD]-}|j |j |j /y)aQ Add custom mocks for tests that should be repeated on each test. Should be called during `MockingTestCase.setUp`, after `super().setUp()`. Args: mocks (`mock.Mock` or list of `mock.Mock`): Mocks that should be added to the `TestCase` after `TestCase.setUpClass` has been run N)rTtuplelistrstart addCleanupstop)r rms r> add_mockszMockingTestCase.add_mockssH) >UUG  $A GGI OOAFF # $r=N) r r rrrrMockrr#r<r=r>rrs)" $uTYYTYY%?@ $r=rc8t}|djj|j}t |j }|dj }t |jdD]}tj|||ryy)NrFT) r clonetodevicer rGrangeshaperIequal)tensorstatetensorsis r>are_the_same_tensorsr0s  E D\   ! $ $U\\ 2FVn  "G AY]]_F 7==# ${{71:v. r=ceZdZdZy) _RunOutputc.||_||_||_yN) returncodestdoutstderr)r r5r6r7s r>__init__z_RunOutput.__init__s$  r=N)r r rr8r<r=r>r2r2sr=r2cZK |jd{}|r ||ny%7wr4)readline)streamcallbacklines r> _read_streamr>s- __&&  TN  &s +)+c T K|rtddj|tj|dg|dd|tjj tjj |dd{}ggd fd tj tjt|j fdtjt|j fdg| d{t|j d{S77)7w) Nz Running:  rr9)stdinr6r7envc|jdj}|j|st|||yy)Nutf-8)file)decoderstriprVprint)r=sinkpipelabelquiets r>teez_stream_subprocess..tees:{{7#**, D %D )r=c8|tjdS)Nzstdout:rK)sysr6)loutrMs r>r?z$_stream_subprocess..QSZZ_hAir=c8|tjdS)Nzstderr:rO)rPr7)rQerrrMs r>r?z$_stream_subprocess..rSr=)timeout)) rHjoinasynciocreate_subprocess_exec subprocessPIPEwait create_taskr>r6r7r2) cmdrBrArVrLechoprUrRrMs ` @@@r>_stream_subprocessrbs  mSXXc]+,, A QR!!&&!!&&   A" C C* ,,    QXX7i j k    QXX7i j k    AFFHnc3 //E 6%s7A3D(9D":BD(:D$;D(D& D($D(&D(r_c t|D]$\}}t|tst|||<&t j }|j t||||||} dj|} | jdkDr9dj| j} td| d| jd| | S)N)rBrArVrLr`r@r 'z' failed with returncode z, The combined stderr from workers follows: ) enumeraterTrstrrYget_event_looprun_until_completerbrXr5r7 RuntimeError) r_rBrArVrLr`r/cloopresultcmd_strr7s r>execute_subprocess_asyncros#1 a VCF  ! ! #D  $ $3CugUY]^FhhsmG 16==)y1&2C2C1DE::@ C  Mr=ctjjdd}tjdd|dtj }t |S)z Returns an int value of worker's numerical id under `pytest-xdist`'s concurrent workers `pytest -n N` regime, or 0 if `-n 1` or `pytest-xdist` isn't being used. PYTEST_XDIST_WORKERgw0z^gwrWr)r_r`getresubMint)workers r>pytest_xdist_worker_idrys= ZZ^^15 9F VVFB244 0F v;r=c$d}t}||zS)z Returns a port number that can be fed to `torch.distributed.launch`'s `--master_port` argument. Under `pytest-xdist` it adds a delta number based on a worker id so that concurrent tests don't try to use the same port at once. iget_torch_dist_unique_portr}s D')J * r=c eZdZy)SubprocessCallExceptionN)r r rr<r=r>rrsr=rrXct|D]$\}}t|tst|||<&|tj j } tj|tj|}|rt|dr|jd}|Sy#tj$r>}tddj|d|jj|d}~wwxYw)z Runs `command` with `subprocess.check_output` and will potentially return the `stdout`. Will also properly capture if an error occurred while running `command` N)r7rBrFrDz Command `r@z$` failed with the following error: )rfrTrrgr_r`copyr[ check_outputSTDOUThasattrrFCalledProcessErrorrrXoutput)rX return_stdoutrBr/rkres r> run_commandr s '" 1 a QGAJ  {jjoo ((9J9JPST vx(w/M   ( (%)**PQRQYQYQ`Q`QbPc d  sABC-/9C((C- componentscxttjtj}|j |S)z Get a path within the `accelerate` package's directory. Args: *components: Components of the path to join after the package directory. Returns: `Path`: The path to the requested file or directory. )rinspectgetfilerPparentjoinpath)raccelerate_package_dirs r>path_in_accelerate_packager7s2"'//*"=>EE * ! * *J 77r=exception_classmsgc#Kd} dd}|rt d|d y#t$rR}t||sJd|dt||$|t|vsJd|dt|dYd}~hd}~wwxYww) z Context manager to assert that the right `Exception` class was raised. If `msg` is provided, will check that the message is contained in the raised exception. FNTzExpected exception of type z but got zExpected message 'z' to be in exception but got 'rez but ran without issue.) ExceptionrTtypergAssertionError)rrwas_ranrs r>assert_exceptionrFsGd  :?:KKbcdd d!_-p1L_L]]fgklmgnfo/pp- ?#a&= c$6se;YZ]^_Z`Yaab"c c=ds+A<A< A9AA4/A<4A99A<ctj}tj} |t_||i| |t_|j S#t$r}|d}~wwxYw#|t_wxYw)z` Takes in a `func` with `args` and `kwargs` and returns the captured stdout as a string N)ioStringIOrPr6rgetvalue)funcargsrWcaptured_outputoriginal_stdoutrs r>capture_call_outputrYsjkkmOjjO%$  df%  # # %% % s#A A%A  A%%A(( A5)F)NN)NNNFF)NNFT)FNr4)rYrrr_rtrr[rPrrj contextlibr functoolsrpathlibrtypingrrrIrPr-r utilsr r r r rrrrrrrrrrrrrrrrrrr r!r"r#r$r%r&r'r(r)r*r+r,r-r.r/r0r1r2r3rMrtrJmemory_allocated_funcrr[DEFAULT_LAUNCH_COMMANDrgrprkrqrurxrzr|rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrranyrrrrTestCaserrrr0r2r>rbroryr}rrrgrrrUrrr<r=r>rs   % $+++++++++++\#25@M1 l1D&,,Y\]  &j%@8 K]XwUn\\]Y$sTVVTrqaoeqhjqa`lhg_xcW[]S[[[YYrir[a  79;O;QSgSijk$ ! ##  &(h''(B ,** ,$h''$B&0bl&0R$fp*  i c. 8C 8D 8eieceTee$ &r=