uki8F~UdZddlmZddlZddlmZddlZddlZddlZddl Z ddl Z ddl m Z m Z mZddlmZej$ZdZej(Zdd d Zej.eZeeeeeeeeeffZ d. d/d Z ejBZ!ejDZ" d0 d1d Z# d0 d2d Z$d3dZ%d4dZ&d5dZ'd3dZ(d6dZ) d7 d8dZ*d9dZ+ejXZ,ejZZ-de-_ej\Z.de._ej^Z/de/_ej`Z0ejbZ1ejdZ2ejfZ3ejhZ4ejjZ5ejlZ6ejnZ7ejpZ8ejrZ9ejtZ:ejvZ;ejxZej~Z?ejZ@ejZAejZBejZCd:dZDd:dZEeDe8_FeEe8_GGddejZIGdde ZJiZKdeLd<iZMd eLd!<e jZOd"deIjf d;d#ZQ dd+ZW d?d,ZXejZYejZZejZ[ejZ\ejZ]ejZ^ejd@d-Z`ejZbejejejZeejZfejZgejZhejZiejZjejZkejZlejZmy)AzAn XLA client in Python.) annotationsN)Mapping)AnyProtocolUnion)_jaxiHostCUDA)cpugpuc tdtjtdtjtj ||||||||| S)Nr asynchronousdistributed_clientnode_id num_nodes collectives num_devices"get_local_topology_timeout_minutes#get_global_topology_timeout_minutestransfer_server_factory)register_custom_call_handler_xlaregister_custom_call_targetregister_custom_type_handlerregister_custom_typeget_tfrt_cpu_clientrs L/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/jaxlib/xla_client.pymake_cpu_clientrDsRud&F&FGud&?&?@ ! !+)K*M5  c Btjd|tdi|S))Creates a PJRT C API TopologyDescription.tpu)rget_default_c_api_topologydict) topology_namekwargss r#make_tfrt_tpu_c_api_device_topologyr)bs  ( ( t~f~ NNr c Btj||tdi|S)r"r$)rget_c_api_topologyr&)c_apir'r(s rmake_c_api_device_topologyr-is   t~f~ FFr c,tj|SN)rpjrt_plugin_loaded plugin_names rr0r0ps  --r c2tj||dS)N)r,rload_pjrt_plugin)r2 library_paths rload_pjrt_plugin_dynamicallyr7ts  {L EEr c2tj|d|yr/r4)r2r,s rload_pjrt_plugin_with_c_apir9xs T51r c,tj|Sr/)rpjrt_plugin_initializedr1s rr;r;|s % %k 22r c.tj|y)zInitializes a PJRT plugin. The plugin needs to be loaded first (through load_pjrt_plugin_dynamically or static linking) before this method is called. Args: plugin_name: the name of the PJRT plugin. N)rinitialize_pjrt_pluginr1s rr=r=sk*r c:|i}tj||||S)axCreates a PJRT C API client for a PJRT plugin. It is required that load_pjrt_plugin_dynamically is called once with the same plugin_name before this method is called. Args: plugin_name: the name of the PJRT plugin. options: extra platform-specific options. distributed_client: distributed client. Returns: A PJRT C API client for plugin_name. )rget_c_api_client)r2optionsrrs rmake_c_api_clientrAs-& _G     r chi}d|d<tjddj}tjdd}tjdd}|r|r td|}tjd d}tjd d}|d vrtd |z||d <|rt ||d<|r|dv|d<|rt |dz|d<tjdd}t t ||d<tjdd}t t ||d<|S)zYGenerates the PjRt GPU plugin options. Returns: A dictionary of plugin options. cuda platform_nameXLA_PYTHON_CLIENT_ALLOCATORdefaultXLA_CLIENT_MEM_FRACTIONXLA_PYTHON_CLIENT_MEM_FRACTIONz{XLA_CLIENT_MEM_FRACTION is specified together with XLA_PYTHON_CLIENT_MEM_FRACTION. Remove the latter one, it is deprecated.XLA_PYTHON_CLIENT_PREALLOCATE(XLA_PYTHON_CLIENT_COLLECTIVE_MEM_SIZE_MB)rFplatformbfc cuda_asynczcXLA_PYTHON_CLIENT_ALLOCATOR env var must be "default", "platform", "bfc", or "cuda_async", got "%s" allocatormemory_fraction)falseFalse0 preallocateicollective_memory_size.XLA_PYTHON_CLIENT_ABORT_COLLECTIVES_ON_FAILURErSabort_collectives_on_failure%XLA_PYTHON_CLIENT_USE_TFRT_GPU_CLIENTuse_tfrt_gpu_client)osgetenvlower ValueErrorfloatintbool)r@rOrPdeprecated_memory_fractionrTrUabortuse_trft_gpu_clients r generate_pjrt_gpu_plugin_optionsrdsV'#'/ii5yAGGI)II7+990"BB  +-6 7 #'+!&!7G (0GGGM(+,B(Cw(OG $% ))Dc J%,0U,<' () "I3O#',?(@#A'  .r a A Shape is an object defined in C++ that duck types like the following class: class Shape: '''Represents an XLA shape. A shape is either an array shape, having rank-many integer dimensions and an element type (represented by a Numpy dtype), or it is a tuple shape, having a shape for every tuple component: type shape = TupleShape of shape list | ArrayShape of { dimensions: int list; element_type: dtype } ''' @staticmethod def tuple_shape(tuple_shapes) -> Shape: "Construct a tuple shape." @staticmethod def array_shape(element_type, dimensions, minor_to_major=None) -> Shape: @staticmethod def from_pyval(pyval) -> Shape: "Returns a Shape that describes a tuple-tree of Numpy arrays." def __init__(self, str) -> Shape: "Parses a shape string." def __eq__(self, other: Shape) -> bool: def __ne__(self, other: Shape) -> bool: def __hash__(self): def __repr__(self): def is_tuple(self) -> bool: def is_array(self) -> bool: def tuple_shapes(self) -> [Shape]: def numpy_dtype(self) -> np.dtype: "Like element_type(), but returns dtype('O') for a tuple shape." def xla_element_type(self) -> PrimitiveType: def element_type(self) -> np.dtype: def dimensions(self) -> (int, int, ...): def rank(self) -> int: def with_major_to_minor_layout_if_absent(self) -> Shape: "Returns a copy with missing layouts set to major-to-minor." def to_serialized_proto(self) -> bytes: "Returns 'shape' as a serialized proto." z A ProgramShape is a C++ object that duck types like the following class. class ProgramShape: def __init__(self, parameter_shapes, result_shape): def parameter_shapes(self) -> [Shape]: def result_shape(self) -> Shape: def __repr__(self): a A DeviceAssignment is a C++ object with the following signature. def create(assignment): '''Builds a device assignment. Args: assignment: a 2D numpy array of device ordinal integers, indexed by [replica][computation_in_replica]. Returns: A device assignment. ''' def replica_count(): '''Returns the number of replicas.''' def computation_count(): '''Returns the number of computations per replica.''' cr~|j|}|jDcgc]}|d c}Scc}w)Nr)execute_sharded%disassemble_into_single_device_arraysself argumentsdeviceresultsxs rLoadedExecutable_executernXs6   +'EEG H1!A$ HH Hs 4c~|j|d}|jDcgc]}|d c}|jjdfScc}w)NT) with_tokensr)rfrg consume_token get_tokenrhs r#LoadedExecutable_execute_with_tokenrs^sY   ='BBDEqtE ''* Es AceZdZdZdZy)CustomCallTargetTraitsrN)__name__ __module__ __qualname__DEFAULTCOMMAND_BUFFER_COMPATIBLEr$r rruruks '  r ruc0eZdZ d ddZy)CustomCallHandlercyr/r$)rinamefnrL api_versiontraitss r__call__zCustomCallHandler.__call__wsr N)..) rr_rrurstrrrrLrreturnNonerwrxryrr$r rr}r}usI'*   %        r r}zdict[str, CustomCallHandler]_custom_callback_handlerz=dict[str, list[tuple[str, Any, int, CustomCallTargetTraits]]]_custom_callbackr ctj||}t5|tvrt||||||n)tj |gj ||||fdddy#1swYyxYw)aRegisters a custom call target. Args: name: bytes containing the name of the function. fn: a PyCapsule object containing the function pointer. platform: the target platform. api_version: the XLA FFI version to use. Supported versions are: 0 for the untyped FFI and 1 for the typed FFI. traits: custom call traits corresponding to XLA FFI handler traits. N)xla_platform_namesget_custom_callback_lockrr setdefaultappend)rrrLrrxla_platform_names rrrs|&),,Xx@4401 %{F!!"3R8?? [& ) s AA++A4c $tj||}t5|tvr tj d| dddy|t|<|t vr&t |D]\}}}}||||||t |=dddy#1swYyxYw)a1Registers a custom handler and use it to register existing custom calls. If a custom call handler for the platform already exist, calling this method is a no-op and it will not register a new handler. Args: platform: the target platform. handler: the function to register a custom call. zKCustom call handler for %s is already register. Will not register a new oneN)rrrrloggerdebugr)rLhandlerrrrrrs rrrs),,Xx@ .44 ll     . .3:./,,+;> g  s AA&&A/ctj||}t5|tvr tj d| dddy|t|<|t vr!t |D]\}}|||t |=dddy#1swYyxYw)a:Register a custom type id handler and use it to register existing type ids. If a custom type id handler for the platform already exist, calling this method is a no-op and it will not register a new handler. Args: platform: the target platform. handler: the function to register a custom type id. zNCustom type id handler for %s is already register. Will not register a new oneN)rrrrrrr)rLrrrcapsules rrrs),,Xx@ -33 ll     - -29-.O+*+<=-$g + , - - -s B2BB c#Ktj}tj| dtj|y#tj|wxYww)zVContext manager that overwrites and restores the current thread's execution_stream_id.N)rget_execution_stream_idset_execution_stream_id)new_idsaveds rexecution_stream_idrsH  & & (%v&(   'D  's*A!AA!AA!) TNrrvNNNNN)rClient)rH)r'rrDeviceTopology)r,rr'rrr)r2rrr`)r2rr6rrr)r2rr,rrr)r2rrr)NNN)r2rr@z_NameValueMapping | Nonerz$_xla.DistributedRuntimeClient | Nonerz*_xla.TransferServerInterfaceFactory | None)r_NameValueMappingr/) rrrrrLrrr_rrurr)rLrrr}rr)r )rrrrrLrrr)rLrrrrr)rr_)n__doc__ __future__ratexitcollections.abcr contextlibenumloggingrZ threadingtypingrrrjaxlibrr ifrt_programs_versionifrt_version_number _ifrt_versionr getLoggerrwrrr_listr^r`rrrget_topology_for_devicesr)r-r0r7r9r;r=rArd PrimitiveTypeShape ProgramShapeDeviceAssignmentDeviceCompileOptionsHostBufferSemanticsXlaComputationrMemoryArray ArrayImplLoadedExecutable Executable DeviceList OpSharding HloShardingSharding NamedShardingSingleDeviceSharding PmapSharding GSPMDSharding PjRtLayoutAutotuneCacheModernrsexecuteexecute_with_tokenIntFlagrur}r__annotations__rLockrrzrrrrrrrr register_custom_call_partitioner encode_inspect_sharding_callbackhlo_sharding_util+register_custom_call_as_batch_partitionable TracebackFramecontextmanagerrJaxRuntimeErrorXlaRuntimeErrorregistercollect_garbagearray_result_handler+batched_copy_array_to_devices_with_shardingbatched_device_putreorder_shardsbatched_block_until_ready"check_and_canonicalize_memory_kindLayoutcustom_call_targetsArrayCopySemanticsr$r rrs" #  ''""   ((      8 $CsCcE4'G!HHI '+(,  4$$88OOO&(G G"GG.F23+)-?CJN  %=H :)X""  . `   ((& $$..8$$    NN (( __ __ __  =="" 00  "" __ **I 4&I# T\\   :<6;' ( %;%C%C     #   >..-. .8( ;=7<"$$%y~~'   ,--/- -8$(#H#H #'#H#H **44, NN   ((&&$$%0044,,,$$ ::%)%L%L" ..,,r