ukil fddlmZddlmZmZddlZddlmZmZm Z ddl m Z ddl mZddlmZmZmZmZeedfZej0ZeedfZeeZe d d  dd Ze d d  ddZe d d ddZeej>Gdd Zy)) annotations)MappingSequenceN)safe_zip use_cpp_classcache) xla_bridge) xla_client)are_hlo_shardings_equalget_num_ways_dim_shardedis_hlo_sharding_replicatedop_sharding_to_indices.iF)max_sizetrace_context_in_keyShardingc|j|}|jr|S|jjDcic]}||| c}Scc}wN)devices_indices_mapis_fully_addressable_internal_device_listaddressable_device_list)sharding global_shape global_mapds L/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/jax/_src/sharding.py _addressable_devices_indices_mapr!s\++L9* "" 11II K Z]  KK Ks Act|j||jt|}tjj j |jvs|jr tdt||t|j}tt|j|S)Nzidevice_indices_map doesn't work with unreduced. Please file a bug at https://github.com/jax-ml/jax/issues) shard_shape_to_xla_hlo_shardinglenxc OpShardingType UNREDUCEDsubgroup_types is_unreducedNotImplementedErrorr_device_assignmentdictr)sr hlo_shardingindicess rcommon_devices_indices_mapr.*s-- ''L(9:,mm""l&A&A&CC!  0 11 #<#&q';';#< >' hq++W5 66c|jt|}t|r|S|jr|St |\}}t|t|k(sJt|t|fg}t t ||D]L\}\}} t||\} } | dk7rtd|d|d|d|d|d|d |j| Nt|S#t$rtwxYw) Nrz Sharding z implies that array axis z is partitioned z" times, but the dimension size is z (full shape: z , per-dimension tiling factors: z should evenly divide the shape)) r r!r r'r enumeraterdivmod TypeErrorr( ValueErrorappendtuple) selfrr, partitions_outdimr+pquotient remainders r_common_shard_shaper?9s.**3|+<=, -  *<8-*a ZC - -SJ\AR/SS - #x jAB kc6Aq "1aLh A~  dV4SE9I C1!5&(++5,7   JJx  s   s C++C;c>eZdZdZeddZeddZeddZeddZeddZ ddZ eddZ edd Z dd Z dd Zedd Zej"dd Z ddZddZeddZej"ddZd dZd!dZy)"rzADescribes how a :class:`jax.Array` is laid out across devices. ctd)zThe set of devices that this :class:`Sharding` spans. In multi-controller JAX, the set of devices is global, i.e., includes non-addressable devices from other processes. (Subclasses should implement this method.r(r7s r device_setzSharding.device_setZ H IIr/ctd)zIs this sharding fully replicated? A sharding is fully replicated if each device has a complete copy of the entire data. rBrCrDs ris_fully_replicatedzSharding.is_fully_replicatedcrFr/ctd)zIs this sharding fully addressable? A sharding is fully addressable if the current process can address all of the devices named in the :class:`Sharding`. ``is_fully_addressable`` is equivalent to "is_local" in multi-process JAX. rBrCrDs rrzSharding.is_fully_addressablels H IIr/ctd)z-Number of devices that the sharding contains.rBrCrDs r num_deviceszSharding.num_devicesv H IIr/ctd)z(Returns the memory kind of the sharding.rBrCrDs r memory_kindzSharding.memory_kind{rLr/ctd)z?Returns a new Sharding instance with the specified memory kind.z'Subclasses should implement this methodrC)r7kinds rwith_memory_kindzSharding.with_memory_kinds G HHr/ctdNrBrCrDs rr)zSharding._device_assignment H IIr/ctdrSrCrDs rrzSharding._internal_device_listrTr/ctdrSrCr7num_dimensionss rr zSharding._to_xla_hlo_sharding H IIr/ctdrSrCrWs r_to_sdy_shardingzSharding._to_sdy_shardingrYr/cy)NTrDs r _is_concretezSharding._is_concretes r/ctjdk(r |jS|jDchc]+}|j|jjk(r|-c}Scc}w)zdThe set of devices in the :class:`Sharding` that are addressable by the current process. )xb process_countrE process_indexclient)r7rs raddressable_deviceszSharding.addressable_devicess[  Q __ rs#-88%)88 c3h  eSjv&51KK&+K0MK2K51 7 7$ 7)? 72 75124r{{AAAr/