uki^ddlZddlmZmZddlmZmZddlZddl Z ddl m Z ddl m Z ddl m Z ddlmZddlmZdd lmZdd lmZdd lmZmZdd lmZmZmZdd lmZmZm Z edZ!Gdde jDjFZ$eddddejJddde&deddde&e'ze(e)e*dfzdejVdedzdedefdefdZ,eddddejJdddedeedeeezddde&e'ze(e)e*dfzdejVdedzdedefdefdZ,e!ddddejJddddde&e'ze(e)e*dfzdejVdedzdedefdef d Z,iZ-d!Z.edd"d#e&dede'e&ze(e)e*dfzde)e(e)e*dfeffd$Z/edd"dedeedeeezde'e&ze(e)e*dfzde)e(e)e*dfeff d%Z/e!dd"de'e&ze(e)e*dfzde)e(e)e*dfeffd&Z/d'Z0ejJdfde(ed(ee)e)e*dfe1e&e&ffd)Z2d*Z3ejhe-ejj<y)+N)overloadAny)CallableSequence)api)core)dtypes) shape_poly)lax)util) auto_axes)canonicalize_sharding NamedSharding)Array ArrayLike DTypeLike)partition_list set_moduleunzip2z jax.numpyceZdZdZdZy) UnoptimizedzUnoptimized path for einsum.c&dgt|dz zS)N)rr)len)selfinputsargskwargss P/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/jax/_src/numpy/einsum.py__call__zUnoptimized.__call__'s 8s6{Q ''N)__name__ __module__ __qualname____doc__r r!rrr%s $(r!rauto)outoptimize precisionpreferred_element_type _dot_general out_sharding subscriptoperandsr(r).r*r+r,returncyNr&)r.r(r)r*r+r,r-r/s reinsumr3*s r!arraxescyr2r&) r4r5r(r)r*r+r,r-r/s rr3r36s r!c|g|}| tdt|dtr|dnd}|durdn|dur tn|} t d|D}|D chc]M} t| trt j | D]"} tj| s t| $O} } } | stj} n.tt| }tj|t } | |dd| d\}}t d |D}t#|}t%|d }|t|t&s td t)j*t,d d }|t)j.||}t1t3j4d |}|dkDr0|.t7||j8j:||||||dS|||||||Scc} } w)aYEinstein summation JAX implementation of :func:`numpy.einsum`. ``einsum`` is a powerful and generic API for computing various reductions, inner products, outer products, axis reorderings, and combinations thereof across one or more input arrays. It has a somewhat complicated overloaded API; the arguments below reflect the most common calling convention. The Examples section below demonstrates some of the alternative calling conventions. Args: subscripts: string containing axes names separated by commas. *operands: sequence of one or more arrays corresponding to the subscripts. optimize: specify how to optimize the order of computation. In JAX this defaults to ``"auto"`` which produces optimized expressions via the opt_einsum_ package. Other options are ``True`` (same as ``"optimal"``), ``False`` (unoptimized), or any string supported by ``opt_einsum``, which includes ``"optimal"``, ``"greedy"``, ``"eager"``, and others. It may also be a pre-computed path (see :func:`~jax.numpy.einsum_path`). precision: either ``None`` (default), which means the default precision for the backend, a :class:`~jax.lax.Precision` enum value (``Precision.DEFAULT``, ``Precision.HIGH`` or ``Precision.HIGHEST``). preferred_element_type: either ``None`` (default), which means the default accumulation type for the input types, or a datatype, indicating to accumulate results to and return a result with that datatype. out: unsupported by JAX _dot_general: optionally override the ``dot_general`` callable used by ``einsum``. This parameter is experimental, and may be removed without warning at any time. Returns: array containing the result of the einstein summation. See also: :func:`jax.numpy.einsum_path` Examples: The mechanics of ``einsum`` are perhaps best demonstrated by example. Here we show how to use ``einsum`` to compute a number of quantities from one or more arrays. For more discussion and examples of ``einsum``, see the documentation of :func:`numpy.einsum`. >>> M = jnp.arange(16).reshape(4, 4) >>> x = jnp.arange(4) >>> y = jnp.array([5, 4, 3, 2]) **Vector product** >>> jnp.einsum('i,i', x, y) Array(16, dtype=int32) >>> jnp.vecdot(x, y) Array(16, dtype=int32) Here are some alternative ``einsum`` calling conventions to compute the same result: >>> jnp.einsum('i,i->', x, y) # explicit form Array(16, dtype=int32) >>> jnp.einsum(x, (0,), y, (0,)) # implicit form via indices Array(16, dtype=int32) >>> jnp.einsum(x, (0,), y, (0,), ()) # explicit form via indices Array(16, dtype=int32) **Matrix product** >>> jnp.einsum('ij,j->i', M, x) # explicit form Array([14, 38, 62, 86], dtype=int32) >>> jnp.matmul(M, x) Array([14, 38, 62, 86], dtype=int32) Here are some alternative ``einsum`` calling conventions to compute the same result: >>> jnp.einsum('ij,j', M, x) # implicit form Array([14, 38, 62, 86], dtype=int32) >>> jnp.einsum(M, (0, 1), x, (1,), (0,)) # explicit form via indices Array([14, 38, 62, 86], dtype=int32) >>> jnp.einsum(M, (0, 1), x, (1,)) # implicit form via indices Array([14, 38, 62, 86], dtype=int32) **Outer product** >>> jnp.einsum("i,j->ij", x, y) Array([[ 0, 0, 0, 0], [ 5, 4, 3, 2], [10, 8, 6, 4], [15, 12, 9, 6]], dtype=int32) >>> jnp.outer(x, y) Array([[ 0, 0, 0, 0], [ 5, 4, 3, 2], [10, 8, 6, 4], [15, 12, 9, 6]], dtype=int32) Some other ways of computing outer products: >>> jnp.einsum("i,j", x, y) # implicit form Array([[ 0, 0, 0, 0], [ 5, 4, 3, 2], [10, 8, 6, 4], [15, 12, 9, 6]], dtype=int32) >>> jnp.einsum(x, (0,), y, (1,), (0, 1)) # explicit form via indices Array([[ 0, 0, 0, 0], [ 5, 4, 3, 2], [10, 8, 6, 4], [15, 12, 9, 6]], dtype=int32) >>> jnp.einsum(x, (0,), y, (1,)) # implicit form via indices Array([[ 0, 0, 0, 0], [ 5, 4, 3, 2], [10, 8, 6, 4], [15, 12, 9, 6]], dtype=int32) **1D array sum** >>> jnp.einsum("i->", x) # requires explicit form Array(6, dtype=int32) >>> jnp.einsum(x, (0,), ()) # explicit form via indices Array(6, dtype=int32) >>> jnp.sum(x) Array(6, dtype=int32) **Sum along an axis** >>> jnp.einsum("...j->...", M) # requires explicit form Array([ 6, 22, 38, 54], dtype=int32) >>> jnp.einsum(M, (..., 0), (...,)) # explicit form via indices Array([ 6, 22, 38, 54], dtype=int32) >>> M.sum(-1) Array([ 6, 22, 38, 54], dtype=int32) **Matrix transpose** >>> y = jnp.array([[1, 2, 3], ... [4, 5, 6]]) >>> jnp.einsum("ij->ji", y) # explicit form Array([[1, 4], [2, 5], [3, 6]], dtype=int32) >>> jnp.einsum("ji", y) # implicit form Array([[1, 4], [2, 5], [3, 6]], dtype=int32) >>> jnp.einsum(y, (1, 0)) # implicit form via indices Array([[1, 4], [2, 5], [3, 6]], dtype=int32) >>> jnp.einsum(y, (0, 1), (1, 0)) # explicit form via indices Array([[1, 4], [2, 5], [3, 6]], dtype=int32) >>> jnp.transpose(y) Array([[1, 4], [2, 5], [3, 6]], dtype=int32) **Matrix diagonal** >>> jnp.einsum("ii->i", M) Array([ 0, 5, 10, 15], dtype=int32) >>> jnp.diagonal(M) Array([ 0, 5, 10, 15], dtype=int32) **Matrix trace** >>> jnp.einsum("ii", M) Array(30, dtype=int32) >>> jnp.trace(M) Array(30, dtype=int32) **Tensor products** >>> x = jnp.arange(30).reshape(2, 3, 5) >>> y = jnp.arange(60).reshape(3, 4, 5) >>> jnp.einsum('ijk,jlk->il', x, y) # explicit form Array([[ 3340, 3865, 4390, 4915], [ 8290, 9940, 11590, 13240]], dtype=int32) >>> jnp.tensordot(x, y, axes=[(1, 2), (0, 2)]) Array([[ 3340, 3865, 4390, 4915], [ 8290, 9940, 11590, 13240]], dtype=int32) >>> jnp.einsum('ijk,jlk', x, y) # implicit form Array([[ 3340, 3865, 4390, 4915], [ 8290, 9940, 11590, 13240]], dtype=int32) >>> jnp.einsum(x, (0, 1, 2), y, (1, 3, 2), (0, 3)) # explicit form via indices Array([[ 3340, 3865, 4390, 4915], [ 8290, 9940, 11590, 13240]], dtype=int32) >>> jnp.einsum(x, (0, 1, 2), y, (1, 3, 2)) # implicit form via indices Array([[ 3340, 3865, 4390, 4915], [ 8290, 9940, 11590, 13240]], dtype=int32) **Chained dot products** >>> w = jnp.arange(5, 9).reshape(2, 2) >>> x = jnp.arange(6).reshape(2, 3) >>> y = jnp.arange(-2, 4).reshape(3, 2) >>> z = jnp.array([[2, 4, 6], [3, 5, 7]]) >>> jnp.einsum('ij,jk,kl,lm->im', w, x, y, z) Array([[ 481, 831, 1181], [ 651, 1125, 1599]], dtype=int32) >>> jnp.einsum(w, (0, 1), x, (1, 2), y, (2, 3), z, (3, 4)) # implicit, via indices Array([[ 481, 831, 1181], [ 651, 1125, 1599]], dtype=int32) >>> w @ x @ y @ z # direct chain of matmuls Array([[ 481, 831, 1181], [ 651, 1125, 1599]], dtype=int32) >>> jnp.linalg.multi_dot([w, x, y, z]) Array([[ 481, 831, 1181], [ 651, 1125, 1599]], dtype=int32) .. _opt_einsum: https://github.com/dgasmith/opt_einsum Nz2The 'out' argument to jnp.einsum is not supported.rToptimalFc3XK|]"}t|dr|jn|$yw) __jax_array__N)hasattrr:).0ops r zeinsum..%s/'*1_)E2##%2M's(*) einsum_calluse_blasr)c3@K|]^}}}}|t||fywr2) frozenset)r<abc_s rr>zeinsum..6s#L 1a!9Q<+Lsr3zJ`out_sharding` argument of `einsum` only supports NamedSharding instances.)r)static_argnumsinline)namer)r5r-) contractionsr*r+r,r-)NotImplementedError isinstancestrrtuplenpshaperis_constant_dimtype opt_einsum contract_pathnextiter_poly_einsum_handlersget_default_poly_einsum_handlerrrrrjit_einsum named_calllistr ensure_arraylike_tupler mesh explicit_axes) subscriptsr(r)r*r+r,r-r/spec path_typer=dnon_constant_dim_typesrXtyrNnum_contractions jit_einsumoperand_arrayss rr3r3Csv $8 $(_ R SS"8A;4!$$#t+i(eBSYa)'%''( ! 2s(;xx| 4#7#7#: 1g  ,,M d)* +B)--b2NOM( tdYH(LL|LL,&&|X>,j}&M   wwwtL*  6J33HhGH.l6 9    , ,! <93,   nlI,lL JJOs G1z/_default_poly_einsum_handler..ZsEd1gn!3Es!) collections namedtupler;rRrTrp enumerateidrWrX) r/rroxdummiesirhmapping out_dummiesrNcontract_operandss rr]r]Xs  7G*< =%4< >/07#5EQWWEEqww O)*+ >' >"+G"4 5$!QRUAX 5' 5(66J6J+|9DEAx1/EE L ((  > 5EsAC/C$C )r)recyr2r&rer)r/s r einsum_pathras ),r!cyr2r&)r4r5r)r/s rrrhs ),r!cpt|tr|rdn t}tj|g|d|iS)aEvaluates the optimal contraction path without evaluating the einsum. JAX implementation of :func:`numpy.einsum_path`. This function calls into the opt_einsum_ package, and makes use of its optimization routines. Args: subscripts: string containing axes names separated by commas. *operands: sequence of one or more arrays corresponding to the subscripts. optimize: specify how to optimize the order of computation. In JAX this defaults to ``"auto"``. Other options are ``True`` (same as ``"optimize"``), ``False`` (unoptimized), or any string supported by ``opt_einsum``, which includes ``"optimize"``,, ``"greedy"``, ``"eager"``, and others. Returns: A tuple containing the path that may be passed to :func:`~jax.numpy.einsum`, and a printable object representing this optimal path. Examples: >>> key1, key2, key3 = jax.random.split(jax.random.key(0), 3) >>> x = jax.random.randint(key1, minval=-5, maxval=5, shape=(2, 3)) >>> y = jax.random.randint(key2, minval=-5, maxval=5, shape=(3, 100)) >>> z = jax.random.randint(key3, minval=-5, maxval=5, shape=(100, 5)) >>> path, path_info = jnp.einsum_path("ij,jk,kl", x, y, z, optimize="optimal") >>> print(path) [(1, 2), (0, 1)] >>> print(path_info) Complete contraction: ij,jk,kl->il Naive scaling: 4 Optimized scaling: 3 Naive FLOP count: 9.000e+3 Optimized FLOP count: 3.060e+3 Theoretical speedup: 2.941e+0 Largest intermediate: 1.500e+1 elements -------------------------------------------------------------------------------- scaling BLAS current remaining -------------------------------------------------------------------------------- 3 GEMM kl,jk->lj ij,lj->il 3 GEMM lj,ij->il il->il Use the computed path in :func:`~jax.numpy.einsum`: >>> jnp.einsum("ij,jk,kl", x, y, z, optimize=path) Array([[-754, 324, -142, 82, 50], [ 408, -50, 87, -29, 7]], dtype=int32) .. _opt_einsum: https://github.com/dgasmith/opt_einsum r8r))rPboolrrWrXrs rrrps5j$$y+-H  ! !* Kx K( KKr!cp|jtjtj |Sr2) translaterQ maketransdictfromkeys)scharss r _removecharsrs# S]]4==#78 99r!rNc ()*+,-tj|ddi\}ntjdd}fd--fd}-fd}d} |D]\} } } t| } | j d \}}|j d }t | d k(rb|j | d }|\*tj*}| Dcgc] }||d k(s |}}||*|\}*||*||\}*nt | d k(rnt|j | \(+|\),| ()tj+,\()| +,tj()\+,tj)}tj,}| Dcgc]}||d k(r ||d k(r|}}|()|\()| Dcgc]}||d k(r ||d k(r|}}|+,|\+,|()||,z\()|+,||)z\+,t)t,z}| Dcgc] }||vs| } }t)t,z}|Dcgc] }||vs| }}t),fd|D\}}t()+,fd| Ds'Jd(jd)d+jd,dj|}t),fd| D\} }!|dj| z}"t!)|"}#t!,|"}$||$z|#z**|k(r|!| f||ff}%indi}&|+(|%|fdi|&}nH||#z|$z*| |!f||ff}%*|k7r t#*|nindi}&|(+|%|fdi|&}n t%dt *t |cxk(rt t*k(sJJt*t|k(sJ*|k7r*t'*fd|D}'t)j*||'}|j-|t)j.|d |Scc}wcc}wcc}wcc}wcc}w)Nreturn_weak_type_flagTr3FcHtj||jk7r|j}t j |t jd|j|jtk7rtj|Stj|S)Nr) r result_typerpastyper reducerSarrayradd bitwise_or)rxr5r-r+s rsumz_einsum..sums| !34? (() *a :: 288Aqww AGGtO l AD l r!c|r3|Dcgc]}|j|}}||}t||}||fScc}wr2)indexr)operandnamesuniquesrMr5rs r sum_uniquesz_einsum..sum_uniquessI,3 4Dekk$ 4d 4GT"g5'*e E>5s;c |jD]\}}|dkDs t|Dcgc] \}}||k(s |}}}tjt j d|j |} tj| |tj|d}||vr ||}|j|d} ||dd}|j|d|dz }||fScc}}w)Nrrr) itemsrvr _deltarSrprTselect full_likereplace) rrcounts keep_namesrMcountrznr5eyers r sum_repeatsz_einsum..sum_repeatss||~ 5 e '.=s C(C(ctj}tt|jDcgc]1\}}||j |d xs|dk(xs |||d3}}}t |tt|j\}} tj||djfd| DfScc}}w)Nrrrc3(K|] }| ywr2r&)r<rzrs rr>z9_einsum..filter_singleton_dims..s3PE!H3Ps) rdefinitely_equalrvmapfindrTrrarangendimr squeezejoin) rr other_shape other_nameseqrzjkeep sqez_axes keep_axess ` rfilter_singleton_dimsz&_einsum..filter_singleton_dimss  B!#k&6&6">? A17==#Q' ' K17 KbQ6K K AD A)$U7<<5H0IJIy ;;w *BGG3Pi3P,P PP As6Cz->,rrrGc3bK|]&}j|j|f(ywr2)rr<r lhs_names rhs_namess rr>z_einsum..s0$:()&/^^A%6 q8I$J$:,/c3K|]M}|vxrC|vxr=jj|jj|k(Oywr2)rTr)r<rMlhsrrhsrs rr>z_einsum..se&   Mdi/ M )//$'(CIIiood6K,LL M&sAAz-Incompatible reduction dimensions: lhs.shape=z lhs_names=z rhs.shape=z rhs_names=rc3bK|]&}j|j|f(ywr2rrs rr>z_einsum..%s0"=&'$-??1#5yq7I"J"=rr-r+zjax.numpy.einsum does not support simultaneous contraction of 3 or more operands. Typically this means you've passed an unsupported path to the einsum optimize parameter.c3@K|]}j|ywr2r)r<rMrs rr>z_einsum..Ls>5;;t$>s)r r!check_and_canonicalize_user_dtypesortedsplitrpoprtCounterrrSrTsetrallrr_get_inverse_shardingrOrRr transposeappend_convert_element_type).r/rNr*r+r,r-output_weak_typerrroperand_indicescontracted_names_seteinstrcontracted_names input_str result_names input_namesrrrMr lhs_counts rhs_counts lhs_uniques rhs_uniqueslhs_or_rhs_namesrxlhs_and_rhs_names batch_names lhs_batch rhs_batchbatch_names_strlhs_contrhs_cont deleted_namesremaining_lhs_namesremaining_rhs_namesdimension_numbersdot_out_shardingpermrrrrrrs. ` ` @@@@@@rr_r_sj#/5/A/A 0/)-0/,,$EE Q8Dm3o+V23$ll40I|//#&K ?q  _Q/0gfe""5)f#3H$fTla6GHgH"7E7;ngu#7E6<Hngu _  "X\\?3hc3(i-S)RXXc]-68nc9,S)RXXc]-68nc9&&y1j&&y1j'7Id"4(A-*T2Ba2GIkI"3 ;?nc9&6Id"4(A-*T2Ba2GIkI"3 ;?nc9#3 :#/)#;=nc9"3 :#/)#;=nc9Y#i.8%5O>N9N!OOi.3y>9 ,G15F0FQGkG#$:-8$::i &%&&: yykYK8yykYK 9 :& ,o!"=+;"==h%0@(AAm(MB(MB  336IIe , &1Iy3IJ"."6B+\: sC):I36L3!13 "558KK&1Iy3IJ'38M.lE<P) #/"6B+\: sC):I47M4"24  * ++ u:\* =c#e*o == == = u:\* ** *  >> >d gt,g OOG[m^ " "8A;0F#3 55EI*IIPGs0 P+P+P0P5' P:1P: P?P?c>tt|jkDr4|j|jjt}|jt fd|D}t |j j|S)N)rfc3FK|]}j|ywr2r)r<rMrrfs rr>z(_get_inverse_sharding..Xs!H$tL..t45Hs!) partitions)rrfupdate_normalized_spec_for_avalrRrrc)r-rr inverse_specrfs ` @rrrSs\..//&&33C 4EF'HL   $H%HH, |(($+++*N OOr!)6rttypingrrcollections.abcrrnumpyrSrWjax._srcrrr jax._src.exportr jax._src.laxr jax._src.numpyr jax._src.pjitr jax._src.sharding_implsrrjax._src.typingrrr jax._src.utilrrrexportpaths PathOptimizerr dot_generalrQrrarRrs PrecisionLiker3r[r]rrrBr_r_einsum_contract_path_DimExprr&r!rrs .&#H77<< K (*""00(  39#'/3),     Dj4c3h00     &, 3:&      39#'/3),    3- 8C=(    Dj4c3h00    &, 3:&     39#'/3),MJ MJDj4c3h00 MJ   MJ &, MJ3:&MJ MJMJd) 58,,,Sj4c3h00, 4c3h # %& , ,  58 , , 3-,8C=(,Sj4c3h00 ,  4c3h # %& , ,4:6LSj4c3h006L T%S/ "C '( 6L6Lp: d55kd55sCx)C.#!EFGd5LP.8-M-Mj))*r!