uki>DddlmZmZddlZddlmZddlmZddlmZddlm Z ddlm Z ddlm Z dd lm Z dd lmZdd lmZdd lmZdd lmZddlmZddlmZddlmZe j.eej.eej2ecZZej8ecZZe j>GddZ dZ!e jDdedefde jFde$de$dee%deejLf dZ'dejPfdZ)dejPfdZ*dejVd e,dejPd!ede-ejPee%fffd"Z.dejPfd#Z/dee%d$ej`fd%Z1ejdd&Z3d'e3_4e3jke)e3jme*ejne3ejpe)d'(e.ejre3<e/ejte3<e1ejve3<d)ejxd*ejdd+ejdd,ejxfd-Z=ejdd.Z>d'e>_4e>jke)e>jme*y)/)CallableSequenceN)Any)api_util)core)custom_api_util)errors) linear_util)source_info_util)traceback_util) tree_util)util)ad)batching)mlir) partial_evalceZdZUdZedefed<eeed<edefdzed<dddedefdeefd Z e jZ dedefd edeffd Z ejd Zy) custom_dceaCustomize the DCE behavior of a JAX-transformable function. JAX uses dead code elimination (DCE) to remove unused computations from a JAX program. This typically works transparently when the program is completely specified by known JAX operations, but opaque kernels like calls to :py:func:`~jax.experimental.pallas.pallas_call` or :py:func:`~jax.ffi.ffi_call`, for example, may cause problems. In JAX, DCE is performed when a function is staged out using :py:func:`jax.jit`, so it won't be applied when running JAX in eager mode. Similarly, the ``custom_dce`` decorator requires that both the decorated function and the custom DCE rule be compatible with :py:func:`~jax.jit`. This decorator allows users to customize the DCE behavior of a function by defining a custom DCE rule. For a ``custom_dce`` wrapped function ``f(*args)``, the signature of the DCE rule is ``dce_rule(used_outs, *args)`` where ``used_outs`` is a Pytree with the same structure as the output of ``f``, and each leaf is is a ``bool`` indicating which outputs should be computed. The remaining arguments ``*args`` are the original arguments to ``f``. The rule ``dce_rule`` should return a Pytree with the same structure as the original output of ``f``, but any unused outputs can be replaced with ``None``. For example:: >>> @jax.experimental.custom_dce.custom_dce ... def f(x, y): ... return jnp.sin(x) * y, x * jnp.sin(y) ... >>> @f.def_dce ... def f_dce_rule(used_outs, x, y): ... return ( ... jnp.sin(x) * y if used_outs[0] else None, ... x * jnp.sin(y) if used_outs[1] else None, ... ) In this example, ``used_outs`` is a ``tuple`` with two ``bool`` values, indicating which outputs are required. The DCE rule only computes the required outputs, replacing the unused outputs with ``None``. If the ``static_argnums`` argument is provided to ``custom_dce``, the indicated arguments are treated as static when the function is traced, and they will be moved to the front when calling the DCE rule. For example, if ``fun`` takes 2 arguments ``fun(x, y)``, and ``static_argnums`` is ``(1,)``, then the DCE rule will be called as ``dce_rule(y, used_outs, x)``. .funstatic_argnumsNdce_rulercZtj||||_||_d|_yN) functoolsupdate_wrapperrrr)selfrrs N/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/jax/_src/custom_dce.py__init__zcustom_dce.__init__]s*T3'DH(DDMreturnc||_|S)a2Define a custom DCE rule for this function. Args: dce_rule: A function that takes (a) any arguments indicated as static using ``static_argnums``, (b) a Pytree of ``bool`` values (``used_outs``) indicating which outputs should be computed, and (c) the rest of the (non-static) arguments to the original function. The rule should return a Pytree with with the same structure as the output of the original function, but any unused outputs (as indicated by ``used_outs``) can be replaced with ``None``. )r)rrs rdef_dcezcustom_dce.def_dcegsDM Or!c tjjjt ddtjjt j dj|ij}t j dj|ij} t jj||}jrtj}|D]}t||tt|Dcgc] }||vs| }}t jt!j"j|||d \} } jDcgc]}|| } }t j$t!j"j|| nDt!j"j|} t!j"j||} t'j(| \} t j*| \} | Dcgc]}t-j.|c}t0j2d t4d t6t,j8t:t4fffd }t1j<| \}}}t1j>t1j@|}|jBtEjFg|| t|||d}t'jH|S#t$r!}td|jd||d}~wwxYwcc}wcc}wcc}w)Nz,No DCE rule defined for custom_dce function z using def_dce.rrcustom_dce_rulez9The input arguments to the custom_dce-decorated function zT could not be resolved to positional-only arguments. Binding failed with the error:  debug_infoF)require_static_args_hashable used_outsr"c jD]}|s|jt |  \}}jJt j | \}}}t |tt}t j|dgt|jz\}}t |tt}tj|||fS)NT)storesresetflatten_dce_rulerpetrace_to_jaxpr_dynamicswap_primitives custom_dce_p dce_sential_p dce_jaxprlenoutvarsr ClosedJaxpr)r*store flat_rule rule_out_treer4_ dce_constsused_insrfun_namein_avalsin_tree out_avalsout_tree rule_namers rdce_jaxpr_thunkz,custom_dce.__call__..dce_jaxpr_thunks??%  ++-"2      * "i]] && &!#!:!: X"iJ")\=IiLL dVc)"3"344i")]LIi   i 4h >>r! num_consts fun_jaxprrD)%rr>rrAttributeErrorrr(rresolve_kwargs TypeError func_namesetcheck_for_tracersranger5argnums_partiallu wrap_initprepend_static_argsr tree_flattenflatten_fun_nokwargsrget_avalr/_memoizebooltupler7rr0 close_jaxprconvert_constvars_jaxprrAr2bindtree_unflatten)rargskwargsdebug debug_ruleeri dyn_argnumsrdyn_args static_args args_flatflat_funxrDjaxprr;consts closed_callout_flatrr>r?r@rArBrCs` @@@@@@@r__call__zcustom_dce.__call__ys}}TXX&H }}  8 C   dmm,I    dhh $b/3/B/B DE$$%6 %)2484G4GIJ  $ $TXXtV rCr*rAc|tj||} tj||} || g| } t| tr5t | t tj |k(r t | } ttj|tg|jz} tjtj| d\} }gfd}tj| \}} tj|| | dg}t!| ||D]\}}}}|s |ur9td|d|d tj"|d |j%d t'j(|t'j*|x}sMtd|d|d|dtj"|d|j%d |j%d |j-||j/||S#t$r td|d|d|d|d |d |d dwxYw)Nrcttj|d}||jg|z|S|j|g|z|S)Nr)r5r rSextend)rhd num_leavesrlsentinels rappendz flatten_dce_rule..appends_Y++A.q12JyQ]ooxj:-. H ooqcJ&' Hr!c |duSrr)rhs rz"flatten_dce_rule..s T r!)is_leafzCustom DCE rule z for function za must produce an output with the same container (pytree) structure as the output of the function z'. But, the DCE rule returned structure z, while z returned .zy must produce values for all of the required outputs (as specified by the used_outs argument to the rule). But, at outputzC, the DCE rule returned None instead of an output with shape/dtype zR must produce an output with the same shapes/dtypes as the output of the function z. But, at outputz3, the DCE rule returned an output with shape/dtype z was expected.)r r\rylistr5treedef_childrenrXobjectrrunzip2tree_flatten_with_pathrStree_map ValueErrorrJzipkeystr str_shortr typematchrUrr8)r~r8r>rCr*r@rBrArf py_used_outspy_argspy_outdummykeypathsr;r rule_treeresultskpusedavalvalaval_rlrs @@rr.r.sk))(I>,  $ $Wi 8' \ $G $&#f+  *2#6]FX(  " "8fhZ(:M:M-M N% I<>+;*>$s); ; <  YK~hZ@Q J&y'7'7';&<=1161B0C8 ^^  n  .  NN3'*++i .?   9+^H:> z!H +XhZz(1 >   s H)H;rGc2tj||Sr)r jaxpr_as_funrGr]r;s rcustom_dce_implr2s %  9 %t ,,r!c4~|j|jfSr)rAeffectsrs rcustom_dce_abstract_evalr6s   i// //r! axis_datarFrDc   |Dcgc]}|tjuc} t ||Dcgc]!\}}}|rtj||dn|#}}}}tj| d\} t j dtdttjttff fd } tj|| | d} Dcgc]}|rdntj} }| | fScc}wcc}}}wcc}w)NrFr*r"c . |\}}t| Dcgc] \}}|s | }}}tj|t| dDcgc] \}}|s | c}}|\}}tdt||DsJ||fScc}}wcc}}w)Nc3,K|] \}}||k(ywrr).0abs r zGcustom_dce_batching..batched_dce_jaxpr_thunk..`sI$!QqAvIs)rr batch_jaxprall) r*r4r=rrused_out_batcheddce_jaxpr_batcheddce_out_batchedrrD in_batchedrF out_batcheds rbatched_dce_jaxpr_thunkz4custom_dce_batching..batched_dce_jaxpr_thunkMs*95Ix),Y )DMgdAMM)1)=)=h :;(?@IwtQDI *& I#o7G"HI II I h &&N Js B B  BBrE)r not_mappedrmoveaxisrr/rVrWrXrr7rr2r[)rr]dimsrFrGrDrrrhbatched_fun_jaxprrrlout_dimsrrs` ` ` @@rcustom_dce_batchingr;s)7;;,,,;*T40   !Q%&h1a 1, $ $,#7#7J$ [;;'' T  x~ -.'''*   !- ( 8C C!1a(--- C( C 8 M< HDsC?&DD c |Dcgc]}t|tj }}t||Dcgc] \}}|s | }}}tj||d\}}t j jtjt j||jjg||} tj| t|g\} } t!| } t| |D cgc]1\} }|r t#| ntjj%| 3} } }| | fScc}wcc}}wcc}} w)NFr')ryrZeror jvp_jaxprrcall_pr[rPrQrrir(r split_listr5iternextfrom_primal_value)primalstangentsrGr;tin_nznzrout_nzout out_primals out_tangentsout_tangents_iterps rcustom_dce_jvprms./7 8!z!RWW% % 8% 8 1 8EBRa 8( 8ll9eU;)V ll4$$Y/'oo88: .new_dce_jaxpr_thunks = H_$$ 3y>C $667M .}=I|!00<HHl8}  l ""r!rrE)rr/ has_effectsr5invarsrparamsrVrWrXrr7rrrrr6dict new_jaxpr_eqnr2r source_infoctx)r*rrFrr;rrvr6 new_paramsnew_eqnrDrir=s` @@@rr&r&s Ys 3 7S_ $d **^ 6C O #S ((zz,'*JJ01/#Y//%;; # # T  x~ -. # #oocjj:,7)!V 62 ;'$dA ;& ;!)S[[9 B74TQ B' B jj) *      mm oo gg ' 3::l+ +h 6 ??# < Bs F0F00 F6;F6custom_dce_callT)multiple_resultsrioldnewr"cg}|jD]C}|j|ur"|j|j|3|j|E|j|S)N) primitive)eqns)rrrreplace)rirrnew_eqnsrs rr1r1s^( ZZc }}oockkCk01ooc  H %%r! dce_sential)?collections.abcrrrtypingrjax._srcrrrr r rPr r r rjax._src.interpretersrrrrr/register_exclusion__file__safe_mapmap unsafe_mapsafe_zipr unsafe_zipregister_custom_decorator_typerrMtransformation_with_aux2StorestrrW AbstractValuer.r7rrAxisDatarsrXrrJaxprEqnr& Primitiver2rdef_impldef_effectful_abstract_evalregister_lowering lower_funfancy_primitive_batchersprimitive_jvps dce_rulesJaxprr1r3rr!rrs/$&%#$*&4###H-!!!(+--Z--Z//k:k:0k:\ . GSG 88GG G ~ G**+GGT-d&6&6-0t/?/?0 /  /  /  /c5)9)98D>)I#JJK/d#D4D4D#2,@x~,@DMM,@^t~~/0 $  o& (()AB.$..4H3F!!,/"0,, \ & :: & NN &15 & ZZ &}- !%  ' ))*BCr!