uki+UdZddlmZddlmZmZddlZddlZddlZddl m Z m Z ddl m Z ddl mZddl mZdd l mZdd l mZdd l mZdd l mZdd l mZddl mZddl mZddl mZddl mZddlmZddlmZddlmZddlmZ ddl!m"Z"ddl#m$Z$ddl%m&Z&m'Z'm(Z(m)Z)ddl*m+Z+m,Z,ddl-Z.ej^e0Z1ejddZ3de3_4ejjjme3ejne8cZ8Z9ejte;cZ;Z<ejzdGddZ> dDdZ?e3jejeje3e3j dDd ZDd!ZEeEeje3<d"ZGeGeje3<ejeje3eje3<dEd#ZK dFd$ZL dGd%ZMeje3eMd&'d(ZOde,dd) dHd*ZPejdd+ZQdeQ_4ejjjmeQGd,d-ejZSGd.d/ejZTeSZUeTZVejjeSejjeTejjeSejjeTejjeTejjeT dId0Z\eQjejejeQeQj dId1Z^d2Z_e_ejeQ<d3Z`e`ejeQ<d4ZaeaejeQ<d5ZbejeQebd&'dd&d6 dJd7ZcdKd8Zddd9 dLd:Zedd9 dMd;ZfdNd<ZgiZhd=eid><dOd?Zjejehej<d@ehej<dd9 dPdAZmdd&ddB dQdCZny)RzModule for JAX callbacks.) annotations)CallableSequenceN)Anycast)api)config)core)dispatch)dtypes)effects)ffi) pickle_util)sharding_impls) tree_util)util) xla_bridge)ad)batching)mlir) xla_client)ir)hlo)SdyArray SdyArrayListSdyDimSingleDeviceSharding)Array DeprecatedArg pure_callbackT)frozenc.eZdZUdZded<ded<ddZy) _FlatCallbacka/A Python function callable with flat arguments and results. An instance of this class is used as a parameter for the callback primitives. We prefer it to an anonymous flattened function because it produces equal objects when we call the same Python function with the same argument structure. Callable[..., Any] callback_funcztree_util.PyTreeDefin_treectj|j|\}}tj|j|i|SN)rtree_unflattenr& tree_leavesr%)self flat_argsargskwargss L/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/jax/_src/callback.py__call__z_FlatCallback.__call__Ds@++DLL)DLD&  !3!3!3T!DV!D EEN)r,rreturnzSequence[Array])__name__ __module__ __qualname____doc____annotations__r0r1r/r#r#8s$# Fr1r#c~~~ tjd^}}tj||}t j |5 tjtj||cdddS#t$r}td|d}~wwxYw#t$rtjdwxYw#1swYyxYw)Ncpubackendzjax.pure_callback failed to find a local CPU device to place the inputs on. Make sure "cpu" is listed in --jax_platforms or the JAX_PLATFORMS environment variable.zjax.pure_callback failedxb local_devices RuntimeErrorr device_putr default_devicertree_mapnpasarray BaseExceptionlogger exception) result_avalscallbacksharding vmap_methodr- cpu_device_es r/pure_callback_implrPIs \ %%e4NJ j )$ Z(     HdO <     /       12    5A:B: &B: B BB B77B::Cc~~~~|Sr(r8)rJrIrKrLavalss r/pure_callback_abstract_evalrTfs Xx r1c~~td)NzgPure callbacks do not support JVP. Please use `jax.custom_jvp` to use callbacks while taking gradients. ValueErrorr-r.s r/pure_callback_jvp_rulerYr FM NNr1c~~td)NzmPure callbacks do not support transpose. Please use `jax.custom_vjp` to use callbacks while taking gradients.rVrXs r/pure_callback_transpose_ruler\|rZr1c dg}|r4|Dcgc])}t|tjs|j+}}t |Dcgc]}t dt gdg|zd!c}Scc}wcc}w)zGReturns an SdyArrayList with `max(1, len(avals))` replicated shardings.rr8F)axesis_open mesh_shape dim_shardingslogical_device_ids) isinstancer ShapedArrayndimrrr)rSndimsxrfs r/!_get_sdy_array_list_for_callbacksris #% " FjD4D4D&EQVV FE F .3 4&*R784?!4 55 G4sA.A.$A3c,t|tjr|jt |j j k7r td| tdtjjr t|}|Stj}tjjj|_|St|tj"r|qt|t$stdt!|t't)|j*}|j,}| t/d |j1|}nd}tjjr3t7d t9|}t;|t=d g|f gz}|Stj}tjjj>|_d g|_ |g|_!|Sy#t2$r}t5d|d|d|d}~wwxYw) Nz~callbacks are only supported in spmd computations when all mesh axes are partitioned manually (no partial automatic sharding).zEcallbacks do not support specifying sharding inside spmd computationsz:pure_callback only supports SingleDeviceSharding, but got z9Please file a bug at https://github.com/jax-ml/jax/issuesz6Sharding provided to pure_callback specifies a device z' that is not in the device assignment ()rr8r`)"rdrSPMDAxisContext manual_axes frozensetmesh axis_namesNotImplementedErrorr use_shardy_partitionervaluerixc OpShardingTypeMANUALtypeShardingContextrnextiter device_setdevice_assignmentAssertionErrorindex IndexErrorrWmaxlenrrMAXIMALtile_assignment_dimensionstile_assignment_devices) axis_contextrK avals_out op_shardingdevicer~ device_indexrOnum_sdy_shardingss r/_callback_op_shardingrs# n<<=9\->->-I-I#JJ  L    $$**5i@k MMOk++22k  n<<= "6 7! X !  D,,-.f&88  " GI I.(..v6 l $$**aY0 !2 ". 263"34k  MMOk++33k01sk,-9Nk)  ;. x"#1 &'-. ..sG22 H;HHc fd}t|jj||j}t ||dt ||j |jdd| \}}}|S)Nc2tt|ddS)N)rJrK)tuplerP)r,rJparamss r/ _callbackz)pure_callback_lowering.._callbacks+      r1Fhas_side_effect returns_tokenrK)rmodule_contextrremit_python_callbacklistavals_in) ctxrJrKr-rrrresultrNs ` ` r/pure_callback_loweringrsj& %%x@+%   4j ll mm ,&!Q -r1F) cacheablectj|j}tj||k7r t dy)NzQresult_shape_dtypes cannot specify 64-bit types when `jax_enable_x64` is disabled)rDdtyper canonicalize_dtyperW) shape_dtypedts r/_check_shape_dtypers@ xx !!"" r"b( [ ]])r1)rK vectorizedrLct|ts tdgd}||vrtd|d|tj||f\}} tj t |tj d|} tj| \} } tj|t|| t| ||d} tj| | S)aCalls a pure Python callback. Works under :func:`jit`/:func:`~vmap`/etc. For more explanation, see `External Callbacks`_. ``pure_callback`` enables calling a Python function in JIT-ed JAX functions. The input ``callback`` will be passed JAX arrays placed on a local CPU, and it should also return JAX arrays on CPU. The callback is treated as functionally pure, meaning it has no side-effects and its output value depends only on its argument values. As a consequence, it is safe to be called multiple times (e.g. when transformed by :func:`~vmap` or :func:`~pmap`), or not to be called at all when e.g. the output of a `jit`-decorated function has no data dependence on its value. Pure callbacks may also be reordered if data-dependence allows. .. warning:: In the context of JAX transformations, Python exceptions should be considered side-effects: this means that intentionally raising an error within a `pure_callback` breaks the API contract, and the behavior of the resulting program is undefined. When `vmap`-ed the behavior will depend on the value of the ``vmap_method``. * Calling :func:`~jax.vmap` on a callback without an explicit ``vmap_method`` raises a ``NotImplementedError``. * ``vmap_method="sequential"`` uses :func:`~jax.lax.map` to loop over the batched arguments, calling ``callback`` once for each batch element. * ``vmap_method="sequential_unrolled"`` is like ``sequential``, but the loop is unrolled. * ``vmap_method="expand_dims"`` calls ``callback`` with new axes of size ``1`` added as the leading dimension unbatched inputs. * ``vmap_method="broadcast_all"`` behaves like ``expand_dims``, but the inputs are tiled to the expected batched shape. If necessary, the legacy behavior provided by the removed ``vectorized=True`` argument can be recovered using ``vmap_method="legacy_vectorized"``. The current default behavior is to use ``vmap_method="sequential"`` when not specified, but this behavior is deprecated, and in the future, the default will be to raise a ``NotImplementedError`` unless ``vmap_method`` is explicitly specified. Args: callback: function to execute on the host. The callback is assumed to be a pure function (i.e. one without side-effects): if an impure function is passed, it may behave in unexpected ways, particularly under transformation. The callable will be passed PyTrees of arrays as arguments, and should return a PyTree of arrays that matches ``result_shape_dtypes``. result_shape_dtypes: pytree whose leaves have ``shape`` and ``dtype`` attributes, whose structure matches the expected output of the callback function at runtime. :class:`jax.ShapeDtypeStruct` is often used to define leaf values. *args: arguments to be passed to the callback function sharding: optional sharding that specifies the device from which the callback should be invoked. vmap_method: string specifying how the callback transforms under :func:`~jax.vmap` as described above. **kwargs: keyword arguments to be passed to the callback function Returns: result: a pytree of :class:`jax.Array` objects whose structure matches that of ``result_shape_dtypes``. See Also: - :func:`jax.experimental.io_callback`: callback designed for impure functions. - :func:`jax.debug.callback`: callback designed for general-purpose debugging. - :func:`jax.debug.print`: callback designed for printing. Examples: The behavior of ``pure_callback`` under :func:`~jax.vmap` is controlled by the ``vmap_method`` argument as described above. It is useful to consider some explicit examples that demonstrate the semantics. For example, consider the following function: >>> def callback(x, y): ... print(jnp.shape(x), jnp.shape(y)) ... return x + y >>> def fun(x, y, *, vmap_method): ... shape = jnp.broadcast_shapes(jnp.shape(x), jnp.shape(y)) ... dtype = jnp.result_type(x, y) ... out_type = jax.ShapeDtypeStruct(shape, dtype) ... return jax.pure_callback(callback, out_type, x, y, ... vmap_method=vmap_method) Calling this with ``vmap_method="expand_dims"`` adds a new axis of size ``1`` to ``y``: >>> from functools import partial >>> x = jnp.arange(4) >>> y = 1.0 >>> jax.vmap(partial(fun, vmap_method="expand_dims"), in_axes=(0, None))(x, y) (4,) (1,) Array([1., 2., 3., 4.], dtype=float32) Whereas, ``vmap_method="broadcast_all"`` adds an axis of size ``4`` to ``y``: >>> jax.vmap(partial(fun, vmap_method="broadcast_all"), ... in_axes=(0, None))(x, y) (4,) (4,) Array([1., 2., 3., 4.], dtype=float32) .. _External Callbacks: https://docs.jax.dev/en/latest/external-callbacks.html zdThe 'vectorized' argument of jax.pure_callback was removed in JAX v0.6.0. Use 'vmap_method' instead.) sequentialsequential_unrolled expand_dims broadcast_alllegacy_vectorizedNz.vmap_method must be on of the allowed methods z , but got: cVtj|j|jSr(r reshaperrhs r/zpure_callback..s  !''2r1)rJrIrKrL) rdrrWr tree_flattenrCrpure_callback_pbindr#rr))rJresult_shape_dtypesrKrrLr-r.allowed_vmap_methodsr,r&rIflat_result_avalsout_treeout_flats r/r r sf J .  - ..F,,  89M8NO= " ##!--tVn=)W ')<=##24GI, ) 6 6| DX  ! !Xw/*+ (  ! !(H 55r1 io_callbackceZdZdZy)IOEffectcy)NIOr8rNs r/rzIOEffect.r1Nr3r4r5__str__r8r1r/rrs 'r1rceZdZdZy)OrderedIOEffectcy)N OrderedIOr8rs r/rzOrderedIOEffect.rr1Nrr8r1r/rrs !'r1rc~~~ tjd^}}tj||}t j |5 tjtj||cdddS#t$r}td|d}~wwxYw#t$rtjdwxYw#1swYyxYw)Nr:r;zjax.io_callback failed to find a local CPU device to place the inputs on. Make sure "cpu" is listed in --jax_platforms or the JAX_PLATFORMS environment variable.zjax.io_callback failedr=)rIrJrKorderedr-rMrNrOs r/io_callback_implrsHg %%e4NJ j )$ Z(     HdO <     /       /0    rQc.~~~|rtnt}||hfSr()_OrderedIOEffect _IOEffect)rJrIrKrrSeffects r/io_callback_abstract_evalrs" Xx& I& x r1c~~td)Nz IO callbacks do not support JVP.rVrXs r/io_callback_jvp_rulers F566r1c~~td)Nz&IO callbacks do not support transpose.rVrXs r/io_callback_transpose_rulers F;<._batch_funs7   j., GF   v8+7 HHr1)r) jax._src.lax.control_flow.loopsrrWr not_mappedzipmoveaxisrpartition_listr)r-dimsrJrIrKrlax_mapdargdimnew_argsrrout_valsrrs ``` @@r/io_callback_batching_rulers= 9 ::6:;,,,;*=@t_N19cH///cS!,-N(N!%!4!4Z!J.,HHZ .( 4#h-' ''<Ns B;5Cc fd}t|jj||j}r|jj t }t|||t||j|jdd| \} }} |jtjt |i| St||dt||j|jdd| \} } } | S)Nc 4tt|ddS)N)rJrKr)rr)r,rJrrs r/rz'io_callback_lowering.._callbacks.       r1TrF) rrrr tokens_ingetrrrrset_tokens_outrTokenSet) rrJrKrr-rrrtokenrrNs ` ` ` r/io_callback_loweringrs & %%x@+ MM  . /E+   T    FE1t}}&6%>?@ -(   T    LFAq -r1)rKrc6tj||f\}}tjt|tj|\}} t d|} t j |t||t| ||d} tj| | S)a"Calls an impure Python callback. For more explanation, see `External Callbacks`_. Args: callback: function to execute on the host. It is assumed to be an impure function. If ``callback`` is pure, using :func:`jax.pure_callback` instead may lead to more efficient execution. result_shape_dtypes: pytree whose leaves have ``shape`` and ``dtype`` attributes, whose structure matches the expected output of the callback function at runtime. :class:`jax.ShapeDtypeStruct` is often used to define leaf values. *args: arguments to be passed to the callback function sharding: optional sharding that specifies the device from which the callback should be invoked. ordered: boolean specifying whether sequential calls to callback must be ordered. **kwargs: keyword arguments to be passed to the callback function Returns: result: a pytree of :class:`jax.Array` objects whose structure matches that of ``result_shape_dtypes``. See Also: - :func:`jax.pure_callback`: callback designed for pure functions. - :func:`jax.debug.callback`: callback designed for general-purpose debugging. - :func:`jax.debug.print`: callback designed for printing. .. _External Callbacks: https://docs.jax.dev/en/latest/notebooks/external_callbacks.html cVtj|j|jSr(rrs r/rzio_callback..CsD$4$4QWWagg$Fr1)rJrIrKr) rrrCrrrrr#rr)) rJrrKrr-r.r,r&flat_shape_dtypesrrrs r/rrsH!--tVn=)W ')<= ) 6 67J KXF+-   Xw/*+ (  ! !(H 55r1c&td|DS)Nc3&K|] }|dk( yw)rNr8).0rs r/ z!is_empty_shape..Ps Q!V s)any)ss r/is_empty_shaperOs Q  r1rKc tjj|tj}tj |g||t jjd}t jjtt jjt|t jjt||jd<|tjjr[t!|t"sJt%|j&dk\sJt#t)dg|j&dj*g}tj,|||j.S) NTis_host_transfer_xla_host_transfer_handler_name_xla_host_transfer_rendezvousmhlo.frontend_attributesrlr8rr`)r ChannelHandlerrSEND_TO_HOST_TYPESendOprBoolAttrDictAttrdict StringAttrstr attributesr rsrtrdrr shardingsrrc set_shardingr)channelroperandnamerKchannel_handlesend_ops r/ send_to_hostrSs.$$(($2H2HI. JJy%)+)> @'35;;?? *,--*;*;CI*F(* (9(9#d)(DF4G'/0 $$** , // / ## $ )) ) 2!)!3!3A!6!I!I KLMh gx( r1c tjj|tj}tj tj |tjjg||tjjd}tjjttjjt|tjjt||jd<|tj j"rit%|t&sJt)|j*dk\sJt'|j*dt-dg|j*dj.g}tj0|||j2\}}||fS) NTrrrrlrr8r`)rrrrRECV_FROM_HOST_TYPERecvOpaval_to_ir_type TokenTyperrrrrr r r rsrtrdrrr rrcr results)r rout_avalrrKrrecv_oprs r/receive_from_hostrssj$$(($2J2JK. JJ,,X6 ))+-.3^)+)> @'46;;?? *,--*;*;CI*F(* (9(9#d)(DF4G'/0 $$** , // / ## $ )) )   Q  2!)!3!3A!6!I!I KLMh  gx(//-&% r1c tt||S#t$r}tdt||d}~wwxYw)NzNo xla_shape_handler for type: )_xla_shape_handlersryKeyError TypeError)avalerrs r/_aval_to_xla_shaper"sGM tDz *4 00 M 5d4j\B CLMs ?:?z9dict[type[core.AbstractValue], Callable[[Any], xc.Shape]]rctj|}|jtjk(rt jdn |j}t jj||jS)Nbool) r physical_avalrr float0rDruShape array_shaper)r rs r/_make_array_shaper)sP  D !$"jjFMM9"((6 tzz%   eTZZ 00r1c>tjjSr()rur' token_shapers r/rrsBHH4H4H4Jr1c |xstj}|} g} |s| fd} |jj} t j dt j}tjt jdt j}g|t|}t| |||j| }| j| nL|D]G}|jj}t||||j| }| j|Ig}g}| r|s| fd} t j dt j}t|g}|jj}t||||j| \}}|j|n||D]w}|jj}t!|tj sJt||||j| \}}|j||j|y|j#| ||| |t$j&}|jj)|||fS)Nc~Sr(r8)r-callback_without_argss r/_wrapped_callbackz4_emit_tpu_python_callback.._wrapped_callbacks  " $$r1)rlrlrc|y)N)gr8)r-callback_without_resultss r/r/z4_emit_tpu_python_callback.._wrapped_callbacks% r1r8)r create_tokenr new_channelr rerDfloat32r ir_constantzerosr"rr3appendrrd,make_python_callback_from_host_send_and_recvrdumpsadd_host_callback)r<rrJroperands operand_avalsoperand_shapesrI result_shapesrrKr/ send_channels send_channeldummy_send_avaldummy_send_valrr  recv_channelsoutputsdummy_recv_avalrN result_avalout ifrt_callbackr.r1s @@r/_emit_tpu_python_callbackrIsF  %3##%%-  .%%%113L&&tRZZ8O%%bhhq"**&=>NK~K'9/'JKN unh>O>O"* ,E&$""..0g7E7H4E4E$,.e7# $ - '< 1&&r2::6O'89M  ,,.G ):):XOHE1!#$ ""..0g  T%5%5 66 6$We[%-%6%6KjeS nnS7# $FF [&&(-&&}5 %r1)r partitionedrKc t|jjdkDr td|jjddvrt dd|r dvrtddr t d t t j|jj} D cgc] } t| } } |D cgc] } t| } } fd }d k(rtjt| D cgc]\} }t| js| |f c}} \}}t| |||||| |||| \}}t!|}Dcgc]`}t|jr>t#j$t'j(|j|j* n t-|b}}||dfSdvrdnd}|rdnd}d|d|d}|r|fd}|g|}t.j0j2rN| Lt|j4dkDr4t7| t8r$t9t;dgdg| j<} t?j@|tBjDg|jFtBjDg|j4}|}|jjI|t'jJt|jjLdz }tOjP|||g|dt'jJ|i}| t#jR|| |jT}|r|^}}|||fScc} wcc} wcc}} wcc}w)aEmits MLIR that calls back to a provided Python function. Args: ctx: The lowering context. callback: The Python callback function. token: The token to use for the callback. operands: The operands to the callback. operand_avals: The abstract values of the operands. result_avals: The abstract values of the results. has_side_effect: Whether the callback has side effects. returns_token: Whether the callback should return a token. partitioned: If True, then `callback` is called on local shards only. If False, then `callback` is called on all shards. sharding: The sharding of the callback. Returns: A tuple of MLIR result values, a new token (if any), and the host callback object. rlz+multi-platform lowering for python_callbackr>r:tpucudarocmz&`EmitPythonCallback` not supported on z backend.>r:rMrNz(Partitioned callback not implemented on z6Partitioned callback not supported with return values.c H|}t|tk7r-tdjtt|td|D}t t |D]\}\}}|j |j k7r(td|d|j d|j |j|jk7sdtd|d|jd|jdk(rtdt |D}|S|S) NzDMismatched number of outputs from callback. Expected: {}, Actual: {}c3lK|],}tjtj|.ywr()r canonicalize_valuerDrE)ras r/rzBemit_python_callback.._wrapped_callback..+s#P!V..rzz!}=Ps24z)Incorrect output shape for return value #z : Expected: z , Actual: z)Incorrect output dtype for return value #rLc3PK|]\}}t|js| ywr()rr)rout_valrFs r/rzBemit_python_callback.._wrapped_callback..;s*!4"g{ 1 12 !4s$&)rr@formatr enumeraterrr) r-rirTrnon_empty_out_valsrJplatformrIs r/r/z/emit_python_callback.._wrapped_callback$sCH 8}L))  %%+VC ,=s8}%M OOPxPPH"+C,,G"HD GX (.. (7s;!( 7==/ CD D (.. (7s;!( 7==/ CD D D5 !!4&)(L&A!44  or1rL)rrK)rN>rMrNgpur: _partitionedxla_ffi_python_rc|g|Sr(r8)rr-callback_without_tokens r/r/z/emit_python_callback.._wrapped_callbackYs 4,d3 44r1r8r`)rr)rr)+rr platformsrrrWrruClient get_backendr"runzip2rrrrIr|rr5rDr6rr{r rsrtrrdrrr  dataclassesreplacer abstract_tokenrr:uint64host_callbacksrbuild_ffi_lowering_functionr r)rrJrr;r<rIrrrJrKr<r r>r=r/rnon_empty_result_avalsnon_empty_result_shapesnon_empty_outputsnon_empty_outputs_iterrFrDr partitioncall_target_namerHrrrr`rYs ` ` @@r/rrs@    % %&* K LL    ) )! ,( 33  0 )D FF..  4XJi H JJ O PPBIIs'9'9'E'E'GH'8DE%d+E-E9FG&t,G.G>6:kk|];C+ D%djj) u C+7,33 9'%- 7#h  8u ""34()  +++ , +"3"3;;L;LMN267M2N O)G) E4  005e& +n)yk&C .5!!H%%++    " x .  !# %     h    %%5 5&&77 C$-&&}5 ))C**99:Q> ?% 3 * *%  ,  ,))E* ,& fh' NN' OEG % &&IFGHC+)s6MM# #M( "A%M.)rJr#rKSingleDeviceSharding | NonerL str | None)rSSequence[core.ShapedArray]r2r)rKrq)rJr#rKrq)rJr$rrr-rrKrqrzbool | None | DeprecatedArgrLrrr.r)rJr#rKrqrr$) rJr$rrr-rrKrqrr$r.r)rz core.Shaper2r$) r intr hlo.TokenTyperrrr rK#SdyArrayList | xc.OpSharding | Noner2zir.Value) r rtrrurcore.ShapedArrayrr rKrvr2ztuple[ir.Value, ir.Value])r zcore.AbstractValuer2xc.Shape)r rwr2rx)r<z xc.Clientrmlir.LoweringRuleContextr Any | Noner;Sequence[ir.Value]r<rsr=Sequence[xc.Shape]rIrsr>r|rr$rKrvr2ztuple[Sequence[ir.Value], Any])rryrrzr;r{r<rsrIrsrr$rr$rJr$rKrvr2z(tuple[Sequence[mlir.IrValues], Any, Any])or6 __future__rcollections.abcrrre functoolsloggingtypingrrjax._srcrr r r r r rrrrrrr>jax._src.interpretersrrr jax._src.librrujax._src.lib.mlirrjax._src.lib.mlir.dialectsrjax._src.sharding_implsrrrrjax._src.typingrrnumpyrD getLoggerr3rG Primitivermultiple_results%prim_requires_devices_during_loweringaddsafe_mapr unsafe_mapsafe_zipr unsafe_zip dataclassr#rPdef_implpartialapply_primitivedef_abstract_evalrTrYprimitive_jvpsr\primitive_transposesffi_batching_ruleprimitive_batchersrirrregister_loweringrr rEffectrrrrlowerable_effectsadd_typecontrol_flow_allowed_effectsordered_effectsshardable_ordered_effectsrdef_effectful_abstract_evalrrrrrrrrrr"rr7r)re AbstractTokenrIrr8r1r/rs; ". #%$*&) *XX0   8 $!$..1#' ..22?C--Z--Zd# F F$ F   *    2***8+C+C+:<=""*   #N&</"N ,H(0Ay/@/@?0O, 5B7BJ'3N>(>%P]-1.;o" J6 J6J6 J6* J6 , J6  J6J6^}- !% ..22=Aw~~"gnn" J "$""8,""?3$$--h7$$--o> 1!!**?;  *    2(y(()A)A)689**    *    +  7$8- =*D &( .GM*(X}&:eL -1 06 0606 06* 06  06  06f 59      2 L59 " " ""  " 2 ""JM79491):D$$%*JD&&'59E E !E  E ! E . E'E-E&EE2E$Eb48Q' !Q' Q'! Q' . Q' - Q'Q'Q'Q'2Q'.Q'r1