uki 6PdZddlZddlZddlmZddlmZddlmZddl m Z m Z m Z m Z mZmZmZmZmZmZddlmZmZmZmZeZeZe ZeefdZddded fd Zej<ed Zddded fd Z ej<e d Z!ej<e d Z"ddddeefdZ#dZ$e$ejJZ&e$e Z'e$ejPZ)e$e dZ*e$e dZ+e$e Z,e$eZ-e$eZ.e$eZ/e$eZ0e$eZ1d!dZ2e2ejfejh Z5e2ejldZ7dZ8e2ejlde8Z9dZ:e:Z:dZ;e;Z;dZd#dZ?dZ@dZAd ZBy)$aStax is a small but flexible neural net specification library from scratch. You likely do not mean to import this module! Stax is intended as an example library only. There are a number of other much more fully-featured neural network libraries for JAX, including `Flax`_ from Google, and `Haiku`_ from DeepMind. .. _Haiku: https://github.com/deepmind/dm-haiku .. _Flax: https://github.com/google/flax N)lax)random) relu log_softmaxsoftmaxsoftplussigmoidelu leaky_reluselugelu standardize) glorot_normalnormaloneszerosc$fd}d}||fS)z?Layer constructor function for a dense (fully-connected) layer.c|dd fz}tj|\}}||d f| f}}|||ffS)N)rsplit) rng input_shape output_shapek1k2WbW_initb_initout_dims U/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/jax/example_libraries/stax.pyinit_funzDense..init_fun6sYs#wj0L \\# FB "{20 16"wj3IqA !Q c>|\}}tj|||zSN)jnpdot)paramsinputskwargsrrs r! apply_funzDense..apply_fun;s! DAq 7761  !!r#)r rrr"r+s``` r!Denser-4s " 9 r#VALIDgư>c  \ dtz xs xs*t jd jd   f d} fd}||fS)z.init_funIs\*6>?01!"SH89SKs 34*+,?L?//\7G5FHL7?@!a3h(A-@J@ \\# FB "l #VB %;qA !Q ? A 2B$"B)c L|\}}tj|||zSN)r@)rconv_general_dilated) r(r)r*rrr@onerErGs r!r+zGeneralConv..apply_funTs9 DAq  # #FAwc6G IKL MMr#lenrr8) r@rCrArGrErrr"r+rBrLrDrFs ``````` @@@@r! GeneralConvrOAsj"3(Hh s<  # Ns'  L]8>>##6s8KL&   M 9 r#)NHWCHWIOrPc  \ dtz}xs|xs*t jd jd   f d}fd} || fS)zGLayer construction function for a general transposed-convolution layer.r0r2r3c\ t}Dcgc]-}|dk(rn#|dk(r|jdn t|/}}tj|| }Dcgc] }|dk(rnd }}t j |\}} || ||} } || | ffScc}wcc}wr5)r7r8r9rconv_transpose_shape_tuplerrr;s r!r"z&GeneralConvTranspose..init_funds\*6>?01!"SH89SKs 34*+,?L?11\7G5FHL7?@!a3h(A-@J@ \\# FB "l #VB %;qA !Q ? ArHcH|\}}tj|||zSrJ)rconv_transpose)r(r)r*rrr@rErGs r!r+z'GeneralConvTranspose..apply_funos5 DAq   fa'0A CEF GGr#rM) r@rCrArGrErrrLr"r+rBrDrFs ``````` @@@r!GeneralConvTransposerW\sj"3(Hh s<  # Ns'  L]8>>##6s8KL&   G 9 r#)NHCHIOrX)rr1gh㈵>Tc fdfd tjrfn fd}fd}||fS)zzBatchNorm..|s6)C"7rr#cr ||SdSr]r,)rr^ gamma_initscales r!razBatchNorm..}sU:c5#9r#ct fdt|D}tj|\}}||||}}|||ffS)Nc32K|]\}}|vs |ywr%r,).0idaxiss r! z.BatchNorm..init_fun..sH1!4-!Hs )tuple enumeraterr) rrr^rrbetagamma _beta_init _gamma_initrjs r!r"zBatchNorm..init_funsP H + 6H HE \\# FBR'R)?%D u %%r#c|\}}tfdttj|D}t | }r r|||z||zSr|||zS r|||zS|S)Nc3@K|]}|vrdn tdywr%)slice)rgrhrjs r!rkz/BatchNorm..apply_fun..s NaqDyteDk1Ns)epsilon)rlranger&ndimr) r(xr*rnroedzrjr`rurds r!r+zBatchNorm..apply_funsyKD% N5!;MN NBAtW-A %b A R 88 a$r(l" U2Y]" Hr#)r&isscalar) rjrur`rdr_rcr"r+rprqs `````` @@r! BatchNormr|ys;J*J+LL&$D$&   9 r#c d}fd}||fS)z?Layer that applies a scalar function elementwise on its inputs.c |dfSr]r,rrs r!razelementwise.. {B&7r#c|fiSr%r,)r(r)r*fun fun_kwargss r!razelementwise..ss6/HZ/Hr#r,)rrr"r+s`` r! elementwisers 7(H) 9 r#r)rjcdfd }|S)Nc<xsdtz r  nddtdzf}n"jdjdf}t|D]}d|dz|dzd|dz|dz fd}  fd}||fS) z0Layer construction function for a pooling layer.r0Nrr1Nr6ctj|}dtz}tj||||}|dfS)Nr0r,)rpadtype_to_padsrNreduce_window_shape_tuple)rr padding_valsr out_shaperErG window_shapes r!r"z6_pooling_layer..PoolingLayer..init_funsS((l)0';l C % %d//\7L$Fi ]r#cTtj| }r ||S|Sr%)r reduce_window) r(r)r*outinit_valrEreducerrescalespecrGrs r!r+z7_pooling_layer..PoolingLayer..apply_funs6   fh%w 0c+2WS&$ ';;r#)rNr8sorted) rrGrErnon_spatial_axesrhr"r+rrrrescalers ```` @r! PoolingLayerz$_pooling_layer..PoolingLayers1L 11G:Bh|Wg6G |C -11C$**S/9 $ %1!"1%,|AB/??l d"WQR[0g1<< Y r#)Nr.Nr,)rrrrs``` r!_pooling_layerrs4 r#cfd}|S)Nc|djdz fn"|jd|jdftfdtjD}t j |j }tj|dtj }tD]}t j||}||z S)Nrr1rr6c3DK|]}|vrj|ywr%r^)rgrhr)rs r!rkz=_normalize_by_window_size..rescale..s*8&66!,,q/8s )dtyper) rwr8rlrvr&rrrraddr expand_dims) outputsr)r spatial_shaperL window_sizesrhrdimsrErGs ` @r!rz*_normalize_by_window_size..rescales |FKK!O+C$**S/98#(#588M ((= 5C$$S"cggtWgNL $ %6__\15l6 \ !!r#r,)rrGrErs``` r!_normalize_by_window_sizers" .r#cd}d}||fS)zCLayer construction function for flattening all but the leading dim.cD|dtj|ddf}|dfS)Nrr1r,)mathprod)rrrs r!r"zFlatten..init_funs*q>499[_#==L  r#cLtj||jddfS)Nrr)r&reshaper^r(r)r*s r!r+zFlatten..apply_funs ;;v Q4 55r#r,r"r+s r!Flattenrs6 9 r#cd}d}||fS)z2Layer construction function for an identity layer.c |dfSr]r,rs r!razIdentity..rr#c|Sr%r,rs r!razIdentity..svr#r,rs r!Identityrs 7(5) 9 r#c fd}fd}||fS)z0Layer construction function for a fan-out layer.c|gzdfSr]r,)rrnums r!razFanOut..s }s':B&?r#c|gzSr%r,)r(r)r*rs r!razFanOut..s x#~r#r,)rr"r+s` r!FanOutrs ?(=) 9 r#cd}d}||fS)z3Layer construction function for a fan-in sum layer.c|ddfS)Nrr,r,rs r!razFanInSum..s{1~r&:r#ct|Sr%)sumrs r!razFanInSum..s s6{r#r,rs r!FanInSumrs :(:) 9 r#c fd}fd}||fS)z=Layer construction function for a fan-in concatenation layer.ct|dztfd|D}|dd|fz|ddzdz}|dfS)Nrc3(K|] }| ywr%r,)rgr^axs r!rkz0FanInConcat..init_fun..s9EeBi9sr1r,)rNr)rr concat_sizerrrjs @r!r"zFanInConcat..init_funs[ KN# #B9[99KAs#{n4{1~bde7LLI b=r#c0tj|Sr%)r& concatenate)r(r)r*rjs r!r+zFanInConcat..apply_funs ??64 ((r#r,)rjr"r+s` r! FanInConcatrs ) 9 r#c d}fd}||fS)z@Layer construction function for a dropout layer with given rate.c |dfSr]r,rs r!r"zDropout..init_funs ?r#c|jdd}| d}t|dk(r;tj||j}t j ||z dS|S)NrzDropout layer requires apply_fun to be called with a PRNG key argument. That is, instead of `apply_fun(params, inputs)`, call it like `apply_fun(params, inputs, rng)` where `rng` is a PRNG key (e.g. from `jax.random.key`).trainr)get ValueErrorr bernoullir^r&where)r(r)r*rmsgkeepmoderates r!r+zDropout..apply_fun sf **UD !C {6c sO w   c4 6d YYtVd]A .. mr#r,)rrr"r+s`` r!Dropoutrs  9 r#cRt|t|\fd}fd}||fS)a Combinator for composing layers in serial. Args: *layers: a sequence of layers, each an (init_fun, apply_fun) pair. Returns: A new layer, meaning an (init_fun, apply_fun) pair, representing the serial composition of the given sequence of layers. cg}D]7}tj|\}}|||\}}|j|9||fSr%)rrappend)rrr(r" layer_rngparam init_funss r!r"zserial..init_fun(sS F||C(nc9#I{;k5 mmE  r#c|jdd}|tj|ndz}t||D]\}}}|||fd|i|}|SNrr%poprrzip) r(r)r*rrngsrr apply_funsnlayerss r!r+zserial..apply_fun/sg **UD !C),6<<W %g>ODz6485UC5&4c4V4f5 Mr#rNrlayersr"r+rrrs @@@r!serialrs3 K'v,)Z 9 r#cTt|t|\fd}fd}||fS)aCombinator for composing layers in parallel. The layer resulting from this combinator is often used with the FanOut and FanInSum layers. Args: *layers: a sequence of layers, each an (init_fun, apply_fun) pair. Returns: A new layer, meaning an (init_fun, apply_fun) pair, representing the parallel composition of the given sequence of layers. In particular, the returned layer takes a sequence of inputs and returns a sequence of outputs with the same length as the argument `layers`. c tj|}tt||Dcgc]\}}}|||c}}}Scc}}}wr%)rrr)rrrinitr^rrs r!r"zparallel..init_funIsS <<W %D  4577&6dCc5!7 887sA c |jdd}|tj| nd z}t |||Dcgc]\}}}}|||fd|i|c}}}}Scc}}}}wrr) r(r)r*rrfprxrrrs r!r+zparallel..apply_funMsj **UD !C),6<<W %g>OD7::vvW[7\ ] ]Aq!Aa $ $V $ ]] ]sA" rrs @@@r!parallelr8s4 K'v,)Z8^ 9 r#c fd}fd}||fS)aCombinator to delay layer constructor pair until input shapes are known. Args: make_layer: a one-argument function that takes an input shape as an argument (a tuple of positive integers) and returns an (init_fun, apply_fun) pair. Returns: A new layer, meaning an (init_fun, apply_fun) pair, representing the same layer as returned by `make_layer` but with its construction delayed until input shapes are known. c(|d||S)Nrr,)rr make_layers r!r"z!shape_dependent..init_fun`s %:k "1 %c; 77r#c>|jd||fi|S)Nr1r)r(r)r*rs r!r+z"shape_dependent..apply_funbs% &:fll #A &vv @ @@r#r,)rr"r+s` r!shape_dependentrTs8A 9 r#r%)r)r)C__doc__ functoolsrjaxrr jax.numpynumpyr&jax.nnrrrrr r r r r rjax.nn.initializersrrrrglorotrandn logsoftmaxr-rOpartialConvrWConv1DTranspose ConvTransposer|rtanhTanhReluexpExp LogSoftmaxSoftmaxSoftplusSigmoidElu LeakyReluSeluGelurmaxinfMaxPoolrSumPoolrAvgPoolrrrrrrrrrr,r#r!rs  999BB   (/&( gdd|0y&>?"&wt &t 0$)##$8:OP! !!"6":< d4t$0 3884#'' 2 . gB ' x  g #  # 448 377( + " %" "&? @ ) : : .88r#