bi3ddlmZddlZddlmZddlmZddlZddl Z ddl m Z m Z ddlmZmZddlmZddlmZdd lmZdd lmZmZer eZdd Zdd Zdd ZdZddZddZ y)) annotationsN)Lock) TYPE_CHECKING)BlockwiseDepDict blockwise)meta_lib_from_arraytolist)pyarrow_strings_enabled)HighLevelGraph)tokenize) ensure_dictis_series_likec jdkDrtdj|)ddlm}t ||s td|j }|tj}tjdd| tjj}ntj|r tdtfd |DsRt!t#|j%jj}td jd |jj&}|Dcgc]}||vr||dnd }}nPjd k(rtj|s||j)g|j|St+|d k(r3|j-tj.gj||Stdtj0jd r td|4jdk(r!tt3jd ndg}nCt+|jd k7r(tdt+|djd djgt+|z}t5||D cic]\} } | tj.g| } } } |j-| ||Scc}wcc} } w)z8Create empty DataFrame or Series which has correct dtypezCfrom_array does not input more than 2D array, got array with shape Nr)Indexz3'index' must be an instance of dask.dataframe.Indexnamesz+For a struct dtype, columns must be a list.c3NK|]}|jjvywN)dtyper).0ixs O/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/dask/dataframe/io/io.py z#_meta_from_array..1s9AQ!''--'9s"%zdtype z doesn't have fields f8)namerindex)r)columnsrz?For a 1d array, columns must be a scalar or single element listz Shape along axis 1 must be knownz:Number of column names must match width of the array. Got z names for z columns)ndim ValueErrorshapedask.dataframer isinstance_metar DataFramegetattrrlistrnpisscalarallsortedset differencefields_constructor_slicedlen _constructorarrayisnanrangezip) rrrmetarextrar/ndtypescdtdatas ` r_meta_from_arrayr>s vvz&'gg 0   (%'RS S  |"1%//1qww&2 ?177==)G [[ !JK K9993w<22177==ABEvaggY.CE7KL LAHIA!v+&)A,47II 1 ;;w 7?++u, \Q $$177+WE%  M   88AGGAJ ?@ @ ?121d5,-1#G \QWWQZ '7|nK |8E ''S\)58&5I J'1bArxx"% % JD J   T7%  @@5J2 Ks K9"K>c \t|trtj|}||fd|i|S)aCreate a Dask partition for either a DataFrame or Series. Designed to be used with :func:`dask.blockwise.blockwise`. ``data`` is the array from which the partition will be created. ``index`` can be: 1. ``None``, in which case each partition has an independent RangeIndex 2. a `tuple` with two elements, the start and stop values for a RangeIndex for this partition, which gives a continuously varying RangeIndex over the whole Dask DataFrame 3. an instance of a ``pandas.Index`` or a subclass thereof The ``kwargs`` _must_ contain an ``initializer`` key which is set by calling ``type(meta)``. r)r$tuplepd RangeIndex)r=r initializerkwargss r_partition_from_arrayrERs1% u% t 35 3F 33c t||||}dt||z}|g}|j|jdk(rdndg}|j|ji}||j |jdk7r4dj |j |jd}t||j} |j||j|jdg|j f||j<ntjt|jr dgt!|j"dd zz} nt|j"d} dg} d} i} t%|j"dD]0\} }| |z } | | | f| | f<| | k(r| d z} | j| 2|jt'| dgt)|r$|j*|jt-|d }n|j.t-|d }t1t2|dg||d d|}t5j6|||}ddlm}ddlm}m }ddl!m"}ddl#m$}tKt!| d z D cgc]} || f}} ||tM|||| |||}tOr|||jPS|Scc} w)auCreate a Dask DataFrame from a Dask Array. Converts a 2d array into a DataFrame and a 1d array into a Series. Parameters ---------- x : da.Array columns : list or string list of column names if DataFrame, single string if Series index : dask.dataframe.Index, optional An optional *dask* Index to use for the output Series or DataFrame. The default output index depends on whether `x` has any unknown chunks. If there are any unknown chunks, the output has ``None`` for all the divisions (one per chunk). If all the chunks are known, a default index with known divisions is created. Specifying `index` can be useful if you're conforming a Dask Array to an existing dask Series or DataFrame, and you would like the indices to match. meta : object, optional An optional `meta` parameter can be passed for dask to specify the concrete dataframe type to be returned. By default, pandas DataFrame is used. Examples -------- >>> import dask.array as da >>> import dask.dataframe as dd >>> x = da.ones((4, 2), chunks=(2, 2)) >>> df = dd.io.from_dask_array(x, columns=['a', 'b']) >>> df.compute() a b 0 1.0 1.0 1 1.0 1.0 2 1.0 1.0 3 1.0 1.0 See Also -------- dask.bag.to_dataframe: from dask.bag )r7zfrom-dask-array-rijrNrz@The index and array have different numbers of blocks. ({} != {})r)mapping)rrrC)rrCT) numblocks concatenate) dependencies)optimize) from_graphnew_collection)ArrowStringConversion) key_split))r>r rr rJ npartitionsformatr! divisionsappendextend_namer)r4sumr"r1chunks enumeraterrrtyperrrEr from_collectionsdask.array.optimizationrM$dask.dataframe.dask_expr._collectionrNrOdask.dataframe.dask_expr._exprrP dask.utilsrQr5r r expr)rrrr7rgraph_dependenciesarrays_and_indicesrJmsgrT n_elementsstop index_mappingr incrementrDblkgraphrMrNrOrPrQkeysresults rfrom_dask_arrayrmfsV AwD 9D G 4 4D&&!&&A+$3?%I     A .#VE$5$5q{{1~F S/ !OO !!%(!!5;;"45"'"3"3!5 %++ #agg, Fc!((1+.23 !% C  %ahhqk2 #LAy I D#,Q<"6M1$ z!    T " # !!#3M#JC"PQd77DIId4jQ!\\$t*E          C  + +D#DV WE0OD$$S^a%78 9!T1I 9D 9 U#T*  $ F 3FKK@AA M :s9 K cy)zA dummy function to link results together in a graph We use this to enforce an artificial sequential ordering on tasks that don't explicitly pass around a shared resource N)tokenrls r_linkrqs rFc t|tjr|dk(r(tt t |j |S|dk(rP|r>> L = ['A', 'B', 'C', 'D', 'E', 'F'] >>> sorted_division_locations(L, chunksize=2) (['A', 'C', 'E', 'F'], [0, 2, 4, 6]) >>> sorted_division_locations(L, chunksize=3) (['A', 'D', 'F'], [0, 3, 6]) >>> L = ['A', 'A', 'A', 'A', 'B', 'B', 'B', 'C'] >>> sorted_division_locations(L, chunksize=3) (['A', 'B', 'C', 'C'], [0, 4, 7, 8]) >>> sorted_division_locations(L, chunksize=2) (['A', 'B', 'C', 'C'], [0, 4, 7, 8]) >>> sorted_division_locations(['A'], chunksize=2) (['A', 'A'], [0, 1]) Nz;Exactly one of npartitions and chunksize must be specified.FrTc&t|kzSr)int)ind chunksizeresiduals r chunksizesz-sorted_division_locations..chunksizes2s3sX~...rFr) r!r r,r-r1bisect bisect_leftrmaxrU)seqrRr seq_unique duplicates enforce_exactroffsetssubtract_driftrrT locationsrrdrift divs_remaindiv offs_remainposrs ` @rsorted_division_locationsrs. td!23VWW +CC!JZ3s8+JM7AB!6%%c1-BB#CG (C ##* HNH + s8k)/QII1 A C E2?+I.TK c#h,!f {((SV<"'lS0 ,;44C Aa&Cgcl#CC )B- q),s7|);C %SQ # " "5C NUVDV9W!WXq c!ZI7%?@@A   S !   S !CW c#h,^SW SX i ]CsH2)NNN)NN)Fr@)! __future__rr threadingrtypingrnumpyr)pandasrAdask.blockwiserrdask.dataframe.dispatchrr dask.dataframe.utilsr dask.highlevelgraphr dask.tokenizer r`r rlockr>rErmrqrrrorFrrs["  6?8."2 v7At4({|;&q rF