bi>ddlmZddlZddlmZmZddlmZddlZddl Z ddl m Z ddl mZ ddlmZdZdZdd Zd Zd Zd Z dd ZeGddeZeGddeZy#e$rdZYEwxYw)) annotationsN)Protocolruntime_checkable)uuid4)LocalFileSystem)VersioncD|xrt|txs t|S)z'Check if an fsspec file-system is local) isinstancer_is_local_fs_pyarrowfss R/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/dask/dataframe/io/utils.py _is_local_fsrs&  2' $  #c|rz%_get_pyarrow_dtypes...s 1 1 3rNspandasutf8columns field_namename numpy_type pandas_type)datetime datetimetzmetadatatimezonezM8[ns]dtypecategory)dask.dataframe.io.parquet.arrowrgetr%jsonloadsdecoderangelenr pdSeriesdt tz_localizer(typeNotImplementedError)schema categories dtype_backendr type_mapperhas_pandas_metadatapandas_metadatacpandas_metadata_dtypestzdtypesifield numpy_dtypecats r_get_pyarrow_dtypesrE's((R488 3 !//5V)v:V**V__Y%?%F%Fv%NO%((B7"  EE,fd 3 4a o E" " %((B7  #==!J- EE,fd 3 4a m6G6GD7   "$ F 3v; )q  ::/ /zzRIIb144@@EJJPVV5UZZ@  )%**5 )uzz#)& %C$F3K % MM"  2'  s-GA G%G"" G.-G.c |Dcic]*}|tjg|j|d,}}|xsgDcgc]}|j|}}t |dk(rd}nGt |dk(r|d}|ddk7r,|d|_n!tj j||}tj||}|r||j_ |Scc}wcc}w) aNGet the final metadata for the dask.dataframe Parameters ---------- to_read_columns : list All the columns to end up with, including index names file_dtypes : dict Mapping from column name to dtype for every element of ``to_read_columns`` index_cols : list Subset of ``to_read_columns`` that should move to the index column_index_names : list The values for df.columns.name for a MultiIndex in the columns, or df.index.name for a regular Index in the columns Returns ------- meta : DataFrame int64r'rN__index_level_0__)names)index) r1r2r+popr0r MultiIndex from_arrays DataFramerrJ) to_read_columns file_dtypes index_colscolumn_index_namesr=dataindexesrKdfs r_meta_from_dtypesrW]s,FU @A299R{q': ;; D %/$4"5qtxx{5G5 7|q ZA   a=/ /#AEJ ))')D d% (B-  I% 6s /CC!c*tjS)z0Simple utility function to get random hex string)rhexrr_guidr[s 7;;rc,||S|j|S)z5Helper function to place an object on a context stack) enter_context)objstacks r _set_contextr`s }   s ##rc |!|Dcgc]}t||fi||c}S|xsij}|jdd}|dk(r|t|st t j t dkDr|j||jddxsdgt|z}|jdd} | dk7rtd| d t||D cgc](\}} ttj|f|| d ||*c} }S|+|Dcgc]}t|j|fi||!c}S|Dcgc]}tt|fi||c}Scc}wcc} }wcc}wcc}w) aReturn a list of open-file objects given a list of input-file paths. WARNING: This utility is experimental, and is meant for internal ``dask.dataframe`` use only. Parameters ---------- paths : list(str) Remote or local path of the parquet file fs : fsspec object, optional File-system instance to use for file handling context_stack : contextlib.ExitStack, Optional Context manager to use for open files. open_file_func : callable, optional Callable function to use for file opening. If this argument is specified, ``open_file_func(path, **kwargs)`` will be used to open each file in ``paths``. Default is ``fs.open``. precache_options : dict, optional Dictionary of key-word arguments to use for precaching. If ``precache_options`` contains ``{"method": "parquet"}``, ``fsspec.parquet.open_parquet_file`` will be used for remote storage. **kwargs : Key-word arguments to pass to the appropriate open function Nmethodparquetz 2021.11.0 row_groups cache_typepartsz<'parts' `cache_type` required for 'parquet' precaching, got .)r rd)r`copyrLrrfsspec __version__updater0 ValueErrorzipfsspec_parquetopen_parquet_fileopen) pathsr context_stackopen_file_funcprecache_optionskwargspathprecacherdrergss r_open_input_filesrysF!  77 G  ).B446##Hd3HI NR F&& ''+*> > &'ZZ d3LU8K ZZ g6  "|1& ! 3  c 00"      QVW WRWWT4V4mDWWJO P$Ld-f-} = PPO 2  X PsE,*-E1!$E7 E<c,eZdZdZedZdZdZy)DataFrameIOFunctionzkDataFrame IO function with projectable columns Enables column projection in ``DataFrameIOLayer``. ct)z$Return the current column projectionr6selfs rrzDataFrameIOFunction.columnss "!rct)zUReturn a new DataFrameIOFunction object with a new column projection r})rrs rproject_columnsz#DataFrameIOFunction.project_columnss "!rct)z Return a new DataFrame partitionr})rargsrus r__call__zDataFrameIOFunction.__call__s!!rN)__name__ __module__ __qualname____doc__propertyrrrrZrrr{r{s% """ "rr{ceZdZddZddZy) SupportsLockcyrrZr~s racquirezSupportsLock.acquirercyrrZr~s rreleasezSupportsLock.releaserrN)returnobject)rrrrrrZrrrrs$$rrr)NNNN) __future__rr,typingrruuidrripandasr1fsspec.implementations.localrpackaging.versionrfsspec.parquetrcrn ImportErrorrr rErWr[r`ryr{rrZrrrs" . 8%+  3l'T $  KQ\"("",%8%%WNsA00A:9A: