bi6,ddlmZddlZGddZy))OptionalNceZdZdZ ddeedeedeedeededeed ed eefd Zd Z d Z dZ de jfdZedee je je jffdZeddee je jffdZddZddZde jfdZeddededefdZy)IncrementalPCAay An implementation of Incremental Principal Components Analysis (IPCA) that leverages PyTorch for GPU acceleration. Adapted from https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/decomposition/_incremental_pca.py This class provides methods to fit the model on data incrementally in batches, and to transform new data based on the principal components learned during the fitting process. Args: n_components (int, optional): Number of components to keep. If `None`, it's set to the minimum of the number of samples and features. Defaults to None. copy (bool): If False, input data will be overwritten. Defaults to True. batch_size (int, optional): The number of samples to use for each batch. Only needed if self.fit is called. If `None`, it's inferred from the data and set to `5 * n_features`. Defaults to None. svd_driver (str, optional): name of the cuSOLVER method to be used for torch.linalg.svd. This keyword argument only works on CUDA inputs. Available options are: None, gesvd, gesvdj, and gesvda. Defaults to None. lowrank (bool, optional): Whether to use torch.svd_lowrank instead of torch.linalg.svd which can be faster. Defaults to False. lowrank_q (int, optional): For an adequate approximation of n_components, this parameter defaults to n_components * 2. lowrank_niter (int, optional): Number of subspace iterations to conduct for torch.svd_lowrank. Defaults to 4. lowrank_seed (int, optional): Seed for making results of torch.svd_lowrank reproducible. N n_componentscopy batch_size svd_driverlowrank lowrank_q lowrank_niter lowrank_seedc ||_||_||_||_||_||_||_||_d|_|jr|jyy)N) rrrr r r r r n_features__validate_lowrank_params) selfrrrr r r r r s U/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/peft/utils/incremental_pca.py__init__zIncrementalPCA.__init__.s_) $$ "*( <<  ) ) + c|j,|j td|jdz|_y|j|jkr tdy)NzKn_components must be specified when using lowrank mode with lowrank_q=None.z8lowrank_q must be greater than or equal to n_components.)r r ValueError)rs rrz'IncrementalPCA._validate_lowrank_paramsGsY >> !  ( !noo!..2DN ^^d// /WX X0rcZtjj|d|jS)NF) full_matricesdriver)torchlinalgsvdr rXs r _svd_fn_fullzIncrementalPCA._svd_fn_fullOs"||tOOrcJ|jdu}tjj|5|rtj|jtj ||j |j\}}}|||jfcdddS#1swYyxYw)N)enabled)qniter) r rrandomfork_rng manual_seed svd_lowrankr r mH)rr seed_enabledUSVs r_svd_fn_lowrankzIncrementalPCA._svd_fn_lowrankRs((4 \\ " "< " 8 !!$"3"34''T^^4CUCUVGAq!a:    s A BB"returnc:tjtjg}t|tjs&tj |tj}n|j r|j}|j\}}|jnU|j|kDrtd|jd|d|j|kDrtd|jd||j|vr|jtj}|S)z Validates and converts the input data `X` to the appropriate tensor format. Args: X (torch.Tensor): Input data. Returns: torch.Tensor: Converted to appropriate format. )dtypez n_components=z invalid for n_features=z<, need more rows than columns for IncrementalPCA processing.z6 must be less or equal to the batch number of samples ) rfloat32float64 isinstanceTensortensorrcloneshaperrr1to)rr valid_dtypes n_samples n_featuress r_validate_datazIncrementalPCA._validate_dataZs u}}5 !U\\* Qemm4A YY A ! :    $     + 1 122J:,WMM    * 1 122hirhst  77, &U]]#Arc.|jddk(r|||fS|dkDr| td| tdtj|jdg|j}||z}|>tj |jdtj |j}n||z}|jdtj }||z|z }||z } || z } | jdtj j} | j| jdtj } | | |z z} || |z } n?||z}|j|z }|| z||z ||z |z jzz}||z } || |fS)aj Computes the incremental mean and variance for the data `X`. Args: X (torch.Tensor): The batch input data tensor with shape (n_samples, n_features). last_mean (torch.Tensor): The previous mean tensor with shape (n_features,). last_variance (torch.Tensor): The previous variance tensor with shape (n_features,). last_sample_count (torch.Tensor): The count tensor of samples processed before the current batch. Returns: Tuple[torch.Tensor, torch.Tensor, torch.Tensor]: Updated mean, variance tensors, and total sample count. rz6last_mean should not be None if last_sample_count > 0.z:last_variance should not be None if last_sample_count > 0.device)r1r@)dimr1) r8rrr6r@zerosr3sumsquaresquare_double)r last_mean last_variancelast_sample_countnew_sample_countupdated_sample_countlast_sumnew_sum updated_meanTtemp correctionnew_unnormalized_varianceupdated_variancelast_unnormalized_variancelast_over_new_countupdated_unnormalized_variances r_incremental_mean_and_varz(IncrementalPCA._incremental_mean_and_var}s 771:?m->> > q   !YZZ$ !]^^ << QXXF03CC  {{1771:U]]188TH #44H%%AU]]%3 7*.BB & &1uXX!5==X9@@B  $(HH%--H$H!!Z2B%BB!  8;OO )69J)J &"3":":"! )) )o .2-K-K tzz499d&:&:. *'?  MA"ZZq1N  A%*ZZ1E1E1L1L1NQ`1`dm0m%n "4 ^8STO ))..w7$:J:JJ#A <<++A.HAq"((+HAq"q"u=2T_q%89#$a4%))Go4M*N#N .1 1 12 !"5D$5$5 6  #56I8I8I#J )ABUDDUDU)V&   Y $; ;#5d6G6G6I#J#O#O#QD  $)<<AHH#ED  rc||jz }tj|j|jj j |jS)a Applies dimensionality reduction to `X`. The input data `X` is projected on the first principal components previously extracted from a training set. Args: X (torch.Tensor): New data tensor with shape (n_samples, n_features) to be transformed. Returns: torch.Tensor: Transformed data tensor with shape (n_samples, n_components). )rsrmmrGrqrPr9r1rs r transformzIncrementalPCA.transform-sC  Nxx D$4$4$6$67::177CCrnrjc#Kd}tt||zD] }||z}||z|kDrt|||}"||krt||yyw)aGenerator to create slices containing `batch_size` elements from 0 to `n`. The last slice may contain less than `batch_size` elements, when `batch_size` does not divide `n`. Args: n (int): Size of the sequence. batch_size (int): Number of elements in each batch. min_batch_size (int, optional): Minimum number of elements in each batch. Defaults to 0. Yields: slice: A slice of `batch_size` elements. rN)r_intslice)rrrjstart_ends rrlzIncrementalPCA.gen_batches<sps1 ?+, A*$C^#a's# #E   19q/ ! sAA)NTNNFNN)T)r)__name__ __module__ __qualname____doc__rrboolstrrrr r.rr5r= staticmethodtuplerXrgrormrrlrrrrsZ6'+#$($(#'&*,sm,tn,SM , SM ,  ,C=,,sm,2YP!5<<!F6D u||U\\5<<7 86D6Dp% ell8R2S2,EN Dell D"s""S""rr)typingrrrrrrrs ~"~"r