bi`ddlmZmZmZmZddlZddlmZmZm Z m Z ddl m Z m Z GddZy))AnyCallableOptionalUnionN)GenerationConfigPreTrainedTokenizerPreTrainedTokenizerFastset_seed)SUPPORTED_ARCHITECTURESPreTrainedModelWrapperc eZdZ ddedeeefdeee gee fde de de e de d e ed dfd Z dd eee ej"eej"eee fd ede ee ej&fd eee fdZy)BestOfNSamplerNmodel tokenizerqueries_to_scoreslength_sampler sample_sizeseed n_candidatesgeneration_configreturnc 6| t|t|ttfst dt |t|t st dt |dt ||_||_||_ ||_ ||_ ||_ ||_ y)a Initialize the sampler for best-of-n generation Args: model (`PreTrainedModelWrapper`): The pretrained model to use for generation tokenizer (`PreTrainedTokenizer` or `PreTrainedTokenizerFast`): Tokenizer associated with the pretrained model queries_to_scores (`Callable[[list[str]], list[float]]`): Callable that takes a list of generated texts and returns the associated reward scores length_sampler (`Any`): Sampler used to sample the length of the generated text sample_size (`int`): Number of samples to generate for each query seed (`int`, *optional*): Random seed used to control generation n_candidates (`int`): Number of candidates to return for each query generation_config (`GenerationConfig`, *optional*): Generation config passed to the underlying model's `generate` method. See `GenerationConfig` (https://huggingface.co/docs/transformers/v4.29.1/en/main_classes/text_generation#transformers.GenerationConfig) for more details NzHtokenizer must be a PreTrainedTokenizer or PreTrainedTokenizerFast, got z,model must be a PreTrainedModelWrapper, got z - supported architectures are: )r isinstancerr ValueErrortyper rrrr gen_configrr) selfrrrrrrrrs W/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/trl/extras/best_of_n_sampler.py__init__zBestOfNSampler.__init__sD   TN)&9;R%STZ[_`i[jZkl %"9;>tE{mKkmDlEF  "!2,+&(tokenized_queryskip_special_tokensdevicec d}t|tjr!|jdk(r|j d}nt|t rt |d}|tur%tj|j d}nb|tjur|Dcgc]}|jd}}n1|Dcgc]&}tj|jd(}}g} |D]}|j|jdf}|jj|j|f|j|j d|j#} |j$j'| |} tj|j)| } | j+|j,j.D cgc]} | |  } } | j1| | Scc}wcc}wcc} w)a Generate the best of n samples for input queries Args: tokenized_query (`list[int]` or `torch.Tensor` or `list[torch.Tensor]` or `list[int]`): represents either a single tokenized query (a single tensor or a list of integers) or a batch of tokenized queries (a list of tensors or a list of lists of integers) skip_special_tokens (`bool`): Whether to remove the special tokens from the output device (`str` or `torch.device`, *optional*): The device on which the model will be loaded **generation_kwargs (`dict`, *optional*): Additional keyword arguments passed along to the underlying model's `generate` method. This is used to override generation config Returns: list[list[str]]: A list of lists of generated texts Nr)r&)max_new_tokensr)r#)rtorchTensorndim unsqueezelistrinttensorreshaperepeatrrgeneratetorrsqueezer batch_decodertopkrindicesappend) rr"r#r$generation_kwargsqueries element_typer/queryresultoutputscoresis rr2zBestOfNSampler.generateOs2 ou|| 49M9MQR9R%//2G  . 23Ls",,7AA!D-APQv6>>'2QQM\]E5<<.66w?]] "EllD$4$4a#89G(TZZ(( 6"#224"&//$  gi  ^^00M`0aF\\$"8"8"@AF)/T5F5F)G)O)OPAfQiPFP MM& ! " 'R]QsG/>+G4 G9)Nr&N)TN)__name__ __module__ __qualname__r rrr rr-strfloatrr.rrr r)r*boolr$r2r!rrrs"8<5)%5),.EEF5)$T#YKe$<= 5)  5)  5)sm5)5)$$455) 5)t%)59 5tCy%,,U\\8JDQUVYQZO[\5"5sELL012 5 d3i 5r!r)typingrrrrr) transformersrrr r modelsr r rrHr!rrLs%21 aaDmmr!