biS NddlmZddlmZdZdZd dZdZedk(reyy) )CustomArgumentParser)merge_fsdp_weightsa!Utility to merge the weights from multiple FSDP checkpoints into a single combined checkpoint. Should be used if `SHARDED_STATE_DICT` was used for the model. Weights will be saved to `{output_path}`. This is a CPU-bound process and requires enough RAM to load the entire model state dict.crt|j|j|j |jyN)rcheckpoint_directory output_pathunsafe_serializationremove_checkpoint_dir)argss T/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/accelerate/commands/merge.py merge_commandr s/ !!4#3#39R9R5RTXTnTnNc>||jdt}ntt}|jdtd|jdtd|jdd d d |jd d dd ||j t |S)Nz merge-weights) descriptionrz5A directory containing sharded weights saved by FSDP.)typehelprzHThe path to save the merged weights. Defaults to the current directory. z--unsafe_serialization store_trueFzZWhether to save the merged weights as `.bin` rather than `.safetensors` (not recommended).)actiondefaultrz--remove_checkpoint_dirz9Whether to remove the checkpoint directory after merging.)rrr)func) add_parserrr add_argumentstr set_defaultsr ) subparsersparsers r merge_command_parserr s&&K&P%+> .S?vw   W    i   ! H / MrcNt}|j}t|yr)r parse_argsr )rr s r mainr >s ! #F    D$r__main__r) accelerate.commands.utilsraccelerate.utilsrrr rr __name__rr r&s8 ;/\  <  zFr