biddlZddlZddlmZmZmZmZddlZddl Z ddl Z ddl m cmZddl mZmZmZddlmZmZddlmZmZmZee j j ej4e j6ee j j eej4ee j6fZeZdefdZd Z Gd d eZ!Gd d e!Z"Gdde!Z#Gdde!Z$y)N)ListOptionalTupleUnion)Image ImageFilterImageOps) ConfigMixinregister_to_config) CONFIG_NAMEPIL_INTERPOLATION deprecatereturnct|tjjxs:t|tjt j fxr|jdvS)a Checks if the input is a valid image. A valid image can be: - A `PIL.Image.Image`. - A 2D or 3D `np.ndarray` or `torch.Tensor` (grayscale or color image). Args: image (`Union[PIL.Image.Image, np.ndarray, torch.Tensor]`): The image to validate. It can be a PIL image, a NumPy array, or a torch tensor. Returns: `bool`: `True` if the input is a valid image, `False` otherwise. )) isinstancePILrnpndarraytorchTensorndimimages T/home/cdr/jupyterlab/.venv/lib/python3.12/site-packages/diffusers/image_processor.pyis_valid_imager)sG eSYY__ - wEBJJPUP\P\C]1^1wchcmcmqwcwwct|tjtjfr|j dk(ryt |ryt|trtd|DSy)a Checks if the input is a valid image or list of images. The input can be one of the following formats: - A 4D tensor or numpy array (batch of images). - A valid single image: `PIL.Image.Image`, 2D `np.ndarray` or `torch.Tensor` (grayscale image), 3D `np.ndarray` or `torch.Tensor`. - A list of valid images. Args: images (`Union[np.ndarray, torch.Tensor, PIL.Image.Image, List]`): The image(s) to check. Can be a batch of images (4D tensor/array), a single image, or a list of valid images. Returns: `bool`: `True` if the input is valid, `False` otherwise. Tc32K|]}t|ywN)r).0rs r z+is_valid_image_imagelist..Ts=U>%(=F) rrrrrrrlistallimagess ris_valid_image_imagelistr+<sR&&2::u||45&++:J   FD !=f=== rc eZdZdZeZe d-dededede dededed ed effd Z e d e jd eej j fdZe d eeej j ej j fd e jfdZe d e jd ej*fdZe d ej*d e jfdZe d ee jej*fd ee jej*ffdZe d ee jej*fd ee jej*ffdZe dej j d ej j fdZe dej j d ej j fdZe d.dej j ded ej j fdZe d/dej j dedefdZdej j deded ej j fdZdej j deded ej j fdZ d0deej j e jej*fdedede d eej j e jej*ff d Z dej j d ej j fd!Z! d1d ej*d"e"eed ej*fd#Z# d2deej j e jej*fde"ede"ed e$eeffd$Z% d3de&de"ede"ede d%e"e$eeeefd ej*f d&Z' d4dej*d'e d"e"eed eej j e jej*ffd(Z( d1d)ej j d*ej j dej j d+e"e$eeeefd ej j f d,Z)xZ*S)5VaeImageProcessorai Image processor for VAE. Args: do_resize (`bool`, *optional*, defaults to `True`): Whether to downscale the image's (height, width) dimensions to multiples of `vae_scale_factor`. Can accept `height` and `width` arguments from [`image_processor.VaeImageProcessor.preprocess`] method. vae_scale_factor (`int`, *optional*, defaults to `8`): VAE scale factor. If `do_resize` is `True`, the image is automatically resized to multiples of this factor. resample (`str`, *optional*, defaults to `lanczos`): Resampling filter to use when resizing the image. do_normalize (`bool`, *optional*, defaults to `True`): Whether to normalize the image to [-1,1]. do_binarize (`bool`, *optional*, defaults to `False`): Whether to binarize the image to 0/1. do_convert_rgb (`bool`, *optional*, defaults to be `False`): Whether to convert the images to RGB format. do_convert_grayscale (`bool`, *optional*, defaults to be `False`): Whether to convert the images to grayscale format. do_resizevae_scale_factorvae_latent_channelsresample reducing_gap do_normalize do_binarizedo_convert_rgbdo_convert_grayscalec Dt ||r| r tddyy)Nz`do_convert_rgb` and `do_convert_grayscale` can not both be set to `True`, if you intended to convert the image into RGB format, please set `do_convert_grayscale = False`.z` if you intended to convert the image into grayscale format, please set `do_convert_rgb = False`)super__init__ ValueError) selfr.r/r0r1r2r3r4r5r6 __class__s rr9zVaeImageProcessor.__init__ps3  2tr 3>rr*rcV|jdk(r|d}|dzjjd}|jddk(r4|Dcgc]'}t j |j d)}}|S|Dcgc]}t j |}}|Scc}wcc}w) a Convert a numpy image or a batch of images to a PIL image. Args: images (`np.ndarray`): The image array to convert to PIL format. Returns: `List[PIL.Image.Image]`: A list of PIL images. rN.uint8r Lmoderroundastypeshaper fromarraysqueezer*r pil_imagess r numpy_to_pilzVaeImageProcessor.numpy_to_pils ;;! I&F3,%%'..w7 << q RXY%//%--/DYJY?EEU%//%0EJE ZEs ,B!B&ct|ts|g}|Dcgc]7}tj|j tj dz 9}}tj |d}|Scc}w)aT Convert a PIL image or a list of PIL images to NumPy arrays. Args: images (`PIL.Image.Image` or `List[PIL.Image.Image]`): The PIL image or list of images to convert to NumPy format. Returns: `np.ndarray`: A NumPy array representation of the images. go@raxisrr'rarrayrGfloat32stackr*rs r pil_to_numpyzVaeImageProcessor.pil_to_numpys_&$'XFJPQ"((5/((4u<QQ&q) R? rc|jjddddjj}|S)a Convert a PyTorch tensor to a NumPy image. Args: images (`torch.Tensor`): The PyTorch tensor to convert to NumPy format. Returns: `np.ndarray`: A NumPy array representation of the images. rrrr )cpupermutefloatnumpyr)s r pt_to_numpyzVaeImageProcessor.pt_to_numpys7%%aAq1779??A rcd|zdz S)a Normalize an image array to [-1,1]. Args: images (`np.ndarray` or `torch.Tensor`): The image array to normalize. Returns: `np.ndarray` or `torch.Tensor`: The normalized image array. g@g?r)s r normalizezVaeImageProcessor.normalizesV|c!!rc2|dzdzjddS)a Denormalize an image array to [0,1]. Args: images (`np.ndarray` or `torch.Tensor`): The image array to denormalize. Returns: `np.ndarray` or `torch.Tensor`: The denormalized image array. ?rr )clampr)s r denormalizezVaeImageProcessor.denormalizes  s"))!Q//rrc(|jd}|S)z Converts a PIL image to RGB format. Args: image (`PIL.Image.Image`): The PIL image to convert to RGB. Returns: `PIL.Image.Image`: The RGB-converted PIL image. RGBconvertrs rconvert_to_rgbz VaeImageProcessor.convert_to_rgbs e$ rc(|jd}|S)z Converts a given PIL image to grayscale. Args: image (`PIL.Image.Image`): The input image to convert. Returns: `PIL.Image.Image`: The image converted to grayscale. rBrkrs rconvert_to_grayscalez&VaeImageProcessor.convert_to_grayscales c" r blur_factorcN|jtj|}|S)a Applies Gaussian blur to an image. Args: image (`PIL.Image.Image`): The PIL image to convert to grayscale. Returns: `PIL.Image.Image`: The grayscale-converted PIL image. )filterr GaussianBlur)rrps rblurzVaeImageProcessor.blurs" [55kBC r mask_imagewidthheightc |jd}tj|}|j\}}d}t |D]#}|dd|fdk(j sn|dz }%d} t t |D]#}|dd|fdk(j sn| dz } %d} t |D]}||dk(j sn| dz } !d} t t |D]}||dk(j sn| dz } !tt||z dtt| |z dtt|| z |z|tt|| z |z|f\} } }}|| z || z z }||z }||kDr|| z |z }t||| z z }| |dzz} |||dzz z }||jk\r||jz }||z}| |z} | dkr || z}| | z} ||jk\r|j}n~|| z |z}t||| z z }| |dzz} |||dzz z }||jk\r||jz }||z}| |z} | dkr || z}| | z} ||jk\r |j}| | ||fS)a Finds a rectangular region that contains all masked ares in an image, and expands region to match the aspect ratio of the original image; for example, if user drew mask in a 128x32 region, and the dimensions for processing are 512x512, the region will be expanded to 128x128. Args: mask_image (PIL.Image.Image): Mask image. width (int): Width of the image to be processed. height (int): Height of the image to be processed. pad (int, optional): Padding to be added to the crop region. Defaults to 0. Returns: tuple: (x1, y1, x2, y2) represent a rectangular region that contains all masked ares in an image and matches the original aspect ratio. rBrNr r) rlrrRrHranger(reversedintmaxminrwrv)rurvrwpadmaskhw crop_lefti crop_rightcrop_top crop_bottomx1y1x2y2ratio_crop_regionratio_processingdesired_heightdesired_height_diffdiff desired_widthdesired_width_diffs rget_crop_regionz!VaeImageProcessor.get_crop_region%s$ '', xx #zz1 q AAJ!O((* NI   %(# AAJ!O((* !OJ  q AGqL%%' MH   %(# AGqL%%' 1 K  IOQ' ( HsNA& ' A NS(!, - A Oc)1- .  BB "Wb1 6> / / 2g)99N"%nR&@"A  %* *B %(;q(@@ @BZ&&&J---d d AvbbZ&&&&&"W(88M!$]b2g%>!?  $) )B $'9Q'>> >BZ%%%J,,,d d AvbbZ%%%%%2r2~rc ||z }|j|jz }||kr|n|j|z|jz}||k\r|n|j|z|jz}|j||ftd}t j d||f} | j ||dz|dzz |dz|dzz f||kr|dz|dzz } | dkDrm| j |j|| fdd|dfd| j |j|| fd|j||jfd| |zf| S||kDr}|dz|dzz } | dkDrm| j |j| |fddd|fd| j |j| |f|jd|j|f| |zdf| S)a* Resize the image to fit within the specified width and height, maintaining the aspect ratio, and then center the image within the dimensions, filling empty with data from image. Args: image (`PIL.Image.Image`): The image to resize and fill. width (`int`): The width to resize the image to. height (`int`): The height to resize the image to. Returns: `PIL.Image.Image`: The resized and filled image. lanczosr1rjrboxr)rrrvrwresizerrnewpaste) r;rrvrwratio src_ratiosrc_wsrc_hresizedres fill_height fill_widths r_resize_and_fillz"VaeImageProcessor._resize_and_fills.KK%,,. * f0D 0T9,%,,2F%++2U,,u~8I)8T,Uiiv/ ' UaZ 71uPQz9QR S 9  A+ 2KQ '..%)=Aq%QRCS.TZ` a NNE;#7aQVX_XfXf=gNhK%/0 Y !eqj0JA~ '..*f)=Aq!VCT.U[a b NNJ#7gmmQPWP]P]_e=fNg#e+Q/  rc||z }|j|jz }||kDr|n|j|z|jz}||kr|n|j|z|jz}|j||ftd}t j d||f} | j ||dz|dzz |dz|dzz f| S)a Resize the image to fit within the specified width and height, maintaining the aspect ratio, and then center the image within the dimensions, cropping the excess. Args: image (`PIL.Image.Image`): The image to resize and crop. width (`int`): The width to resize the image to. height (`int`): The height to resize the image to. Returns: `PIL.Image.Image`: The resized and cropped image. rrrjrrr) r;rrvrwrrrrrrs r_resize_and_cropz"VaeImageProcessor._resize_and_crops,KK%,,. * f0D 0T9,%,,2F%++2U,,u~8I)8T,Uiiv/ ' UaZ 71uPQz9QR S r resize_modec>|dk7r2t|tjjstd|t|tjjr|dk(rG|j ||ft |j j|j j}|S|dk(r|j|||}|S|dk(r|j|||}|Std|dt|tjr/tjjj|||f}|St|t j"rO|j%|}tjjj|||f}|j'|}|S) a Resize image. Args: image (`PIL.Image.Image`, `np.ndarray` or `torch.Tensor`): The image input, can be a PIL image, numpy array or pytorch tensor. height (`int`): The height to resize to. width (`int`): The width to resize to. resize_mode (`str`, *optional*, defaults to `default`): The resize mode to use, can be one of `default` or `fill`. If `default`, will resize the image to fit within the specified width and height, and it may not maintaining the original aspect ratio. If `fill`, will resize the image to fit within the specified width and height, maintaining the aspect ratio, and then center the image within the dimensions, filling empty with data from image. If `crop`, will resize the image to fit within the specified width and height, maintaining the aspect ratio, and then center the image within the dimensions, cropping the excess. Note that resize_mode `fill` and `crop` are only supported for PIL image input. Returns: `PIL.Image.Image`, `np.ndarray` or `torch.Tensor`: The resized image. defaultz2Only PIL image input is supported for resize_mode )r1r2fillcropz resize_mode  is not supported)size)rrrr:rrconfigr1r2rrrrnn functional interpolaterrr[ra)r;rrwrvrs rrzVaeImageProcessor.resizes< ) #Juciioo,NQR]Q^_` ` eSYY__ -i' FO.t{{/C/CD!%!9!9%0 '&--eUFC$ #&--eUFC  !< }.^s* 56ADKK000 0 s&))rrrrwrrrHrv)r;rrwrvs` rget_default_height_widthz*VaeImageProcessor.get_default_height_width5s2 >%1E5<<0QQ =%1 E5<<0 A A ;@&/ vu}r crops_coordsc D tjjtjtj f}|j jrt|tj tjfr||jdk(rmt|tj r|jd}nA|jddk(rtj|d}ntj|d}t|tr`t|dtjrC|djdk(r1tjdt tj"|d}t|tr`t|dtj rC|djdk(r1tjdt t j$|d}t'|s$t)d d j+d |Dt|ts|g}t|dtjjr||Dcgc]}|j-|}}|j j.r;|j1|d||\}}|Dcgc]}|j3|||| }}|j j4r|Dcgc]}|j7|}}n4|j jr|Dcgc]}|j9|}}|j;|}|j=|}nt|dtjr|djdk(rtj"|dntj>|d}|j=|}|j1|||\}}|j j.r|j3|||}nt|dtj r|djdk(rt j$|dnt j>|d}|j jr |jdk(r|jd}|jd}||j j@k(r|S|j1|||\}}|j j.r|j3|||}|j jB} | rR|jEdkr?tjd |jEd|jGdt d} | r|jI|}|j jJr|jM|}|Scc}wcc}wcc}wcc}w)a Preprocess the image input. Args: image (`PipelineImageInput`): The image input, accepted formats are PIL images, NumPy arrays, PyTorch tensors; Also accept list of supported formats. height (`int`, *optional*): The height in preprocessed image. If `None`, will use the `get_default_height_width()` to get default height. width (`int`, *optional*): The width in preprocessed. If `None`, will use get_default_height_width()` to get the default width. resize_mode (`str`, *optional*, defaults to `default`): The resize mode, can be one of `default` or `fill`. If `default`, will resize the image to fit within the specified width and height, and it may not maintaining the original aspect ratio. If `fill`, will resize the image to fit within the specified width and height, maintaining the aspect ratio, and then center the image within the dimensions, filling empty with data from image. If `crop`, will resize the image to fit within the specified width and height, maintaining the aspect ratio, and then center the image within the dimensions, cropping the excess. Note that resize_mode `fill` and `crop` are only supported for PIL image input. crops_coords (`List[Tuple[int, int, int, int]]`, *optional*, defaults to `None`): The crop coordinates for each image in the batch. If `None`, will not crop the image. Returns: `torch.Tensor`: The preprocessed image. rr rArrOr!zPassing `image` as a list of 4d np.ndarray is deprecated.Please concatenate the list along the batch dimension and pass it as a single 4d np.ndarrayzPassing `image` as a list of 4d torch.Tensor is deprecated.Please concatenate the list along the batch dimension and pass it as a single 4d torch.Tensorz9Input is in incorrect format. Currently, we only support , c32K|]}t|ywr#)str)r$rs rr%z/VaeImageProcessor.preprocess..sUxabVYZ[V\Uxr&)rPassing `image` as torch tensor with value range in [-1,1] is deprecated. The expected value range for image tensor is [0,1] when passing as pytorch tensor or numpy Array. You passed `image` with value range [,]F)'rrrrrrrr6rr unsqueezerH expand_dimsr'warningswarn FutureWarning concatenatecatr+r:joinrr.rrr5rmrorVr[rTr0r3r}r|rdr4r) r;rrwrvrrsupported_formatsrchannelr3s r preprocesszVaeImageProcessor.preprocessdsF!YY__bjj%,,G ;; + + 55<?A,,Q/??11?DE!2215EE%%e,E$$U+E a"** -5:1X]]a5GBNN5q1RXXV[bcMdE$$U+E 99%OMFE{{$$ E659 a%,, /05a 0BEIIe!, TY`aHbE{{//EJJ!O*kk!nG$++999  99%OMFE{{$$ E659{{// EIIK!O MMgglgpgpgrfsstuzu~u~vAuBBCD  !L NN5)E ;; " "MM%(E e>`?EsVV V?V output_typec:t|tjstdt |d|dvrd|d}t dd|d d }|d k(r|S|j ||}|d k(r|S|j|}|d k(r|S|d k(r|j|Sy) Postprocess the image output from tensor to `output_type`. Args: image (`torch.Tensor`): The image input, should be a pytorch tensor with shape `B x C x H x W`. output_type (`str`, *optional*, defaults to `pil`): The output type of the image, can be one of `pil`, `np`, `pt`, `latent`. do_denormalize (`List[bool]`, *optional*, defaults to `None`): Whether to denormalize the image to [0,1]. If `None`, will use the value of `do_normalize` in the `VaeImageProcessor` config. Returns: `PIL.Image.Image`, `np.ndarray` or `torch.Tensor`: The postprocessed image. 1Input for postprocessing is in incorrect format: . We only support pytorch tensorlatentptrpilthe output_type v is outdated and has been set to `np`. Please make sure to set it to one of these instead: `pil`, `np`, `pt`, `latent`Unsupported output_type1.0.0F standard_warnrrrrN) rrrr:typerrrarM)r;rrrdeprecation_messages r postprocesszVaeImageProcessor.postprocesss,%.CDK=Ppq  ; ;";-0..  /:M]b cK ( "L//~F $ L  ' $ L % $$U+ + rr init_image crop_coordsc|j|j}}tjj d||f}|j |j dj dtj|j d|j d}|m|\}} } } | |z } | | z } tjj d||f}|j|| | d}|j ||| f|j d}|j d}|j||j d}|S)a Applies an overlay of the mask and the inpainted image on the original image. Args: mask (`PIL.Image.Image`): The mask image that highlights regions to overlay. init_image (`PIL.Image.Image`): The original image to which the overlay is applied. image (`PIL.Image.Image`): The image to overlay onto the original. crop_coords (`Tuple[int, int, int, int]`, *optional*): Coordinates to crop the image. If provided, the image will be cropped accordingly. Returns: `PIL.Image.Image`: The final image with the overlay applied. RGBaRGBArB)rr)rwrvrrj) rvrwrrrrrlr invertralpha_composite)r;rrrrrvrwinit_image_maskedryrrrr base_images r apply_overlayzVaeImageProcessor.apply_overlays,2#((**;*;vIIMM&5&/B 2 26 : B B6 JQYQ`Q`aeamamnqarQst-55f=  "&LAq"bQAQAvv?JKKaqfKME   UQF +&&u-E f% /0 e$ r) Tr!rNTFFF)r!)r)rr#)NN)NNrNrN)+__name__ __module__ __qualname____doc__r config_namer boolr{rr9 staticmethodrrrrrrMrrVrrr[rardrhrmrortrrrrrrrrrPipelineImageInputrrr __classcell__r<s@rr-r-Xs*K !#$! !!$%*!     #(RZZD,A.U4 #8#))//#IJrzz&BJJ5<<$ ELL RZZ   "% ELL 89 "eBJJPUP\P\D\>] " " 0E"**ell":; 0bjjRWR^R^F^@_ 0 0ciioo#))// CIIOO  CIIOO#ciioo WCIIOOWCWWWr2yy22 2  2hyy   L% :SYY__bjj%,,>?:: :  : syy ELL8 9 :xciioo#))//$LP ll 4?- -} - sCx -d!%#$<@ A!A A} A  A uS#sC%789 A AL!/3 0,||0,0,!d, 0, syy ELL8 9 0,n<@ -iioo-IIOO-yy - eCc3$678 -  -rr-c eZdZdZeZe ddedede deffd Z e de jdeej j fd Ze deeej j ej j fde jfd Ze d ee jej*fdee jej*ffd Zde jdeej j fd Z dd ej*de deeedeej j e jej*ffdZ ddeej*ej j e jfdeej*ej j e jfdeedeedeedej*f dZxZS)VaeImageProcessorLDM3Da Image processor for VAE LDM3D. Args: do_resize (`bool`, *optional*, defaults to `True`): Whether to downscale the image's (height, width) dimensions to multiples of `vae_scale_factor`. vae_scale_factor (`int`, *optional*, defaults to `8`): VAE scale factor. If `do_resize` is `True`, the image is automatically resized to multiples of this factor. resample (`str`, *optional*, defaults to `lanczos`): Resampling filter to use when resizing the image. do_normalize (`bool`, *optional*, defaults to `True`): Whether to normalize the image to [-1,1]. r.r/r1r3c"t|yr#r8r9)r;r.r/r1r3r<s rr9zVaeImageProcessorLDM3D.__init__Zs rr*rc n|jdk(r|d}|dzjjd}|jddk(r4|Dcgc]'}t j |j d)}}|S|Dcgc]#}t j |d d d d d df%}}|Scc}wcc}w) ab Convert a NumPy image or a batch of images to a list of PIL images. Args: images (`np.ndarray`): The input NumPy array of images, which can be a single image or a batch. Returns: `List[PIL.Image.Image]`: A list of PIL images converted from the input NumPy array. rr>r?r@rAr rBrCNrErKs rrMz#VaeImageProcessorLDM3D.numpy_to_pilds ;;! I&F3,%%'..w7 << q RXY%//%--/DYJYIOOu%//%1bqb/:OJO ZOs ,B-(B2ct|ts|g}|Dcgc]7}tj|j tj dz 9}}tj |d}|Scc}w)aI Convert a PIL image or a list of PIL images to NumPy arrays. Args: images (`Union[List[PIL.Image.Image], PIL.Image.Image]`): The input image or list of images to be converted. Returns: `np.ndarray`: A NumPy array of the converted images. irrOrQrUs rdepth_pil_to_numpyz)VaeImageProcessorLDM3D.depth_pil_to_numpy|s_&$'XFPVWu"((5/((4 BWW&q) XrWrc:|dddddfdz|dddddfzS)a* Convert an RGB-like depth image to a depth map. Args: image (`Union[np.ndarray, torch.Tensor]`): The RGB-like depth image to convert. Returns: `Union[np.ndarray, torch.Tensor]`: The corresponding depth map. Nr rrcrs rrgblike_to_depthmapz*VaeImageProcessorLDM3D.rgblike_to_depthmaps(Q1W~$uQ1W~55rc|jdk(r|d}|ddddddddf}|jddk(rW|dzjjd}|Dcgc](}t j |j |d *}}|S|jdd k(rH|d zjtj}|Dcgc]}t j |d }}|Std cc}wcc}w) au Convert a NumPy depth image or a batch of images to a list of PIL images. Args: images (`np.ndarray`): The input NumPy array of depth images, which can be a single image or a batch. Returns: `List[PIL.Image.Image]`: A list of PIL images converted from the input NumPy depth images. rr>NrAr?r@zI;16rCr!g@z Not supported) rrHrFrGrrIrruint16 Exception)r;r* images_depth image_depthrLs rnumpy_to_depthz%VaeImageProcessorLDM3D.numpy_to_depths ;;! I&FaAqrk* << q (3.557>>wGLgsXc 8 8 EFSJ \\"  "(72::299ELWcd %//+FCdJdO, , es -C4C9rrc Pt|tjstdt |d|dvrd|d}t dd|d d }|j ||}|j|}|d k(rw|jd d k(rAtj|Dcgc]}|j|d d d d dd f!c}d}n|d d d d d d dd f}|d d d d d d d df|fS|dk(r"|j||j|fStd|dcc}w)rrrrrrrrFrrrArNrrrOrz This type r)rrrr:rrrrarHrrTrrMrr)r;rrrrimrs rrz"VaeImageProcessorLDM3D.postprocesssL,%.CDK=Ppq  ; ;";-0..  /:M]b cK//~F  ' $ {{2!# hhY^'_SU(@(@Aq!"H(N'_fgh #Aq!QRK0 Aq"1"%{2 2 % $$U+T-@-@-GG Gj 5FGH H(`s$D#rgbdepthrwrv target_resc tjjtjtj f|j jrDt|tj tjfr|jdk(r tdt|r|g}|g}n[t|trtfd|Ds7td|Dcgc] }t|c}ddjt|dtjjr|j j r td|j j"s|r]|s|j%|d||n|\}}|Dcgc]}|j'|||}}|Dcgc]}|j'|||}}|j)|}|j+|}|j-|}|j+|}ngt|dtjr!|djdk(rtj.|d ntj0|d }|j+|}|j%|||\}}|j j"r|j'|||}|djdk(rtj.|d ntj0|d }|j+|}|j%|||\}}|j j"r<|j'|||}n(t|dtj r td|j j2}|j5dkrA|r?t7j8d |j5d |j;d t<d }|r"|j?|}|j?|}|j j@r"|jC|}|jC|}||fScc}wcc}wcc}w)a` Preprocess the image input. Accepted formats are PIL images, NumPy arrays, or PyTorch tensors. Args: rgb (`Union[torch.Tensor, PIL.Image.Image, np.ndarray]`): The RGB input image, which can be a single image or a batch. depth (`Union[torch.Tensor, PIL.Image.Image, np.ndarray]`): The depth input image, which can be a single image or a batch. height (`Optional[int]`, *optional*, defaults to `None`): The desired height of the processed image. If `None`, defaults to the height of the input image. width (`Optional[int]`, *optional*, defaults to `None`): The desired width of the processed image. If `None`, defaults to the width of the input image. target_res (`Optional[int]`, *optional*, defaults to `None`): Target resolution for resizing the images. If specified, overrides height and width. Returns: `Tuple[torch.Tensor, torch.Tensor]`: A tuple containing the processed RGB and depth images as PyTorch tensors. rzThis is not yet supportedc36K|]}t|ywr#)r)r$rrs rr%z4VaeImageProcessorLDM3D.preprocess..s/^UV 1>O0P/^szInput is in incorrect format: z. Currently, we only support rrr!rOrrrF)"rrrrrrrr6rrrr'r(r:rrr5r.rrrVr[rrrTr3r}rrr|rrdr4r) r;r r rwrvr rr3rs @rrz!VaeImageProcessorLDM3D.preprocesss6!YY__bjj%,,G ;; + + 3rzz@Z0[`c`h`hlm`m78 8 c, -%CGES$'C/^Z]/^,^031Ga$q'1G0HHefjfofoqBgCfDE  c!fciioo .{{)) ;<<{{$$ \f = =c!ffe Tlv >ABt{{1fe4BB@EF1Q6FF##C(C""3'C++E2E$$U+E A +14Q1A"..1-rxxPSZ[G\C""3'C 99#vuMMFE{{$$kk#vu558V[[A5EBNN5q1288TY`aKbE$$U+E 99%OMFE{{$$ E659 A -78 84{{// 779q=\ MMggjgngngpfqqrsvszszs|r}}~@  !L ..%CNN5)E ;; " "--$CMM%(EEzW2HCFsP7 P<$Q)TrrTr)NNN)rrrrr rr rr{rr9rrrrrrrMrrrrrrrrrrrs@rrrIs8 K !!!    RZZD,A.5ciioo)> )O#PUWU_U_& 65U\\)A#B 6uRZZY^YeYeMeGf 6 6RZZD4I>!/3 0I||0I0I!d, 0I syy ELL8 9 0Il!%#$( q 5<<"**< =qU\\399??BJJ>?q q } q SM q qrrceZdZdZeZe ddedede dededef fd Z e d e jd ed ed efd ZxZS)IPAdapterMaskProcessora Image processor for IP Adapter image masks. Args: do_resize (`bool`, *optional*, defaults to `True`): Whether to downscale the image's (height, width) dimensions to multiples of `vae_scale_factor`. vae_scale_factor (`int`, *optional*, defaults to `8`): VAE scale factor. If `do_resize` is `True`, the image is automatically resized to multiples of this factor. resample (`str`, *optional*, defaults to `lanczos`): Resampling filter to use when resizing the image. do_normalize (`bool`, *optional*, defaults to `False`): Whether to normalize the image to [-1,1]. do_binarize (`bool`, *optional*, defaults to `True`): Whether to binarize the image to 0/1. do_convert_grayscale (`bool`, *optional*, defaults to be `True`): Whether to convert the images to grayscale format. r.r/r1r3r4r6c0t|||||||yN)r.r/r1r3r4r6rr;r.r/r1r3r4r6r<s rr9zIPAdapterMaskProcessor.__init__w) -%#!5  rr batch_size num_queriesvalue_embed_dimcH|jd}|jd}||z }ttj||z }t|t|t|zdk7z}||z}t j |j d||fdjd} | jd|kr| j|dd} | j| jdd} ||z} | |krDtjdtt j| d|| jdz fd } | |kDr%tjdt| d d d |f} | j| jd| jddjdd|} | S) a Downsamples the provided mask tensor to match the expected dimensions for scaled dot-product attention. If the aspect ratio of the mask does not match the aspect ratio of the output image, a warning is issued. Args: mask (`torch.Tensor`): The input mask tensor generated with `IPAdapterMaskProcessor.preprocess()`. batch_size (`int`): The batch size. num_queries (`int`): The number of queries. value_embed_dim (`int`): The dimensionality of the value embeddings. Returns: `torch.Tensor`: The downsampled mask tensor. r rrbicubic)rrDrAzThe aspect ratio of the mask does not match the aspect ratio of the output image. Please update your masks or adjust the output size for optimal performance.g)valueN)rHr{mathsqrtFrrrJrepeatviewrr UserWarningr~) rrrro_ho_wrmask_hmask_wmask_downsampledownsampled_areas r downsamplez!IPAdapterMaskProcessor.downsamples*jjmjjmc TYY{U234VsK#f+$=!#CDD&--q(9@PW`aiijkl   #j 0-44ZAFO)../D/DQ/GL!F? k ) MM^   eeOaG\G\]^G_9_5`hklO k ) MM^  .a+o>O*../D/DQ/GI^I^_`Iacdell q/ r)TrrFTT)rrrrr rr rr{rr9rrrr'rrs@rrras&K !!" %)         #  $;;3;S;[^;;rrceZdZdZe ddedededededef fd Ze d ed ed e d e eeffd Z e de jdeded e jfdZxZS)PixArtImageProcessora Image processor for PixArt image resize and crop. Args: do_resize (`bool`, *optional*, defaults to `True`): Whether to downscale the image's (height, width) dimensions to multiples of `vae_scale_factor`. Can accept `height` and `width` arguments from [`image_processor.VaeImageProcessor.preprocess`] method. vae_scale_factor (`int`, *optional*, defaults to `8`): VAE scale factor. If `do_resize` is `True`, the image is automatically resized to multiples of this factor. resample (`str`, *optional*, defaults to `lanczos`): Resampling filter to use when resizing the image. do_normalize (`bool`, *optional*, defaults to `True`): Whether to normalize the image to [-1,1]. do_binarize (`bool`, *optional*, defaults to `False`): Whether to binarize the image to 0/1. do_convert_rgb (`bool`, *optional*, defaults to be `False`): Whether to convert the images to RGB format. do_convert_grayscale (`bool`, *optional*, defaults to be `False`): Whether to convert the images to grayscale format. r.r/r1r3r4r6c0t|||||||yrrrs rr9zPixArtImageProcessor.__init__rrrwrvratiosrct||z t|jfd}||}t|dt|dfS)a Returns the binned height and width based on the aspect ratio. Args: height (`int`): The height of the image. width (`int`): The width of the image. ratios (`dict`): A dictionary where keys are aspect ratios and values are tuples of (height, width). Returns: `Tuple[int, int]`: The closest binned height and width. c2tt|z Sr#)absr_)rars rz@PixArtImageProcessor.classify_height_width_bin..sSuPRAR=Sr)keyrr )r_r}keysr{)rwrvr+ closest_ratio default_hwr/s @rclassify_height_width_binz.PixArtImageProcessor.classify_height_width_binsM6E> "FKKM/ST M* :a=!3z!}#555rsamples new_width new_heightc>|jd|jd}}||k7s||k7rtt||z ||z }t||z}t||z}tj|||fdd}||z dz}||z} ||z dz} | |z} |dddd| | || f}|S)a Resizes and crops a tensor of images to the specified dimensions. Args: samples (`torch.Tensor`): A tensor of shape (N, C, H, W) where N is the batch size, C is the number of channels, H is the height, and W is the width. new_width (`int`): The desired width of the output images. new_height (`int`): The desired height of the output images. Returns: `torch.Tensor`: A tensor containing the resized and cropped images. rrbilinearF)rrD align_cornersN)rHr|r{rr) r6r7r8 orig_height orig_widthr resized_widthresized_heightstart_xend_xstart_yend_ys rresize_and_crop_tensorz+PixArtImageProcessor.resize_and_crop_tensors#*--"2GMM!4DZ  * $ i(? [0)j2HIE U 23M u!45Nmm~}=J^cG %y0Q6Gi'E% 2q8Gj(EaGEM75=@AGr)TrrTFF)rrrrr rr{rr9rdictrr5rrrDrrs@rr)r)s* !!!!%*         #  $6#6c646ERUWZRZO66"" ""RU"Z_ZfZf""rr))%rrtypingrrrrr`r PIL.Imagerrtorch.nn.functionalrrrrrr configuration_utilsr r utilsr rrrrrPipelineDepthInputrrr+r-rrr)rcrrrLs // ,,@<<IIOOJJ LL (xTx&8n  n bU.Upe.eP^,^r