|h#nddlZddlmZddlmZddlmZmZddlm Z ddl m Z ddl m Z Gd d eZy) N)Image)SegmentationPredictor) DEFAULT_CFGchecks)box_iou) scale_masks)adjust_bboxes_to_image_bordercHeZdZdZeddffd ZfdZddZdZdZ xZ S) FastSAMPredictora FastSAMPredictor is specialized for fast SAM (Segment Anything Model) segmentation prediction tasks. This class extends the SegmentationPredictor, customizing the prediction pipeline specifically for fast SAM. It adjusts post-processing steps to incorporate mask prediction and non-maximum suppression while optimizing for single-class segmentation. Attributes: prompts (dict): Dictionary containing prompt information for segmentation (bboxes, points, labels, texts). device (torch.device): Device on which model and tensors are processed. clip_model (Any, optional): CLIP model for text-based prompting, loaded on demand. clip_preprocess (Any, optional): CLIP preprocessing function for images, loaded on demand. Methods: postprocess: Apply postprocessing to FastSAM predictions and handle prompts. prompt: Perform image segmentation inference based on various prompt types. set_prompts: Set prompts to be used during inference. Nc6t||||i|_y)a> Initialize the FastSAMPredictor with configuration and callbacks. This initializes a predictor specialized for Fast SAM (Segment Anything Model) segmentation tasks. The predictor extends SegmentationPredictor with custom post-processing for mask prediction and non-maximum suppression optimized for single-class segmentation. Args: cfg (dict): Configuration for the predictor. overrides (dict, optional): Configuration overrides. _callbacks (list, optional): List of callback functions. N)super__init__prompts)selfcfg overrides _callbacks __class__s a/var/www/html/test/engine/venv/lib/python3.12/site-packages/ultralytics/models/fastsam/predict.pyrzFastSAMPredictor.__init__"s i4 c|jjdd}|jjdd}|jjdd}|jjdd}t | |||}|D]} t j dd| j d| j dg|djtj} t| jj| j } t jt| d| d kDj} | jdk7s| | jj| <|j!||||| S) a Apply postprocessing to FastSAM predictions and handle prompts. Args: preds (List[torch.Tensor]): Raw predictions from the model. img (torch.Tensor): Input image tensor that was fed to the model. orig_imgs (List[numpy.ndarray]): Original images before preprocessing. Returns: (List[Results]): Processed results with prompts applied. bboxesNpointslabelstextsrr )devicedtypeg?)rrrr)rpopr postprocesstorchtensor orig_shaperfloat32r boxesxyxynonzerorflattennumelprompt)rpredsimg orig_imgsrrrrresultsresultfull_boxr%idxrs rr zFastSAMPredictor.postprocess2sF!!(D1!!(D1!!(D1   $/'%eS)< 2F||Av((+V->->q-AB5QR8??bgboboH2&,,2C2CVEVEVWE-- > DEMMOCyy{a)1 !!#& 2{{76&W\{]]rc  ||||Sg}t|ts|g}|D]}t|dk(r|j|$|jj }|j dd|jk7rt|d|jd}tjt|tj|j} |tj|tj|j}|jdk(r|dn|}|dddf|dddfz |dddf|dddfz z} tj |D cgc],} |dd| d| d| d| dfj#d.c} } tj"|d} | dddf| z| z }d | tj$| |z d<|tj|tj|j}|jdk(r|dn|}|"tj&|j d}tj|tj|j}t|t|k(sJd t|d t||j#dk(r9tj&t|tj|jn8tjt|tj|j}t)||D]9\}}t||tj*|dd|d|dfd d<;| |z} |8t|t,r|g}gg}}t/|j0j2j5D]v\}} d | D\}}}}||j#dkr|j|<|jt7j8|j:||||dddfx|j=||}tj$|d}t|rB|tj>||jdtA|kj#dz }d | |<|j|| |Scc} w)a Perform image segmentation inference based on cues like bounding boxes, points, and text prompts. Args: results (Results | List[Results]): Original inference results from FastSAM models without any prompts. bboxes (np.ndarray | List, optional): Bounding boxes with shape (N, 4), in XYXY format. points (np.ndarray | List, optional): Points indicating object locations with shape (N, 2), in pixels. labels (np.ndarray | List, optional): Labels for point prompts, shape (N, ). 1 = foreground, 0 = background. texts (str | List[str], optional): Textual prompts, a list containing string objects. Returns: (List[Results]): Output results filtered and determined by the provided prompts. Nrr )rr)r r4)dimTz5Expected `labels` with same size as `point`, but got z and )as_tuplec32K|]}t|yw)N)int).0xs r z*FastSAMPredictor.prompt..s%8c!f%8sdr)! isinstancelistlenappendmasksdatashaper#rr!zerosboolr as_tensorint32ndimstacksumargmaxoneszipr'str enumerater%r&tolistr fromarrayorig_img_clip_inferencer"r8)rr.rrrrprompt_resultsr/rCr1 bbox_areasb mask_areasfull_mask_areasunion point_idxpointlabelcrop_ims filter_idxix1y1x2y2 similaritytext_idxs rr*zFastSAMPredictor.promptNsK >fnN'4(iG3 /F6{a%%f-LL%%E{{12&"3"33#E$K1B1BCAF++c&kDKKPC!u{{4;;W)/)9v$QTlVAqD\9fQTlVTUWXTX\>YZ "[[fl)mab%1Q4!A$;!qt 0K*L*P*PU[*P*\)mn "'))Ev">"1d7+o= J?CELLe!3;<!u{{4;;W)/)9v>"ZZ Q8Fu{{4;;W6{c&k1KCPVK=X]^abh^i]jk1 zz|q(JJs6{%**T[[QS[ 4;;W %($7kLE5_cdi_jIemmE!U1XuQx2G,HSWXYZ[\ky  eS)"GE')2*%fll&7&7&>&>&@AZDAq%8a%8NBBQx||~,"))!, OOEOOFOOBrE2b5RVTVRVDV4W$XY Z "11(EB  << ;z?j!Md!SWZ[cWd!d i ijk llH $H  ! !&+ .g3 /jQ*ns 1S c ddl}t |dr t |ds*|j d|j \|_|_tj|Dcgc],}|j|j|j .c}}|j|j|j }|jj|}|jj|}||jdd z}||jdd z}||dddfzj!dS#t$rtjdddl}YYwxYwcc}w) a Perform CLIP inference to calculate similarity between images and text prompts. Args: images (List[PIL.Image]): List of source images, each should be PIL.Image with RGB channel order. texts (List[str]): List of prompt texts, each should be a string object. Returns: (torch.Tensor): Similarity matrix between given images and texts with shape (M, N). rNz+git+https://github.com/ultralytics/CLIP.git clip_modelclip_preprocesszViT-B/32r>r=T)r5keepdim)clip ImportErrorrcheck_requirementshasattrloadrrirjr!rKtotokenize encode_image encode_textnormrL)rimagesrrlimagetokenized_textimage_features text_featuress rrUz FastSAMPredictor._clip_inferences7  l+WTCT5U48IIjQUQ\Q\I4] 1DOT1W]^ed2259<rs+ A1--0f,fr