|h)&ddlmZddlmZddlmZmZmZmZddl Z ddl m Z ddl mZmZmZddlmZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZmZddl m!Z!m"Z"GddeZ#Gdde#eZ$y))deepcopy)Path)AnyDictOptionalUnionN) functional)YOLOConcatDatasetbuild_dataloaderbuild_yolo_dataset)LoadVisualPrompt)check_det_dataset)DetectionValidator)SegmentationValidator) YOLOEDetect) YOLOEModel)LOGGERTQDM) select_devicesmart_inference_modecreZdZdZedej jjde dejfdZ de e efde e efffd Zde e efdej jjfd Ze dd eedeee e fd ee d ede e eff fd ZxZS)YOLOEDetectValidatora A validator class for YOLOE detection models that handles both text and visual prompt embeddings. This class extends DetectionValidator to provide specialized validation functionality for YOLOE models. It supports validation using either text prompts or visual prompt embeddings extracted from training samples, enabling flexible evaluation strategies for prompt-based object detection. Attributes: device (torch.device): The device on which validation is performed. args (namespace): Configuration arguments for validation. dataloader (DataLoader): DataLoader for validation data. Methods: get_visual_pe: Extract visual prompt embeddings from training samples. preprocess: Preprocess batch data ensuring visuals are on the same device as images. get_vpe_dataloader: Create a dataloader for LVIS training visual prompt samples. __call__: Run validation using either text or visual prompt embeddings. Examples: Validate with text prompts >>> validator = YOLOEDetectValidator() >>> stats = validator(model=model, load_vp=False) Validate with visual prompts >>> stats = validator(model=model, refer_data="path/to/data.yaml", load_vp=True) dataloadermodelreturnc bt|tsJt|jjdj Dcgc]}|j ddd}}tjt||jdj|j}tjt|}d}|D]f}|djdjtjj!} tj"| t| } || z }h|j|j}t%|t|| } | D]}|j'|}|j)|d |d  } |d} t+| j,dD]}|d| |k(jdjtjj!d} tj.| j,d|jdz}| |dt| | D]-}||xx| |||k(j1d||z z cc</ t3j4||dk7dd||dk7<d||dk(<|j7dScc}w)a Extract visual prompt embeddings from training samples. This method processes a dataloader to compute visual prompt embeddings for each class using a YOLOE model. It normalizes the embeddings and handles cases where no samples exist for a class by setting their embeddings to zero. Args: dataloader (torch.utils.data.DataLoader): The dataloader providing training samples. model (YOLOEModel): The YOLOE model from which to extract visual prompt embeddings. Returns: (torch.Tensor): Visual prompt embeddings with shape (1, num_classes, embed_dim). names/r)devicez)Get visual prompt embeddings from samplescls) minlength)totaldescimgvisuals)visual batch_idxT)sortedN)dimp) isinstancerlistdatasetdatavaluessplittorchzeroslenrembedr!squeezetointuniquebincountr preprocess get_visual_perangeshapeonessumF normalize unsqueeze)selfrrnamer visual_pecls_visual_numr%batchr"countpbarpredsr)ipad_clscs `/var/www/html/test/engine/venv/lib/python3.12/site-packages/ultralytics/models/yolo/yoloe/val.pyr>z"YOLOEDetectValidator.get_visual_pe1s %,,,37 8J8J8O8OPW8X8_8_8a3bc4C#A&ccKKE EKKO,A,A$++V SZ0:  $E,&&r*--eii8??ACNN3#e*=E e #N $ (**4;;7Jc*oDA VEOOE*E''e U9=M'NEk*I5;;q>* VEl9>2::2>AA%))LSS[_S`**U[[^DKKH2M&) #c(#VAaLE!HW\$:$>$>q$ANSTDU$UULV  V V*+Y~QR?R5SY[_`)a .A%&)* .A%&""1%%?dsJ,rJcvt||}d|vr$|dj|dj|d<|S)zIPreprocess batch data, ensuring visuals are on the same device as images.r'r&)superr=r9r!)rFrJ __class__s rQr=zYOLOEDetectValidator.preprocesscsC"5)  $Y/225<3F3FGE)  r1c t|j|j|jj|jd|jj|dd}t |t r5|jD]%}|jjt'n#|jjtt||jj|jjddS)a Create a dataloader for LVIS training visual prompt samples. This method prepares a dataloader for visual prompt embeddings (VPE) using the specified dataset. It applies necessary transformations including LoadVisualPrompt and configurations to the dataset for validation purposes. Args: data (dict): Dataset configuration dictionary containing paths and settings. Returns: (torch.utils.data.DataLoader): The dataloader for visual prompt samples. valF)moderectr )shufflerank) r argsgetr3rJr.r datasets transformsappendr r workers)rFr1r0ds rQget_vpe_dataloaderz'YOLOEDetectValidator.get_vpe_dataloaderjs% II HHTYY__dhhuo 6 IIOO    g0 1%% 8 ##$4$67 8    % %&6&8 9  IIOO II     rUtrainer refer_dataload_vpcT|"|j|_|jj}t|jjj dj Dcgc]}|jddd}}|rUtjdd|j_ |j|j|}|j||n8tjd|j|}|j||t |A||} | S| |sJdt#|jj|_t%|t&t(frdd lm} | ||jd }|j/j1|jt3|xs|jj } t| dj Dcgc]}|jddd}}|rvtjdd|j_ |j5| } |j| |}|j||t |At7| } | St%|j8d t:r)t=|j8d drt |A||Stjd|j|}|j||t |At7| } | Scc}wcc}w)a Run validation on the model using either text or visual prompt embeddings. This method validates the model using either text prompts or visual prompts, depending on the load_vp flag. It supports validation during training (using a trainer object) or standalone validation with a provided model. For visual prompts, reference data can be specified to extract embeddings from a different dataset. Args: trainer (object, optional): Trainer object containing the model and device. model (YOLOEModel | str, optional): Model to validate. Required if trainer is not provided. refer_data (str, optional): Path to reference data for visual prompts. load_vp (bool): Whether to load visual prompts. If False, text prompts are used. Returns: (dict): Validation statistics containing metrics computed during validation. rrrrz!Validate using the visual prompt.FzValidate using the text prompt.z5Refer data is only used for visual prompt validation.)attempt_load_weightsT)r!inplace)rr lrpc)r!emar/rr0r1r2r3rinfor\halfr> set_classes get_text_perS__call__rr.strrultralytics.nn.tasksrhevalr9rrcrrrhasattr)rFrdrrerfrGrvpetpestatsrhr1rrTs rQrpzYOLOEDetectValidator.__call__s0  !..DKKKOOE7;DOO''.!!%-G$We4E< 9%W WWw' (8(89DK%#t-E,U4;;PTU JJLOODKK ($Z%A499>>BD7;DM^w'77 =>''.!!%-(x(? Um0Us (L 1L%)NNNF)__name__ __module__ __qualname____doc__rr4utilsr1 DataLoaderrTensorr>rrqrr=rcrrboolrp __classcell__)rTs@rQrrs6/& (8(8(C(C/&J/&[`[g[g/&/&bS#X4S>! tCH~! %++:J:J:U:U! F"&26$( D#Dj#o./DSM D  D c3h DDrUrceZdZdZy)YOLOESegValidatorzRYOLOE segmentation validator that supports both text and visual prompt embeddings.N)rxryrzr{rUrQrrs\rUr)%copyrpathlibrtypingrrrrr4torch.nnr rCultralytics.datar r r ultralytics.data.augmentr ultralytics.data.utilsrultralytics.models.yolo.detectrultralytics.models.yolo.segmentrultralytics.nn.modules.headrrrrultralytics.utilsrrultralytics.utils.torch_utilsrrrrrrUrQrsW-- $TT54=A3+*M}-}@ ,.C rU