|h7*ddlZddlmZmZddlmZddlmZmZmZm Z ddl Z ddl m Z m Z ddlmZddlmZmZddlmZdd lmZmZmZdd lmZd d lmZd dlmZGddeZGddeZ GddeeZ!Gdde e!Z"Gdde!Z#y)N)copydeepcopy)Path)DictListOptionalUnion)YOLOConcatDatasetbuild_yolo_dataset)LoadVisualPrompt)DetectionTrainerDetectionValidator) YOLOEModel) DEFAULT_CFGLOGGERRANK) de_parallel)WorldTrainerFromScratch)YOLOEDetectValidatorcfeZdZdZeddfdeeffd Zd defdZ dZ d de d e d ee fd Z xZS) YOLOETrainerae A trainer class for YOLOE object detection models. This class extends DetectionTrainer to provide specialized training functionality for YOLOE models, including custom model initialization, validation, and dataset building with multi-modal support. Attributes: loss_names (tuple): Names of loss components used during training. Methods: get_model: Initialize and return a YOLOEModel with specified configuration. get_validator: Return a YOLOEDetectValidator for model validation. build_dataset: Build YOLO dataset with multi-modal support for training. N overridesc:|i}d|d<t||||y)aQ Initialize the YOLOE Trainer with specified configurations. This method sets up the YOLOE trainer with the provided configuration and overrides, initializing the training environment, model, and callbacks for YOLOE object detection training. Args: cfg (dict): Configuration dictionary with default training settings from DEFAULT_CFG. overrides (dict, optional): Dictionary of parameter overrides for the default configuration. _callbacks (list, optional): List of callback functions to be applied during training. NF overlap_mask)super__init__)selfcfgr _callbacks __class__s b/var/www/html/test/engine/venv/lib/python3.12/site-packages/ultralytics/models/yolo/yoloe/train.pyrzYOLOETrainer.__init__%s+  I$) .! i4verbosectt|tr|dn||jdt |jdd|xr t dk(}|r|j ||S)aX Return a YOLOEModel initialized with the specified configuration and weights. Args: cfg (dict | str, optional): Model configuration. Can be a dictionary containing a 'yaml_file' key, a direct path to a YAML file, or None to use default configuration. weights (str | Path, optional): Path to pretrained weights file to load into the model. verbose (bool): Whether to display model information during initialization. Returns: (YOLOEModel): The initialized YOLOE model. Notes: - The number of classes (nc) is hard-coded to a maximum of 80 following the official configuration. - The nc parameter here represents the maximum number of different text samples in one image, rather than the actual number of classes. yaml_filechannelsncPchr)r%)r isinstancedictdataminrload)rr weightsr%models r# get_modelzYOLOETrainer.get_model6sa( *3 5C 3yy$499T?B'*    JJw  r$cd|_t|j|jt |j |j S)z9Return a YOLOEDetectValidator for YOLOE model validation.boxclsdflsave_dirargsr!) loss_namesr test_loaderr<rr= callbacksrs r# get_validatorzYOLOETrainer.get_validatorUs8-#   t}}4 ?W[WeWe  r$img_pathmodebatchc tt|jr-t|jjjndd}t |j |||j||dk(||dk(S)a Build YOLO Dataset. Args: img_path (str): Path to the folder containing images. mode (str): 'train' mode or 'val' mode, users are able to customize different augmentations for each mode. batch (int, optional): Size of batches, this is for rectangular training. Returns: (Dataset): YOLO dataset configured for training or validation. r valtrain)rDrectstride multi_modal)maxintr4rrKr r=r0)rrCrDrEgss r# build_datasetzYOLOETrainer.build_dataset\shTZZ[,33779QOQS T! IIx 45=Y[imqxix  r$NNTrIN)__name__ __module__ __qualname____doc__rrrrboolr5rBstrrNrP __classcell__r"s@r#rrsR 'DUY58D>5">  c  xPS} r$rceZdZdZddefdZy)YOLOEPETraineraM Fine-tune YOLOE model using linear probing approach. This trainer freezes most model layers and only trains specific projection layers for efficient fine-tuning on new datasets while preserving pretrained features. Methods: get_model: Initialize YOLOEModel with frozen layers except projection layers. Nr%c tt|tr|dn||jd|jd|xr tdk(}|j d`|Jd|r|j||jt|jdj}|j|}|j|||j dj|jt|j dj dd j#d |j dj dd <t|j dj d d j#d |j dj d d <t|j dj d d j#d |j dj d d <|`|j%|S) a Return YOLOEModel initialized with specified config and weights. Args: cfg (dict | str, optional): Model configuration. weights (str, optional): Path to pretrained weights. verbose (bool): Whether to display model information. Returns: (YOLOEModel): Initialized model with frozen layers except for specific projection layers. r'r(r)r+r,z7Pretrained weights must be provided for linear probing.namesrrTr)rr.r/r0rr4savper2evallistvalues get_text_pe set_classesfusepercv3requires_grad_rI)rr r3r%r4r^tpes r#r5zYOLOEPETrainer.get_modelys *3 5C 3yy$yy*   KKO !"]$]]"  JJw  TYYw'..01& %% BUXX&$,U[[_-@-@-CA-F$G$V$VW[$\ BAq!$,U[[_-@-@-CA-F$G$V$VW[$\ BAq!$,U[[_-@-@-CA-F$G$V$VW[$\ BAq! H  r$rQ)rSrTrUrVrWr5r$r#r\r\ns((r$r\c\eZdZdZd deeeefdedeefdZ dZ deeded e fd Z y) YOLOETrainerFromScratcha Train YOLOE models from scratch with text embedding support. This trainer combines YOLOE training capabilities with world training features, enabling training from scratch with text embeddings and grounding datasets. Methods: build_dataset: Build datasets for training with grounding support. preprocess_batch: Process batches with text features. generate_text_embeddings: Generate and cache text embeddings for training. NrCrDrEc2tj||||S)a Build YOLO Dataset for training or validation. This method constructs appropriate datasets based on the mode and input paths, handling both standard YOLO datasets and grounding datasets with different formats. Args: img_path (List[str] | str): Path to the folder containing images or list of paths. mode (str): 'train' mode or 'val' mode, allowing customized augmentations for each mode. batch (int, optional): Size of batches, used for rectangular training/validation. Returns: (YOLOConcatDataset | Dataset): The constructed dataset for training or validation. )rrP)rrCrDrEs r#rPz%YOLOETrainerFromScratch.build_datasets'44T8T5QQr$cltj||}ttj|d}t j |Dcgc]}|j|c}j|j}|jt|dd|jd}||d<|Scc}w)zKProcess batch for training, moving text features to the appropriate device.textsr+ txt_feats) r preprocess_batchra itertoolschaintorchstacktext_embeddingstodevicereshapelenshape)rrErotextrps r#rqz(YOLOETrainerFromScratch.preprocess_batchs 11$>Y__eGn56KK N!5!5d!; NORRSWS^S^_ %%c%.&92yr?RS &k !OsB1ro cache_dircPd}|d|jddjdddz }|jratjd|dt j ||j }t|jt|k(r|Stjd |d|jJt|jj||d d }tt||jd}t j|||S)ak Generate text embeddings for a list of text samples. Args: texts (List[str]): List of text samples to encode. batch (int): Batch size for processing. cache_dir (Path): Directory to save/load cached embeddings. Returns: (dict): Dictionary mapping text samples to their embeddings. zmobileclip:blttext_embeddings_:_/z.ptzReading existed cache from '') map_locationzCaching text embeddings to 'TF)without_reprtacache_clip_modelr)replaceexistsrinfortr2rxsortedkeysr4rrcr/zipsqueezesave)rrorEr}r4 cache_pathtxt_maprps r#generate_text_embeddingsz0YOLOETrainerFromScratch.generate_text_embeddingss!#3EMM#s4K4S4STWY\4]3^^a!bb     KK6zl!D Ejj$++FGglln%6 2:,a@Azz%%% +77uUYlq7r s5)"3"3A"678 7J'r$rR) rSrTrUrVr rrXrrNrPrqrrrjr$r#rlrls] ReDIsN&;R3RYabeYfR"d3iPTr$rlc(eZdZdZdZdZdefdZy)YOLOEPEFreeTrainera Train prompt-free YOLOE model. This trainer combines linear probing capabilities with from-scratch training for prompt-free YOLOE models that don't require text prompts during inference. Methods: get_validator: Return standard DetectionValidator for validation. preprocess_batch: Preprocess batches without text features. set_text_embeddings: Set text embeddings for datasets (no-op for prompt-free). cd|_t|j|jt |j |j S)z6Return a DetectionValidator for YOLO model validation.r7r;)r>rr?r<rr=r@rAs r#rBz YOLOEPEFreeTrainer.get_validators8-!   t}}4 ?W[WeWe  r$c2tj||}|S)z_Preprocess a batch of images for YOLOE training, adjusting formatting and dimensions as needed.)r rq)rrEs r#rqz#YOLOEPEFreeTrainer.preprocess_batchs 11$> r$rEcy)a8 Set text embeddings for datasets to accelerate training by caching category names. This method collects unique category names from all datasets, generates text embeddings for them, and caches these embeddings to improve training efficiency. The embeddings are stored in a file in the parent directory of the first dataset's image path. Args: datasets (List[Dataset]): List of datasets containing category names to process. batch (int): Batch size for processing text embeddings. Notes: The method creates a dictionary mapping text samples to their embeddings and stores it at the path specified by 'cache_path'. If the cache file already exists, it will be loaded instead of regenerating the embeddings. Nrj)rdatasetsrEs r#set_text_embeddingsz&YOLOEPEFreeTrainer.set_text_embeddingss" r$N)rSrTrUrVrBrqrNrrjr$r#rrs    3 r$rc\eZdZdZddeeeefdedeeffd Z fdZ fdZ xZ S) YOLOEVPTrainera{ Train YOLOE model with visual prompts. This trainer extends YOLOETrainerFromScratch to support visual prompt-based training, where visual cues are provided alongside images to guide the detection process. Methods: build_dataset: Build dataset with visual prompt loading transforms. preprocess_batch: Preprocess batches with visual prompts. rCrDrEct||||}t|tr6|jD]%}|j j t'|S|j j t|S)a  Build YOLO Dataset for training or validation with visual prompts. Args: img_path (List[str] | str): Path to the folder containing images or list of paths. mode (str): 'train' mode or 'val' mode, allowing customized augmentations for each mode. batch (int, optional): Size of batches, used for rectangular training/validation. Returns: (Dataset): YOLO dataset configured for training or validation, with visual prompts for training mode. )rrPr.r r transformsappendr )rrCrDrEdatasetdr"s r#rPzYOLOEVPTrainer.build_dataset st''$> g0 1%% 8 ##$4$67 8    % %&6&8 9r$cjt|t|jjt rI|jjj D]%}|jjt'y|jjjjty)zPClose mosaic augmentation and add visual prompt loading to the training dataset.N) r_close_dataloader_mosaicr. train_loaderrr rrrr )rrr"s r#rz'YOLOEVPTrainer._close_dataloader_mosaic4s (* d''//1B C&&..77 8 ##$4$67 8    % % 0 0 7 78H8J Kr$cht||}|dj|j|d<|S)zaPreprocess a batch of images for YOLOE training, moving visual prompts to the appropriate device.visuals)rrqrwrx)rrEr"s r#rqzYOLOEVPTrainer.preprocess_batch=s5(/ +..t{{;i r$rR) rSrTrUrVr rrXrrNrPrrqrYrZs@r#rrsF eDIsN&;3YabeYf(Lr$r)$rrrrpathlibrtypingrrrr rtultralytics.datar r ultralytics.data.augmentr ultralytics.models.yolo.detectr rultralytics.nn.tasksrultralytics.utilsrrrultralytics.utils.torch_utilsrworld.train_worldrrHrrr\rlrrrjr$r#rs.. B5O+7757%V #V r3%3l@l,C@F* )@* Z-,-r$