hhSddlmZmZmZmZmZddlZddlmZddl mcm Z ddl m Z mZddlmZddlmZGddej&ZGd d eZy) )AnyDictListOptionalTupleN) FocalLoss VarifocalLoss)bbox_iou)HungarianMatcherceZdZdZ d"dedeeeefde de de de ded ed effd Z d#d e jd e jde jdededeee jff dZ d#de jde jdedeee jffdZ d$de jd e jde jde jdeedeeededee jdee jdeee jffdZedeedeee je jfe jffdZde jde jdeedee je jffdZ d%de jd e jde jde jdeedee jdee jdedeeedeee jffdZ d#de jd e jdeeefded edeee jff d!ZxZS)&DETRLossa DETR (DEtection TRansformer) Loss class for calculating various loss components. This class computes classification loss, bounding box loss, GIoU loss, and optionally auxiliary losses for the DETR object detection model. Attributes: nc (int): Number of classes. loss_gain (Dict[str, float]): Coefficients for different loss components. aux_loss (bool): Whether to compute auxiliary losses. use_fl (bool): Whether to use FocalLoss. use_vfl (bool): Whether to use VarifocalLoss. use_uni_match (bool): Whether to use a fixed layer for auxiliary branch label assignment. uni_match_ind (int): Index of fixed layer to use if use_uni_match is True. matcher (HungarianMatcher): Object to compute matching cost and indices. fl (FocalLoss | None): Focal Loss object if use_fl is True, otherwise None. vfl (VarifocalLoss | None): Varifocal Loss object if use_vfl is True, otherwise None. device (torch.device): Device on which tensors are stored. nc loss_gainaux_lossuse_fluse_vfl use_uni_match uni_match_indgammaalphac  t || ddddddd}||_tdddd|_||_||_|r t|| nd|_|r t|| nd|_ ||_ ||_ d|_ y) a Initialize DETR loss function with customizable components and gains. Uses default loss_gain if not provided. Initializes HungarianMatcher with preset cost gains. Supports auxiliary losses and various loss types. Args: nc (int): Number of classes. loss_gain (Dict[str, float], optional): Coefficients for different loss components. aux_loss (bool): Whether to use auxiliary losses from each decoder layer. use_fl (bool): Whether to use FocalLoss. use_vfl (bool): Whether to use VarifocalLoss. use_uni_match (bool): Whether to use fixed layer for auxiliary branch label assignment. uni_match_ind (int): Index of fixed layer for uni_match. gamma (float): The focusing parameter that controls how much the loss focuses on hard-to-classify examples. alpha (float): The balancing factor used to address class imbalance. Nr g?)classbboxgiou no_objectmaskdice)rrr) cost_gain)super__init__rr matcherrrrflr vflrrdevice) selfrrrrrrrrr __class__s [/var/www/html/dev/engine/venv/lib/python3.12/site-packages/ultralytics/models/utils/loss.pyr#zDETRLoss.__init__$s:   "#QUV`abI'AqRS2TU "  -3)E5)29=.t** pred_scorestargets gt_scoresnum_gtspostfixreturncd|}|jdd\}}tj|||jdzftj|j } | j d|jdd| dddf} |j||d| z}|jrU|r |jr|j||| } n |j|| j} | t|d|z z} n:tjd ||jdj!} || j#|j$d ziS) a} Compute classification loss based on predictions, target values, and ground truth scores. Args: pred_scores (torch.Tensor): Predicted class scores with shape (B, N, C). targets (torch.Tensor): Target class indices with shape (B, N). gt_scores (torch.Tensor): Ground truth confidence scores with shape (B, N). num_gts (int): Number of ground truth objects. postfix (str, optional): String to append to the loss name for identification in multi-loss scenarios. Returns: (Dict[str, torch.Tensor]): Dictionary containing classification loss value. Notes: The function supports different classification loss types: - Varifocal Loss (if self.vfl is True and num_gts > 0) - Focal Loss (if self.fl is True) - BCE Loss (default fallback) loss_classNrr )dtyper'.none reductionr)shapetorchzerosrint64r'scatter_ unsqueezeviewr%r&floatmaxnnBCEWithLogitsLossmeansumsqueezer) r(r,r-r.r/r0 name_classbsnqone_hotloss_clss r*_get_loss_classzDETRLoss._get_loss_classPs2."'+ ""2A&B++r2tww{35;;w~~^G--b115#ss(#NN2r1-7 7748888KGD77; @ GQ", ,H=r++f=k9UZZ[\]aacHH,,.1HHIIr+ pred_bboxes gt_bboxescFd|}d|}i}t|dk(rJtjd|j||<tjd|j||<|S|jdt j ||dzt|z ||<d t||d d z ||<||jt|z ||<|jd ||z||<|jDcic]\}}||jc}}Scc}}w) aQ Compute bounding box and GIoU losses for predicted and ground truth bounding boxes. Args: pred_bboxes (torch.Tensor): Predicted bounding boxes with shape (N, 4). gt_bboxes (torch.Tensor): Ground truth bounding boxes with shape (N, 4). postfix (str, optional): String to append to the loss names for identification in multi-loss scenarios. Returns: (Dict[str, torch.Tensor]): Dictionary containing: - loss_bbox{postfix}: L1 loss between predicted and ground truth boxes, scaled by the bbox loss gain. - loss_giou{postfix}: GIoU loss between predicted and ground truth boxes, scaled by the giou loss gain. Notes: If no ground truth boxes are provided (empty list), zero-valued tensors are returned for both losses. loss_bbox loss_giourr'rrEr7g?T)xywhGIoUr) lenr:tensorr'rFl1_lossr rEitemsrF) r(rMrNr0 name_bbox name_gioulosskvs r*_get_loss_bboxzDETRLoss._get_loss_bboxzs( y) y)  y>Q #ll3t{{CDO#ll3t{{CDOK..0199[)_d3eehkluhvvYidQU VVYy/--/#i.@Y..04 ?BY+/::<841a199;888s>Dgt_cls gt_groups match_indicesmasksgt_maskc tj|dnd|j} |O|jrC|j ||j ||j ||||||j nd| }t t||D]d\} \} } ||| nd}|j| | ||||| || }| dxx|d|z cc<| d xx|d |z cc<| d xx|d |z cc<fd || dd|| d d|| d i} | S)a[ Get auxiliary losses for intermediate decoder layers. Args: pred_bboxes (torch.Tensor): Predicted bounding boxes from auxiliary layers. pred_scores (torch.Tensor): Predicted scores from auxiliary layers. gt_bboxes (torch.Tensor): Ground truth bounding boxes. gt_cls (torch.Tensor): Ground truth classes. gt_groups (List[int]): Number of ground truths per image. match_indices (List[Tuple], optional): Pre-computed matching indices. postfix (str, optional): String to append to loss names. masks (torch.Tensor, optional): Predicted masks if using segmentation. gt_mask (torch.Tensor, optional): Ground truth masks if using segmentation. Returns: (Dict[str, torch.Tensor]): Dictionary of auxiliary losses. NrrSrdre)rdrer0rcrr3r rPrrQloss_class_aux loss_bbox_aux loss_giou_aux) r:r;r'rr$r enumeratezip _get_loss)r(rMr,rNrarbrcr0rdrer]i aux_bboxes aux_scores aux_masksloss_s r* _get_loss_auxzDETRLoss._get_loss_auxst<{{ 11qASAST  T%7%7 LLD../D../383DeD../$)M,5Sk5R+S 4 'A' J$)$5a4INN+# E Guz'34 4G Guy 23 3G Guy 23 3G 4,WI &QG9 %tAwG9 %tAw  r+c htjt|Dcgc]\}\}}tj|| c}}}}tj|Dcgc]\}}| c}}}tj|Dcgc]\}}| c}}}||f|fScc}}}wcc}}wcc}}w)at Extract batch indices, source indices, and destination indices from match indices. Args: match_indices (List[Tuple]): List of tuples containing matched indices. Returns: batch_idx (Tuple[torch.Tensor, torch.Tensor]): Tuple containing (batch_idx, src_idx). dst_idx (torch.Tensor): Destination indices. )r:catrl full_like)rcrosrc_ batch_idxsrc_idxdstdst_idxs r* _get_indexzDETRLoss._get_indexsII)TaJbcc;1hsAusA6cd ))?Xc1S?@))?XaS?@7#W,,d??s#B!  B(  B. c tjt||Dcgc]J\}\}}t|dkDr||n.tjd|j d|j Lc}}}}tjt||Dcgc]J\}\}}t|dkDr||n.tjd|j d|j Lc}}}} || fScc}}}wcc}}}w)a Assign predicted bounding boxes to ground truth bounding boxes based on match indices. Args: pred_bboxes (torch.Tensor): Predicted bounding boxes. gt_bboxes (torch.Tensor): Ground truth bounding boxes. match_indices (List[Tuple]): List of tuples containing matched indices. Returns: pred_assigned (torch.Tensor): Assigned predicted bounding boxes. gt_assigned (torch.Tensor): Assigned ground truth bounding boxes. rr5rS)r:rvrmrVr;r9r') r(rMrNrctrory pred_assignedj gt_assigneds r*_get_assigned_bboxeszDETRLoss._get_assigned_bboxess "%[-!@  Av1A ! Aqwwr{4;;(WW  ii"%Y !>  Av1A ! Aqwwr{4;;(WW  k))   s AC3 AC: c 6| |j|||||||} |j| \} } || || }}|jdd\} } tj| | f|j |j |j}|| || <tj| | g|j }t|r.t|j|djd|| <i|j|||t|||j|||S) a% Calculate losses for a single prediction layer. Args: pred_bboxes (torch.Tensor): Predicted bounding boxes. pred_scores (torch.Tensor): Predicted class scores. gt_bboxes (torch.Tensor): Ground truth bounding boxes. gt_cls (torch.Tensor): Ground truth classes. gt_groups (List[int]): Number of ground truths per image. masks (torch.Tensor, optional): Predicted masks if using segmentation. gt_mask (torch.Tensor, optional): Ground truth masks if using segmentation. postfix (str, optional): String to append to loss names. match_indices (List[Tuple], optional): Pre-computed matching indices. Returns: (Dict[str, torch.Tensor]): Dictionary of losses. Nrhr)r'r4rST)rTr5)r$r~r9r:fullrr'r4r;rVr detachrFrLr`)r(rMr,rNrarbrdrer0rcidxgt_idxrHrIr-r.s r*rnzDETRLoss._get_loss2s):   LL[)VYe]d)Moom4 V!,S!19V3DY ""2A&B**b"Xtww{7I7IQWQ]Q]^f~ KKR1C1CD y>%k&8&8&:IDQYYZ\]IcN "";C NT[\ !!+y'B  r+batchkwargsc |j|_|jdd}|d|d|d} }}|j|d|d||| ||} |jr,| j |j |dd|dd||| ||| S)a Calculate loss for predicted bounding boxes and scores. Args: pred_bboxes (torch.Tensor): Predicted bounding boxes, shape (L, B, N, 4). pred_scores (torch.Tensor): Predicted class scores, shape (L, B, N, C). batch (Dict[str, Any]): Batch information containing cls, bboxes, and gt_groups. postfix (str, optional): Postfix for loss names. **kwargs (Any): Additional arguments, may include 'match_indices'. Returns: (Dict[str, torch.Tensor]): Computed losses, including main and auxiliary (if enabled). Notes: Uses last elements of pred_bboxes and pred_scores for main loss, and the rest for auxiliary losses if self.aux_loss is True. rcNclsbboxesrbr5r0rc)r'getrnrupdatert) r(rMr,rr0rrcrarNrb total_losss r*forwardzDETRLoss.forwardes2"((  ?D9 ',U|U8_eKFX9 ^^ O[_iT[kx$  ==   ""$k#2&6 69Vcel  r+) PNTTFFrg?g?))NrNN)NNrN)__name__ __module__ __qualname____doc__intrrstrr@boolr#r:TensorrLr`rrrt staticmethodr~rrnrr __classcell__r)s@r*rrs,04#* *De,-* *  *  *****Zwy(J <<(J27,,(JKP<<(Jbe(Jps(J c5<< (JVRT!9 <<!949LL!9KN!9 c5<< !9R04(,*.F\\F\\F<< F  F 9 F U ,FF %F%,,'F c5<< FP-$u+-%ellELL>X8Y[`[g[g8g2h-- * <<*49LL*QUV[Q\* u||U\\) **H)-*./31 \\1 \\1 << 1  1 9 1  %1 %,,'1 1  U ,1  c5<< 1 p (\\(\\(CH~ (  (  ( c5<< (r+rc\eZdZdZ d deej ej fdeee fde ej de ej de eee fdeeej ff fd Z e d e ej d ed e ede eej ej ffd ZxZS)RTDETRDetectionLossa# Real-Time DeepTracker (RT-DETR) Detection Loss class that extends the DETRLoss. This class computes the detection loss for the RT-DETR model, which includes the standard detection loss as well as an additional denoising training loss when provided with denoising metadata. predsr dn_bboxes dn_scoresdn_metar1c |\}}t||||}|c|d|d} } t|dt| k(sJ|j| | |d} t||||d| } |j | |S|j |j D cic]'} | dt jd|j)c} |Scc} w)av Forward pass to compute detection loss with optional denoising loss. Args: preds (Tuple[torch.Tensor, torch.Tensor]): Tuple containing predicted bounding boxes and scores. batch (Dict[str, Any]): Batch data containing ground truth information. dn_bboxes (torch.Tensor, optional): Denoising bounding boxes. dn_scores (torch.Tensor, optional): Denoising scores. dn_meta (Dict[str, Any], optional): Metadata for denoising. Returns: (Dict[str, torch.Tensor]): Dictionary containing total loss and denoising loss if applicable. dn_pos_idx dn_num_grouprb_dnrrRrS) r"rrVget_dn_match_indicesrkeysr:rWr')r(rrrrrrMr,rrrrcdn_lossr^r)s r*rzRTDETRDetectionLoss.forwards*$) [W_[+uE   '.|'U Ju[)*c*o= ==!55j,PUVaPbcMgoiE5`monG   g &    YcYhYhYjkTU!Cy%,,s4;;*OOk lls,Crrrbcdg}tjdg|ddjd}t|D]\}}|dkDrtj|tj ||z}|j |}t||t|k(s#Jdt||dt|d|j|||f|jtjdgtj tjdgtj f|S) a Get match indices for denoising. Args: dn_pos_idx (List[torch.Tensor]): List of tensors containing positive indices for denoising. dn_num_group (int): Number of denoising groups. gt_groups (List[int]): List of integers representing number of ground truths per image. Returns: (List[Tuple[torch.Tensor, torch.Tensor]]): List of tuples containing matched indices for denoising. rNr5)endr4z"Expected the same length, but got z and z respectively.)r4) r: as_tensorcumsum_rlarangelongrepeatrVappendr;)rrrbdn_match_indices idx_groupsronum_gtrs r*rz(RTDETRDetectionLoss.get_dn_match_indicess __a%9)CR.%9:BB1E "9- rIAvz& CjQRmS|4:a=)S[88Z]9K8LERUV\R]Q^^lm8!''A(?@ ''aS )KU[[Z[Y\didndnMo(pq r r+)NNN)rrrrrr:rrrrrrrrrrrrs@r*rrs-1,0,0 'U\\5<</0'CH~'ELL) ' ELL) ' $sCx.) ' c5<< 'R & 69 FJ3i eELL%,,./ 0  r+r)typingrrrrrr:torch.nnrBtorch.nn.functional functionalrXultralytics.utils.lossrr ultralytics.utils.metricsr opsr Modulerrr+r*rsC43 ;.!~ryy~B L (L r+