hh,dZddlZddlZddlmZmZmZmZddlZddl m Z ddl m cm Z ddlmZmZddlmZmZmZmZddlmZmZddlmZmZmZmZmZmZm Z dd l!m"Z"m#Z#dd l$m%Z%m&Z&m'Z'dd l(m)Z)m*Z*d Z+Gd de jXZ-Gdde-Z.Gdde-Z/Gdde-Z0Gdde jXZ1Gdde-Z2Gdde jXZ3Gdde-Z4Gdde4Z5Gdd e jXZ6Gd!d"e-Z7y)#zModel head modules.N)ListOptionalTupleUnion) constant_xavier_uniform_) TORCH_1_10 dist2bbox dist2rbox make_anchors)fuse_conv_and_bnsmart_inference_mode)DFLSAVPEBNContrastiveHeadContrastiveHeadProtoResidual SwiGLUFFN)ConvDWConv)MLPDeformableTransformerDecoder!DeformableTransformerDecoderLayer)bias_init_with_prob linear_init) DetectSegmentPoseClassifyOBB RTDETRDecoder v10Detect YOLOEDetect YOLOESegmentc eZdZdZdZdZdZdZdZdZ e jdZ e jdZ dZdZddedeffd Zd ee j(d eee j(effd Zd ee j(d eeeffd Zd ee j(d e j(fd ZdZdde j(de j(ded e j(fdZedde j(deded e j(fdZxZS)ra YOLO Detect head for object detection models. This class implements the detection head used in YOLO models for predicting bounding boxes and class probabilities. It supports both training and inference modes, with optional end-to-end detection capabilities. Attributes: dynamic (bool): Force grid reconstruction. export (bool): Export mode flag. format (str): Export format. end2end (bool): End-to-end detection mode. max_det (int): Maximum detections per image. shape (tuple): Input shape. anchors (torch.Tensor): Anchor points. strides (torch.Tensor): Feature map strides. legacy (bool): Backward compatibility for v3/v5/v8/v9 models. xyxy (bool): Output format, xyxy or xywh. nc (int): Number of classes. nl (int): Number of detection layers. reg_max (int): DFL channels. no (int): Number of outputs per anchor. stride (torch.Tensor): Strides computed during build. cv2 (nn.ModuleList): Convolution layers for box regression. cv3 (nn.ModuleList): Convolution layers for classification. dfl (nn.Module): Distribution Focal Loss layer. one2one_cv2 (nn.ModuleList): One-to-one convolution layers for box regression. one2one_cv3 (nn.ModuleList): One-to-one convolution layers for classification. Methods: forward: Perform forward pass and return predictions. forward_end2end: Perform forward pass for end-to-end detection. bias_init: Initialize detection head biases. decode_bboxes: Decode bounding boxes from predictions. postprocess: Post-process model predictions. Examples: Create a detection head for 80 classes >>> detect = Detect(nc=80, ch=(256, 512, 1024)) >>> x = [torch.randn(1, 256, 80, 80), torch.randn(1, 512, 40, 40), torch.randn(1, 1024, 20, 20)] >>> outputs = detect(x) FN,rncchcnt|_t|_d_|j dzz_tjj_ td|ddzj dzft|dtjdctjfd|D_jrtjfd|Dntjfd|D_j dkDrt#j ntj$_j(rIt+j,j_t+j,j _y y ) z Initialize the YOLO detection layer with specified number of classes and channels. Args: nc (int): Number of classes. ch (tuple): Tuple of channel sizes from backbone feature maps. rdc 3K|]S}tjt|dtdtjdjzdUyw)r-rN)nn SequentialrConv2dreg_max).0xc2selfs Y/var/www/html/dev/engine/venv/lib/python3.12/site-packages/ultralytics/nn/modules/head.py z"Detect.__init__..]sJ! cdBMM$q"a.$r2q/299RT\\IY[\;] ^! sAAc 3K|]P}tjt|dtdtjjdRywr0rNr1r2rr3r)r5r6c3r8s r9r:z"Detect.__init__..asEphi"--QARQSUW[W^W^`aIbcpAAc 32K|]}tjtjt||dt|dtjtdtdtjj dywr<)r1r2rrr3r)r>s r9r:z"Detect.__init__..csu   MM&Aq/42q>BMM&R"3T"b!_EIIb$''1-sBBrN)super__init__r)lennlr4notorchzerosstridemaxminr1 ModuleListcv2legacycv3rIdentitydflend2endcopydeepcopy one2one_cv2 one2one_cv3)r8r)r*r7r? __class__s` @@r9rCzDetect.__init__NsI b' t||a''kk$''* b"Q%1*dllQ&678#beSRUEV:WB==! hj!   {{ MMpmop p   )- q(83t||$bkkm <<#}}TXX6D #}}TXX6D  r6returnc`|jr|j|St|jD]I}t j |j ||||j|||fd||<K|jr|S|j|}|jr|S||fS)HConcatenate and return predicted bounding boxes and class probabilities.r) rRforward_end2endrangerErGcatrMrOtraining _inferenceexport)r8r6iys r9forwardzDetect.forwardrs <<''* *tww HA99kdhhqk!A$/!QqT1BCQGAaD H ==H OOA KKq+aV+rXc |Dcgc]}|j}}t|jDcgc]F}tj|j ||||j |||fdH}}t|jD]I}tj|j||||j|||fd||<K|jr||dS|j|}|j|jddd|j|j}|jr|S|||dfScc}wcc}w)av Perform forward pass of the v10Detect module. Args: x (List[torch.Tensor]): Input feature maps from different levels. Returns: outputs (dict | tuple): Training mode returns dict with one2many and one2one outputs. Inference mode returns processed detections or tuple with detections and raw outputs. r)one2manyone2oner)detachr]rErGr^rUrVrMrOr_r` postprocesspermutemax_detr)ra)r8r6xix_detachrbrgrcs r9r\zDetect.forward_end2end~sK+,,BBIIK,,hmnrnunuhv cdEII*t''*8A;79L9I9I!9LXVW[9YZ\] ^  tww HA99kdhhqk!A$/!QqT1BCQGAaD H == !g6 6 OOG $   QYYq!Q/tww GKKqMaaG)L%MM- s EA Ec |dj}tj|Dcgc]"}|j|d|jd$c}d}|j dk7rM|j s|j|k7r2dt||jdD\|_ |_ ||_|jr?|j dvr1|ddd|jd zf}|dd|jd zdf}n.|j|jd z|jfd \}}|jr|j d vr|d}|d }tj||||g|j j#d d d } |j|jd| zz } |j%|j'|| z|jj)d| ddddfz} n|jr|j dk(r|j%|j'||jz|jj)d|jzd} | j+d d|j-j/ddd fS|j%|j'||jj)d|jz} tj| |j-fd Scc}w)aM Decode predicted bounding boxes and class probabilities based on multiple-level feature maps. Args: x (List[torch.Tensor]): List of feature maps from different detection layers. Returns: (torch.Tensor): Concatenated tensor of decoded bounding boxes and class probabilities. rrhimxc3@K|]}|jddywrrN transposer5r6s r9r:z$Detect._inference..s)g!++a*;)g?>pbtfjstfliteedgetpu saved_modelNr-r>r{r|r0deviceF)xywh)shaperGr^viewrFformatdynamicr rIanchorsstridesrar4splitr)tensorrreshape decode_bboxesrQ unsqueezerusigmoidrk) r8r6rrmx_catboxclsgrid_hgrid_w grid_sizenormdboxs r9r`zDetect._inferences!  AFb27758TWWb9FJ ;;% T\\TZZ55H)g\RSUYU`U`beEf)g &DL$,DJ ;;4;;*\\-T\\A---.C4<BHC ;;4;;*??1XF1XF ffff%EcjjYaabcefhijI<<4;;q>I#=>D%%dhhsmd&:DLL>!Q')>)>q!Q)GG G%%dhhsmT\\5K5KA5NORVR^R^^Dyy$ .227Gs'K;c|}t|j|j|jD]q\}}}d|djj ddt jd|jz d|z dzz |djj d|js|jrt|j|j|jD]q\}}}d|djj ddt jd|jz d|z dzz |djj d|jsyy)BInitialize Detect() biases, WARNING: requires stride availability.?rpNrh) ziprMrOrIbiasdatamathlogr)rRrUrVr8mabss r9 bias_initzDetect.bias_inits 155!%%2 JGAq!!$AbEJJOOA &*hhq144x37q./H&IAbEJJOOFadd # J <<q}}ammQXXF N1a%(" "*.((1qtt8sQw1n3L*M" !$$' N rXbboxesrrcZt|||xr|jxs |j dS)z'Decode bounding boxes from predictions.r)rdim)r rRxyxy)r8rrrs r9rzDetect.decode_bboxess)t/WT\\=VTYY8W]^__rXpredsrlcX|j\}}}|jd|gd\}}|jdjt ||dj d}|j d|jddd}|j d|jdd|}|jdjt ||\}}tj|d} tj|| ||zf|d||zdjgdS)a% Post-process YOLO model predictions. Args: preds (torch.Tensor): Raw predictions with shape (batch_size, num_anchors, 4 + nc) with last dimension format [x, y, w, h, class_probs]. max_det (int): Maximum detections per image. nc (int, optional): Number of classes. Returns: (torch.Tensor): Processed predictions with shape (batch_size, min(max_det, num_anchors), 6) and last dimension format [x, y, w, h, max_class_prob, class_index]. r-rprr)rindex).N) rramaxtopkrKrgatherrepeatflattenrGaranger^float) rrlr) batch_sizer_boxesscoresrrbs r9rjzDetect.postprocesss"' GQ QG 4 v  #((Wg)>?BLLRP %,,q!Q*? @1ELLAr,BCq)..s7G/DE  LL $Y /yy%5B;/ 1BURZQZD[DaDaDcdjlmmrXPT)r) __name__ __module__ __qualname____doc__rrarrRrlrrGemptyrrrNrintrrCrTensorrrddictr\r`rboolr staticmethodrj __classcell__rWs@r9rrsV(TG F FGG Eekk!nGekk!nG F D"73"7"7H ,ell+ ,d5<<6H%6O0P ,Nell!3NdEk8JN0'3D.'35<<'3R N`ELL`5<<`t`_d_k_k`n5<<n#n3n nnrXrc eZdZdZd dedededeffd Zdeejde eeejffd Z xZ S) ra YOLO Segment head for segmentation models. This class extends the Detect head to include mask prediction capabilities for instance segmentation tasks. Attributes: nm (int): Number of masks. npr (int): Number of protos. proto (Proto): Prototype generation module. cv4 (nn.ModuleList): Convolution layers for mask coefficients. Methods: forward: Return model outputs and mask coefficients. Examples: Create a segmentation head >>> segment = Segment(nc=80, nm=32, npr=256, ch=(256, 512, 1024)) >>> x = [torch.randn(1, 256, 80, 80), torch.randn(1, 512, 40, 40), torch.randn(1, 1024, 20, 20)] >>> outputs = segment(x) r)nmnprr*ct|||_|_t |djj_t |ddzjtjfd|D_ y)aN Initialize the YOLO model attributes such as the number of masks, prototypes, and the convolution layers. Args: nc (int): Number of classes. nm (int): Number of masks. npr (int): Number of protos. ch (tuple): Tuple of channel sizes from backbone feature maps. rr-c 3K|]P}tjt|dtdtjjdRywr<r1r2rr3rr5r6c4r8s r9r:z#Segment.__init__..H woptAr1~tBAPRPYPYZ\^b^e^eghPi!j wr@N) rBrCrrrprotorJr1rLcv4)r8r)rrr*rrWs` @r9rCzSegment.__init__sm R 2a5$((DGG4 A!TWW %== wtv wwrXr6rYc |j|d}|jd}tjt |j Dcgc]5}|j |||j||jd7c}d}tj||}|jr|||fS|jrtj||gd|fStj|d|gd|d||ffScc}w)gReturn model outputs and mask coefficients if training, otherwise return outputs and mask coefficients.rrprhr) rrrGr^r]rErrrrrdr_ra)r8r6pbsrbmcs r9rdzSegment.forwards JJqt  WWQZ YYtwwXA  AaD)..r477B?XZ[ \ NN4 # ==b!8O-1[[ 1b'1%q)guyy!A$PRUV?WZ[\]Z^`bdeYf>gg Ys :C<)r r) rrrrrrrCrrGrrrdrrs@r9rrs`*x3xxxux$ hell+ heT%,,=O6O0P hrXrceZdZdZd dededeffd Zdeejde ejeffdZ d ejd ejdejfd Z xZ S) r"a YOLO OBB detection head for detection with rotation models. This class extends the Detect head to include oriented bounding box prediction with rotation angles. Attributes: ne (int): Number of extra parameters. cv4 (nn.ModuleList): Convolution layers for angle prediction. angle (torch.Tensor): Predicted rotation angles. Methods: forward: Concatenate and return predicted bounding boxes and class probabilities. decode_bboxes: Decode rotated bounding boxes. Examples: Create an OBB detection head >>> obb = OBB(nc=80, ne=1, ch=(256, 512, 1024)) >>> x = [torch.randn(1, 256, 80, 80), torch.randn(1, 512, 40, 40), torch.randn(1, 1024, 20, 20)] >>> outputs = obb(x) r)ner*ct|||_t|ddzjt j fd|D_y)a  Initialize OBB with number of classes `nc` and layer channels `ch`. Args: nc (int): Number of classes. ne (int): Number of extra parameters. ch (tuple): Tuple of channel sizes from backbone feature maps. rr-c 3K|]P}tjt|dtdtjjdRywr<)r1r2rr3rrs r9r:zOBB.__init__..@rr@N)rBrCrrJr1rLr)r8r)rr*rrWs` @r9rCz OBB.__init__3sJ R  A!TWW %== wtv wwrXr6rYc F|djd}tjt|jDcgc]5}|j |||j ||jd7c}d}|jdz tjz}|js||_ tj||}|jr||fS|jrtj||gdStj|d|gd|d|ffScc}w)r[rrprhg?r)rrGr^r]rErrrrrpir_anglerrdra)r8r6rrbrs r9rdz OBB.forwardBs qTZZ] ERVRYRYN[q;488A;qt,11"dggrB[]^_4'4772}}DJ NN4 # ==e8O+/;;uyy!UQ'hUYY!e}VW=X[\]^[_afZg>> pose = Pose(nc=80, kpt_shape=(17, 3), ch=(256, 512, 1024)) >>> x = [torch.randn(1, 256, 80, 80), torch.randn(1, 512, 40, 40), torch.randn(1, 1024, 20, 20)] >>> outputs = pose(x) r) kpt_shaper*ct|||_|d|dz_t |ddzjt j fd|D_y)aC Initialize YOLO network with default parameters and Convolutional Layers. Args: nc (int): Number of classes. kpt_shape (tuple): Number of keypoints, number of dims (2 for x,y or 3 for x,y,visible). ch (tuple): Tuple of channel sizes from backbone feature maps. rrr-c 3K|]P}tjt|dtdtjjdRywr<)r1r2rr3nkrs r9r:z Pose.__init__..yrr@N)rBrCrrrJr1rLr)r8r)rr*rrWs` @r9rCz Pose.__init__ks^ R "A,1- A!TWW %== wtv wwrXr6rYc |djd}tjt|jDcgc]5}|j |||j ||jd7c}d}tj||}|jr||fS|j||}|jrtj||gdStj|d|gd|d|ffScc}w)z?Perform forward pass through YOLO model and return predictions.rrpr) rrGr^r]rErrrrrdr_ kpts_decodera)r8r6rrbkptpred_kpts r9rdz Pose.forward{s qTZZ]ii%PTPWPW.YQ!QqT*//DGGR@Y[]^ NN4 # ==c6M##B,.2kkuyy!X*l 1Q4QYJZ\]@^abcdaegj`k?ll Zs:C9rkptsc\|jd}|jrd|jdvr|j|g|jd}|jd|jd}}t j ||g|jjddd}|j|jd|zz }|ddddddfd z|jd z z|z} nM|j|g|jd}|ddddddfd z|jd z z|jz} |dk(r2t j| |ddddddfjfd} | j||jdS|j}|dk(r$|dddd|fj|dddd|f<|dddd|fd z|jdd z z|jz|dddd|f<|dddd|fd z|jdd z z|jz|dddd|f<|S) z"Decode keypoints from predictions.r>r{r|rprhr0r~rN@rx)rrarrrrGrrrrrIrr^rrclone) r8rrndimrcrrrrrs r9rzPose.kpts_decodes.~~a  ;;{{ DIIb64>>626!%A 1 !LL&&)9!((KSSTUWXZ[\ ||t{{1~ 'ABq!RaRx[3&$,,*<=EDIIb64>>626q!RaRx[3&$,,*<=MqyIIq!Aq!A#I,"6"6"891=66"dggr* * Aqy !!QWW* 5 5 7!QWW* q!'T'z]S0DLLOc4IJdllZAaDjMq!'T'z]S0DLLOc4IJdllZAaDjMHrX)r)r0r)rrrrrrrCrrGrrrdrrrs@r9r r Usu*x3xxUx mell+mellE6I0Jmc%,,rXr c eZdZdZdZd dededededeedef fd Zd ee e je jfd ee je ffd Z xZS)r!a YOLO classification head, i.e. x(b,c1,20,20) to x(b,c2). This class implements a classification head that transforms feature maps into class predictions. Attributes: export (bool): Export mode flag. conv (Conv): Convolutional layer for feature transformation. pool (nn.AdaptiveAvgPool2d): Global average pooling layer. drop (nn.Dropout): Dropout layer for regularization. linear (nn.Linear): Linear layer for final classification. Methods: forward: Perform forward pass of the YOLO model on input image data. Examples: Create a classification head >>> classify = Classify(c1=1024, c2=1000) >>> x = torch.randn(1, 1024, 20, 20) >>> output = classify(x) Fc1r7krrgct|d}t|||||||_t j d|_t jdd|_t j|||_ y)a Initialize YOLO classification head to transform input tensor from (b,c1,20,20) to (b,c2) shape. Args: c1 (int): Number of input channels. c2 (int): Number of output classes. k (int, optional): Kernel size. s (int, optional): Stride. p (int, optional): Padding. g (int, optional): Groups. irT)rinplaceN) rBrCrconvr1AdaptiveAvgPool2dpoolDropoutdropLinearlinear) r8rr7rrrrc_rWs r9rCzClassify.__init__sa  RAq!, ((+ JJd3 iiB' rXr6rYc Jt|trtj|d}|j |j |j |j|jd}|jr|S|jd}|jr|S||fS)z;Perform forward pass of the YOLO model on input image data.r) isinstancelistrGr^rrrrrr_softmaxra)r8r6rcs r9rdzClassify.forwards| a  !QA KK $))DIIaL"9"A"A!"DE F ==H IIaLKKq+aV+rX)rrNr)rrrrrarrrCrrrGrrrdrrs@r9r!r!s|,F(3(C(C((HSM(]`(&,tELL15<<?@,U5<>> world_detect = WorldDetect(nc=80, embed=512, with_bn=False, ch=(256, 512, 1024)) >>> x = [torch.randn(1, 256, 80, 80), torch.randn(1, 512, 40, 40), torch.randn(1, 1024, 20, 20)] >>> text = torch.randn(1, 80, 512) >>> outputs = world_detect(x, text) r)embedwith_bnr*ct|||t|dt|jdt j fd|D|_t j fd|D|_y)] Initialize YOLO detection layer with nc classes and layer channels ch. Args: nc (int): Number of classes. embed (int): Embedding dimension. with_bn (bool): Whether to use batch normalization in contrastive head. ch (tuple): Tuple of channel sizes from backbone feature maps. rr.c 3K|]F}tjt|dtdtjdHywr<r1r2rr3r5r6r?rs r9r:z'WorldDetect.__init__..sB umntAr1~tBAPRPYPYZ\^cefPg!h uA Ac3LK|]}r tn tywNrrr5rrrs r9r:z'WorldDetect.__init__.." h`aW!25!9/J[![ h!$N) rBrCrJrKr)r1rLrOrr8r)rrr*r?rWs `` @r9rCzWorldDetect.__init__s] R ADGGS) *== urt uu== heg hhrXr6textrYc t|jD]]}tj|j||||j ||j ||||fd||<_|jr|S|j|jdzz|_ |j|}|jr|S||fS)r[rr-) r]rErGr^rMrrOr_r)r4rFr`ra)r8r6rrbrcs r9rdzWorldDetect.forwardstww [A99kdhhqk!A$/!XXa[1=NPT1UVXYZAaD [ ==H''DLL1,, OOA KKq+aV+rXc|}t|j|j|jD]"\}}}d|djj dd$y)rrrpN)rrMrOrIrrrs r9rzWorldDetect.bias_init sI 155!%%2 %GAq!!$AbEJJOOA  %rXrFr)rrrrrrrrCrrGrrrdrrrs@r9rrso, i3 iC i iRW i,ell+,5<<,E$u||J\^cJcDd,%rXrc eZdZdZddej dej dej deffd Zdejdejfd Z d e jd e jd e deee jffd ZxZS)LRPCHeada] Lightweight Region Proposal and Classification Head for efficient object detection. This head combines region proposal filtering with classification to enable efficient detection with dynamic vocabulary support. Attributes: vocab (nn.Module): Vocabulary/classification layer. pf (nn.Module): Proposal filter module. loc (nn.Module): Localization module. enabled (bool): Whether the head is enabled. Methods: conv2linear: Convert a 1x1 convolutional layer to a linear layer. forward: Process classification and localization features to generate detection proposals. Examples: Create an LRPC head >>> vocab = nn.Conv2d(256, 80, 1) >>> pf = nn.Conv2d(256, 1, 1) >>> loc = nn.Conv2d(256, 4, 1) >>> head = LRPCHead(vocab, pf, loc, enabled=True) vocabpflocenabledct||r|j|n||_||_||_||_y)a` Initialize LRPCHead with vocabulary, proposal filter, and localization components. Args: vocab (nn.Module): Vocabulary/classification module. pf (nn.Module): Proposal filter module. loc (nn.Module): Localization module. enabled (bool): Whether to enable the head functionality. N)rBrC conv2linearrrrr)r8rrrrrWs r9rCzLRPCHead.__init__-s; 07T%%e,U  rXrrYcxt|tjr|jdk(sJtj|j |j }|jj|j dj|j_ |jj|j_ |S)z4Convert a 1x1 convolutional layer to a linear layer.)rrrp) rr1r3 kernel_sizer in_channels out_channelsweightrrr)r8rrs r9r!zLRPCHead.conv2linear=s$ *t/?/?6/III4++T->->?![[--d.?.?DII 99>>  rXcls_featloc_featconfc|jr|j|djd}|j|kD}|jdj dd}|j |r |dd|fn!||j djz}|j||j ddf|fS|j |}|j|}||jdftj|jd|jdz|jtjfS) zQProcess classification and localization features to generate detection proposals.)rrrrhrpNr0)rdtype)rrrrrurrrrrGonesrrr)r8r'r(r)pf_scoremasks r9rdzLRPCHead.forwardEs <<wwx(.66q9H##%,D''*44R>> yoloe_detect = YOLOEDetect(nc=80, embed=512, with_bn=True, ch=(256, 512, 1024)) >>> x = [torch.randn(1, 256, 80, 80), torch.randn(1, 512, 40, 40), torch.randn(1, 1024, 20, 20)] >>> cls_pe = torch.randn(1, 80, 512) >>> outputs = yoloe_detect(x, cls_pe) Fr)rrr*ct|||t|dt|jdksJdusJ|j rt jfd|Dnt jfd|D|_t jfd|D|_ tt|_ t||_|_y)r rr.Tc 3K|]F}tjt|dtdtjdHywr<r r s r9r:z'YOLOEDetect.__init__..s?nfg"--QARQSUW\^_I`anr c 3K|]}tjtjt||dt|dtjtdtdtjdywr<)r1r2rrr3r s r9r:z'YOLOEDetect.__init__..sq   MM&Aq/42q>BMM&R"3T"b!_EIIb%+sB B c3LK|]}r tn tywrrrs r9r:z'YOLOEDetect.__init__..rrN)rBrCrJrKr)rNr1rLrOrrrreprtarsavperrs `` @r9rCzYOLOEDetect.__init__vs R ADGGS) *U{{${{ MMnkmn n   == heg hhy67 2r5)  rX txt_featscF|jry|jrJ|jtjj d}t |j|jD]\}}t|tjsJt|tsJ|d}t|tjsJ|j}|j}|j }||j#z}t%||}|j&j(j dj d} |jj(} || z} || j+dj-dzj d} tj.| |z} tj|j0| j2ddj5dj|j&j6}|j&j(j9| j-dj-d|jj(j9| | z||d<|j;|`tj>|_d|_y)z>Fuse text features with model weights for efficient inference.Nrrpr)r#FT) is_fusedr_torGfloat32squeezerrOrrr1r2rr3 logit_scalerrexpr r&rrr ones_liker$rrequires_grad_rcopy_fuser6rP) r8r8cls_headbn_headrr>rrtwrb1b2s r9rCzYOLOEDetect.fuses  == ==  LL/77: !$TXXtxx!8!  Hgh 6 66g'89 99BB$t+B $$GGAJ !  &DKK&&'  KK   " "1;;r?#<#.s)w!++a*;)wrwrxr-rprhr>r{r|r0r~N.)r:r]rErOrMrlrpcrrargetattrappendrr rIrrrGr^rr4rrrrrrQrr)r8r6rRmasksrbr'r(r/rrrmrrrrrrrrcs r9 forward_lrpczYOLOEDetect.forward_lrpcs}}QQQ}tww A"txx{1Q4(H"txx{1Q4(HdiilH5 55%1(T\\AwW[]cejOkJAaD$ LL  !Q  <<4::.)w\abRc\]STUVSWRceiepepruEv)w &DL$,DJiiaPAE!HdllQ.>CPRSTii+2A+Q/ ;;4;;*??1XF1XF ffff%EcjjYaabcefhijI<<4;;q>I#=>D%%dhhsmd&:DLL>> yoloe_segment = YOLOESegment(nc=80, nm=32, npr=256, embed=512, with_bn=True, ch=(256, 512, 1024)) >>> x = [torch.randn(1, 256, 80, 80), torch.randn(1, 512, 40, 40), torch.randn(1, 1024, 20, 20)] >>> text = torch.randn(1, 80, 512) >>> outputs = yoloe_segment(x, text) r)rrrrr*ct|||||_|_t |djj_t |ddzjtjfd|D_ y)a Initialize YOLOESegment with class count, mask parameters, and embedding dimensions. Args: nc (int): Number of classes. nm (int): Number of masks. npr (int): Number of protos. embed (int): Embedding dimension. with_bn (bool): Whether to use batch normalization in contrastive head. ch (tuple): Tuple of channel sizes from backbone feature maps. rr-c 3K|]P}tjt|dtdtjjdRywr<r)r5r6c5r8s r9r:z(YOLOESegment.__init__..?rr@N) rBrCrrrrrJr1rLcv5) r8r)rrrrr*rbrWs ` @r9rCzYOLOESegment.__init__+sq UGR02a5$((DGG4 A!TWW %== wtv wwrXr6rrYc |j|d}|jd}tjt |j Dcgc]5}|j |||j||jd7c}d}t|d}|stj|||}ntj|||d\}}|jr|||fS|r2|jr|js|jzn|df}|jrtj||gd|fStj|d|gd|d||ffScc}w) rrrprhrUT)rR.r)rrrGr^r]rErcrrr\r%rdr_rarr) r8r6rrrrbrhas_lrpcr/s r9rdzYOLOESegment.forwardAs; JJqt  WWQZ YYtwwXA  AaD)..r477B?XZ[ \4(##D!T2A!))$4T)JGAt ==b!8O &*kk$,,"txxz/BsTXyMB-1[[ 1b'1%q)guyy!A$PRUV?WZ[\]Z^`bdeYf>ggYs :E)rrrrFr)rrrrrrrrCrrGrrrdrrs@r9r&r&s0prxx #x/2xADxUYxglx,hell+h5<<hE%QVQ]Q]J]D^hrXr&ceZdZdZdZddddddd d d ej d d dddfdedededededededede dejdedede de de ffd Z d3d e ejd!eed"eeejffd#Zd$ej(d%d&fd'e e ed(e d)ej*d*ed+e d"eejejff d,Zd e ejd"eeje e effd-Z d4d.ejd'e e ed/eejd0eejd"eejejejejff d1Zd2ZxZS)5r#a Real-Time Deformable Transformer Decoder (RTDETRDecoder) module for object detection. This decoder module utilizes Transformer architecture along with deformable convolutions to predict bounding boxes and class labels for objects in an image. It integrates features from multiple layers and runs through a series of Transformer decoder layers to output the final predictions. Attributes: export (bool): Export mode flag. hidden_dim (int): Dimension of hidden layers. nhead (int): Number of heads in multi-head attention. nl (int): Number of feature levels. nc (int): Number of classes. num_queries (int): Number of query points. num_decoder_layers (int): Number of decoder layers. input_proj (nn.ModuleList): Input projection layers for backbone features. decoder (DeformableTransformerDecoder): Transformer decoder module. denoising_class_embed (nn.Embedding): Class embeddings for denoising. num_denoising (int): Number of denoising queries. label_noise_ratio (float): Label noise ratio for training. box_noise_scale (float): Box noise scale for training. learnt_init_query (bool): Whether to learn initial query embeddings. tgt_embed (nn.Embedding): Target embeddings for queries. query_pos_head (MLP): Query position head. enc_output (nn.Sequential): Encoder output layers. enc_score_head (nn.Linear): Encoder score prediction head. enc_bbox_head (MLP): Encoder bbox prediction head. dec_score_head (nn.ModuleList): Decoder score prediction heads. dec_bbox_head (nn.ModuleList): Decoder bbox prediction heads. Methods: forward: Run forward pass and return bounding box and classification scores. Examples: Create an RTDETRDecoder >>> decoder = RTDETRDecoder(nc=80, ch=(512, 1024, 2048), hd=256, nq=300) >>> x = [torch.randn(1, 512, 64, 64), torch.randn(1, 1024, 32, 32), torch.randn(1, 2048, 16, 16)] >>> outputs = decoder(x) Fr)rirr(r-rgrrpr.rxrr)r*hdnqndpnhndld_ffndropoutacteval_idxndlabel_noise_ratiobox_noise_scalelearnt_init_queryc t||_||_t ||_||_||_||_tjfd|D|_ t||| | |j |}t||| |_tj||_| |_| |_||_||_|rtj||_t-ddzd|_tj0tj2tj4|_tj2||_t-dd|_tjt=|Dcgc]}tj2|c}|_tjt=|Dcgc]}t-ddc}|_ |jCycc}wcc}w)a Initialize the RTDETRDecoder module with the given parameters. Args: nc (int): Number of classes. ch (tuple): Channels in the backbone feature maps. hd (int): Dimension of hidden layers. nq (int): Number of query points. ndp (int): Number of decoder points. nh (int): Number of heads in multi-head attention. ndl (int): Number of decoder layers. d_ffn (int): Dimension of the feed-forward networks. dropout (float): Dropout rate. act (nn.Module): Activation function. eval_idx (int): Evaluation index. nd (int): Number of denoising. label_noise_ratio (float): Label noise ratio. box_noise_scale (float): Box noise scale. learnt_init_query (bool): Whether to learn initial query embeddings. c 3K|]D}tjtj|ddtjFyw)rF)rN)r1r2r3 BatchNorm2d)r5r6rjs r9r:z)RTDETRDecoder.__init__..s9'wop bii2qu6UWYWeWefhWi(j'wsA A r-rh) num_layersr0N)"rBrC hidden_dimnheadrDrEr) num_queriesnum_decoder_layersr1rL input_projrrdecoder Embeddingdenoising_class_embed num_denoisingrtrurv tgt_embedrquery_pos_headr2r LayerNorm enc_outputenc_score_head enc_bbox_headr]dec_score_head dec_bbox_head_reset_parameters)r8r)r*rjrkrlrmrnrorprqrrrsrtrurv decoder_layerrrWs ` r9rCzRTDETRDecoder.__init__sN  b'"%--'wtv'ww :"b%RUW[W^W^`cd 3B sHU &(\\"b%9"!2."3 \\"b1DN!!QVRA>-- "b(92<<;KL iiB/ Rq9!mmc ,S1RYYr2->,ST]]RWX[R\+]QCBa,H+]^  -T+]s HH r6batchrYc zddlm}|j|\}}|||j|j|j j |j|j|j|j\}}}} |j||||\} } } } |j| | |||j|j|j|\}}||| | | f}|jr|St!j"|j%d|j%dj'fd}|j(r|S||fS)a* Run the forward pass of the module, returning bounding box and classification scores for the input. Args: x (List[torch.Tensor]): List of feature maps from the backbone. batch (dict, optional): Batch information for training. Returns: outputs (tuple | torch.Tensor): During training, returns a tuple of bounding boxes, scores, and other metadata. During inference, returns a tensor of shape (bs, 300, 4+nc) containing bounding boxes and class scores. r) get_cdn_group) attn_maskrp)ultralytics.models.utils.opsr_get_encoder_inputr)r}rr&rrtrur__get_decoder_inputrrrrrGr^r=rra)r8r6rrfeatsshapesdn_embeddn_bboxrdn_metar refer_bbox enc_bboxes enc_scores dec_bboxes dec_scoresrcs r9rdzRTDETRDecoder.forwardsF ?//2 v1>  GG     & & - -     " "  MM 1 -'9g594K4KESY[cel4m1z:z"&             ". "  J  J G C ==H IIz))!,j.@.@.C.K.K.MNPR SKKq+aV+rXg?cpu{Gz?rrr,repsc.g}t|D]\}\}} tj|||} tj| ||} trtj| | dntj| | \} } tj | | gd}tj | |g||}|jddz|z }tj||||zd|zz}|jtj||gdjd|| zd tj|d }||kD|d |z kzjdd }tj|d |z z }|j|td }||fS)aL Generate anchor bounding boxes for given shapes with specific grid size and validate them. Args: shapes (list): List of feature map shapes. grid_size (float, optional): Base size of grid cells. dtype (torch.dtype, optional): Data type for tensors. device (str, optional): Device to create tensors on. eps (float, optional): Small value for numerical stability. Returns: anchors (torch.Tensor): Generated anchor boxes. valid_mask (torch.Tensor): Valid mask for anchors. )endr,rij)indexingrpr,rrrxrr-rT)keepdiminf) enumeraterGrr meshgridstackrrr@rWr^rallr masked_fillr)r8rrr,rrrrbhrGsysxgrid_ygrid_xgrid_xyvalid_WHwh valid_masks r9_generate_anchorszRTDETRDecoder._generate_anchorss~,"6* LIAv1!5@B!5@BFPU^^BTBV[VdVdegikVlNFFkk66"2B7G||QF%GH((+c1X=GfE QUXZ[U[\B NN599gr]B7<DAq LLa00Aq9 : MM1a& !  " %#f} CsB,rrrc|jd}|j||j|j\}}|j ||z}|j |} t j| jdj|jdjjd} t j|| jjdjd|jjd} || | fj||jd} |dd| fj||jd} |j!| | z}|j#}|t j$||gd}| | | fj||jd}|j&r6|j(j*jdj|ddn| }|j,r,|j/}|j&s|j/}|t j$||gd}||||fS)a Generate and prepare the input required for the decoder from the provided features and shapes. Args: feats (torch.Tensor): Processed features from encoder. shapes (list): List of feature map shapes. dn_embed (torch.Tensor, optional): Denoising embeddings. dn_bbox (torch.Tensor, optional): Denoising bounding boxes. Returns: embeddings (torch.Tensor): Query embeddings for decoder. refer_bbox (torch.Tensor): Reference bounding boxes. enc_bboxes (torch.Tensor): Encoded bounding boxes. enc_scores (torch.Tensor): Encoded scores. rrrprr)rr,N)rrr,rrrrGrrJvaluesr}indicesrrrrrrr^rvrr&r_ri)r8rrrrrrrfeaturesenc_outputs_scorestopk_ind batch_indtop_k_features top_k_anchorsrrr embeddingss r9rz RTDETRDecoder._get_decoder_inputFs,[[^"44V5;;W\WcWc4d??:#56!00:::044R8??AQAQWXYaaffgijLLRx~~>HHLSSTUW[WgWghmmnpq ")X"56;;B@P@PRTU8 ,11"d6F6FK ''7-G '')  GZ#8!>r1aHhv ==#**,J))'..0  Hj#91=J:z:==rXctddz |jz}t|jj|t|j j djdt|j j djdt|j|jD]a\}}t|j|t|j djdt|j djdct|jdt|jdj|jrt|jjt|j j djt|j j dj|j"D]}t|djy)zhInitialize or reset the parameters of the model's various components with predefined weights and biases.rrrprrrN)rr)rrrrlayersr&rrrrrrrvrrr)r8bias_clscls_reg_layers r9rzRTDETRDecoder._reset_parameterss't,r1DGG; $%%**H5$$$++B/66<$$$++B/44c:d1143E3EF 1JD$ dii * dkk"o,,c 2 dkk"o**C 0  1 DOOA&'*112  ! ! DNN11 2++2215<<=++2215<<=__ -E E!HOO , -rXr)NN)rrrrrar1ReLUrrrr0rrCrrGrrrrrdr<r,strrrrrrrs@r9r#r#WsS&PF% #&!$"'#M! M! M!  M!  M!  M! M!M!M!M!YYM!M! M!!M! !M!" #M!^0,ell+0,HTN0,eTY[`[g[gTgNh0,j "]] &#T#Y&#&#{{ &#  &#  &# u||U\\) *&#PD$65tTXY\T]A^;_>,0*. 8>||8>T#Y8>5<<( 8> %,,' 8> u||U\\5<<E F 8>t-rXr#c8eZdZdZdZddedeffd ZdZxZ S)r$a v10 Detection head from https://arxiv.org/pdf/2405.14458. This class implements the YOLOv10 detection head with dual-assignment training and consistent dual predictions for improved efficiency and performance. Attributes: end2end (bool): End-to-end detection mode. max_det (int): Maximum number of detections. cv3 (nn.ModuleList): Light classification head layers. one2one_cv3 (nn.ModuleList): One-to-one classification head layers. Methods: __init__: Initialize the v10Detect object with specified number of classes and input channels. forward: Perform forward pass of the v10Detect module. bias_init: Initialize biases of the Detect module. fuse: Remove the one2many head for inference optimization. Examples: Create a v10Detect head >>> v10_detect = v10Detect(nc=80, ch=(256, 512, 1024)) >>> x = [torch.randn(1, 256, 80, 80), torch.randn(1, 512, 40, 40), torch.randn(1, 1024, 20, 20)] >>> outputs = v10_detect(x) Tr)r*ct||t|dtjdt j fd|D_tjj_ y)z Initialize the v10Detect object with the specified number of classes and input channels. Args: nc (int): Number of classes. ch (tuple): Tuple of channel sizes from backbone feature maps. rr.c 3:K|]}tjtjt||d|t|dtjtdtdtjjdyw)r0)rrNr=r>s r9r:z%v10Detect.__init__..sz!  MM d1aa0$q"a.A d2r13T"b!_E "dggq) ! sBBN) rBrCrJrKr)r1rLrOrSrTrV)r8r)r*r?rWs` @r9rCzv10Detect.__init__se R ADGGS) *==!  !   ==2rXctjtjg|jzx|_|_y)z4Remove the one2many head for inference optimization.N)r1rLrPrErMrO)r8s r9rCzv10Detect.fuses* mmR[[]Odgg,EFF48rXr) rrrrrRrrrCrCrrs@r9r$r$s(2G3333*GrXr$)8rrSrtypingrrrrrGtorch.nnr1torch.nn.functional functionalrL torch.nn.initrrultralytics.utils.talr r r r ultralytics.utils.torch_utilsr rblockrrrrrrrrrr transformerrrrutilsrr__all__r0rrr"r r!rrr%r&r#r$rrXr9rs // 4PPP]]]]]3 uNnRYYNnb1hf1hh5=&5=pK6K\4,ryy4,n6%&6%t>ryy>B{A&{A|Ah;AhH-BII-D 3G3GrX