|hz dZddlZddlmZmZddlZddlmZddlmcm Z ddl m Z m Z ddlmZddlmZmZmZdZGd d ej*ZGd d eZGd dej*ZGddej*ZGddej*ZGddej*ZGddej*ZGddej*ZGddej*ZGddej*Zy)zTransformer modules.N)ListOptional) constant_xavier_uniform_)Conv) _get_clonesinverse_sigmoid#multi_scale_deformable_attn_pytorch) TransformerEncoderLayerTransformerLayerTransformerBlockMLPBlock LayerNorm2dAIFIDeformableTransformerDecoder!DeformableTransformerDecoderLayer MSDeformAttnMLPc eZdZdZdddej dfdededed ed ejd e f fd Z e dd e jdee jde jfdZ dde jdee jdee jdee jde jf dZ dde jdee jdee jdee jde jf dZ dde jdee jdee jdee jde jf dZxZS)r a A single layer of the transformer encoder. This class implements a standard transformer encoder layer with multi-head attention and feedforward network, supporting both pre-normalization and post-normalization configurations. Attributes: ma (nn.MultiheadAttention): Multi-head attention module. fc1 (nn.Linear): First linear layer in the feedforward network. fc2 (nn.Linear): Second linear layer in the feedforward network. norm1 (nn.LayerNorm): Layer normalization after attention. norm2 (nn.LayerNorm): Layer normalization after feedforward network. dropout (nn.Dropout): Dropout layer for the feedforward network. dropout1 (nn.Dropout): Dropout layer after attention. dropout2 (nn.Dropout): Dropout layer after feedforward network. act (nn.Module): Activation function. normalize_before (bool): Whether to apply normalization before attention and feedforward. Fc1cm num_headsdropoutactnormalize_beforect|ddlm}|s t dt j |||d|_t j|||_ t j|||_ t j||_ t j||_ t j||_t j||_t j||_||_||_y)a Initialize the TransformerEncoderLayer with specified parameters. Args: c1 (int): Input dimension. cm (int): Hidden dimension in the feedforward network. num_heads (int): Number of attention heads. dropout (float): Dropout probability. act (nn.Module): Activation function. normalize_before (bool): Whether to apply normalization before attention and feedforward. ) TORCH_1_9z]TransformerEncoderLayer() requires torch>=1.9 to use nn.MultiheadAttention(batch_first=True).T)r batch_firstN)super__init__utils.torch_utilsr"ModuleNotFoundErrornnMultiheadAttentionmaLinearfc1fc2 LayerNormnorm1norm2Dropoutrdropout1dropout2rr) selfrrrrrrr" __class__s a/var/www/html/test/engine/venv/lib/python3.12/site-packages/ultralytics/nn/modules/transformer.pyr%z TransformerEncoderLayer.__init__1s( 2%o ''IwTXY99R$99R$\\"% \\"% zz'*  7+  7+  0tensorposreturnc||S||zS)z2Add position embeddings to the tensor if provided.r8r9s r6with_pos_embedz&TransformerEncoderLayer.with_pos_embedZv6&3,6r7srcsrc_masksrc_key_padding_maskc l|j||x}}|j|||||d}||j|z}|j|}|j |j |j |j|}||j|z}|j|S)a Perform forward pass with post-normalization. Args: src (torch.Tensor): Input tensor. src_mask (torch.Tensor, optional): Mask for the src sequence. src_key_padding_mask (torch.Tensor, optional): Mask for the src keys per batch. pos (torch.Tensor, optional): Positional encoding. Returns: (torch.Tensor): Output tensor after attention and feedforward. value attn_maskkey_padding_maskr) r>r*r2r/r-rrr,r3r0)r4r@rArBr9qksrc2s r6 forward_postz$TransformerEncoderLayer.forward_post_s&##C--Awwq!3(MawbcdeDMM$''jjoxx TXXdhhsm%<=>DMM$''zz#r7c l|j|}|j||x}}|j|||||d}||j|z}|j |}|j |j |j|j|}||j|zS)a Perform forward pass with pre-normalization. Args: src (torch.Tensor): Input tensor. src_mask (torch.Tensor, optional): Mask for the src sequence. src_key_padding_mask (torch.Tensor, optional): Mask for the src keys per batch. pos (torch.Tensor, optional): Positional encoding. Returns: (torch.Tensor): Output tensor after attention and feedforward. rDr) r/r>r*r2r0r-rrr,r3)r4r@rArBr9rJrHrIs r6 forward_prez#TransformerEncoderLayer.forward_prezs&zz###D#..Awwq!48NbwcdefDMM$''zz#xx TXXdhhtn%=>?T]]4(((r7cj|jr|j||||S|j||||S)a Forward propagate the input through the encoder module. Args: src (torch.Tensor): Input tensor. src_mask (torch.Tensor, optional): Mask for the src sequence. src_key_padding_mask (torch.Tensor, optional): Mask for the src keys per batch. pos (torch.Tensor, optional): Positional encoding. Returns: (torch.Tensor): Output tensor after transformer encoder layer. )rrMrK)r4r@rArBr9s r6forwardzTransformerEncoderLayer.forwards=&  ##C3GM M  h0DcJJr7NNNN)__name__ __module__ __qualname____doc__r(GELUintfloatModuleboolr% staticmethodtorchTensorrr>rKrMrO __classcell__r5s@r6r r s, !&'1 '1 '1 '1  '1 YY '1'1R7u||7(5<<2H7TYT`T`77,07;&*  \\5<<('u||4  ell #   <,07;&* ) \\)5<<()'u||4 ) ell # )  )<,07;&* K \\K5<<(K'u||4 K ell # K  Kr7r c eZdZdZdddej dfdededed ed ejd e f fd Z d e jde jffd Z e ddedededede jf dZxZS)rz AIFI transformer layer for 2D data with positional embeddings. This class extends TransformerEncoderLayer to work with 2D feature maps by adding 2D sine-cosine positional embeddings and handling the spatial dimensions appropriately. rrrFrrrrrrc.t|||||||y)a Initialize the AIFI instance with specified parameters. Args: c1 (int): Input dimension. cm (int): Hidden dimension in the feedforward network. num_heads (int): Number of attention heads. dropout (float): Dropout probability. act (nn.Module): Activation function. normalize_before (bool): Whether to apply normalization before attention and feedforward. N)r$r%)r4rrrrrrr5s r6r%z AIFI.__init__s( RGS:JKr7xr:cd|jdd\}}}|j|||}t| |j dj ddd|j |j|j}|j dddjd|||gjS)z Forward pass for the AIFI transformer layer. Args: x (torch.Tensor): Input tensor with shape [B, C, H, W]. Returns: (torch.Tensor): Output tensor with shape [B, C, H, W]. rNr)devicedtype)r9) shape"build_2d_sincos_position_embeddingr$rOflattenpermutetorerfview contiguous)r4rbchw pos_embedr5s r6rOz AIFI.forwards''!"+1a;;Aq!D GOAIIaL00Aq9y||STS[S[cdcjcj|?kO lyyAq!&&Aq!}5@@BBr7rqrp embed_dim temperaturec|dzdk(sJdtj|tj}tj|tj}tj||d\}}|dz}tj|tj|z }d||zz }|j d|d z}|j d|d z} tj tj |tj|tj | tj| gd d S) a Build 2D sine-cosine position embedding. Args: w (int): Width of the feature map. h (int): Height of the feature map. embed_dim (int): Embedding dimension. temperature (float): Temperature for the sine/cosine functions. Returns: (torch.Tensor): Position embedding with shape [1, embed_dim, h*w]. rzHEmbed dimension must be divisible by 4 for 2D sin-cos position embeddingrfij)indexingg?.NNr)r\arangefloat32meshgridrjcatsincos) rqrprsrtgrid_wgrid_hpos_dimomegaout_wout_hs r6riz'AIFI.build_2d_sincos_position_embeddings 1}!m#mm!au}}5au}}5Fq. WEMM:WD{E)* +eDk9 +eDk9yy%))E*EIIe,NPUPYPYZ_P`acdefjkkr7)g@)rRrSrTrUr(rVrWrXrYrZr%r\r]rOr[rir^r_s@r6rrs !&L L L L  L YY LL,CC%,,C CJl ll#&l;@l llr7rcdeZdZdZdedeffd ZdejdejfdZxZ S)r zeTransformer layer https://arxiv.org/abs/2010.11929 (LayerNorm layers removed for better performance).rorc|t|tj||d|_tj||d|_tj||d|_tj|||_tj||d|_ tj||d|_ y)z Initialize a self-attention mechanism using linear transformations and multi-head attention. Args: c (int): Input and output channel dimension. num_heads (int): Number of attention heads. F)bias)rsrN) r$r%r(r+rHrIvr)r*r,r-)r4rorr5s r6r%zTransformerLayer.__init__s 1ae,1ae,1ae,''!yI99Q.99Q.r7rbr:c|j|j||j||j|d|z}|j |j ||zS)z Apply a transformer block to the input x and return the output. Args: x (torch.Tensor): Input tensor. Returns: (torch.Tensor): Output tensor after transformer layer. r)r*rHrIrr-r,r4rbs r6rOzTransformerLayer.forward sT GGDFF1Itvvay$&&) 4Q 7! ;xx $q((r7 rRrSrTrUrWr%r\r]rOr^r_s@r6r r s4o/#/#/ ) )%,, )r7r cleZdZdZdedededeffd Zdejdejfd ZxZ S) ra Vision Transformer block based on https://arxiv.org/abs/2010.11929. This class implements a complete transformer block with optional convolution layer for channel adjustment, learnable position embedding, and multiple transformer layers. Attributes: conv (Conv, optional): Convolution layer if input and output channels differ. linear (nn.Linear): Learnable position embedding. tr (nn.Sequential): Sequential container of transformer layers. c2 (int): Output channel dimension. rc2r num_layersct|d|_|k7rt||_t j |_t jfdt|D|_ |_ y)aU Initialize a Transformer module with position embedding and specified number of heads and layers. Args: c1 (int): Input channel dimension. c2 (int): Output channel dimension. num_heads (int): Number of attention heads. num_layers (int): Number of transformer layers. Nc36K|]}tywrP)r ).0_rrs r6 z,TransformerBlock.__init__..7s!]a"22y"A!]s) r$r%convrr(r+linear Sequentialrangetrr)r4rrrrr5s `` r6r%zTransformerBlock.__init__(s`  8R DIiiB' --!]5Q[K\!]^r7rbr:cB|j|j|}|j\}}}}|jdjddd}|j ||j |zjdddj ||j||S)z Forward propagate the input through the transformer block. Args: x (torch.Tensor): Input tensor with shape [b, c1, w, h]. Returns: (torch.Tensor): Output tensor with shape [b, c2, w, h]. rdrr)rrhrjrkrrreshaper)r4rbbrrqrpps r6rOzTransformerBlock.forward:s 99  ! AWW 1a IIaL Aq )wwq4;;q>)*221a;CCAtwwPQSTUUr7rr_s@r6rrsG 3CCS$VV%,,Vr7rc|eZdZdZej fdedeffd ZdejdejfdZ xZ S)rz+A single block of a multi-layer perceptron. embedding_dimmlp_dimct|tj|||_tj|||_||_y)a  Initialize the MLPBlock with specified embedding dimension, MLP dimension, and activation function. Args: embedding_dim (int): Input and output dimension. mlp_dim (int): Hidden dimension. act (nn.Module): Activation function. N)r$r%r(r+lin1lin2r)r4rrrr5s r6r%zMLPBlock.__init__Ns= IImW5 IIg}5 5r7rbr:c`|j|j|j|S)z Forward pass for the MLPBlock. Args: x (torch.Tensor): Input tensor. Returns: (torch.Tensor): Output tensor after MLP block. )rrrrs r6rOzMLPBlock.forward\s$yy$))A,/00r7) rRrSrTrUr(rVrWr%r\r]rOr^r_s@r6rrKs;5=?WW c C  1 1%,, 1r7rc eZdZdZej dfdededededef fd Zd e jd e jfd Z xZ S) ra A simple multi-layer perceptron (also called FFN). This class implements a configurable MLP with multiple linear layers, activation functions, and optional sigmoid output activation. Attributes: num_layers (int): Number of layers in the MLP. layers (nn.ModuleList): List of linear layers. sigmoid (bool): Whether to apply sigmoid to the output. act (nn.Module): Activation function. F input_dim hidden_dim output_dimrsigmoidct|||_|g|dz z}tjdt |g|z||gzD|_||_||_y)a Initialize the MLP with specified input, hidden, output dimensions and number of layers. Args: input_dim (int): Input dimension. hidden_dim (int): Hidden dimension. output_dim (int): Output dimension. num_layers (int): Number of layers. act (nn.Module): Activation function. sigmoid (bool): Whether to apply sigmoid to the output. rc3NK|]\}}tj||ywrP)r(r+)rnrIs r6rzMLP.__init__..s#g1BIIaO#gs#%N) r$r%rr( ModuleListziplayersrr) r4rrrrrrrpr5s r6r%z MLP.__init__wse $ LJN +mm#gYKRSOUVZdYeUe@f#gg  5r7rbr:ct|jD]J\}}||jdz kr+t|dt j ||n||}Lt|ddr|j S|S)z Forward pass for the entire MLP. Args: x (torch.Tensor): Input tensor. Returns: (torch.Tensor): Output tensor after MLP. rrrF) enumeraterrgetattrr(ReLUr)r4rbilayers r6rOz MLP.forwardsx"$++. cHAu=>STAT=T/eRWWY/a9Z_`aZbA c%dIu=qyy{D1Dr7) rRrSrTrUr(rrWrZr%r\r]rOr^r_s@r6rrisc VXU\U\ns*-;>LOgk* E E%,, Er7rcfeZdZdZddedeffd ZdejdejfdZ xZ S) rau 2D Layer Normalization module inspired by Detectron2 and ConvNeXt implementations. This class implements layer normalization for 2D feature maps, normalizing across the channel dimension while preserving spatial dimensions. Attributes: weight (nn.Parameter): Learnable scale parameter. bias (nn.Parameter): Learnable bias parameter. eps (float): Small constant for numerical stability. References: https://github.com/facebookresearch/detectron2/blob/main/detectron2/layers/batch_norm.py https://github.com/facebookresearch/ConvNeXt/blob/main/models/convnext.py num_channelsepsct|tjt j ||_tjt j||_||_ y)z Initialize LayerNorm2d with the given parameters. Args: num_channels (int): Number of channels in the input. eps (float): Small constant for numerical stability. N) r$r%r( Parameterr\onesweightzerosrr)r4rrr5s r6r%zLayerNorm2d.__init__sI ll5::l#;< LL\!:; r7rbr:c|jdd}||z jdjdd}||z tj||jzz }|j ddddf|z|j ddddfzS)z Perform forward pass for 2D layer normalization. Args: x (torch.Tensor): Input tensor. Returns: (torch.Tensor): Normalized output tensor. rTkeepdimrdN)meanpowr\sqrtrrr)r4rbuss r6rOzLayerNorm2d.forwards FF1dF # UKKN  4  0 UejjTXX. .{{1dD=)A- !T4-0HHHr7)gư>) rRrSrTrUrWrXr%r\r]rOr^r_s@r6rrs9 S u  I I%,, Ir7rc eZdZdZddedededeffd ZdZ ddejd ejd ejd e d e ejd ejf dZ xZ S)ra Multiscale Deformable Attention Module based on Deformable-DETR and PaddleDetection implementations. This module implements multiscale deformable attention that can attend to features at multiple scales with learnable sampling locations and attention weights. Attributes: im2col_step (int): Step size for im2col operations. d_model (int): Model dimension. n_levels (int): Number of feature levels. n_heads (int): Number of attention heads. n_points (int): Number of sampling points per attention head per feature level. sampling_offsets (nn.Linear): Linear layer for generating sampling offsets. attention_weights (nn.Linear): Linear layer for generating attention weights. value_proj (nn.Linear): Linear layer for projecting values. output_proj (nn.Linear): Linear layer for projecting output. References: https://github.com/fundamentalvision/Deformable-DETR/blob/main/models/ops/modules/ms_deform_attn.py d_modeln_levelsn_headsn_pointsct|||zdk7rtd|d|||z}||z|k(sJdd|_||_||_||_||_tj|||z|zdz|_ tj|||z|z|_ tj|||_ tj|||_ |jy)aG Initialize MSDeformAttn with the given parameters. Args: d_model (int): Model dimension. n_levels (int): Number of feature levels. n_heads (int): Number of attention heads. n_points (int): Number of sampling points per attention head per feature level. rz.d_model must be divisible by n_heads, but got z and z(`d_model` must be divisible by `n_heads`@rdN)r$r% ValueError im2col_steprrrrr(r+sampling_offsetsattention_weights value_proj output_proj_reset_parameters)r4rrrr _d_per_headr5s r6r%zMSDeformAttn.__init__s  W  !MgYV[\c[def f( W$/[1[[/      " '7X3E3PST3T U!#7Gh4F4Q!R))GW599Wg6  r7cHt|jjjdt j |j tjdtjz|j z z}t j|j|jgd}||jjdddz j|j ddd j!d|j"|j$d}t'|j$D]}|d d d d |d d fxx|dzzcc<t j(5t+j,|jd|j_d d d t|j0jjdt|j0j.jdt3|j4jjt|j4j.jdt3|j6jjt|j6j.jdy #1swYxYw) zReset module parameters.rrwg@rgTrrrrdN)rrrdatar\r{rr|mathpistackrrabsmaxrmrepeatrrrno_gradr(rrrrrr)r4thetas grid_initrs r6rzMSDeformAttn._reset_parameterss$''..33S9dll%--@C$''MTXT`T`D`aKKvzz| qA A T$,,1a ( VAt}}dmmQ 7  t}}% +A aAqj !QU * ! + ]]_ J)+innR6H)ID ! ! & J$((//44c:$((--22C8..334$//&&++S1((//445$""'',,c2 J Js 4JJ!query refer_bboxrE value_shapes value_maskr:c |jdd\}}|jd}td|D|k(sJ|j|}||j|dt d}|j |||j |j|j z}|j|j |||j |j|jd} |j|j |||j |j|jz} tj| dj |||j |j|j} |jd} | dk(rdtj||j |j"j%d} | | ddddddddfz } |ddddddddddf| z}nQ| d k(r=| |jz |ddddddddddfzd z} |ddddddddddf| z}nt'd | d t)|||| }|j+|S) a Perform forward pass for multiscale deformable attention. Args: query (torch.Tensor): Query tensor with shape [bs, query_length, C]. refer_bbox (torch.Tensor): Reference bounding boxes with shape [bs, query_length, n_levels, 2], range in [0, 1], top-left (0,0), bottom-right (1, 1), including padding area. value (torch.Tensor): Value tensor with shape [bs, value_length, C]. value_shapes (list): List with shape [n_levels, 2], [(H_0, W_0), (H_1, W_1), ..., (H_{L-1}, W_{L-1})]. value_mask (torch.Tensor, optional): Mask tensor with shape [bs, value_length], True for non-padding elements, False for padding elements. Returns: (torch.Tensor): Output tensor with shape [bs, Length_{query}, C]. References: https://github.com/PaddlePaddle/PaddleDetection/blob/develop/ppdet/modeling/transformers/deformable_transformer.py Nrdrc32K|]}|d|dzyw)rrNr<)rrs r6rz'MSDeformAttn.forward../s511Q4!A$;5srzrrg)rfrervg?z5Last dim of reference_points must be 2 or 4, but got .)rhsumr masked_fillrXrmrrrrrrFsoftmaxr\ as_tensorrfrefliprr r)r4rrrErrbslen_qlen_vrr num_pointsoffset_normalizeraddsampling_locationsoutputs r6rOzMSDeformAttn.forwardsH4KKO E A5 55>>>&  !%%j&;U1XFE 2udllDLLDLL4PQ007<>r5$,,X\XeXehlhuhuXuvII&7<AA"eT\\[_[h[hjnjwjwx%%b) ? % EKKX]XdXd e j jkm n "%6tT4DRS7S%TTC!+Aq$4,B!Cc!I  1_"T]]2Z1dAtUVUW@W5XX[^^C!+Aq$4!,C!Ds!J TU_T``abc c4ULJ\^op''r7)rrvrrvrP) rRrSrTrUrWr%rr\r]rrrOr^r_s@r6rrs*!!S!s!Z]!>36.2 1(||1(LL1(|| 1(  1( U\\* 1( 1(r7rceZdZdZddddej ddfdeded ed ed ejd ed effd Z e de jde e jde jfdZde jde jfdZ dde jde jde jdede e jde e jde e jde jfdZxZS)ra; Deformable Transformer Decoder Layer inspired by PaddleDetection and Deformable-DETR implementations. This class implements a single decoder layer with self-attention, cross-attention using multiscale deformable attention, and a feedforward network. Attributes: self_attn (nn.MultiheadAttention): Self-attention module. dropout1 (nn.Dropout): Dropout after self-attention. norm1 (nn.LayerNorm): Layer normalization after self-attention. cross_attn (MSDeformAttn): Cross-attention module. dropout2 (nn.Dropout): Dropout after cross-attention. norm2 (nn.LayerNorm): Layer normalization after cross-attention. linear1 (nn.Linear): First linear layer in the feedforward network. act (nn.Module): Activation function. dropout3 (nn.Dropout): Dropout in the feedforward network. linear2 (nn.Linear): Second linear layer in the feedforward network. dropout4 (nn.Dropout): Dropout after the feedforward network. norm3 (nn.LayerNorm): Layer normalization after the feedforward network. References: https://github.com/PaddlePaddle/PaddleDetection/blob/develop/ppdet/modeling/transformers/deformable_transformer.py https://github.com/fundamentalvision/Deformable-DETR/blob/main/models/deformable_transformer.py rrirrvrrd_ffnrrrrcht|tj||||_tj ||_tj||_t|||||_ tj ||_ tj||_ tj|||_||_tj ||_tj|||_tj ||_tj||_y)a Initialize the DeformableTransformerDecoderLayer with the given parameters. Args: d_model (int): Model dimension. n_heads (int): Number of attention heads. d_ffn (int): Dimension of the feedforward network. dropout (float): Dropout probability. act (nn.Module): Activation function. n_levels (int): Number of feature levels. n_points (int): Number of sampling points. )rN)r$r%r(r) self_attnr1r2r.r/r cross_attnr3r0r+linear1rdropout3linear2dropout4norm3) r4rrrrrrrr5s r6r%z*DeformableTransformerDecoderLayer.__init__as, ..wQ 7+ \\'* 'w'8L 7+ \\'* yy%0  7+ yy0  7+ \\'* r7r8r9r:c||S||zS)z;Add positional embeddings to the input tensor, if provided.r<r=s r6r>z0DeformableTransformerDecoderLayer.with_pos_embedr?r7tgtc |j|j|j|j|}||j |z}|j |S)z Perform forward pass through the Feed-Forward Network part of the layer. Args: tgt (torch.Tensor): Input tensor. Returns: (torch.Tensor): Output tensor after FFN. )rrrrrr)r4rtgt2s r6 forward_ffnz-DeformableTransformerDecoderLayer.forward_ffnsN||DMM$((4<<3D*EFGDMM$''zz#r7embedrfeatsshapes padding_maskrF query_posc|j||x}} |j|jdd| jdd|jdd|djdd} ||j| z}|j |}|j |j|||j d|||} ||j| z}|j|}|j|S)aH Perform the forward pass through the entire decoder layer. Args: embed (torch.Tensor): Input embeddings. refer_bbox (torch.Tensor): Reference bounding boxes. feats (torch.Tensor): Feature maps. shapes (list): Feature shapes. padding_mask (torch.Tensor, optional): Padding mask. attn_mask (torch.Tensor, optional): Attention mask. query_pos (torch.Tensor, optional): Query position embeddings. Returns: (torch.Tensor): Output tensor after decoder layer. rr)rFrd) r>r transposer2r/r unsqueezer3r0r) r4rrrrr rFr rHrIrs r6rOz)DeformableTransformerDecoderLayer.forwards4##E955AnnQ[[A. Aq0A5??STVWCXdmnn  )Aq/  c** 5!oo   y 1:3G3G3JESY[g  c** 5!&&r7rQ)rRrSrTrUr(rrWrXrYr%r[r\r]rr>rrrOr^r_s@r6rrGsS6 (+(+(+ (+  (+ YY (+(+(+T7u||7(5<<2H7U\\77 u||  (04,0,0)'||)'LL)'|| )'  )' u||, )'ELL))'ELL))' )'r7rc(eZdZdZddedej dedeffd Z ddejdejd ejd e d ej d ej d ej de ejde ejfdZ xZ S)rav Deformable Transformer Decoder based on PaddleDetection implementation. This class implements a complete deformable transformer decoder with multiple decoder layers and prediction heads for bounding box regression and classification. Attributes: layers (nn.ModuleList): List of decoder layers. num_layers (int): Number of decoder layers. hidden_dim (int): Hidden dimension. eval_idx (int): Index of the layer to use during evaluation. References: https://github.com/PaddlePaddle/PaddleDetection/blob/develop/ppdet/modeling/transformers/deformable_transformer.py r decoder_layerreval_idxct|t|||_||_||_|dk\r||_y||z|_y)aU Initialize the DeformableTransformerDecoder with the given parameters. Args: hidden_dim (int): Hidden dimension. decoder_layer (nn.Module): Decoder layer module. num_layers (int): Number of decoder layers. eval_idx (int): Index of the layer to use during evaluation. rN)r$r%r rrrr)r4rrrrr5s r6r%z%DeformableTransformerDecoder.__init__sD !-< $$$,M zH7L r7rrrr bbox_head score_headpos_mlprFr c |} g} g} d} |j}t|jD]\}}|| |||| |||} ||| }tj|t |z}|j rb| j ||| |dk(r| j |nm| j tj|t | zn<||jk(r-| j ||| | j |n#|} |j r|jn|}tj| tj| fS)a Perform the forward pass through the entire decoder. Args: embed (torch.Tensor): Decoder embeddings. refer_bbox (torch.Tensor): Reference bounding boxes. feats (torch.Tensor): Image features. shapes (list): Feature shapes. bbox_head (nn.Module): Bounding box prediction head. score_head (nn.Module): Score prediction head. pos_mlp (nn.Module): Position MLP. attn_mask (torch.Tensor, optional): Attention mask. padding_mask (torch.Tensor, optional): Padding mask. Returns: dec_bboxes (torch.Tensor): Decoded bounding boxes. dec_cls (torch.Tensor): Decoded classification scores. Nr) rrrr\r trainingappendrdetachr)r4rrrrrrrrFr r dec_bboxesdec_clslast_refined_bboxrrbbox refined_bboxs r6rOz$DeformableTransformerDecoder.forwardsN<  '') !$++. RHAu6:uflIW^_iWjkF9Q<'D == 0K)KLL}}}z!}V456%%l3%%emmD?K\;]4]&^_dmm#}z!}V45!!,/ , 26--,,.\J% R({{:& G(<<r%s ! 4TT MKbiiMK`Il "IlX)ryy)B.Vryy.Vb1ryy1