ujhT'RddlmZmZddlZdgZdej dej fdZGddejjZ Gd d ejjZ Gd d ejjZ Gd dejjZ y))OptionalTupleN Conformerlengthsreturnc|jd}ttj|j }tj ||j |jj|||jdk\}|S)Nr)devicedtype) shapeinttorchmaxitemaranger r expand unsqueeze)r batch_size max_length padding_masks Y/var/www/html/dev/engine/venv/lib/python3.12/site-packages/torchaudio/models/conformer.py_lengths_to_padding_maskr sqq!JUYYw',,./J<< 7>>W^^J   1 L ceZdZdZ d dededededededd ffd Zd ejdejfd Z xZ S)_ConvolutionModuleaConformer convolution module. Args: input_dim (int): input dimension. num_channels (int): number of depthwise convolution layer input channels. depthwise_kernel_size (int): kernel size of depthwise convolution layer. dropout (float, optional): dropout probability. (Default: 0.0) bias (bool, optional): indicates whether to add bias term to each convolution layer. (Default: ``False``) use_group_norm (bool, optional): use GroupNorm rather than BatchNorm. (Default: ``False``) input_dim num_channelsdepthwise_kernel_sizedropoutbiasuse_group_normrNct||dz dzdk7r tdtjj ||_tjjtjj|d|zddd|tjjdtjj|||d|dz dz|||r!tjjd|ntjj|tjjtjj||ddd| tjj||_y) Nr rz __classcell__r9s@rrrsx  $- - -  # -  -  - -  - ^ !U\\ !ell !rrc neZdZdZd dedededdffd Zdejdejfd Z xZ S) _FeedForwardModulezPositionwise feed forward layer. Args: input_dim (int): input dimension. hidden_dim (int): hidden dimension. dropout (float, optional): dropout probability. (Default: 0.0) r hidden_dimrrNc t|tjj tjj |tjj ||dtjjtjj|tjj ||dtjj||_ y)NT)r ) r*r+rr-r0r.Linearr5r6r7)r8rrLrr9s rr+z_FeedForwardModule.__init__ds ((-- HH  y ) HHOOIzO = HHMMO HH  W % HHOOJ O = HH  W %  rr:c$|j|S)z Args: input (torch.Tensor): with shape `(*, D)`. Returns: torch.Tensor: output, with shape `(*, D)`. )r7)r8r:s rr>z_FeedForwardModule.forwardosu%%r)r@) rArBrCrDr rEr+rrGr>rHrIs@rrKrK[sE  #  3    QU  &U\\&ell&rrKceZdZdZ ddededededededed d ffd Zd ejd ejfd Z d ejde ejd ejfdZ xZ S)ConformerLayeraConformer layer that constitutes Conformer. Args: input_dim (int): input dimension. ffn_dim (int): hidden layer dimension of feedforward network. num_attention_heads (int): number of attention heads. depthwise_conv_kernel_size (int): kernel size of depthwise convolution layer. dropout (float, optional): dropout probability. (Default: 0.0) use_group_norm (bool, optional): use ``GroupNorm`` rather than ``BatchNorm1d`` in the convolution module. (Default: ``False``) convolution_first (bool, optional): apply the convolution module ahead of the attention module. (Default: ``False``) rffn_dimnum_attention_headsdepthwise_conv_kernel_sizerr!convolution_firstrNct|t||||_tj j ||_tj j||||_ tj j||_ t||||d||_ t||||_tj j ||_||_y)N)rT)rrrrr r!)r*r+rKffn1rr-r.self_attn_layer_normMultiheadAttention self_attnr6self_attn_dropoutr conv_moduleffn2final_layer_normrU) r8rrRrSrTrr!rUr9s rr+zConformerLayer.__init__s &y'7K $)HH$6$6y$A!44Y@S]d4e!&!1!1'!:-""<)  'y'7K % 2 29 =!2rr:c~|}|jdd}|j|}|jdd}||z}|S)Nrr )r<r\)r8r:residuals r_apply_convolutionz!ConformerLayer._apply_convolutionsF1%  '1%5  rkey_padding_maskc|}|j|}|dz|z}|jr|j|}|}|j|}|j ||||d\}}|j |}||z}|js|j|}|}|j |}|dz|z}|j|}|S)a  Args: input (torch.Tensor): input, with shape `(T, B, D)`. key_padding_mask (torch.Tensor or None): key padding mask to use in self attention layer. Returns: torch.Tensor: output, with shape `(T, B, D)`. g?F)querykeyvaluerb need_weights)rWrUrarXrZr[r]r^)r8r:rbr`r=_s rr>zConformerLayer.forwards IIe  Gh   ! !''*A  % %a (~~-  1  " "1 % L%%''*A IIaL Gh   ! !! $rr?)rArBrCrDr rErFr+rrGrarr>rHrIs@rrQrQzs ($"'333! 3 %( 3  33 3 3> $U\\$Xell=S$X]XdXd$rrQceZdZdZ ddededededededed effd Zd ejd ejd e ejejffdZ xZ S)ra(Conformer architecture introduced in *Conformer: Convolution-augmented Transformer for Speech Recognition* :cite:`gulati2020conformer`. Args: input_dim (int): input dimension. num_heads (int): number of attention heads in each Conformer layer. ffn_dim (int): hidden layer dimension of feedforward networks. num_layers (int): number of Conformer layers to instantiate. depthwise_conv_kernel_size (int): kernel size of each Conformer layer's depthwise convolution layer. dropout (float, optional): dropout probability. (Default: 0.0) use_group_norm (bool, optional): use ``GroupNorm`` rather than ``BatchNorm1d`` in the convolution module. (Default: ``False``) convolution_first (bool, optional): apply the convolution module ahead of the attention module. (Default: ``False``) Examples: >>> conformer = Conformer( >>> input_dim=80, >>> num_heads=4, >>> ffn_dim=128, >>> num_layers=4, >>> depthwise_conv_kernel_size=31, >>> ) >>> lengths = torch.randint(1, 400, (10,)) # (batch,) >>> input = torch.rand(10, int(lengths.max()), input_dim) # (batch, num_frames, input_dim) >>> output = conformer(input, lengths) r num_headsrR num_layersrTrr!rUc t |tjj t |D cgc]} t |||||||c} |_ycc} w)N)rr!rU)r*r+rr- ModuleListrangerQconformer_layers) r8rrjrRrkrTrr!rUrhr9s rr+zConformer.__init__se  % 3 3z* .##1&7  !  sAr:rrct|}|jdd}|jD] }|||} |jdd|fS)aX Args: input (torch.Tensor): with shape `(B, T, input_dim)`. lengths (torch.Tensor): with shape `(B,)` and i-th element representing number of valid frames for i-th batch element in ``input``. Returns: (torch.Tensor, torch.Tensor) torch.Tensor output frames, with shape `(B, T, input_dim)` torch.Tensor output lengths, with shape `(B,)` and i-th element representing number of valid frames for i-th batch element in output frames. rr )rr<ro)r8r:rencoder_padding_maskr=layers rr>zConformer.forwardsW 8@ OOAq !** /Ea-.A /{{1a '))rr?) rArBrCrDr rErFr+rrGrr>rHrIs@rrrsH$"'       %(      8*U\\*ELL*U5<rws"  -ellu||F!F!R&&>ZUXX__ZzN*N*r