'}hddlmZddlZddlmZmZmZddlZgdZeddZ edd Z ddd Z d dd Z dd Z dd Zy)) annotationsN)OptionalTupleTypeVar)fuse_conv_bn_evalfuse_conv_bn_weightsfuse_linear_bn_evalfuse_linear_bn_weightsConvTztorch.nn.modules.conv._ConvNd)boundLinearTztorch.nn.Linearc l|js |jrJdtj|}|j |jJt |j |j|j|j|j|j |j|\|_|_|S)a+Fuse a convolutional module and a BatchNorm module into a single, new convolutional module. Args: conv (torch.nn.modules.conv._ConvNd): A convolutional module. bn (torch.nn.modules.batchnorm._BatchNorm): A BatchNorm module. transpose (bool, optional): If True, transpose the convolutional weight. Defaults to False. Returns: torch.nn.modules.conv._ConvNd: The fused convolutional module. .. note:: Both ``conv`` and ``bn`` must be in eval mode, and ``bn`` must have its running buffers computed. Fusion only for eval!) trainingcopydeepcopy running_mean running_varrweightbiaseps)convbn transpose fused_convs T/var/www/html/test/engine/venv/lib/python3.12/site-packages/torch/nn/utils/fusion.pyrr s F/FF -t$J ?? &2>>+EE E)=:?? BGGY*P&Jz c|j}| |jn|} |tj|}|tj|}|tj|}tj||z} |r"ddgdgt |j dz zz} n!ddgdgt |j dz zz} ||| zj| zj|} ||z | z|z|zj| } tjj| |jtjj| |jfS)aFuse convolutional module parameters and BatchNorm module parameters into new convolutional module parameters. Args: conv_w (torch.Tensor): Convolutional weight. conv_b (Optional[torch.Tensor]): Convolutional bias. bn_rm (torch.Tensor): BatchNorm running mean. bn_rv (torch.Tensor): BatchNorm running variance. bn_eps (float): BatchNorm epsilon. bn_w (Optional[torch.Tensor]): BatchNorm weight. bn_b (Optional[torch.Tensor]): BatchNorm bias. transpose (bool, optional): If True, transpose the conv weight. Defaults to False. Returns: Tuple[torch.nn.Parameter, torch.nn.Parameter]: Fused convolutional weight and bias. )dtype) r"torch zeros_like ones_likersqrtlenshapereshapetonn Parameter requires_grad)conv_wconv_bbn_rmbn_rvbn_epsbn_wbn_brconv_weight_dtypeconv_bias_dtype bn_var_rsqrtr( fused_conv_w fused_conv_bs rrr%sG2 &,&8fll>OO ~!!%( |u% |&;;uv~.LB1#V\\!2Q!677Q1#V\\!2Q!677d\1::5AAEEL]E^Le^|3d:TAEEOE\L <)=)=>@R@RS_agauau@v rc |js |jrJdtj|} |j|jk(s|jdk(sJd|j |j Jt|j|j|j |j |j|j|j\|_|_ |S)aFuse a linear module and a BatchNorm module into a single, new linear module. Args: linear (torch.nn.Linear): A Linear module. bn (torch.nn.modules.batchnorm._BatchNorm): A BatchNorm module. Returns: torch.nn.Linear: The fused linear module. .. note:: Both ``linear`` and ``bn`` must be in eval mode, and ``bn`` must have its running buffers computed. rrzGTo fuse, linear.out_features == bn.num_features or bn.num_features == 1) rrr out_features num_featuresrrr rrr)linearr fused_linears rr r Ts2;;H1HH /==(L  r."//Q2FQPQF ?? &2>>+EE E-C\.. BGG.E*L* rcL|tj|}|tj||zz}||jdz}||z |z|z} tjj ||j tjj | |j fS)aFuse linear module parameters and BatchNorm module parameters into new linear module parameters. Args: linear_w (torch.Tensor): Linear weight. linear_b (Optional[torch.Tensor]): Linear bias. bn_rm (torch.Tensor): BatchNorm running mean. bn_rv (torch.Tensor): BatchNorm running variance. bn_eps (float): BatchNorm epsilon. bn_w (torch.Tensor): BatchNorm weight. bn_b (torch.Tensor): BatchNorm bias. transpose (bool, optional): If True, transpose the conv weight. Defaults to False. Returns: Tuple[torch.nn.Parameter, torch.nn.Parameter]: Fused linear weight and bias. r )r#r$r& unsqueezer+r,r-) linear_wlinear_br0r1r2r3r4bn_scalefused_wfused_bs rr r ys0##E*ekk%&.11H++B//G%8+d2G 88  gx'='= >@R@RSZ\d\r\r@s ssr)F)rr r%torch.nn.modules.batchnorm._BatchNormrboolreturnr )r. torch.Tensorr/Optional[torch.Tensor]r0rIr1rIr2floatr3rJr4rJrrGrH-Tuple[torch.nn.Parameter, torch.nn.Parameter])r=r rrFrHr )rArIrBrJr0rIr1rIr2rKr3rIr4rIrHrL) __future__rrtypingrrrr#__all__r r rrr r rrrQs" ++ h>? )#4 5@- - "- -  -  - ! - !--3-^#Jtt$t t  t  t  t t3tr