论文《AKConv:具有任意采样形状和任意数目参数的卷积核》翻译

AKConv: Convolutional Kernel with Arbitrary Sampled Shapes and

Arbitrary Number of Parameters

Abstract. Neural networks based on convolutional operations have achieved remarkable results

in the field of deep learning, but there are two inherent flaws in standard convolutional

operations. On the one hand, the convolution operation be confined to a local window and

cannot capture information from other locations, and its sampled shapes is fixed. On the other

hand, the size of the convolutional kernel is fixed to

, which is a fixed square shape, and

AKConv, we define initial positions for convolutional kernels of arbitrary size by means of a

new coordinate generation algorithm. To adapt to changes for targets, we introduce offsets to

adjust the shape of the samples at each position. Moreover, we explore the effect of the neural

network by using the AKConv with the same size and different initial sampled shapes. AKConv

completes the process of efficient feature extraction by irregular convolutional operations and

brings more exploration options for convolutional sampling shapes. Object detection

experiments on representative datasets COCO2017, VOC 7+12 and VisDrone-DET2021 fully

demonstrate the advantages of AKConv. AKConv can be used as a plug-and-play convolutional

operation to replace convolutional operations to improve network performance. The code for

the relevant tasks can be found at https://github.com/CV-ZhangXin/AKConv.

models rely heavily on convolutional operations, which efficiently extract local features in

images and ensure model complexity.

Despite the fact that CNNs have achieved many successes in classification [8], object

detection [9], semantic segmentation [10], etc., they still have some limitations. One of the most

notable limitations concerns the choice of convolutional sample shape and size. Standard

convolution operations tend to rely on square kernels with fixed sampling locations, such as 1

× 1, 3 × 3, 5 × 5 and 7 × 7, etc. The sampling position of the regular kernel is not deformable

and cannot be dynamically changed in response to changes in the shape of the object.

Deformable Conv [11, 12] enhances network performance with offset to flexibly adjust the

sampling shape of the convolution kernel, which adapts to the change of the target. For instance,

in [13, 14, 15], they utilized it to to align features. Zhao et al. [16] improved the effectively of

convolutional operations and Deformable Conv shows a squared growth trend with the increase

of the convolution kernel size, which is not a friendly way of growth to the hardware

environment. Therefore, after careful analysis of standard convolution operations and

Deformable Conv, we propose Alterable Kernel Convolution (AKConv). Unlike standard

regular convolution, AKConv is a novel convolutional operations, which can extract features

using efficient convolution kernels with any number of parameters such as (1, 2, 3, 4, 5, 6, 7...),

which is not implemented by standard convolution and Deformable Convolution. AKConv can

easily be used to replace the standard convolutional operations in a network to improve network

performance. Importantly, AKConv allows the number of convolutional parameters to trend

linearly up or down, which is beneficial to hardware environments, and it can be used as an

with sufficient resources. Fig. 1 shows that the regular convolutional kernel makes the number

of parameters to show a square increasing trend, while AKConv only shows a linear increasing

VisDrone-DET2021 [25] fully demonstrate the advantages of AKConv. In summary, our

contributions are as follows:

sampled coordinate for convolutional kernels of arbitrary sizes.

2. To adapt to the different variations of the target, we adjust the sampling position of the

irregular convolutional kernel by the obtained offsets.

3. Compared to regular convolution kernels, the proposed AKConv realizes the function

of irregular convolution kernels to extract features, providing convolution kernels with arbitrary

sampling shapes and sizes for a variety of varying targets, which makes up for the shortcomings

of regular convolutions.

2 Related works

In recent years, many works have considered and analyzed standard convolutional

address these shortcomings, they proposed the Involution operator, which inverts the features

of the convolutional operation to improve network performance. Qi et al. [27] proposed the

freedom, leading to the model losing a small percentage of fine structure features, which poses

a great challenge for the task of segmenting elongated tubular structures, therefore, they

Conv. Unlike using a convolutional kernel for every layers, the Dynamic Conv dynamically

aggregated multiple parallel convolutional kernels based on their attention. The Dynamic Conv

provided greater representation of features. Tan et al. [32] argued that kernel size is often

neglected in CNNS, which may affect the accuracy and efficiency of the network. Second,

using only layer-by-layer convolution does not utilize the full potential of convolutional

networks. Therefore, they proposed MixConv, which naturally mixes multiple kernel sizes in a

single convolution to improve performance of networks.

Although these methods improve the performance of convolutional operations, they are

still limited to regular convolutional operations and do not allow multiple variations of

convolutional sample shapes. In contrast, our proposed AKConv can efficiently extract features

using a convolutional kernel with arbitrary number of parameters and sample shapes.

However, the sampling grid is regular, while AKConv targets irregularly shaped

convolutional kernels. Therefore, to allow irregular convolutional kernels to have a sampling

grid, we create an algorithm for arbitrary size convolution, which generates the initial sampling

sampling grid, then the irregular grids is created for the remaining sampling points, and finally,

we stitch them to generate the overall sampling grid. The pseudo code is as in Algorithm 1.

As shown in Fig. 2, it shown that the initial sampled coordinates is generated for arbitrary

size convolution. The sampling grid of the regular convolution is centered at the (0, 0) point.

While the irregular convolution has no center at many sizes, to adapt to the size of the

convolution used, we set the upper left corner (0, 0) point as the sampling origin in the algorithm.

After defining the initial coordinates Pn for the irregular convolution, the corresponding

convolution operation at position P0 can be defined as follows:

Here, w denotes the convolutional parameter. However, the irregular convolution

operations are impossible to realize, because irregular sampling coordinates cannot be matched

to the corresponding size convolution operations, e.g., convolution of sizes 5, 7, and 13.

Cleverly, our proposed AKConv realizes it.

3.2 Alterable convolutional operation

increases their number of convolution parameters tends to increase by a square, which is not

friendly for the hardware environment. Therefore, we propose a novel Alterable convolutional

operation (AKConv). As shown in Fig. 3, it illustrates the overall structure of an AKConv of

size 5.

Similar to Deformable Conv, in AKConv, the offset of the corresponding kernel are first

obtained by convolution operations, which has the dimensions (B, 2N, H, W), where N is the

convolution kernel size. Take Fig. 3 as an example, N = 5. Then the modified coordinates are