Hi Aiman,
Designing a neural network architecture like AlexNet involves a mix of empirical experimentation, domain knowledge, and some established principles rather than a formula. The choices of filter size, number of kernels (filters), stride, and pooling are typically made based on the following considerations:
- Large filter size captures more spatial information but are computationally expensive.
- Small filter size captures finer details and reduce the number of inputs allowing for deeper networks. Eg: AlexNet starts with an 11x11 filter to capture large spatial features in the first layer, followed by smaller 5x5 filters to refine the features in subsequent layers.
- Similarly, more kernels capture more features at each layer and typically increases with depth to capture more complex patterns.
- The stride controls the step size of the filter as it convolves over the input. Larger strides reduce the spatial dimensions quickly, leading to faster computations but potentially losing spatial resolution.
- Pooling reduces spatial dimensions and computational load, while retaining important features.
The output size of a convolutional operation however can be calculated with the formula:
O = ((W – K + 2P) / S) + 1
Where O is the output size, W is the input size, K is the kernal size, P is the padding and S is the stride. This formula helps to determine the dimensions of the output feature map, which is essential for designing and understanding the architecture of a CNN. In practice, designing architectures often involves iterating over different configurations. The choices of designing are guided by empirical testing, computational constraints and domain knowledge.
Hope this helps!
