Great question!
You're right, sometimes the routine augmentations do not add additional value. In such cases, often I rely upon an optimizer to find optimal hyperparameters like learning rate, mini-batch size,momentum etc. See the documentation https://www.mathworks.com/help/deeplearning/examples/deep-learning-using-bayesian-optimization.html.
With regards to your question on finding the best net structures, it is a area of research and often words like AutoML are used for such workflows. What I mentioned earlier, regarding bayesian optimization, is also a part of AutoML. The downside of trying to use an automated technique to find the best network structure is that it is computationally very very expensive. Having said that, we've an example in the documentation that shows how you can do a parameter sweep on the depth of a network --> https://www.mathworks.com/help/deeplearning/examples/use-parfeval-to-train-multiple-deep-learning-networks.html
