Design of an improved model for natural image classification using AugMix, SE-ResNeXt, and MAML

Abstract

Retaining the effectiveness but improving the efficiency of natural image classification is of prime necessity in recent times, with the surge in demand for deploying these models in practical applications, ensuring accuracy and generalization. Classic deep learning classifiers suffer from limited robustness, generalization, and failure to adapt to new tasks and domains. These shortcomings restrict their practically effective deployment by the availability of different diversified and unseen data. In this work, the authors introduce an optimized deep learning classifier framework, leveraging state-of-the-art techniques in various key domains. The proposed model harnesses a combination of techniques ranging from AugMix, SE-ResNeXt, MAML, Hyperband, and finally Domain-Adversarial Neural Network (DANN) for performance improvement. AugMix integrates Mixup and CutMix with the stochastic augmentation technique of complex augmentation chains to enhance the model's robustness and generalization. Mixing images with stochastic augmentations and the use of Mixup and CutMix bring further strong regularizations, boosting the robustness metrics by 15-20% and classification accuracy by 3-5% on the unseen natural images and samples. SE-ResNeXt introduces the use of channel-wise attention to enhance the representational power of the model. Squeeze-and-Excitation (SE) blocks are introduced to recalibrate the channel-wise feature responses by weighting informative features and suppressing less useful ones. It boosts the accuracy of models on benchmark CIFAR-100 dataset samples by 2-3% over standard ResNeXt. Execution of Model-Agnostic Meta-Learning enables a model to adapt quickly to a new task based on a small number of examples. MAML meta-learns updated models based on examples of tasks instead of direct model parameters. A 5-7% improvement in accuracy is achieved for different scenarios. Hyperband performs tension-free search of optimal hyperparameters via adaptive resources dealing, which configures the resources only for the promising configurations. Reducing the computational cost of hyperparameter tuning to at most 50% ensures an increase in model accuracy of 2- 3%. The DANN technique uses adversarial training in order to suppress the domain shift between source and target datasets. DANN uses a gradient reversal layer to train feature extractors to produce domain-invariant features, leading to a 10~15% increase in accuracy on target domain datasets compared to non-adaptive methods.