Multi-label multi-class text classification-enhanced attention in transformers with knowledge distillation

Abstract

This scholarly paper introduces an innovative and comprehensive ideology that aims to significantly expand the utility of Named Entity Recognition (NER) through the application of Transformers in various Natural Language Processing (NLP) tasks. One prominent task that necessitates attention is the intricate classification of emails into multiple labels, wherein each label can be associated with not just one but potentially multiple independent classes. Despite the existence of several research methodologies attempting to address numerous challenges in this domain, the industry continues to face a substantial hurdle when it comes to accurately categorizing multi-label texts like financial emails, which can encompass diverse categories such as Payment Information, Invoice Information, Disputes, and more. Considering these challenges, our proposed methodology serves as a breakthrough solution, demonstrating remarkable performance in the classification task across a wide range of datasets, including Financial Emails and Consumer Complaint Datasets. By leveraging the power of advanced Transformers, we have achieved an exceptional accuracy rate of 94% for full match of the multi-label classes, while the accuracy for partial match to individual classes soared to an impressive 97%. This achievement not only highlights the effectiveness of the proposed approach but also showcases its potential to enhance the efficiency and reliability of NER applications in practical settings.