Robustness and Predictive Performance of Homogeneous Ensemble Feature Selection in Text Classification

Introduction

Feature selection helps reduce the dimensionality of a dataset by selecting a subset of attributes that are capable enough to represent the knowledge of the entire dataset. Methods used to carry out data classification gain a lot from feature selection once the redundant, noisy and irrelevant attributes are removed from the dataset. With the availability of so many feature selection techniques choosing a suitable technique for a given dataset is a challenge. Generally, the criterions used to gauge the effectiveness of an attribute selection method are the classification performance and algorithm complexity. Lately the stability of the attribute selection method has also become a metric for the same (Awada, Khoshgoftaar, Dittman, Wald, & Napolitano, 2012). This is due to the requirement in real world applications of choosing a similar set of attributes each time the method is used on the same dataset with a little perturbation (Kalousis, Prados, & Hilario, 2007). Stability is the measure of the robustness of the results, when the dataset composition is changed. Robustness implies that removing or adding a small percentage of data instances should not affect the set of features selected in a significant way. Unstable feature selection methods reduce the confidence of domain experts and confuse them. Researchers have been working in the area of stability and classification performance of feature selection algorithms in the genetic analysis (or bioinformatics) domain. However till date, there is lack of work on stability in the text classification domain and is an open area of research in future. Dittman, Khoshgoftaar, Wald, & Napolitano, 2013 studied the effect of dataset difficulty on the stability of feature selection method in the domain of gene selection from high dimensional micro-array datasets.

It has been demonstrated that there is no lone optimal attribute selection method and that quit a few subsets of attributes are capable of discerning the data just the same (Saeys, Abeel, & Van de Peer, 2008). Instead of using a particular attribute selection technique and using its output, ensemble approach may be used by combining the subsets obtained from various attribute selection methods. The various feature subsets obtained from different methods can be considered as local optima in the feature subset space, whereas the ensemble feature selection may give a more favourable subset (Saeys et al., 2008).

Creating a feature selection ensemble includes (i) Deciding the ensemble components that produce different subsets of features; (ii) Aggregation of the above subsets to a final feature subset based on some rank aggregation strategy.

The data perturbation approach for creating feature selection ensemble involves application of the same feature selector to different perturbed versions of the training data, obtaining different ranks for the same feature. This is followed by rank aggregation procedure to obtain a final rank for each feature.