Abstract : Due to the increasing demand for multivariate data analysis from the various application the dimensionality reduction becomes an important task to represent the data in low dimensional space for the robust data representation. In this paper, multivariate data analyzed by using a new approach SVM and ICA to enhance the classification accuracy in a way that data can be present in more condensed form. Traditional methods are classified into two types namely standalone and hybrid method. Standalone method uses either supervised or unsupervised approach, whereas hybrid method uses both approaches. In this paper we are using SVM (support vector machine) as supervised and ICA (Independent component analysis) as a unsupervised approach for the improvement of the classification on the basis of dimensionality reduction. SVM uses SRM (structural risk minimization) principle which is very effective over ERM (empirical risk minimization) which minimizes an upper bound on the expected risk, as opposed to ERM that minimizes the error on the training data, whereas ICA uses maximum independence maximization to improve performance. The perpendicular or right angel projection is used to avoid the redundancy and to improve the dimensionality reduction. At last step we are using a classification algorithm to classify the data samples and classification accuracy is measured. Experiments are performed for various two classes as well as multiclass dataset and performance of hybrid, standalone approaches are compared.
Keywords: Dimensionality Reduction, Hybrid Methods, Supervised Learning, Unsupervised Learning, Support Vector Machine (SVM), Independent Component Analysis (ICA).
[1] Sangwoo Moon and Hairong Qi ,"Hybrid Dimensionality Reduction Method Based on Support Vector Machines and Independent Component Analysis", IEEE Transactions on Neural networks and Learning Systems,vol. 23, no. 5, may 2012.
[2] A.M.Martinez and A.C.Kak, "PCA versus LDA,"IEEE Trans.Pattern Anal.Mach.Intell. vol.23, no. 2, pp. 228-233, Feb. 2001.
[3] C. J. C. Burges. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2(2):121–167, 1998.
[4] L. Cao, K. Chua, W. Chong, H. Lee, and Q. Gu. A comparison of PCA, KPCA and ICA for dimensionality reduction in support vector machine. Nerocomp, 55:321–336, 2003.
[5] Hyvarinen, A., Karhunen, J., Oja, E.: Independent Component Analysis and Its Application John Wiley & Sons,Inc. ,2001
[6] L.-F. Chen, H.-Y. M. Liao, M.-T. Ko, J.-C. Lin, and G.J. Yu, "Anew LDA-based face recognition system which can solve the small sample size problem," Pattern Recognit., vol. 33, no. 10, pp.1713–1726, 2000.
[7] H. Park, M. Jeon, and J. B. Rosen, "Lower dimensional representation of text data based on centroids and least squares," BIT Numerical Math vol. 43, no. 2, pp. 427–448, 2003.