ICA based on Split Generalized Gaussian

Przemysław Spurek,

Przemysław Rola,

Jacek Tabor,

Aleksander Czechowski,

Andrzej Bedychaj


Independent Component Analysis (ICA) is a method for searching the linear transformation that minimizes the statistical dependence between its components. Most popular ICA methods use kurtosis as a metric of independence (non-Gaussianity) to maximize, such as FastICA and JADE. However, their assumption of fourth-order moment (kurtosis) may not always be satisfied in practice. One of the possible solution is to use third-order moment (skewness)  instead of kurtosis, which was applied in ICA_SG and EcoICA. In this paper we present a competitive approach to ICA based on the Split Generalized Gaussian distribution (SGGD), which is well adapted to heavy-tailed as well as asymmetric data. Consequently, we obtain a method which works better than the classical approaches, in both cases: heavy tails and non-symmetric data.

Słowa kluczowe: ICA, Split Normal distribution, skewness, kurtosis

[1] Beckmann, C.F., Smith, S.M., Probabilistic independent component analysis for functional magnetic resonance imaging. Medical Imaging, IEEE Transactions on, 2004, 23(2), pp. 137–152.

[2] Beckmann, C.F., Smith, S.M., Tensorial extensions of independent component analysis for multisubject fmri analysis. Neuroimage, 2005, 25(1), pp. 294–311. 

[3] Rodriguez, P.A., Calhoun, V.D., Adalı, T., De-noising, phase ambiguity correction and visualization techniques for complex-valued ica of group fmri data. Pattern recognition, 2012, 45(6), pp. 2050–2063.

[4] Brunner, C., Naeem, M., Leeb, R., Graimann, B., Pfurtscheller, G., Spatial filtering and selection of optimized components in four class motor imagery eeg data using independent components analysis. Pattern Recognition Letters, 2007, 28(8), pp. 957–964.

[5] Delorme, A., Sejnowski, T., Makeig, S., Enhanced detection of artifacts in eeg data using higher-order statistics and independent component analysis. Neuroimage, 2007, 34(4), pp. 1443–1449.

[6] Zhang, H., Yang, H., Guan, C., Bayesian learning for spatial filtering in an eeg-based brain–computer interface. IEEE transactions on neural networks and learning systems, 2013, 24(7), pp. 1049–1060.

[7] Choi, S.W., Martin, E.B., Morris, A.J., Lee, I.B., Fault detection based on a maximum-likelihood principal component analysis (pca) mixture. Industrial & engineering chemistry research, 2005, 44(7), pp. 2316–2327.

[8] Kiviluoto, K., Oja, E., Independent component analysis for parallel financial time series. In: ICONIP. vol. 2., 1998, pp. 895–898. 

[9] Haghighi, A.M., Haghighi, I.M., et al., An ica approach to purify components of spatial components of seismic recordings. In: SPE Annual Technical Conference and Exhibition, Society of Petroleum Engineers, 2008.

[10] Yang, J., Gao, X., Zhang, D., Yang, J.y., Kernel ica: An alternative formulation and its application to face recognition. Pattern Recognition, 2005, 38(10), pp. 1784–1787.

[11] Dagher, I., Nachar, R., Face recognition using ipca-ica algorithm. IEEE transactions on pattern analysis and machine intelligence, 2006, 28(6), pp. 996–1000.

[12] Chuang, C.F., Shih, F.Y., Recognizing facial action units using independent component analysis and support vector machine. Pattern recognition, 2006, 39(9), pp. 1795–1798.

[13] Tsai, D.M., Lin, P.C., Lu, C.J., An independent component analysis-based filter design for defect detection in low-contrast surface images. Pattern Recognition, 2006, 39(9), pp. 1679–1694.

[14] Jenssen, R., Eltoft, T., Independent component analysis for texture segmentation. Pattern Recognition, 2003, 36(10), pp. 2301–2315.

[15] Bressan, M., Guillamet, D., Vitria, J., Using an ica representation of local color histograms for object recognition. Pattern Recognition, 2003, 36(3), pp. 691–701. 

[16] Kim, K.I., Franz, M.O., Scholkopf, B., Iterative kernel principal component analysis for image modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(9), pp. 1351–1366.

[17] Luo, B., Wilson, R.C., Hancock, E.R., Spectral embedding of graphs. Pattern recognition, 2003, 36(10), pp. 2213–2230.

[18] Luo, B., Wilson, R.C., Hancock, E.R., The independent and principal component of graph spectra. In: Pattern Recognition, 2002. Proceedings. 16th International Conference on. vol. 2., IEEE, 2002, pp. 164–167.

[19] Lai, Z., Xu, Y., Chen, Q., Yang, J., Zhang, D., Multilinear sparse principal component analysis. IEEE transactions on neural networks and learning systems, 2014, 25(10), pp. 1942–1950.

[20] Cardoso, J.F., Souloumiac, A., Blind beamforming for non-gaussian signals. In: Radar and Signal Processing, IEE Proceedings F. vol. 140., IET, 1993, pp. 362– 370.

[21] Pham, D.T., Garat, P., Blind separation of mixture of independent sources through a quasi-maximum likelihood approach. Signal Processing, IEEE Transactions on, 1997, 45(7), pp. 1712–1725.

[22] Hyv¨arinen, A., Fast and robust fixed-point algorithms for independent component analysis. Neural Networks, IEEE Transactions on, 1999, 10(3), pp. 626–634.

[23] Helwig, N.E., Hong, S., A critique of tensor probabilistic independent component analysis: implications and recommendations for multi-subject fmri data analysis. Journal of neuroscience methods, 2013, 213(2), pp. 263–273.

[24] Song, L., Lu, H., Ecoica: Skewness-based ica via eigenvectors of cumulant operator. In: Asian Conference on Machine Learning, 2016, pp. 445–460.

[25] Spurek, P., Tabor, J., Rola, P., Ociepka, M., Ica based on asymmetry. Pattern Recognition, 2017, 67, pp. 230–244.

[26] Karvanen, J., Eriksson, J., Koivunen, V., Pearson system based method for blind separation. In: Proceedings of Second International Workshop on Independent Component Analysis and Blind Signal Separation (ICA2000), Helsinki, Finland, 2000, pp. 585–590.

[27] Karvanen, J., Koivunen, V., Blind separation methods based on pearson system and its extensions. Signal Processing, 2002, 82(4), pp. 663–673. 

[28] Blaschke, T., Wiskott, L., Cubica: Independent component analysis by simultaneous third-and fourth-order cumulant diagonalization. IEEE Transactions on Signal Processing, 2004, 52(5), pp. 1250–1256.

[29] Virta, J., Nordhausen, K., Oja, H., Projection pursuit for non-gaussian independent components. arXiv preprint arXiv:1612.05445, 2016. 

[30] Azzalini, A., A class of distributions which includes the normal ones. Scandinavian journal of statistics, 1985, pp. 171–178.

[31] Azzalini, A., Dalla Valle, A., The multivariate skew-normal distribution. Biometrika, 1996, 83(4), pp. 715–726.

[32] Villani, M., Larsson, R., The multivariate split normal distribution and asymmetric principal components analysis. Communications in Statistics—Theory and Methods, 2006, 35(6), pp. 1123–1140.

[33] Gibbons, J., Mylroie, S., Estimation of impurity profiles in ion-implanted amorphous targets using joined half-gaussian distributions. Applied Physics Letters, 1973, 22(11), pp. 568–569.

[34] Nandi, A.K., M¨ampel, D., An extension of the generalized gaussian distribution to include asymmetry. Journal of the Franklin Institute, 1995, 332(1), pp. 67–75.

[35] Tesei, A., Regazzoni, C.S., Hos-based generalized noise pdf models for signal detection optimization. Signal Processing, 1998, 65(2), pp. 267–281. 

[36] Pascal, F., Bombrun, L., Tourneret, J.Y., Berthoumieu, Y., Parameter estimation for multivariate generalized gaussian distributions. IEEE Transactions on Signal Processing, 2013, 61(23), pp. 5960–5971.

[37] Tacconi, G., Tesei, A., Regazzoni, C., A new hos-based model for signal detection in non-gaussian noise: an application to underwater acoustic communications. In: OCEANS’95. MTS/IEEE. Challenges of Our Changing Global Environment. Conference Proceedings. vol. 1., IEEE, 1995, pp. 620–625.

[38] Miller, J., Thomas, J., Detectors for discrete-time signals in non-gaussian noise. IEEE Transactions on Information Theory, 1972, 18(2), pp. 241–250. 

[39] Lorenzo-Seva, U., Ten Berge, J.M., Tucker’s congruence coefficient as a meaningful index of factor similarity. Methodology, 2006, 2(2), pp. 57–64.

[40] Amari, S.i., Cichocki, A., Yang, H.H., A new learning algorithm for blind signal separation. In: Advances in neural information processing systems, 1996, pp. 757–763.

[41] Helwig, N.E., ica: Independent Component Analysis. 2015 R package version 1.0-1.

[42] Karvanen, J., PearsonICA. 2008 R package version 1.2-3. 

[43] Hastie, T., Tibshirani, R., ProDenICA: Product Density Estimation for ICA using tilted Gaussian density estimates. 2010 R package version 1.0.

[44] Matilainen, M., Miettinen, J., Nordhausen, K., Oja, H., Taskinen, S., tsBSS: Tools for Blind Source Separation for Time Series. 2016 R package version 0.2.

[45] Miettinen, J., Nordhausen, K., Oja, H., Taskinen, S., fICA: Classical, Reloaded and Adaptive FastICA Algorithms. 2015 R package version 1.0-3.

[46] Nordhausen, K., Oja, H., Tyler, D.E., Virta, J., ICtest: Estimating and Testing the Number of Interesting Components in Linear Dimension Reduction. 2016 R package version 0.2.

[47] Bell, A.J., Sejnowski, T.J., An information-maximization approach to blind separation and blind deconvolution. Neural computation, 1995, 7(6), pp. 1129–1159. 

[48] Stuart, A., Kendall, M.G., et al., The advanced theory of statistics. Charles Griffin, 1968.

[49] Bach, F.R., Jordan, M.I., Kernel independent component analysis. Journal of machine learning research, 2002, 3(Jul), pp. 1–48.

[50] Hastie, T., Tibshirani, R., Friedman, J., The elements of statistical learning 2nd edition, 2009.