Multilinear Filtering Based on a Hierarchical Structure of Covariance Matrices

Andrzej Szwabe,

Pawel Misiorek,

Michal Ciesielczyk


We propose a novel model of multilinear filtering based on a hierarchical structure of covariance matrices – each matrix being extracted from the input tensor in accordance to a specific set-theoretic model of data generalization, such as derivation of expectation values. The experimental analysis results presented in this paper confirm that the investigated approaches to tensor-based data representation and processing outperform the standard collaborative filtering approach in the ‘cold-start’ personalized recommendation scenario (of very sparse input data). Furthermore, it has been shown that the proposed method is superior to standard tensor-based frameworks such as N-way Random Indexing (NRI) and Higher-Order Singular Value Decomposition (HOSVD) in terms of both the AUROC measure and computation time.

Słowa kluczowe: tensor-based data modeling, multilinear PCA, random indexing, dimensionality reduction, multilinear data filtering, higher-order SVD
[1] Nickel M., Tresp V., An Analysis of Tensor Models for Learning on Structured Data. In: Machine Learning and Knowledge Discovery in Databases. 8189 of LNCS. Springer Berlin Heidelberg 2013, pp. 272–287. 
[2] Lu H., Plataniotis K.N., Venetsanopoulos A.N., Multilinear principal component analysis of tensor objects for recognition. In: 18th International Conference on Pattern Recognition, ICPR 2006. vol. 2., 2006, pp. 776–779. 
[3] Sandin F., Emruli B., Sahlgren M., Incremental dimension reduction of tensors with random index. March 2011, pp. 240–56. 
[4] Grasedyck L., Kressner, D., Tobler C., A literature survey of low-rank tensor approximation techniques. GAMM–Mitteilungen, 2013, 36.1, pp. 53–78. 
[5] Baldassarre L., Rosasco L., Barla A., Verri A., Multi-output learning via spectral filtering. Machine learning, 2012, 87(3), pp. 259–301. 
[6] De Lathauwer L., De Moor B., Vandewalle, J., A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl, 2000, 21, pp. 1253–1278. 
[7] Pearl J., Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann, 1988. 
[8] Cohen T., Schvaneveldt R., Widdows D., Reflective Random Indexing and indirect inference: a scalable method for discovery of implicit connections. Journal of Biomedical Informatics, 2010, 43(2), pp. 240–56. 
[9] Ciesielczyk M., Szwabe A., RSVD-based Dimensionality Reduction for Recommender Systems. International Journal of Machine Learning and Computing, 2011, 1(2), pp. 170–175. 
[10] Brin S., Page L., The anatomy of a large-scale hypertextual web search engine. ProceedingWWW7ProceedingsoftheseventhinternationalconferenceonWorld Wide Web 7, 1998, 30(1-7), pp. 107–117. 
[11] Kroonenberg P. M., Three-mode principal component analysis: Theory and applications. vol. 2. DSWO press;, 1983.
[12] Grasedyck L., Hierarchical singular value decomposition of tensors. SIAM Journal on Matrix Analysis and Applications, 2010, 31(4), pp. 2029–2054. 
[13] Kolda T.G., Bader B.W., Tensor decompositions and applications. SIAM review, 2009, 51(3), pp. 455–500. 
[14] Herlocker J.L., Konstan, J., Terveen L.G., Riedl, J., Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems, 2004, 22(1), pp. 5–53.
 [15] Koren Y., Bell R., Volinsky C., Matrix factorization techniques for recommender systems. Computer, 2009, 8, pp. 42–49.