Pairwise versus Pointwise Ranking: A Case Study

Vitalik Melnikov,

Eyke Hüllermeier,

Daniel Kaimann,

Bernd Frick,

Pritha Gupta


Object ranking is one of the most relevant problems in the realm of preference learning and ranking. It is mostly tackled by means of two different techniques, often referred to as pairwise and pointwise ranking. In this paper, we present a case study in which we systematically compare two representatives of these techniques, a method based on the reduction of ranking to binary classification and so-called expected rank regression (ERR). Our experiments are meant to complement existing studies in this field, especially previous evaluations of ERR. And indeed, our results are not fully in agreement with previous findings and partly support different conclusions.

Słowa kluczowe: Preference learning, object ranking, linear regression, logistic regression, hotel rating, TripAdvisor

[1] F¨urnkranz J., H¨ullermeier E., eds. Preference Learning. Springer, 2010.

[2] Cohen W., Schapire R., Singer Y., Learning to order things. Journal of Artificial Intelligence Research, 1999, 10 (1), pp. 243–270.

[3] Kamishima T., Kazawa H., Akaho S., A survey and empirical comparison of object ranking methods. In: F¨urnkranz J., H¨ullermeier E., eds.: Preference Learning. Springer 2010 pp. 181–202.

[4] Har-Peled, S., Roth, D., Zimak, D., Constraint classification: a new approach to multiclass classification. In: Cesa-Bianchi N., Numao M., Reischuk R., eds.: Proceedings of the 13th International Conference on Algorithmic Learning Theory, Springer, 2002, pp. 365–379. 83

[5] Cheng W., H¨uhn J., H¨ullermeier E., Decision tree and instance-based learning for label ranking. In: Proceedings of the 26th International Conference on Machine Learning, Omnipress, 2009, pp. 161–168.

[6] Vembu S., G¨artner T., Label ranking: a survey. In: F¨urnkranz J., H¨ullermeier E., eds.: Preference Learning. Springer 2010 pp. 45–64.

[7] F¨urnkranz J., H¨ullermeier E., Vanderlooy S., Binary decomposition methods for multipartite ranking. In: Proc. of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, Springer, 2009, pp. 359–374.

[8] Kamishima T., Kazawa H., Akaho S., Supervised ordering – an empirical survey.In: Proc. ICDM, 5th IEEE International Conference on Data Mining, Houston, Texas, 2005, pp. 673–676.

[9] Kamishima T., Akaho S., Supervised ordering by regression combined with Thurstone’s model. Artificial Intelligence Review, 2006, 25 (3), pp. 231–246.

[10] Marden J., Analyzing and Modeling Rank Data. Chapman and Hall, London, New York, 1995.