Evaluating the Fairness of News Recommender Algorithms Within Detected User Communities

Abstract

This work addresses the problem of unfair treatment of different user groups in the recommendations they receive from recommendation algorithms. Recommender systems (RS) are algorithms that suggest items to a user that are most likely to be of interest, based on the user’s interaction history with items. Recommendation algorithms are used in a variety of domains, such as recommending music on music streaming platforms. Collaborative filtering RS have been shown in previous studies to be particularly sensitive to data imbalance, resulting in less relevant recommendations for certain user groups. In such group fairness studies, users are grouped according to a sensitive user attribute, typically based on user traits or demographics, and the equitable treatment of these groups is then examined. This thesis explores the fairness of recommendations by quantifying variations between user groups and different collaborative filtering RS, using a dataset from the Austrian online news platform “DER STANDARD”. Unlike related work, these user groups are large user communities discovered in a user network modelled from user interaction data. Graphs with different user relationships are built using data on users’ clicks on news articles and users’ community activities, such as postings, votes for postings and users’ followers. User communities in these networks are identified using two community detection algorithms. The graph with the highest agreement between community detection algorithms in assigning users to communities is selected, and the partition with the highest quality is used to group users based on the communities detected. The extent to which various RS provide different recommendations to users is evaluated, with a focus on differences between large user communities. This study identifies instability in community detection algorithms by observing considerable variations in consensus scores for graph partitions, as well as in the size and number of communities discovered when networks are filtered using different edge weight thresholds. The recommendations to users in the detected user communities vary considerably depending on the collaborative filtering RS and the evaluation metric.

Publications

@inproceedings{10.1145/3708319.3733694,

title = {Beyond Demographics: Evaluating News Recommender Systems Fairness Through Behavioural Communities},

author = {Bernhard Steindl and Thomas Elmar Kolb and Julia Neidhardt},

url = {https://doi.org/10.1145/3708319.3733694},

doi = {10.1145/3708319.3733694},

isbn = {9798400713996},

year  = {2025},

date = {2025-01-01},

urldate = {2025-01-01},

booktitle = {Adjunct Proceedings of the 33rd ACM Conference on User Modeling, Adaptation and Personalization},

pages = {13–17},

publisher = {Association for Computing Machinery},

address = {New York, NY, USA},

series = {UMAP Adjunct '25},

abstract = {Fairness in recommender systems is often framed around demographic attributes. In this work, we explore a novel direction—evaluating fairness across latent behavioural communities derived from user interactions on a real-world news platform. Using graph-based community detection (Louvain and Infomap), we identify large user groups and examine how different network modelling choices affect fairness outcomes in both traditional and fairness-aware recommender systems. Experiments on an Austrian news dataset reveal that small changes in graph construction considerably impact community formation and recommendation quality. Notably, fairness-aware algorithms show only marginal improvements over standard approaches, underscoring the complexity of achieving equitable outcomes in real-world systems and raising important questions for future research.},

keywords = {},

pubstate = {published},

tppubtype = {inproceedings}

}

@mastersthesis{steindl2024fairness,

title = {Evaluating the fairness of news recommender algorithms within detected user communities},

author = {Bernhard Steindl},

url = {https://doi.org/10.34726/hss.2024.115142},

doi = {10.34726/hss.2024.115142},

year  = {2024},

date = {2024-01-01},

pages = {116},

address = {Vienna, Austria},

school = {Technische Universität Wien},

abstract = {This work addresses the problem of unfair treatment of different user groups in the recommendations they receive from recommendation algorithms. Recommender systems (RS) are algorithms that suggest items to a user that are most likely to be of interest, based on the user's interaction history with items. Recommendation algorithms are used in a variety of domains, such as recommending music on music platforms or news in online portals.},

note = {Advisors: Julia Neidhardt, Thomas Elmar Kolb},

keywords = {},

pubstate = {published},

tppubtype = {mastersthesis}

}