Recommendation Debiasing

This website collects recent works and datasets on recommendation debiasing and their codes. We hope this website could help you do search on this topic.

Survey paper and Tutorial

Type	Paper	Conference
Survey Paper	Bias and Debias in Recommender System: A Survey and Future Directions	Arxiv 2021
Tutorial	Bias Issues and Solutions in Recommender System	WWW 2021, Recsys 2021

Datasets

We collect some datasets which include unbiased data and are often used in the research of recommendation debiasing.

Dataset	Paper	Source
Yahoo! R3	Collaborative Prediction and Ranking with Non-Random Missing Data	Data
Coat	Recommendations as Treatments: Debiasing Learning and Evaluation	Data
KuaiRec	KuaiRec: A Fully-observed Dataset for Recommender Systems	Data

More works about Introduction of bias can be found here

Debiasing Strategies

Papers	bias	Methods	Conference	Code
Transfer Learning in Collaborative Recommendation for Bias Reduction	Universal	TJR	Recsys 2021	Tensorflow
AutoDebias: Learning to Debias for Recommendation	Universal	AutoDebias	SIGIR 2021	Pytorch
A General Knowledge Distillation Framework for Counterfactual Recommendation via Uniform Data	Universal	KDCRec	SIGIR 2020	Tensorflow
Causal Embeddings for Recommendation	Universal	CausE	Recsys 2018	Tensorflow
Combating Selection Biases in Recommender Systems with a Few Unbiased Ratings	Selection Bias	LTD	WSDM 2021
Asymmetric tri-training for debiasing missing-not-at-random explicit feedback	Selection Bias	AT	SIGIR 2020
Recommendations as treatments: Debiasing learning and evaluation	Selection Bias	MF-IPS	SIGIR 2020	Python
Doubly robust joint learning for recommendation on data missing not at random	Selection Bias	DR	ICML 2019
The deconfounded recommender: A causal inference approach to recommendation	Selection Bias	Deconfounded model	arXiv 2018
Social recommendation with missing not at random data	Selection Bias	SPMF-MNAR	ICDM 2018
Recommendations as treatments: Debiasing learning and evaluation	Selection Bias	MF-IPS	PMLR 2016	Python
Boosting Response Aware Model-Based Collaborative Filtering	Selection Bias	RAPMF	TKDE 2015	-
Probabilistic matrix factorization with non-random missing data	Selection Bias	MF-MNAR	JMLR 2014	Python
Bayesian Binomial Mixture Model for Collaborative Prediction With Non-Random Missing Data	Selection Bias		RecSys 2014
Evaluation of recommendations: rating-prediction and ranking	Selection Bias		RecSys 2013
Training and testing of recommender systems on data missing not at random	Selection Bias		KDD 2010
Training and testing of recommender systems on data missing not at random	Selection Bias		KDD 2010
Collaborative prediction and ranking with non-random missing data	Selection Bias	MM	RecSys 2009
Collaborative filtering and the missing at random assumption	Selection Bias		UAI 2007	Python
Popularity Bias Is Not Always Evil: Disentangling Benign and Harmful Bias for Recommendation	Conformity Bias	TIDE	Arxiv 2021	To publish
Disentangling user interest and Conformity for recommendation with causal embedding	Conformity Bias	DICE	WWW 2021	Pytorch
Learning personalized preference of strong and weak ties for social recommendation	Conformity Bias	PTPMF	WWW 2017
Are you influenced by others when rating?: Improve rating prediction by conformity modeling	Conformity Bias		RecSys 2016
Xgboost: A scalable tree boosting system	Conformity Bias	XGBoost	KDD 2016	Python
A probabilistic model for using social networks in personalized item recommendation	Conformity Bias	SPF	RecSys 2015	Python
mtrust: discerning multi-faceted trust in a connected world	Conformity Bias	mTrust	WSDM 2012
Learning to recommend with social trust ensemble	Conformity Bias	RSTE	SIGIR 2009
Non-Clicks Mean Irrelevant? Propensity Ratio Scoring As a Correction	Exposure Bias	PRS	WSDM 2021	-
Propensity-Independent Bias Recovery in Offline Learning-to-Rank Systems	Exposure Bias	LTR	SIGIR 2021	-
Mitigating Confounding Bias in Recommendation via Information Bottleneck	Exposure Bias	DIB	Recsys 2021	To come
Debiased Explainable Pairwise Ranking from Implicit Feedback	Exposure Bias	EBPR	Recsys 2021	Pytorch
Top-N Recommendation with Counterfactual User Preference Simulation	Exposure Bias	CPR	CIKM 2021	-
SamWalker++: recommendation with informative sampling strategy	Exposure Bias	SamWalker++	TKDE 2021	Pytorch
Deconfounded Causal Collaborative Filtering	Exposure Bias	DCCF	Arxiv 2021	-
Unbiased recommender learning from missing-not-at-random implicit feedback	Exposure Bias	Rel-MF	WSDM 2020	Python
Lightgcn: Simplifying and powering graph convolution network for recommendation （LightGCN）	Exposure Bias	LightGCN	SIGIR 2020	Python
Reinforced negative sampling over knowledge graph for recommendation	Exposure Bias	KGPolicy	WWW 2020	Python
Fast adaptively weighted matrix factorization for recommendation with implicit feedback	Exposure Bias	FAWMF	AAAI 2020
Correcting for selection bias in learning-to-rank systems	Exposure Bias	Heckman rank/Propensity SVM rank	WWW 2020	Python
Large-scale causal approaches to debiasing post-click conversion rate estimation with multi-task learning	Exposure Bias	Multi-IPW/Multi-DR	WWW 2020
Entire space multi-task modeling via post-click behavior decomposition for conversion rate prediction	Exposure Bias	ESM^2^	SIGIR 2020
Gmcm: Graph-based micro-behavior conversion model for post-click conversion rate estimation	Exposure Bias	GMCM	SIGIR 2020
” click” is not equal to” like”: Counterfactual recommendation for mitigating clickbait issue	Exposure Bias	CR	arXiv 2020
Unbiased Implicit Recommendation and Propensity Estimation via Combinational Joint Learning	Exposure Bias	CJ	Recsys 2020	Tensorflow
Debiasing Item-to-Item Recommendations With Small Annotated Datasets	Exposure Bias	-	Recsys 2020	-
Reinforced negative sampling for recommendation with exposure data	Exposure Bias	RNS	IJCAI 2019	Python
Samwalker: Social recommendation with informative sampling strategy	Exposure Bias	Samwalker	WWW 2019	MATLAB/C++
Samwalker: Social recommendation with informative sampling strategy	Exposure Bias	Samwalker	WWW 2019	MATLAB/C++
Collaborative filtering with social exposure: A modular approach to social recommendation	Exposure Bias	SERec	AAAI 2018	C++
An improved sampler for bayesian personalized ranking by leveraging view data	Exposure Bias	BPR+view	WWW 2018
Unbiased offline recommender evaluation for missing-not-at-random implicit feedback	Exposure Bias		RecSys 2018	Python
Entire space multi-task model: An effective approach for estimating post-click conversion rate	Exposure Bias	ESMM	SIGIR 2018
“Modeling users’ exposure with social knowledge influence and consumption influence for recommendation	Exposure Bias	SoEXBMF	CIKM 2018
Selection of negative samples for one-class matrix factorization	Exposure Bias	Full	SDM 2017	Python
Selection of negative samples for one-class matrix factorization	Exposure Bias	Full	SDM 2017	Python
Neural collaborative filtering （NCF）	Exposure Bias	NCF	WWW 2017	Python
Learning to rank with selection bias in personal search	Exposure Bias	Global/Segmented/ Generalized Bias Model	SIGIR 2016
Modeling user exposure in recommendation	Exposure Bias	Content ExpoMF/Location ExpoMF	WWW 2016	Python
Collaborative denoising auto-encoders for top-n recommender systems （CDAE）	Exposure Bias	CDAE	WSDM 2016	C++
Fast matrix factorization for online recommendation with implicit feedback	Exposure Bias	eALS	SIGIR 2016	Python
Dynamic matrix factorization with priors on unknown values	Exposure Bias		KDD 2015	C++
Logistic matrix factorization for implicit feedback data	Exposure Bias	LogisticMF	NIPS 2014
Improving one-class collaborative filtering by incorporating rich user information	Exposure Bias		CIKM 2010
Bpr: Bayesian personalized ranking from implicit feedback	Exposure Bias	BPR	UAI 2009	Python
Mind the gaps: weighting the unknown in large-scale one-class collaborative filtering	Exposure Bias	GALS/sMMMF	KDD 2009
Collaborative filtering for implicit feedback datasets	Exposure Bias		ICDM 2008	Python
One-class collaborative filtering	Exposure Bias	wALS/sALS	ICDM 2008
A Graph-Enhanced Click Model for Web Search	Position Bias	GraphCM	SIGIR 2021	Pytorch
Adapting Interactional Observation Embedding for Counterfactual Learning to Rank	Position Bias	IOBM	SIGIR 2021	Tensorflow
When Inverse Propensity Scoring does not Work: Affine Corrections for Unbiased Learning to Rank	Position Bias	Affine estimator	CIKM 2020	Tensorflow
A deep recurrent survival model for unbiased ranking	Position Bias	DRSR	SIGIR 2020	Python
Attribute-based propensity for unbiased learning in recommender systems: Algorithm and case studies	Position Bias		KDD 2020
Debiasing grid-based product search in e-commerce	Position Bias		KDD 2020
Cascade model-based propensity estimation for counterfactual learning to rank	Position Bias	CM-IPS	SIGIR 2020	Python
Addressing Trust Bias for Unbiased Learning-to-Rank	Position Bias	TrustPBM	WWW 2019	-
Position bias estimation for unbiased learning to rank in personal search	Position Bias	Regression-based EM	WSDM 2018
Unbiased learning to rank with unbiased propensity estimation	Position Bias	DLA	SIGIR 2018	Python
Unbiased learning-to-rank with biased feedback	Position Bias	Propensity SVM-Rank	WSDM 2017	Link
Multileave gradient descent for fast online learning to rank	Position Bias		WSDM 2016
Learning to rank with selection bias in personal search	Position Bias	Global/Segmented/ Generalized Bias Model	SIGIR 2016
Batch learning from logged bandit feedback through counterfactual risk minimization	Position Bias	POEM	JMLR 2015
Learning socially optimal information systems from egoistic users	Position Bias	SoPer-R/SoPer-S	ECML PKDD 2013
Reusing historical interaction data for faster online learning to rank for ir	Position Bias	RHC/CPS	WSDM 2013	Link
A novel click model and its applications to online advertising	Position Bias	GCM	WSDM 2010
A dynamic bayesian network click model for web search ranking	Position Bias	DBN	WWW 2009
Click chain model in web search	Position Bias	CCM	WWW 2009
A user browsing model to predict search engine click data from past observations.	Position Bias		SIGIR 2008
An experimental comparison of click position-bias models	Position Bias		WSDM 2008
Comparing click logs and editorial labels for training query rewriting	Position Bias		WWW 2007
Popularity bias in ranking and recommendation	Popularity Bias		AIES 2019
Disentangling User Interest and Conformity for Recommendation with Causal Embedding	Popularity Bias	DICE	WWW 2021	Python
The Unfairness of Popularity Bias in Recommendation	Popularity Bias	VAE-CF	SAC 2021	Pytorch
Popularity Bias in Dynamic Recommendation	Popularity Bias	FPC	KDD 2021	Tensorflow
Causal Intervention for Leveraging Popularity Bias in Recommendation	Popularity Bias	PDA	SIGIR 2021	Tensorflow
Deconfounded Recommendation for Alleviating Bias Amplification	Popularity Bias	DecRS	KDD 2021	Pytorch
Popularity Bias Is Not Always Evil: Disentangling Benign and Harmful Bias for Recommendation	Popularity Bias	TIDE	Arxiv 2021	To come
Model-agnostic counterfactual reasoning for eliminating popularity bias in recommender system	Popularity Bias	MACR	KDD 2021	Tensorflow
ESAM: discriminative domain adaptation with non-displayed items to improve long-tail performance	Popularity Bias	ESAM	SIGIR 2020	Python
Unbiased offline recommender evaluation for missing-not-at-random implicit feedback	Popularity Bias		RecSys 2018	Python
An adversarial approach to improve long-tail performance in neural collaborative filtering	Popularity Bias		CIKM 2018
Controlling popularity bias in learning-to-rank recommendation	Popularity Bias		RecSys 2017
Incorporating diversity in a learning to rank recommender system	Popularity Bias		FLAIRS 2016
The limits of popularity-based recommendations, and the role of social ties	Popularity Bias		KDD 2016	C++
Correcting popularity bias by enhancing recommendation neutrality	Popularity Bias		RecSys 2014
Efficiency improvement of neutrality-enhanced recommendation	Popularity Bias		RecSys 2013	Link
Leave No User Behind: Towards Improving the Utility of Recommender Systems for Non-mainstream Users	Unfairness	NAECF	WSDM 2021	Python
User-oriented Fairness in Recommendation	Unfairness		WWW2021	Python
Policy-Gradient Training of Fair and Unbiased Ranking Functions	Unfairness	FULTR	SIGIR 2021	Python
Towards Long-term Fairness in Recommendation	Unfairness	FCPO	WSDM 2021	Python
Towards Personalized Fairness based on Causal Notion	Unfairness		SIGIR 2021
Learning Fair Representations for Recommendation: A Graph-based Perspective	Unfairness	FairGo	WWW 2021	PyTorch
Debiasing Career Recommendations with Neural Fair Collaborative Filtering	Unfairness	NFCF	WWW 2021	PyTorch
Computationally Efficient Optimization of Plackett-Luce Ranking Models for Relevance and Fairness	Unfairness	PL-Rank	SIGIR 2021	Tensorflow
Debayes: a bayesian method for debiasing network embeddings	Unfairness	DeBayes	ICML 2020	Link
Measuring and Mitigating Item Under-Recommendation Bias in Personalized Ranking Systems	Unfairness	DPR	SIGIR 2020	Tensorflow
Controlling fairness and bias in dynamic learning-to-rank	Unfairness	FairCo	SIGIR 2020	Python
Designing fair ranking schemes	Unfairness		SIGMOD 2019
Fairwalk: Towards fair graph embedding	Unfairness	Fairwalk	IJCAI 2019
Fairness in recommendation ranking through pairwise comparisons	Unfairness		KDD 2019
“Compositional fairness constraints for graph embeddings	Unfairness		ICML 2019	Python
Fairness-aware ranking in search & recommendation systems with application to linkedin talent search	Unfairness	PSL	KDD 2019
Counterfactual fairness: Unidentification bound and algorithm.	Unfairness		IJCAI 2019
Privacy-aware recommendation with private-attribute protection using adversarial learning	Unfairness	RAP	WSDM 2019
Policy Learning for Fairness in Ranking	Unfairness	FAIR-PG-RANK	NIPS 2019	Python
Fairness of exposure in rankings	Unfairness		KDD 2018
Fairness-aware tensor-based recommendation	Unfairness	FATR	CIKM 2018	Pyhton
Fairness in decision-making - the causal explanation formula	Unfairness	Ctf-DE/Ctf-IE/Ctf-SE	AAAI 2018
On discrimination discovery and removal in ranked data using causal graph	Unfairness	FRank/FDetect	KDD 2018
A fairness-aware hybrid recommender system	Unfairness	PSL	FATREC 2018
Fair inference on outcomes	Unfairness		AAAI 2018	C++
Equity of attention: Amortizing individual fairness in rankings	Unfairness		SIGIR 2018
Fa*ir: A fair top-k ranking algorithm	Unfairness	FA*IR	CIKM 2017	Java/Python
Beyond parity: Fairness objectives for collaborative filtering	Unfairness		NIPS 2017
Balanced neighborhoods for fairness-aware collaborative recommendation	Unfairness	SLIM	RecSys 2017
Controlling popularity bias in learning-to-rank recommendation	Unfairness		RecSys 2017
Considerations on recommendation independence for a find-good-items task	Unfairness	IERS	Recsys 2017
New fairness metrics for recommendation that embrace differences	Unfairness		FAT/ML 2017
Fairness-aware group recommendation with pareto-efficiency	Unfairness		RecSys 2017
Counterfactual fairness	Unfairness	counterfactual fairness	arXiv 2017	Python
Censoring representations with an adversary	Unfairness	ALFR	ICLR 2016
Model-based approaches for independence-enhanced recommendation	Unfairness		IEEE 2016	Link
Efficiency improvement of neutrality-enhanced recommendation.	Unfairness		RecSys 2013	Link
Learning fair representations	Unfairness		JMLR 2013
Enhancement of the neutrality in recommendation	Unfairness		RecSys 2012
Discrimination-aware data mining	Unfairness		KDD 2008
AutoDebias: Learning to Debias for Recommendation	Loop Effect	AutoDebias	SIGIR 2021	PyTorch
A general knowledge distillation framework for counterfactual recommendation via uniform data	Loop Effect	KDCRec	SIGIR 2020	Python
Influence function for unbiased recommendation	Loop Effect	IF4URec	SIGIR 2020
Jointly learning to recommend and advertise	Loop Effect	RAM	KDD 2020
Counterfactual evaluation of slate recommendations with sequential reward interactions	Loop Effect	RIPS	KDD 2020	Python
Joint policy value learning for recommendation	Loop Effect	Dual Bandit	KDD 2020	Python
Degenerate feedback loops in recommender systems	Loop Effect	Oracle	AIES 2019
When people change their mind: Off-policy evaluation in non-stationary recommendation environments	Loop Effect		WSDM 2019	Python
Top-k off-policy correction for a reinforce recommender system	Loop Effect	REINFORCE	WSDM 2019	Python
Improving ad click prediction by considering non-displayed events	Loop Effect		CIKM 2019	Link
Large-scale interactive recommendation with tree-structured policy gradient	Loop Effect	TPGR	AAAI 2019	Python
Deep reinforcement learning for list-wise recommendations	Loop Effect	LIRD	KDD 2019	Python
Debiasing the human-recommender system feedback loop in collaborative filtering	Loop Effect	propensity MF	WWW 2019
Causal embeddings for recommendation	Loop Effect	CausE	RecSys 2018	Python
Stabilizing reinforcement learning in dynamic environment with application to online recommendation	Loop Effect	Robust DQN	KDD 2018
Recommendations with negative feedback via pairwise deep reinforcement learning	Loop Effect	DEERS	KDD 2018
Drn: A deep reinforcement learning framework for news recommendation	Loop Effect	DDQN	WWW 2018
Deep reinforcement learning for page-wise recommendations	Loop Effect	DeepPage	RecSys 2018
A reinforcement learning framework for explainable recommendation	Loop Effect		ICDM 2018
Interactive social recommendation	Loop Effect	ISR	CIKM 2017
Off-policy evaluation for slate recommendation	Loop Effect	PI	NIPS 2017	Python
Factorization bandits for interactive recommendation.	Loop Effect	FactorUCB	AAAI 2017
Predicting counterfactuals from large historical data and small randomized trials	Loop Effect		WWW 2016
Deconvolving feedbackloops in recommender systems	Loop Effect	Deconvolving feedback	NIPS 2016
Interactive collaborative filtering	Loop Effect	ICF	CIKM 2013	Python
A contextual-bandit approach to personalized news article recommendation	Loop Effect	LinUCB	WWW 2010	Python

Tips

We will keep updating this list, and if you find any missing related work or have any suggestions, please feel free to contact us (cjwustc@ustc.edu.cn).

If you find this repository useful to your research or work, it is really appreciate to star this repository. Thank you!