Social media users are overwhelmed by a large number of updates displayed chronologically in their news feed. Moreover, most updates are irrelevant. Ranking news feed updates by relevance has been proposed to help users catch up with the content they may find interesting. For this matter, supervised learning models have been commonly used to predict relevance. However, no comparative study was made to determine the most suitable models. In this work, we select, analyze, and compare six supervised learning algorithms applied to this case study. Experimental results on Twitter highlight that ensemble learning models are the most appropriate to predict the relevance of updates.