The contributions of this paper are multi-dimensional and highly significant: (i) developing a set of benchmarks for a diverse prediction tasks, (ii) demonstrating the utility of incorporating the vast amount of unlabeled protein data to pre-train models via semi-supervised learning, and (iii) the unlabeled data and pre-trained models made publicly available. This work will make a significant impact on the field by establishing solid benchmarks and facilitate the introduction of challenging protein prediction tasks to the machine learning community. The paper is extremely clearly written, well-structured and very concise. All reviewers are satisfied by the author response.