Originality - The work primarily builds on Koh and Liang's Influence functions work and extends it to evaluating influence over non-random groups of examples instead of single examples. The work is an interesting analytical extension of influence functions work. Contributions and related work are clearly specified and justified. Quality - The theoretical analyses appears sound although I did not go through the proofs in exact detail. The analyses shows empirical as well as theoretically the gaps in estimating influence using Koh and Liang's methods when used to analyze group effects. The analyses is mostly carried out in the regime where a large fraction of samples are removed. The analyses uses newton approximation of influence functions to bound a decomposition of the actual and estimated influence. Clarity - The paper is well written, comprehensive and empirical evaluation is adequate. Significance - The work in and of itself is interesting but significance to the field is low. Nonetheless the analyses has important implications of explainability and other related fields.

This paper examines the question of whether influence functions for determining the impact of a single training point on a model’s predictions generalize in the case where a group of points (e.g., a demographic group) that likely have cross-correlated effects are all removed. This is a fascinating and important problem. The authors demonstrate empirically that across five datasets the actual effect and the influence as approximated by the influence functions are correlated. This is an important result and validation for the real-world use of influence functions in important fairness domains. They then provide an analysis attempting to characterize the conditions under which the approximated influence and the actual effects are correlated, including nice experiments demonstrating that the Newton approximation is highly correlated to the actual effect.

# Originality As far as I can tell, the proposed use of influence functions is new. The experimental design, theoretical analysis, and case studies also appear to be novel. # Quality I think this paper is very well done. The empirical results are thorough, and the limitations of the theoretical analysis are clearly discussed. # Clarity The paper is well written; I enjoyed reading it. # Significance Several recent papers in the ML community have focused on influence functions. The paper presents another use for influence functions, and the case studies at the end of the paper make a compelling case for why group effect estimates are valuable. The paper also presents interesting new empirical and theoretical analyses that will likely be useful in other papers that study influence functions.