NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID:5500
Title:Functional Adversarial Attacks

Reviewer 1


		
#####After Rebuttal##### I think the authors to clarify my concerns. For the second question, I would like to see more instances of functional attacks, which could be also applied to image classification. Given such results would make this paper stronger. I will keep my original rating. ##################### This paper proposes a novel class of threat models for crafting adversarial examples. The paper is well-written. A typical type of functional adversarial attacks is realized by changing the color of images as ReColorAdv. The constrains on this attack and the optimization process is clearly illustrated. Below are some minor concerns: 1. Although the proposed functional adversarial attack is novel, it is somewhat relevant to "blind-spot attack" (Zhang et al., "The Limitations of Adversarial Training and the Blind-Spot Attack", ICLR 2019), and "unrestricted adversarial examples" (Song et al., "Constructing Unrestricted Adversarial Examples with Generative Models", NeurIPS 2018). The authors can discuss the connections between the proposed attack and other attacks, and also the differences. 2. Although the definition of functional threat model is general and flexible, this paper only provides one instance of functional attacks. More examples of functional attacks can make this paper more convincing and interesting. 3. From Table 1, C and C-RGB attacks are less powerful than L_\infty attack (D). S+D attack is more powerful than C+D in most cases, and gets similar performance to C+S+D. So the concern is about the effectiveness of the proposed ReColorAdv attack. It seems that ReColorAdv brings little benefit.

Reviewer 2


		
Update after authors response: Authors addressed my concerns regarding strength of the attack in their response. ---------------------------- Original comments: Originality: Authors propose novel adversarial attack which recolors all pixels of the image in the same way using function f(x). Authors also propose to combine multiple adversarial attack to build stronger adversary. Quality: Overall paper is technically sound. One piece of critics is following. It seems like proposed attack is actually much weaker compared to other attacks (as could be seen from table 1). However this could be compensated by the fact that proposed attack seems to be less noticeable visually and also could be combined with other attacks. Clarity: Paper is clearly written. Significance: Moderate significance. Authors propose interesting new technique for crafting adversarial perturbations which in most cases visually less noticeable compared to L-infinity ball restricted perturbations.

Reviewer 3


		
This paper would be more interesting if it could do more than just adjust the brightness on each channel independently. Would he approach work taking a tripple of values (or ) and perturb those functionally also be effective using this definition? How much more power would that have? The proposed approach is interesting. New threat models are generally useful and help expand the space of valid attacks. I was especially happy to see a discussion around the color space. (However: I was missing a definition of CIELUV --- was it defined anywhere?) It would have been nice to see more discussion around this point, and in particular setting different bounds per channel. For example, it's well studied that humans are more sensitive to changes in green than blue (and so standard compression will compress blue more than green). Can the same be modeled here? My main concern with this paper is it is not technically deep: there is not much novelty gained on top of the basic idea. However, it is evaluated well and clearly presented. There are no major writing issues. Two minor comments: - I am relatively well-versed in the literature, but even I don't know what function "f6" is from Carlini & Wagner [2]. - In 5.1 using the word "transferability" is somewhat confusing as this is often the terminology used for adversarial examples. Reply to Author Response: Thank you for the clarifications and experiments. I am increasing my score as a result.