Adversarial examples

Adversarial examples

• Given a classifier \( f({\bf{x}}) \; \; : \; \; {\bf{x}} \in \mathcal{X} \rightarrow y \in \mathcal{Y} \; \) and original inputs \( {\bf{x}} \in \mathcal{X} \).
• The problem of generating untargeted adversarial examples can be expressed as the optimization:
  
  \[ argmin_{ {\bf{x}}^{*}} L({\bf{x}},{\bf{x}}^{*}), s.t. f({\bf{x}}) \neq f({\bf{x}}^{*}) \]

where \(L(\cdot)\) is a distance metric between examples from the input space.

Adversarial examples

• Similarly, generating a targeted adversarial attack on a classifier can be expressed as:
  
  \[ argmin_{ {\bf{x}}^{*}} L({\bf{x}},{\bf{x}}^{*}), s.t. f({\bf{x}}^{*}) = y_t \]

where \( y_t \in \mathcal{Y} \) is some target label chosen by the attacker.

Adversarial examples

Characteristics

• It can be seen as a strategy for generating training data.
• It is also a method to investigate the weaknesses of classification models, not only DNNs.
• Methods to construct adversarial examples try to add small perturbations to the inputs to fool the network into producing incorrect outputs.

Adversarial examples

Characteristics

• In some scenarios it is possible to detect when an input is adversarial.
• Another research direction is to devise learning methods that make DNNs less vulnerable to adversarial perturbations.
• This is a current direction of active research.

Methods for creating adversarial examples

Adversarial examples

Methods for creating adversarial examples

1. White Box Attacks: The attacker is assumed to have information about the target model such as the architecture, parameters, and training procedure.

Two main groups include Gradient Based Optimization and Constrained Optimization methods.

2. Blackbox attacks: Involve only access to the output of a target model, and not its internals.

Include a variety of optimization methods such as Evolutionary Algorithms and other search-based heuristics.

Methods for creating adversarial examples

L-BFGS targeted attack

1. The objective is to find a perceptually-minimal input perturbation \( \arg min_r ||r||_2 \) where \( r = x^{\prime}-x \), such that the targeted labeled is output.
2. The Limited Memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithm is used to pose the problem as a box-constrained formulation.
3. The optimization problem is to find \( x^{\prime} \) that minimizes:
   \begin{equation} c ||r||_2 + \mathcal{L}(x^{\prime},t) \end{equation}

where \(x^{\prime}, x \in [0,1]^n\), \( \mathcal{L}(x^{\prime},t) \) is the true loss function of the model, and \(t\) is the target misclassification label.

4. The above optimization process is iterated for increasingly large values of \(c\) until the models classifies as the targeted label.

Methods for creating adversarial examples

Fast Gradient Sign Method (FGSM) attack

1. Finds a perturbation such that the loss function of the target model will increase.
2. As a result the classification confidence will decrease.
3. Then, it will more likely that the model will assign the highest probability to another (incorrect) class.
4. FGSM works by calculating the gradient of the loss function with respect to the input, and creating a small perturbation:
   \begin{equation} x^{\prime} = x + \epsilon \cdot sign \left(\nabla_x \mathcal{L}(x, y) \right) \end{equation}

where \( \nabla_x \mathcal{L}(x, y) \) is the first derivative of the loss function with respect to the input x.

5. FGSM does not explicitly optimize for the adversary \(x^{\prime} \) to have a minimal perceptual differenc. Instead, it uses a small \( \epsilon \) to weakly bound the perturbation \( r \).

Methods for creating adversarial examples

Fast Gradient Sign Method (FGSM) attack

Fast Gradient Sign Method (FGSM) attack

Methods for creating adversarial examples

Jacobian-based Saliency Map Attacks (JSMA)

1. It was originally conceived for visualizing how deep neural networks make predictions.
2. The saliency map rates each input feature (e.g., each pixel in an image) by its influence upon the network’s class prediction.
3. JSMA exploits this information by perturbing a small set of input features to cause misclassification.
4. This is in contrast to attacks like the FGSM that modify most, if not all, input features. As such, JSMA attacks tend to find sparse perturbations.

Universal adversarial examples

• A single small image perturbation that fools a state-of-the-art deep neural network classifier on all natural images.

Universal adversarial examples

• Single-class universal perturbations: Expected to fool the target model only for inputs of one particular class.
• Multiple-class universal perturbations: Expected to fool target model only for a particular subset of \(N\) classes, with \(1 < N < k\).
• Fully universal perturbations: Expected to fool a target model for every possible input, with a complete independence of the original class.

Adversarial examples

Applications in other domains

• Adversarial examples has been mainly created for images in computer vision problems.
• However recent research proposes their use for other domains (e.g., audio, graphs problems).
• Audio adversarial examples constructed from voice commands can be used to fool virtual assistants based on speech recognition.
• Speech-to-text systems and medical images can be also negatively affected by adversarial examples.

Adversarial examples in Speech Command Classification

• An audio waveform is transformed to fool the model while keeping the distortion below a given threshold.
• An end-to-end differentiable classification process is assumed from which all the information will be available.
• The gradients of the model’s output can be computed with respect to the input audio waveform and used to create the adversarial example.

Adversarial defenses

Adversarial defenses

Feature squeezing

• The dimensionality of input features is often unnecessarily large, yielding a large attack surface.
• Predictions between squeezed and unsqueezed inputs are compared.
• Examples of squeezed features: color bit-depth reduction and spatial smoothing.

Transferability

• Adversarial examples or perturbations generated to fool a particular model can be used to fool others.
• Different ways of transferability:
  1. Transferability between different models.
  2. Transferability between different machine learning techniques.
  3. Transferability between models that perform different tasks.