secmlt.optimization package

Submodules

secmlt.optimization.constraints module

Constraints for tensors and the corresponding batch-wise projections.

class secmlt.optimization.constraints.ClipConstraint(lb: float = 0.0, ub: float = 1.0)[source]

Bases: Constraint

Box constraint, usually for the input space.

__call__(x: torch.Tensor, *args, **kwargs) → torch.Tensor[source]

Call the projection function.

Parameters:: x (torch.Tensor) – Input samples.
Returns:: Tensor projected onto the box constraint.
Return type:: torch.Tensor

__init__(lb: float = 0.0, ub: float = 1.0) → None[source]

Create box constraint.

Parameters:

lb (float, optional) – Lower bound of the domain, by default 0.0.
ub (float, optional) – Upper bound of the domain, by default 1.0.

class secmlt.optimization.constraints.Constraint[source]

Bases: ABC

Generic constraint.

abstract __call__(x: torch.Tensor, *args, **kwargs) → torch.Tensor[source]

Project onto the constraint.

Parameters:: x (torch.Tensor) – Input tensor.
Returns:: Tensor projected onto the constraint.
Return type:: torch.Tensor

class secmlt.optimization.constraints.L0Constraint(radius: float = 0.0, center: float = 0.0)[source]

Bases: LpConstraint

L0 constraint.

__init__(radius: float = 0.0, center: float = 0.0) → None[source]

Create L0 constraint.

Parameters:

radius (float, optional) – Radius of the constraint, by default 0.0.
center (float, optional) – Center of the constraint, by default 0.0.

project(x: torch.Tensor) → torch.Tensor[source]

Project the samples onto the L0 constraint.

Returns the sample with the top-k components preserved, and the rest set to zero.

Parameters:: x (torch.Tensor) – Input samples.
Returns:: Samples projected onto L0 constraint.
Return type:: torch.Tensor

class secmlt.optimization.constraints.L1Constraint(radius: float = 0.0, center: float = 0.0)[source]

Bases: LpConstraint

L1 constraint.

__init__(radius: float = 0.0, center: float = 0.0) → None[source]

Create L1 constraint.

Parameters:

radius (float, optional) – Radius of the constraint, by default 0.0.
center (float, optional) – Center of the constraint, by default 0.0.

project(x: torch.Tensor) → torch.Tensor[source]

Compute Euclidean projection onto the L1 ball for a batch.

Source: https://gist.github.com/tonyduan/1329998205d88c566588e57e3e2c0c55

min ||x - u||_2 s.t. ||u||_1 <= eps

Inspired by the corresponding numpy version by Adrien Gaidon.

Parameters:: x (torch.Tensor) – Input tensor.
Returns:: Projected tensor.
Return type:: torch.Tensor

Notes

The complexity of this algorithm is in O(dlogd) as it involves sorting x.

References

[1] Efficient Projections onto the l1-Ball for Learning in High Dimensions: John Duchi, Shai Shalev-Shwartz, Yoram Singer, and Tushar Chandra. International Conference on Machine Learning (ICML 2008)

class secmlt.optimization.constraints.L2Constraint(radius: float = 0.0, center: float = 0.0)[source]

Bases: LpConstraint

L2 constraint.

__init__(radius: float = 0.0, center: float = 0.0) → None[source]

Create L2 constraint.

Parameters:

radius (float, optional) – Radius of the constraint, by default 0.0.
center (float, optional) – Center of the constraint, by default 0.0.

project(x: torch.Tensor) → torch.Tensor[source]

Project onto the L2 constraint.

Parameters:: x (torch.Tensor) – Input tensor.
Returns:: Tensor projected onto the L2 constraint.
Return type:: torch.Tensor

class secmlt.optimization.constraints.LInfConstraint(radius: float = 0.0, center: float = 0.0)[source]

Bases: LpConstraint

Linf constraint.

__init__(radius: float = 0.0, center: float = 0.0) → None[source]

Create Linf constraint.

Parameters:

radius (float, optional) – Radius of the constraint, by default 0.0.
center (float, optional) – Center of the constraint, by default 0.0.

project(x: torch.Tensor) → torch.Tensor[source]

Project onto the Linf constraint.

Parameters:: x (torch.Tensor) – Input tensor.
Returns:: Tensor projected onto the Linf constraint.
Return type:: torch.Tensor

class secmlt.optimization.constraints.LpConstraint(radius: float = 0.0, center: float = 0.0, p: str = 'linf')[source]

Bases: Constraint, ABC

Abstract class for Lp constraint.

__call__(x: torch.Tensor, *args, **kwargs) → torch.Tensor[source]

Project the samples onto the Lp constraint.

Parameters:: x (torch.Tensor) – Input tensor.
Returns:: Tensor projected onto the Lp constraint.
Return type:: torch.Tensor

__init__(radius: float = 0.0, center: float = 0.0, p: str = 'linf') → None[source]

Create Lp constraint.

Parameters:

radius (float, optional) – Radius of the constraint, by default 0.0.
center (float, optional) – Center of the constraint, by default 0.0.
p (str, optional) – Value of p for Lp norm, by default LpPerturbationModels.LINF.

abstract project(x: torch.Tensor) → torch.Tensor[source]

Project onto the Lp constraint.

Parameters:: x (torch.Tensor) – Input tensor.
Returns:: Tensor projected onto the Lp constraint.
Return type:: torch.Tensor

secmlt.optimization.gradient_processing module

Processing functions for gradients.

class secmlt.optimization.gradient_processing.GradientProcessing[source]

Bases: ABC

Gradient processing base class.

abstract __call__(grad: torch.Tensor) → torch.Tensor[source]

Process the gradient with the given transformation.

Parameters:: grad (torch.Tensor) – Input gradients.
Returns:: The processed gradients.
Return type:: torch.Tensor

class secmlt.optimization.gradient_processing.LinearProjectionGradientProcessing(perturbation_model: str = 'l2')[source]

Bases: GradientProcessing

Linear projection of the gradient onto Lp balls.

__call__(grad: torch.Tensor) → torch.Tensor[source]

Process gradient with linear projection onto the Lp ball.

Sets the direction by maximizing the scalar product with the gradient over the Lp ball.

Parameters:: grad (torch.Tensor) – Input gradients.
Returns:: The gradient linearly projected onto the Lp ball.
Return type:: torch.Tensor
Raises:: NotImplementedError – Raises NotImplementedError if the norm is not in 2, inf.

__init__(perturbation_model: str = 'l2') → None[source]

Create linear projection for the gradient.

Parameters:: perturbation_model (str, optional) – Perturbation model for the Lp ball, by default LpPerturbationModels.L2.
Raises:: ValueError – Raises ValueError if the perturbation model is not implemented. Available, l1, l2, linf.

secmlt.optimization.initializer module

Initializers for the attacks.

class secmlt.optimization.initializer.Initializer[source]

Bases: object

Initialization for the perturbation delta.

__call__(x: torch.Tensor) → torch.Tensor[source]

Get initialization for the perturbation.

Parameters:: x (torch.Tensor) – Input samples.
Returns:: Initialized perturbation.
Return type:: torch.Tensor

class secmlt.optimization.initializer.RandomLpInitializer(radius: torch.Tensor, perturbation_model: LpPerturbationModels)[source]

Bases: Initializer

Random perturbation initialization in Lp ball.

__call__(x: torch.Tensor) → torch.Tensor[source]

Get random perturbations.

Parameters:: x (torch.Tensor) – Input samples.
Returns:: Initialized random perturbations.
Return type:: torch.Tensor

__init__(radius: torch.Tensor, perturbation_model: LpPerturbationModels) → None[source]

Create random perturbation initializer.

Parameters:

radius (torch.Tensor) – Radius of the Lp ball for the constraint.
perturbation_model (LpPerturbationModels) – Perturbation model for the constraint.

secmlt.optimization.optimizer_factory module

Optimizer creation tools.

class secmlt.optimization.optimizer_factory.OptimizerFactory[source]

Bases: object

Creator class for optimizers.

OPTIMIZERS: ClassVar[dict[str, torch.optim.Optimizer]] = {'adam': torch.optim.Adam, 'sgd': torch.optim.SGD}

static create_adam(lr: float) → partial[torch.optim.Adam][source]

Create the Adam optimizer.

Parameters:: lr (float) – Learning rate.
Returns:: Adam optimizer.
Return type:: functools.partial[Adam]

static create_from_name(optimizer_name: str, lr: float, **kwargs) → partial[torch.optim.Adam] | partial[torch.optim.SGD][source]

Create an optimizer.

Parameters:

optimizer_name (str) – One of the available optimizer names. Available: adam, sgd.
lr (float) – Learning rate.

Returns:

The created optimizer.

Return type:

functools.partial[Adam] | functools.partial[SGD]

Raises:

ValueError – Raises ValueError when the requested optimizer is not in the list of implemented optimizers.

static create_sgd(lr: float) → partial[torch.optim.SGD][source]

Create the SGD optimizer.

Parameters:: lr (float) – Learning rate.
Returns:: SGD optimizer.
Return type:: functools.partial[SGD]

secmlt.optimization.random_perturb module

Random pertubations in Lp balls.

class secmlt.optimization.random_perturb.RandomPerturb(p: str, epsilon: float)[source]

Bases: object

Random perturbation creator.

static __new__(cls, p: str, epsilon: float) → RandomPerturbBase[source]

Creator for random perturbation in Lp norms.

Parameters:

p (str) – p-norm used for the random perturbation shape.
epsilon (float) – Radius of the random perturbation constraint.

Returns:

Random perturbation object.

Return type:

RandomPerturbBase

Raises:

ValueError – Raises ValueError if the norm is not in 0, 1, 2, inf.

class secmlt.optimization.random_perturb.RandomPerturbBase(epsilon: float)[source]

Bases: ABC

Class implementing the random perturbations in Lp balls.

__call__(x: torch.Tensor) → torch.Tensor[source]

Get the perturbations for the given samples.

Parameters:: x (torch.Tensor) – Input samples to perturb.
Returns:: Perturbations (to apply) to the given samples.
Return type:: torch.Tensor

__init__(epsilon: float) → None[source]

Create random perturbation object.

Parameters:: epsilon (float) – Constraint radius.

abstract get_perturb(x: torch.Tensor) → torch.Tensor[source]

Generate random perturbation for the Lp norm.

Parameters:: x (torch.Tensor) – Input samples to perturb.

class secmlt.optimization.random_perturb.RandomPerturbL0(epsilon: float)[source]

Bases: RandomPerturbBase

Random Perturbations for L0 norm.

get_perturb(x: torch.Tensor) → torch.Tensor[source]

Generate random perturbation for the L0 norm.

Parameters:: x (torch.Tensor) – Input samples to perturb.
Returns:: Perturbed samples.
Return type:: torch.Tensor

class secmlt.optimization.random_perturb.RandomPerturbL1(epsilon: float)[source]

Bases: RandomPerturbBase

Random Perturbations for L1 norm.

get_perturb(x: torch.Tensor) → torch.Tensor[source]

Generate random perturbation for the L1 norm.

Parameters:: x (torch.Tensor) – Input samples to perturb.
Returns:: Perturbed samples.
Return type:: torch.Tensor

class secmlt.optimization.random_perturb.RandomPerturbL2(epsilon: float)[source]

Bases: RandomPerturbBase

Random Perturbations for L2 norm.

get_perturb(x: torch.Tensor) → torch.Tensor[source]

Generate random perturbation for the L2 norm.

Parameters:: x (torch.Tensor) – Input samples to perturb.
Returns:: Perturbed samples.
Return type:: torch.Tensor

class secmlt.optimization.random_perturb.RandomPerturbLinf(epsilon: float)[source]

Bases: RandomPerturbBase

Random Perturbations for Linf norm.

get_perturb(x: torch.Tensor) → torch.Tensor[source]

Generate random perturbation for the Linf norm.

Parameters:: x (torch.Tensor) – Input samples to perturb.
Returns:: Perturbed samples.
Return type:: torch.Tensor

Module contents

Optimization functionalities.