secmlt.optimization package

Submodules

secmlt.optimization.constraints module

Constraints for tensors and the corresponding batch-wise projections.

class secmlt.optimization.constraints.ClipConstraint(lb: float = 0.0, ub: float = 1.0)

Bases: Constraint

Box constraint, usually for the input space.

class secmlt.optimization.constraints.Constraint

Bases: ABC

Generic constraint.

class secmlt.optimization.constraints.InputSpaceConstraint(preprocessing: DataProcessing)

Bases: Constraint, ABC

Input space constraint.

Reverts the preprocessing, applies the constraint, and re-applies the preprocessing.

class secmlt.optimization.constraints.L0Constraint(radius: float | Tensor = 0.0)

Bases: LpConstraint

L0 constraint.

project(x: Tensor) → Tensor

Project the samples onto the L0 constraint.

Returns the sample with the top-k components preserved, and the rest set to zero.

Parameters:

x (torch.Tensor) – Input samples.

Returns:

Samples projected onto L0 constraint.

Return type:

torch.Tensor

class secmlt.optimization.constraints.L1Constraint(radius: float | Tensor = 0.0)

Bases: LpConstraint

L1 constraint.

project(x: Tensor) → Tensor

Compute Euclidean projection onto the L1 ball for a batch.

Source: https://gist.github.com/tonyduan/1329998205d88c566588e57e3e2c0c55

min ||x - u||_2 s.t. ||u||_1 <= eps

Inspired by the corresponding numpy version by Adrien Gaidon.

Parameters:

x (torch.Tensor) – Input tensor.

Returns:

Projected tensor.

Return type:

torch.Tensor

Notes

The complexity of this algorithm is in O(dlogd) as it involves sorting x.

References

[1] Efficient Projections onto the l1-Ball for Learning in High Dimensions

John Duchi, Shai Shalev-Shwartz, Yoram Singer, and Tushar Chandra. International Conference on Machine Learning (ICML 2008)

class secmlt.optimization.constraints.L2Constraint(radius: float | Tensor = 0.0)

Bases: LpConstraint

L2 constraint.

project(x: Tensor) → Tensor

Project onto the L2 constraint.

Parameters:

x (torch.Tensor) – Input tensor.

Returns:

Tensor projected onto the L2 constraint.

Return type:

torch.Tensor

class secmlt.optimization.constraints.LInfConstraint(radius: float | Tensor = 0.0)

Bases: LpConstraint

Linf constraint.

project(x: Tensor) → Tensor

Project onto the Linf constraint.

Parameters:

x (torch.Tensor) – Input tensor.

Returns:

Tensor projected onto the Linf constraint.

Return type:

torch.Tensor

class secmlt.optimization.constraints.LpConstraint(radius: float | Tensor = 0.0, p: str = 'linf')

Bases: Constraint, ABC

Abstract class for Lp constraint.

abstract project(x: Tensor) → Tensor

Project onto the Lp constraint.

Parameters:

x (torch.Tensor) – Input tensor.

Returns:

Tensor projected onto the Lp constraint.

Return type:

torch.Tensor

property radius: Tensor

Get radius of the constraint.

class secmlt.optimization.constraints.MaskConstraint(mask: Tensor)

Bases: Constraint

Constraint for keeping components only on the given mask.

class secmlt.optimization.constraints.QuantizationConstraint(preprocessing: DataProcessing = None, levels: list[float] | torch.Tensor | int = 255)

Bases: InputSpaceConstraint

Constraint for ensuring quantized outputs into specified levels.

secmlt.optimization.gradient_processing module

Processing functions for gradients.

class secmlt.optimization.gradient_processing.GradientProcessing

Bases: ABC

Gradient processing base class.

class secmlt.optimization.gradient_processing.LinearProjectionGradientProcessing(perturbation_model: str = 'l2')

Bases: GradientProcessing

Linear projection of the gradient onto Lp balls.

secmlt.optimization.gradient_processing.lin_proj_l1(x: Tensor) → Tensor

Return the linear projection of x onto an L1 unit ball.

Parameters:

x (torch.Tensor) – Input tensor to project.

Returns:

• torch.Tensor

• Linear projection of x onto unit L1 ball.

secmlt.optimization.initializer module

Initializers for the attacks.

class secmlt.optimization.initializer.Initializer

Bases: object

Initialization for the perturbation delta.

class secmlt.optimization.initializer.RandomLpInitializer(radius: Tensor, perturbation_model: LpPerturbationModels)

Bases: Initializer

Random perturbation initialization in Lp ball.

secmlt.optimization.losses module

Difference of Logits Loss.

class secmlt.optimization.losses.LogitDifferenceLoss(weight: Tensor | None = None, reduction: str = 'mean')

Bases: _WeightedLoss

Implements the Difference of Logits Loss.

forward(input: Tensor, target: Tensor) → Tensor

Compute the difference between input and target logits.

The loss is defined as:

\[\mathcal{L}(x, y) = -(z_y - \max_{j \ne y} z_j)\]

where:

• \(z\) are the model’s output logits for input \(x\),

• \(y\) is the true class index,

• \(z_y\) is the logit corresponding to the true class,

• \(\max_{j \ne y} z_j\) is the highest logit among incorrect classes.

This loss encourages the logit of the true class to be greater than that of any other class, and is particularly useful in adversarial attack settings like FMN.

secmlt.optimization.optimizer_factory module

Optimizer creation tools.

class secmlt.optimization.optimizer_factory.OptimizerFactory

Bases: object

Creator class for optimizers.

OPTIMIZERS: ClassVar[dict[str, torch.optim.Optimizer]] = {'adam': <class 'torch.optim.adam.Adam'>, 'sgd': <class 'torch.optim.sgd.SGD'>}

static create_adam(lr: float) → partial[Adam]

Create the Adam optimizer.

Parameters:

lr (float) – Learning rate.

Returns:

Adam optimizer.

Return type:

functools.partial[Adam]

static create_from_name(optimizer_name: str, lr: float, **kwargs) → partial[Adam] | partial[SGD]

Create an optimizer.

Parameters:

• optimizer_name (str) – One of the available optimizer names. Available: adam, sgd.

• lr (float) – Learning rate.

Returns:

The created optimizer.

Return type:

functools.partial[Adam] | functools.partial[SGD]

Raises:

ValueError – Raises ValueError when the requested optimizer is not in the list of implemented optimizers.

static create_sgd(lr: float) → partial[SGD]

Create the SGD optimizer.

Parameters:

lr (float) – Learning rate.

Returns:

SGD optimizer.

Return type:

functools.partial[SGD]

secmlt.optimization.random_perturb module

Random pertubations in Lp balls.

class secmlt.optimization.random_perturb.RandomPerturb(p: str, epsilon: float)

Bases: object

Random perturbation creator.

class secmlt.optimization.random_perturb.RandomPerturbBase(epsilon: float)

Bases: ABC

Class implementing the random perturbations in Lp balls.

abstract get_perturb(x: Tensor) → Tensor

Generate random perturbation for the Lp norm.

Parameters:

x (torch.Tensor) – Input samples to perturb.

class secmlt.optimization.random_perturb.RandomPerturbL0(epsilon: float)

Bases: RandomPerturbBase

Random Perturbations for L0 norm.

get_perturb(x: Tensor) → Tensor

Generate random perturbation for the L0 norm.

Parameters:

x (torch.Tensor) – Input samples to perturb.

Returns:

Perturbed samples.

Return type:

torch.Tensor

class secmlt.optimization.random_perturb.RandomPerturbL1(epsilon: float)

Bases: RandomPerturbBase

Random Perturbations for L1 norm.

get_perturb(x: Tensor) → Tensor

Generate random perturbation for the L1 norm.

Parameters:

x (torch.Tensor) – Input samples to perturb.

Returns:

Perturbed samples.

Return type:

torch.Tensor

class secmlt.optimization.random_perturb.RandomPerturbL2(epsilon: float)

Bases: RandomPerturbBase

Random Perturbations for L2 norm.

get_perturb(x: Tensor) → Tensor

Generate random perturbation for the L2 norm.

Parameters:

x (torch.Tensor) – Input samples to perturb.

Returns:

Perturbed samples.

Return type:

torch.Tensor

class secmlt.optimization.random_perturb.RandomPerturbLinf(epsilon: float)

Bases: RandomPerturbBase

Random Perturbations for Linf norm.

get_perturb(x: Tensor) → Tensor

Generate random perturbation for the Linf norm.

Parameters:

x (torch.Tensor) – Input samples to perturb.

Returns:

Perturbed samples.

Return type:

torch.Tensor

secmlt.optimization.scheduler_factory module

Learning Rate Schedulers creation tools.

class secmlt.optimization.scheduler_factory.LRSchedulerFactory

Bases: object

Creator class for learning rate schedulers.

SCHEDULERS: ClassVar[dict[str, _LRScheduler]] = {'cosine_annealing': <class 'torch.optim.lr_scheduler.CosineAnnealingLR'>, 'no_scheduler': <class 'secmlt.optimization.scheduler_factory.NoScheduler'>}

static create_cosine_annealing() → partial[_LRScheduler]

Create the Cosine Annealing scheduler.

Returns:

Cosine Annealing scheduler.

Return type:

functools.partial[LRScheduler]

static create_from_name(scheduler_name: str, **kwargs) → partial[_LRScheduler]

Create a learning rate scheduler.

Parameters:

scheduler_name (str) – One of the available scheduler names. Available: cosine.

Returns:

The created scheduler.

Return type:

functools.partial[LRScheduler]

Raises:

ValueError – Raises ValueError when the requested scheduler is not in the list of implemented schedulers.

static create_no_scheduler() → partial[_LRScheduler]

Create a NoScheduler instance.

Returns:

NoScheduler instance.

Return type:

functools.partial[LRScheduler]

class secmlt.optimization.scheduler_factory.NoScheduler(optimizer: Optimizer, last_epoch: int | None = -1)

Bases: _LRScheduler

No learning rate scheduler, does nothing.

step(epoch: int | None = None) → None

No operation.

Module contents

Optimization functionalities.