3. API Reference

3.1. Dictionary learning models

class sparsecoding.models.SparseCoding(inference_method, n_basis, n_features, sparsity_penalty=0.2, device=None, check_for_dictionary_nan=False, **kwargs)

Bases: Module

Class for learning a sparse code via dictionary learning

Parameters:

  • inference_method (sparsecoding.InferenceMethod) – Method for inferring coefficients for each image given the dictionary

  • n_basis (int) – Number of basis functions in dictionary

  • n_features (int) – Number of features in data

  • sparsity_penalty (float, default=0.2) – Sparsity penalty

  • dictionary_lr (float, default=1e-2) – Learning rate of dictionary update

  • device (torch.device, default=torch.device("cpu")) – Which device to utilize

  • check_for_dictionary_nan (bool, default=False) – Flag to check for nans in the dictionary after gradient updates and normalizations. Raises ValueError if nan found

checknan(data=tensor(0), name='data')

Check for nan values in dictinary, or data

Parameters:

  • data (array-like, optional) – Data to check for nans

  • name (str, optional) – Name to add to error, if one is thrown

Raises:

ValueError – If nan is found in data

compute_grad_dict(data, a)

Compute gradient of loss function w.r.t. dictionary elements

Parameters:

  • data (array-like, shape [batch_size, n_features]) – input data

  • a (array-like, shape [batch_size, n_basis]) – already-inferred coefficients

Returns:

dictionary_grad – gradient of dictionary

Return type:

array-like, shape [n_features, n_basis]

compute_loss(data, a)

Compute loss given data and inferred coefficients

Parameters:

  • data (array-like, shape [batch_size, n_features])

  • a (array-like, shape [batch_size, n_basis]) – inferred coefficients

Returns:

loss

Return type:

float

get_numpy_dictionary()

Returns dictionary as numpy array

Returns:

dictionary – numpy array dictionary

Return type:

array-like, shape [n_features,n_basis]

learn_dictionary(dataset, n_epoch, batch_size)

Learn dictionary for n_epoch epochs

Parameters:

  • dataset (torch.utils.data.Dataset or array-like, shape [n_samples, n_features]) – Input dataset

  • n_epoch (int) – Iumber of iterations to learn dictionary

  • batch_size (int) – Batch size to do dictionary updates over

Returns:

losses – Model losses (i.e., energy for the Boltzmann enthusiasts) after each dictionary update

Return type:

array-like, shape [nepoch,]

load_dictionary(filename)

Load dictionary from pkl dump

Parameters:

filename (str) – File to load dictionary from

normalize_dictionary()

Normalize columns of dictionary matrix to unit norm.

save_dictionary(filename)

Save dictionary to pkl dump

Parameters:

filename (str) – File to save current dictionary to

set_dictionary(dictionary)

Set model dictionary to passed dictionary

Parameters:

dictionary (array-like, shape [n_features, n_basis]) – Dictionary to set default dictionary to

update_dictionary(data, a)

Compute gradient of loss function w.r.t. dictionary elements, and update

Parameters:

  • data (array-like, shape [batch_size,n_features]) – Input data

  • a (array-like, shape [batch_size, n_basis]) – Already-inferred coefficients

3.2. Inference methods

class sparsecoding.inference.IHT(sparsity, n_iter=10, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Infer coefficients for each image in data using elements dictionary. Method description can be traced to “Iterative Hard Thresholding for Compressed Sensing” (T. Blumensath & M. E. Davies, 2009)

Parameters:

  • sparsity (float) – Sparsity of the solution. The number of active coefficients will be set to ceil(sparsity * data_dim) at the end of each iterative update.

  • n_iter (int, default=10) – number of iterations to run for an inference method

  • return_all_coefficients (str, default=False) – returns all coefficients during inference procedure if True user beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging

  • solver (default=None)

infer(data, dictionary)

Infer coefficients for each image in data using dict elements dictionary using Iterative Hard Thresholding (IHT)

Parameters:

  • data (array-like (batch_size, n_features)) – data to be used in sparse coding

  • dictionary (array-like, (n_features, n_basis)) – dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like (batch_size, n_basis)

class sparsecoding.inference.ISTA(n_iter=100, sparsity_penalty=0.01, stop_early=False, epsilon=0.01, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Iterative shrinkage-thresholding algorithm for solving LASSO problems.

Parameters:

  • n_iter (int, default=100) – Number of iterations to run

  • sparsity_penalty (float, default=0.2)

  • stop_early (bool, default=False) – Stops dynamics early based on change in coefficents

  • epsilon (float, default=1e-2) – Only used if stop_early True, specifies criteria to stop dynamics

  • return_all_coefficients (str, default=False) – Returns all coefficients during inference procedure if True User beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging.

  • solver (default=None)

References

[1] Beck, A., & Teboulle, M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences, 2(1), 183-202.

infer(data, dictionary, coeff_0=None, use_checknan=False)

Infer coefficients for each image in data using dictionary elements. Uses ISTA (Beck & Taboulle 2009), equations 1.4 and 1.5.

Parameters:

  • data (array-like, shape [batch_size, n_features])

  • dictionary (array-like, shape [n_features, n_basis])

  • coeff_0 (array-like, shape [n_samples, n_basis], optional) – Initial coefficient values

  • use_checknan (bool, default=False) – Check for nans in coefficients on each iteration. Setting this to False can speed up inference time.

Returns:

coefficients – First case occurs if return_all_coefficients == “none”. If return_all_coefficients != “none”, returned shape is second case. Returned dimension along dim 1 can be less than n_iter when stop_early==True and stopping criteria met.

Return type:

array-like, shape [n_samples, n_basis] OR [n_samples, n_iter+1, n_basis]

threshold_nonlinearity(u)

Soft threshhold function

Parameters:

u (array-likes, shape [batch_size, n_basis]) – Membrane potentials

Returns:

a – activations

Return type:

array-like, shape [batch_size, n_basis]

class sparsecoding.inference.InferenceMethod(solver)

Bases: object

Base class for inference method.

static checknan(data=tensor(0), name='data')

Check for nan values in data.

Parameters:

  • data (array-like, optional) – Data to check for nans

  • name (str, default="data") – Name to add to error, if one is thrown

Raises:

ValueError – If the nan found in data

grad()

Compute the gradient step.

infer(dictionary, data, coeff_0=None, use_checknan=False)

Infer the coefficients given a dataset and dictionary.

Parameters:

  • dictionary (array-like, shape [n_features,n_basis])

  • data (array-like, shape [n_samples,n_features])

  • coeff_0 (array-like, shape [n_samples,n_basis], optional) – Initial coefficient values.

  • use_checknan (bool, default=False) – Check for nans in coefficients on each iteration

Returns:

coefficients

Return type:

array-like, shape [n_samples,n_basis]

initialize(a)

Define initial coefficients.

class sparsecoding.inference.LCA(n_iter=100, coeff_lr=0.001, threshold=0.1, stop_early=False, epsilon=0.01, solver=None, return_all_coefficients='none', nonnegative=False)

Bases: InferenceMethod

Method implemented according locally competative algorithm (LCA) with the ideal soft thresholding function.

Parameters:

  • n_iter (int, default=100) – Number of iterations to run

  • coeff_lr (float, default=1e-3) – Update rate of coefficient dynamics

  • threshold (float, default=0.1) – Threshold for non-linearity

  • stop_early (bool, default=False) – Stops dynamics early based on change in coefficents

  • epsilon (float, default=1e-2) – Only used if stop_early True, specifies criteria to stop dynamics

  • nonnegative (bool, default=False) – Constrain coefficients to be nonnegative

  • return_all_coefficients (str, {"none", "membrane", "active"}, default="none") – Returns all coefficients during inference procedure if not equal to “none”. If return_all_coefficients==”membrane”, membrane potentials (u) returned. If return_all_coefficients==”active”, active units (a) (output of thresholding function over u) returned. User beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging.

  • solver (default=None)

References

[1] Rozell, C. J., Johnson, D. H., Baraniuk, R. G., & Olshausen, B. A. (2008). Sparse coding via thresholding and local competition in neural circuits. Neural computation, 20(10), 2526-2563.

grad(b, G, u, a)

Compute the gradient step on membrane potentials

Parameters:

  • b (array-like, shape [batch_size, n_coefficients]) – Driver signal for coefficients

  • G (array-like, shape [n_coefficients, n_coefficients]) – Inhibition matrix

  • a (array-like, shape [batch_size, n_coefficients]) – Currently active coefficients

Returns:

du – Gradient of membrane potentials

Return type:

array-like, shape [batch_size, n_coefficients]

infer(data, dictionary, coeff_0=None, use_checknan=False)

Infer coefficients using provided dictionary

Parameters:

  • dictionary (array-like, shape [n_features, n_basis])

  • data (array-like, shape [n_samples, n_features])

  • coeff_0 (array-like, shape [n_samples, n_basis], optional) – Initial coefficient values

  • use_checknan (bool, default=False) – Check for nans in coefficients on each iteration. Setting this to False can speed up inference time.

Returns:

coefficients – First case occurs if return_all_coefficients == “none”. If return_all_coefficients != “none”, returned shape is second case. Returned dimension along dim 1 can be less than n_iter when stop_early==True and stopping criteria met.

Return type:

array-like, shape [n_samples, n_basis] OR [n_samples, n_iter+1, n_basis]

threshold_nonlinearity(u)

Soft threshhold function

Parameters:

u (array-like, shape [batch_size, n_basis]) – Membrane potentials

Returns:

a – Activations

Return type:

array-like, shape [batch_size, n_basis]

class sparsecoding.inference.LSM(n_iter=100, n_iter_LSM=6, beta=0.01, alpha=80.0, sigma=0.005, sparse_threshold=0.01, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Infer latent coefficients generating data given dictionary. Method implemented according to “Group Sparse Coding with a Laplacian Scale Mixture Prior” (P. J. Garrigues & B. A. Olshausen, 2010)

Parameters:

  • n_iter (int, default=100) – Number of iterations to run for an optimizer

  • n_iter_LSM (int, default=6) – Number of iterations to run the outer loop of LSM

  • beta (float, default=0.01) – LSM parameter used to update lambdas

  • alpha (float, default=80.0) – LSM parameter used to update lambdas

  • sigma (float, default=0.005) – LSM parameter used to compute the loss function

  • sparse_threshold (float, default=10**-2) – Threshold used to discard smallest coefficients in the final solution SM parameter used to compute the loss function

  • return_all_coefficients (bool, default=False) – Returns all coefficients during inference procedure if True User beware: If n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging.

  • solver (default=None)

References

[1] Garrigues, P., & Olshausen, B. (2010). Group sparse coding with a laplacian scale mixture prior. Advances in neural information processing systems, 23.

infer(data, dictionary)

Infer coefficients for each image in data using dict elements dictionary using Laplacian Scale Mixture (LSM)

Parameters:

  • data (array-like, shape [batch_size, n_features]) – Data to be used in sparse coding

  • dictionary (array-like, shape [n_features, n_basis]) – Dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like, shape [batch_size, n_basis]

lsm_Loss(data, dictionary, coefficients, lambdas, sigma)

Compute LSM loss according to equation (7) in (P. J. Garrigues & B. A. Olshausen, 2010)

Parameters:

  • data (array-like, shape [batch_size, n_features]) – Data to be used in sparse coding

  • dictionary (array-like, shape [n_features, n_basis]) – Dictionary to be used

  • coefficients (array-like, shape [batch_size, n_basis]) – The current values of coefficients

  • lambdas (array-like, shape [batch_size, n_basis]) – The current values of regularization coefficient for all basis

  • sigma (float, default=0.005) – LSM parameter used to compute the loss functions

Returns:

loss – Loss values for each data sample

Return type:

array-like, shape [batch_size, 1]

class sparsecoding.inference.MP(sparsity, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Infer coefficients for each image in data using elements dictionary. Method description can be traced to “Matching Pursuits with Time-Frequency Dictionaries” (S. G. Mallat & Z. Zhang, 1993)

Parameters:

  • sparsity (scalar (1,)) – sparsity of the solution

  • return_all_coefficients (string (1,) default=False) – returns all coefficients during inference procedure if True user beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging

  • solver (default=None)

infer(data, dictionary)

Infer coefficients for each image in data using dict elements dictionary using Matching Pursuit (MP)

Parameters:

  • data (array-like (batch_size, n_features)) – data to be used in sparse coding

  • dictionary (array-like, (n_features, n_basis)) – dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like (batch_size, n_basis)

class sparsecoding.inference.OMP(sparsity, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Infer coefficients for each image in data using elements dictionary. Method description can be traced to:

“Orthogonal Matching Pursuit: Recursive Function Approximation with Application to Wavelet Decomposition” (Y. Pati & R. Rezaiifar & P. Krishnaprasad, 1993)

Parameters:

  • sparsity (scalar (1,)) – sparsity of the solution

  • return_all_coefficients (string (1,) default=False) – returns all coefficients during inference procedure if True user beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging

  • solver (default=None)

infer(data, dictionary)

Infer coefficients for each image in data using dict elements dictionary using Orthogonal Matching Pursuit (OMP)

Parameters:

  • data (array-like (batch_size, n_features)) – data to be used in sparse coding

  • dictionary (array-like, (n_features, n_basis)) – dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like (batch_size, n_basis)

class sparsecoding.inference.PyTorchOptimizer(optimizer_f, loss_f, n_iter=100, solver=None)

Bases: InferenceMethod

Infer coefficients using provided loss functional and optimizer.

Parameters:

  • optimizer (function handle) –

    Pytorch optimizer handle have single parameter:

    (coefficients)

    where coefficients is of shape [batch_size, n_basis]

  • loss_f (function handle) –

    Must have parameters:

    (data, dictionary, coefficients)

    where data is of shape [batch_size, n_features] and loss_f must return tensor of size [batch_size,]

  • n_iter (int, default=100) – Number of iterations to run for an optimizer

  • solver (default=None)

infer(data, dictionary, coeff_0=None)

Infer coefficients for each image in data using dict elements dictionary by minimizing provided loss function with provided optimizer.

Parameters:

  • data (array-like, shape [batch_size, n_features]) – Data to be used in sparse coding

  • dictionary (array-like, shape [n_features, n_basis]) – Dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like, shape [batch_size, n_basis]

class sparsecoding.inference.Vanilla(n_iter=100, coeff_lr=0.001, sparsity_penalty=0.2, stop_early=False, epsilon=0.01, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Gradient descent with Euler’s method on model in Olshausen & Field (1997) with laplace prior over coefficients (corresponding to L1 norm penalty).

Parameters:

  • n_iter (int, default=100) – Number of iterations to run

  • coeff_lr (float, default=1e-3) – Update rate of coefficient dynamics

  • sparsity_penalty (float, default=0.2)

  • stop_early (bool, default=False) – Stops dynamics early based on change in coefficents

  • epsilon (float, default=1e-2) – Only used if stop_early True, specifies criteria to stop dynamics

  • return_all_coefficients (str, default=False) – Returns all coefficients during inference procedure if True User beware: If n_iter is large, setting this parameter to True Can result in large memory usage/potential exhaustion. This function typically used for debugging.

  • solver (default=None)

References

[1] Olshausen, B. A., & Field, D. J. (1997). Sparse coding with an overcomplete basis set: A strategy employed by V1?. Vision research, 37(23), 3311-3325.

grad(residual, dictionary, a)

Compute the gradient step on coefficients

Parameters:

  • residual (array-like, shape [batch_size, n_features]) – Residual between reconstructed image and original

  • dictionary (array-like, shape [n_features,n_coefficients]) – Dictionary

  • a (array-like, shape [batch_size, n_coefficients]) – Coefficients

Returns:

da – Gradient of membrane potentials

Return type:

array-like, shape [batch_size, n_coefficients]

infer(data, dictionary, coeff_0=None, use_checknan=False)

Infer coefficients using provided dictionary

Parameters:

  • dictionary (array-like, shape [n_features, n_basis]) – Dictionary

  • data (array like, shape [n_samples, n_features])

  • coeff_0 (array-like, shape [n_samples, n_basis], optional) – Initial coefficient values

  • use_checknan (bool, default=False) – check for nans in coefficients on each iteration. Setting this to False can speed up inference time

Returns:

coefficients – First case occurs if return_all_coefficients == “none”. If return_all_coefficients != “none”, returned shape is second case. Returned dimension along dim 1 can be less than n_iter when stop_early==True and stopping criteria met.

Return type:

array-like, shape [n_samples, n_basis] OR [n_samples, n_iter+1, n_basis]

3.3. Visualization tools

sparsecoding.visualization.plot_dictionary(dictionary, color=False, nrow=30, normalize=True, scale_each=True, fig=None, ax=None, title='', size=8)

Plot all elements of dictionary in grid

Parameters:

  • dictionary (array-like, shape [n_features, n_basis]) – Dictionary

  • color (bool, default=False) – Set True if dictionary 3 channel (color)

  • nrow (int, default=30) – Number of dictionary elements in a row

  • normalize (bool, default=True) – Normalize to [0,1] (see https://pytorch.org/vision/main/generated/torchvision.utils.make_grid.html)

  • scale_each (bool, default=True) – Scale each element to [0,1] (see https://pytorch.org/vision/main/generated/torchvision.utils.make_grid.html)

  • fig (matplotlib.pyplot figure handle, optional) – If not provided, new handle created and returned

  • ax (matplotlib.pyplot axes handle, optional) – If not provided, new handle created and returned

  • title (str, optional) – Title of plot

  • size (float, default=8) – Plot size (inches)

Returns:

  • fig (matplotlib.pyplot figure handle)

  • ax (matplotlib.pyplot axes handle)

sparsecoding.visualization.plot_patches(patches, color=False, normalize=True, scale_each=True, fig=None, ax=None, title='', size=8)
Parameters:

  • patches (array-like, shape [batch_size, n_pixels]) – Image patches

  • color (bool, default=False) – Set True if dictionary 3 channel (color)

  • nrow (int, default=30) – Number of dictionary elements in a row

  • normalize (bool, default=True) – Normalize to [0,1] (see https://pytorch.org/vision/main/generated/torchvision.utils.make_grid.html)

  • scale_each (bool, default=True) – Scale each element to [0,1] (see https://pytorch.org/vision/main/generated/torchvision.utils.make_grid.html)

  • fig (matplotlib.pyplot figure handle, optional) – If not provided, new handle created and returned

  • ax (matplotlib.pyplot axes handle, optional) – If not provided, new handle created and returned

  • title (str, optional) – Title of plot

  • size (float, default=8) – Plot size (inches)

Returns:

  • fig (matplotlib.pyplot figure handle)

  • ax (matplotlib.pyplot axes handle)

3.4. Priors

class sparsecoding.priors.L0Prior(prob_distr: Tensor)

Bases: Prior

Prior with a distribution over the l0-norm of the weights.

A class of priors where the weights are binary; the distribution is over the l0-norm of the weight vector (how many weights are active).

Parameters:

prob_distr (Tensor, shape [D], dtype float32) – Probability distribution over the l0-norm of the weights.

property D

Number of weights per sample.

sample(num_samples: int)

Sample weights from the prior.

Parameters:

num_samples (int, default=1) – Number of samples.

Returns:

samples – Sampled weights.

Return type:

Tensor, shape [num_samples, self.D]

class sparsecoding.priors.Prior

Bases: ABC

A distribution over weights.

Parameters:

weights_dim (int) – Number of weights for each sample.

abstract D()

Number of weights per sample.

abstract sample(num_samples: int = 1)

Sample weights from the prior.

Parameters:

num_samples (int, default=1) – Number of samples.

Returns:

samples – Sampled weights.

Return type:

Tensor, shape [num_samples, self.D]

class sparsecoding.priors.SpikeSlabPrior(dim: int, p_spike: float, scale: float, positive_only: bool = True)

Bases: Prior

Prior where weights are drawn from a “spike-and-slab” distribution.

The “spike” is at 0 and the “slab” is Laplacian.

See:

https://wesselb.github.io/assets/write-ups/Bruinsma,%20Spike%20and%20Slab%20Priors.pdf

for a good review of the spike-and-slab model.

Parameters:

  • dim (int) – Number of weights per sample.

  • p_spike (float) – The probability of the weight being 0.

  • scale (float) – The “scale” of the Laplacian distribution (larger is wider).

  • positive_only (bool) – Ensure that the weights are positive by taking the absolute value of weights sampled from the Laplacian.

property D

Number of weights per sample.

sample(num_samples: int)

Sample weights from the prior.

Parameters:

num_samples (int, default=1) – Number of samples.

Returns:

samples – Sampled weights.

Return type:

Tensor, shape [num_samples, self.D]

3.5. Datasets

class sparsecoding.datasets.BarsDataset(patch_size: int, dataset_size: int, prior: Prior)

Bases: Dataset

Toy dataset where the dictionary elements are horizontal and vertical bars.

Dataset elements are formed by taking linear combinations of the dictionary elements, where the weights are sampled according to the input Prior.

Parameters:

  • patch_size (int) – Side length for elements of the dataset.

  • dataset_size (int) – Number of dataset elements to generate.

  • prior (Prior) – Prior distribution on the weights. Should be sparse.

basis

Dictionary elements (horizontal and vertical bars).

Type:

Tensor, shape [2 * patch_size, patch_size, patch_size]

weights

Weights for each of the dataset elements.

Type:

Tensor, shape [dataset_size, 2 * patch_size]

data

Weighted linear combinations of the basis elements.

Type:

Tensor, shape [dataset_size, patch_size, patch_size]

class sparsecoding.datasets.FieldDataset(root: str, patch_size: int = 8, stride: int | None = None)

Bases: Dataset

Dataset used in Olshausen & Field (1996).

Paper:

https://courses.cs.washington.edu/courses/cse528/11sp/Olshausen-nature-paper.pdf Emergence of simple-cell receptive field properties by learning a sparse code for natural images.

Parameters:

  • root (str) – Location to download the dataset to.

  • patch_size (int) – Side length of patches for sparse dictionary learning.

  • stride (int, optional) – Stride for sampling patches. If not specified, set to patch_size (non-overlapping patches).

B = 10
C = 1
H = 512
W = 512

3.6. Data transformations

sparsecoding.transforms.whiten.compute_whitening_stats(X: Tensor)

Given a tensor of data, compute statistics for whitening transform.

Parameters:

X (torch.Tensor) – Input data of size [N, D]

Return type:

Dictionary containing whitening statistics (eigenvalues, eigenvectors, mean)

sparsecoding.transforms.whiten.whiten(X: Tensor, algorithm: str = 'zca', stats: Dict | None = None, n_components: float | None = None, epsilon: float = 0.0, return_W: bool = False) → Tensor

Apply whitening transform to data using pre-computed statistics.

Parameters:

  • X (torch.Tensor) – Input data of shape [N, D] where N are unique data elements of dimensionality D

  • algorithm (str, default="zca") – Whitening transform we want to apply, one of [‘zca’, ‘pca’, or ‘cholesky’]

  • stats (Dict, default=None) – Dict containing precomputed whitening statistics (mean, eigenvectors, eigenvalues)

  • n_components (float, int, default=None) – Number of principal components to keep. If None, keep all components. If int, keep that many components. If float between 0 and 1, keep components that explain that fraction of variance.

  • epsilon (float, default=0.0) – Optional small constant to prevent division by zero

Return type:

Whitened data of shape [N, D]

Notes

See examples/Data_Whitening.ipynb for usage examples, and brief discussion about the different whitening methods

See https://arxiv.org/abs/1512.00809 for extensive details on whitening transformations - Possible TODO: Also gives two additional transforms that have not been implemented

See https://stats.stackexchange.com/questions/117427/what-is-the-difference-between-zca-whitening-and-pca-whitening for details on PCA and ZCA in particular

3.7. Image transformations

class sparsecoding.transforms.images.WhiteningTransform(algorithm: str = 'zca', stats: Dict | None = None, compute_stats: bool = False, **kwargs)

Bases: object

A PyTorch transform for image whitening that can be used in a transform pipeline. Supports frequency, PCA, and ZCA whitening methods.

Parameters:

  • algorithm (str) – One of [‘frequency’, ‘pca’, ‘zca’, ‘cholesky]

  • stats (Dict or None, default=None) – Pre-computed statistics for PCA/ZCA whitening

  • compute_stats (bool, default=False) – If True, will compute stats on first batch seen

  • **kwargs – Additional arguments passed to whitening function

sparsecoding.transforms.images.check_images(images: Tensor, algorithm: str = 'zca')

Verify that tensor is in the shape [N, C, H, W] and C != when using fourier based method

sparsecoding.transforms.images.compute_image_whitening_stats(images: Tensor) → Dict

Wrapper for computing whitening stats of an image dataset

Parameters:

images (torch.Tensor) – Tensor of shape (N, C, H, W)

Return type:

Dictionary containing whitening statistics (eigenvalues, eigenvectors, mean)

sparsecoding.transforms.images.create_frequency_filter(image_size: int, f0_factor: float = 0.4) → Tensor

Create a frequency domain filter for image whitening.

Parameters:

  • image_size (int) – Size of the square image

  • f0_factor (float, default=0.4) – Factor for determining the cutoff frequency

Returns:

torch.Tensor

Return type:

Frequency domain filter

sparsecoding.transforms.images.frequency_whitening(images: Tensor, target_variance: float = 0.1, f0_factor: float = 0.4) → Tensor

Apply frequency domain decorrelation to batched images. Method used in original sparsenet in Olshausen and Field in Nature and http://www.rctn.org/bruno/sparsenet/

Parameters:

  • images (torch.Tensor) – Input images of shape (N, C, H, W)

  • target_variance (float, default=0.1) – Target variance for normalization

  • f0_factor (float, default = 0.4) – Factor for determining filter cutoff frequency

Returns:

torch.Tensor

Return type:

Whitened images

sparsecoding.transforms.images.get_cached_filter(image_size: int, f0_factor: float = 0.4) → Tensor

Get a cached frequency filter for the given image size.

Parameters:

  • image_size (int) – Size of the square image

  • f0_factor (float, default=0.4) – Factor for determining the cutoff frequency

Returns:

torch.Tensor

Return type:

Cached frequency domain filter

sparsecoding.transforms.images.normalize_variance(tensor: Tensor, target_variance: float = 1.0) → Tensor

Normalize the variance of a tensor to a target value.

Parameters:

  • tensor (torch.Tensor) – Input tensor

  • target_variance (float, default=1.0) – Desired variance after normalization

Returns:

torch.Tensor

Return type:

Normalized tensor

sparsecoding.transforms.images.patchify(patch_size: int, image: Tensor, stride: int | None = None)

Break an image into square patches.

Inverse of quilt().

Parameters:

  • patch_size (int) – Patch side length.

  • image (Tensor, shape [*, C, H, W]) – where: C is the number of channels, H is the image height, W is the image width.

  • stride (int, optional) – Stride between patches in pixel space. If not specified, set to patch_size (non-overlapping patches).

Returns:

patches – Non-overlapping patches taken from the input image, where: P is the patch size, N is the number of patches, equal to H//P * W//P, C is the number of channels of the input image.

Return type:

Tensor, shape [*, N, C, P, P]

sparsecoding.transforms.images.quilt(height: int, width: int, patches: Tensor)

Gather square patches into an image.

Inverse of patchify().

Parameters:

  • height (int) – Height for the reconstructed image.

  • width (int) – Width for the reconstructed image.

  • patches (Tensor, shape [*, N, C, P, P]) – Non-overlapping patches from an input image, where: P is the patch size, N is the number of patches, C is the number of channels in the image.

Returns:

image – Image reconstructed by stitching together input patches.

Return type:

Tensor, shape [*, C, height, width]

sparsecoding.transforms.images.sample_random_patches(patch_size: int, num_patches: int, image: Tensor)

Sample random patches from an image.

Parameters:

  • patch_size (int) – Patch side length.

  • num_patches (int) – Number of patches to sample.

  • image (Tensor, shape [*, C, H, W]) – where: C is the number of channels, H is the image height, W is the image width.

Returns:

patches – Sampled patches from the input image(s).

Return type:

Tensor, shape [num_patches, C, patch_size, patch_size]

sparsecoding.transforms.images.whiten_channel(channel: Tensor, filt: Tensor, target_variance: float = 1.0) → Tensor

Apply frequency domain whitening to a single channel.

Parameters:

  • channel (torch.Tensor) – Single channel image tensor

  • filt (torch.Tensor) – Frequency domain filter

  • target_variance (float, default=1.0) – Target variance for normalization

Returns:

torch.Tensor

Return type:

Whitened channel

sparsecoding.transforms.images.whiten_images(images: Tensor, algorithm: str, stats: Dict | None = None, **kwargs) → Tensor

Wrapper for all whitening transformations

Parameters:

  • images (torch.Tensor) – Tensor of shape (N, C, H, W)

  • algorithm (str) – What whitening transform we want to use

  • stats (Dict, default=None) – Dictionary of dataset statistics needed for whitening transformations

Return type:

Tensor of whitened data in shape (N, C, H, W)