sparsecoding package

class sparsecoding.BarsDataset(patch_size: int, dataset_size: int, prior: Prior)

Bases: Dataset

Toy dataset where the dictionary elements are horizontal and vertical bars.

Dataset elements are formed by taking linear combinations of the dictionary elements, where the weights are sampled according to the input Prior.

Parameters:

  • patch_size (int) – Side length for elements of the dataset.

  • dataset_size (int) – Number of dataset elements to generate.

  • prior (Prior) – Prior distribution on the weights. Should be sparse.

basis

Dictionary elements (horizontal and vertical bars).

Type:

Tensor, shape [2 * patch_size, patch_size, patch_size]

weights

Weights for each of the dataset elements.

Type:

Tensor, shape [dataset_size, 2 * patch_size]

data

Weighted linear combinations of the basis elements.

Type:

Tensor, shape [dataset_size, patch_size, patch_size]

class sparsecoding.FieldDataset(root: str, patch_size: int = 8, stride: int | None = None)

Bases: Dataset

Dataset used in Olshausen & Field (1996).

Paper:

https://courses.cs.washington.edu/courses/cse528/11sp/Olshausen-nature-paper.pdf Emergence of simple-cell receptive field properties by learning a sparse code for natural images.

Parameters:

  • root (str) – Location to download the dataset to.

  • patch_size (int) – Side length of patches for sparse dictionary learning.

  • stride (int, optional) – Stride for sampling patches. If not specified, set to patch_size (non-overlapping patches).

B = 10
C = 1
H = 512
W = 512
class sparsecoding.IHT(sparsity, n_iter=10, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Infer coefficients for each image in data using elements dictionary. Method description can be traced to “Iterative Hard Thresholding for Compressed Sensing” (T. Blumensath & M. E. Davies, 2009)

Parameters:

  • sparsity (float) – Sparsity of the solution. The number of active coefficients will be set to ceil(sparsity * data_dim) at the end of each iterative update.

  • n_iter (int, default=10) – number of iterations to run for an inference method

  • return_all_coefficients (str, default=False) – returns all coefficients during inference procedure if True user beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging

  • solver (default=None)

infer(data, dictionary)

Infer coefficients for each image in data using dict elements dictionary using Iterative Hard Thresholding (IHT)

Parameters:

  • data (array-like (batch_size, n_features)) – data to be used in sparse coding

  • dictionary (array-like, (n_features, n_basis)) – dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like (batch_size, n_basis)

class sparsecoding.ISTA(n_iter=100, sparsity_penalty=0.01, stop_early=False, epsilon=0.01, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Iterative shrinkage-thresholding algorithm for solving LASSO problems.

Parameters:

  • n_iter (int, default=100) – Number of iterations to run

  • sparsity_penalty (float, default=0.2)

  • stop_early (bool, default=False) – Stops dynamics early based on change in coefficents

  • epsilon (float, default=1e-2) – Only used if stop_early True, specifies criteria to stop dynamics

  • return_all_coefficients (str, default=False) – Returns all coefficients during inference procedure if True User beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging.

  • solver (default=None)

References

[1] Beck, A., & Teboulle, M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences, 2(1), 183-202.

infer(data, dictionary, coeff_0=None, use_checknan=False)

Infer coefficients for each image in data using dictionary elements. Uses ISTA (Beck & Taboulle 2009), equations 1.4 and 1.5.

Parameters:

  • data (array-like, shape [batch_size, n_features])

  • dictionary (array-like, shape [n_features, n_basis])

  • coeff_0 (array-like, shape [n_samples, n_basis], optional) – Initial coefficient values

  • use_checknan (bool, default=False) – Check for nans in coefficients on each iteration. Setting this to False can speed up inference time.

Returns:

coefficients – First case occurs if return_all_coefficients == “none”. If return_all_coefficients != “none”, returned shape is second case. Returned dimension along dim 1 can be less than n_iter when stop_early==True and stopping criteria met.

Return type:

array-like, shape [n_samples, n_basis] OR [n_samples, n_iter+1, n_basis]

threshold_nonlinearity(u)

Soft threshhold function

Parameters:

u (array-likes, shape [batch_size, n_basis]) – Membrane potentials

Returns:

a – activations

Return type:

array-like, shape [batch_size, n_basis]

class sparsecoding.L0Prior(prob_distr: Tensor)

Bases: Prior

Prior with a distribution over the l0-norm of the weights.

A class of priors where the weights are binary; the distribution is over the l0-norm of the weight vector (how many weights are active).

Parameters:

prob_distr (Tensor, shape [D], dtype float32) – Probability distribution over the l0-norm of the weights.

property D

Number of weights per sample.

sample(num_samples: int)

Sample weights from the prior.

Parameters:

num_samples (int, default=1) – Number of samples.

Returns:

samples – Sampled weights.

Return type:

Tensor, shape [num_samples, self.D]

class sparsecoding.LCA(n_iter=100, coeff_lr=0.001, threshold=0.1, stop_early=False, epsilon=0.01, solver=None, return_all_coefficients='none', nonnegative=False)

Bases: InferenceMethod

Method implemented according locally competative algorithm (LCA) with the ideal soft thresholding function.

Parameters:

  • n_iter (int, default=100) – Number of iterations to run

  • coeff_lr (float, default=1e-3) – Update rate of coefficient dynamics

  • threshold (float, default=0.1) – Threshold for non-linearity

  • stop_early (bool, default=False) – Stops dynamics early based on change in coefficents

  • epsilon (float, default=1e-2) – Only used if stop_early True, specifies criteria to stop dynamics

  • nonnegative (bool, default=False) – Constrain coefficients to be nonnegative

  • return_all_coefficients (str, {"none", "membrane", "active"}, default="none") – Returns all coefficients during inference procedure if not equal to “none”. If return_all_coefficients==”membrane”, membrane potentials (u) returned. If return_all_coefficients==”active”, active units (a) (output of thresholding function over u) returned. User beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging.

  • solver (default=None)

References

[1] Rozell, C. J., Johnson, D. H., Baraniuk, R. G., & Olshausen, B. A. (2008). Sparse coding via thresholding and local competition in neural circuits. Neural computation, 20(10), 2526-2563.

grad(b, G, u, a)

Compute the gradient step on membrane potentials

Parameters:

  • b (array-like, shape [batch_size, n_coefficients]) – Driver signal for coefficients

  • G (array-like, shape [n_coefficients, n_coefficients]) – Inhibition matrix

  • a (array-like, shape [batch_size, n_coefficients]) – Currently active coefficients

Returns:

du – Gradient of membrane potentials

Return type:

array-like, shape [batch_size, n_coefficients]

infer(data, dictionary, coeff_0=None, use_checknan=False)

Infer coefficients using provided dictionary

Parameters:

  • dictionary (array-like, shape [n_features, n_basis])

  • data (array-like, shape [n_samples, n_features])

  • coeff_0 (array-like, shape [n_samples, n_basis], optional) – Initial coefficient values

  • use_checknan (bool, default=False) – Check for nans in coefficients on each iteration. Setting this to False can speed up inference time.

Returns:

coefficients – First case occurs if return_all_coefficients == “none”. If return_all_coefficients != “none”, returned shape is second case. Returned dimension along dim 1 can be less than n_iter when stop_early==True and stopping criteria met.

Return type:

array-like, shape [n_samples, n_basis] OR [n_samples, n_iter+1, n_basis]

threshold_nonlinearity(u)

Soft threshhold function

Parameters:

u (array-like, shape [batch_size, n_basis]) – Membrane potentials

Returns:

a – Activations

Return type:

array-like, shape [batch_size, n_basis]

class sparsecoding.LSM(n_iter=100, n_iter_LSM=6, beta=0.01, alpha=80.0, sigma=0.005, sparse_threshold=0.01, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Infer latent coefficients generating data given dictionary. Method implemented according to “Group Sparse Coding with a Laplacian Scale Mixture Prior” (P. J. Garrigues & B. A. Olshausen, 2010)

Parameters:

  • n_iter (int, default=100) – Number of iterations to run for an optimizer

  • n_iter_LSM (int, default=6) – Number of iterations to run the outer loop of LSM

  • beta (float, default=0.01) – LSM parameter used to update lambdas

  • alpha (float, default=80.0) – LSM parameter used to update lambdas

  • sigma (float, default=0.005) – LSM parameter used to compute the loss function

  • sparse_threshold (float, default=10**-2) – Threshold used to discard smallest coefficients in the final solution SM parameter used to compute the loss function

  • return_all_coefficients (bool, default=False) – Returns all coefficients during inference procedure if True User beware: If n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging.

  • solver (default=None)

References

[1] Garrigues, P., & Olshausen, B. (2010). Group sparse coding with a laplacian scale mixture prior. Advances in neural information processing systems, 23.

infer(data, dictionary)

Infer coefficients for each image in data using dict elements dictionary using Laplacian Scale Mixture (LSM)

Parameters:

  • data (array-like, shape [batch_size, n_features]) – Data to be used in sparse coding

  • dictionary (array-like, shape [n_features, n_basis]) – Dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like, shape [batch_size, n_basis]

lsm_Loss(data, dictionary, coefficients, lambdas, sigma)

Compute LSM loss according to equation (7) in (P. J. Garrigues & B. A. Olshausen, 2010)

Parameters:

  • data (array-like, shape [batch_size, n_features]) – Data to be used in sparse coding

  • dictionary (array-like, shape [n_features, n_basis]) – Dictionary to be used

  • coefficients (array-like, shape [batch_size, n_basis]) – The current values of coefficients

  • lambdas (array-like, shape [batch_size, n_basis]) – The current values of regularization coefficient for all basis

  • sigma (float, default=0.005) – LSM parameter used to compute the loss functions

Returns:

loss – Loss values for each data sample

Return type:

array-like, shape [batch_size, 1]

class sparsecoding.MP(sparsity, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Infer coefficients for each image in data using elements dictionary. Method description can be traced to “Matching Pursuits with Time-Frequency Dictionaries” (S. G. Mallat & Z. Zhang, 1993)

Parameters:

  • sparsity (scalar (1,)) – sparsity of the solution

  • return_all_coefficients (string (1,) default=False) – returns all coefficients during inference procedure if True user beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging

  • solver (default=None)

infer(data, dictionary)

Infer coefficients for each image in data using dict elements dictionary using Matching Pursuit (MP)

Parameters:

  • data (array-like (batch_size, n_features)) – data to be used in sparse coding

  • dictionary (array-like, (n_features, n_basis)) – dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like (batch_size, n_basis)

class sparsecoding.OMP(sparsity, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Infer coefficients for each image in data using elements dictionary. Method description can be traced to:

“Orthogonal Matching Pursuit: Recursive Function Approximation with Application to Wavelet Decomposition” (Y. Pati & R. Rezaiifar & P. Krishnaprasad, 1993)

Parameters:

  • sparsity (scalar (1,)) – sparsity of the solution

  • return_all_coefficients (string (1,) default=False) – returns all coefficients during inference procedure if True user beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging

  • solver (default=None)

infer(data, dictionary)

Infer coefficients for each image in data using dict elements dictionary using Orthogonal Matching Pursuit (OMP)

Parameters:

  • data (array-like (batch_size, n_features)) – data to be used in sparse coding

  • dictionary (array-like, (n_features, n_basis)) – dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like (batch_size, n_basis)

class sparsecoding.PyTorchOptimizer(optimizer_f, loss_f, n_iter=100, solver=None)

Bases: InferenceMethod

Infer coefficients using provided loss functional and optimizer.

Parameters:

  • optimizer (function handle) –

    Pytorch optimizer handle have single parameter:

    (coefficients)

    where coefficients is of shape [batch_size, n_basis]

  • loss_f (function handle) –

    Must have parameters:

    (data, dictionary, coefficients)

    where data is of shape [batch_size, n_features] and loss_f must return tensor of size [batch_size,]

  • n_iter (int, default=100) – Number of iterations to run for an optimizer

  • solver (default=None)

infer(data, dictionary, coeff_0=None)

Infer coefficients for each image in data using dict elements dictionary by minimizing provided loss function with provided optimizer.

Parameters:

  • data (array-like, shape [batch_size, n_features]) – Data to be used in sparse coding

  • dictionary (array-like, shape [n_features, n_basis]) – Dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like, shape [batch_size, n_basis]

class sparsecoding.SparseCoding(inference_method, n_basis, n_features, sparsity_penalty=0.2, device=None, check_for_dictionary_nan=False, **kwargs)

Bases: Module

Class for learning a sparse code via dictionary learning

Parameters:

  • inference_method (sparsecoding.InferenceMethod) – Method for inferring coefficients for each image given the dictionary

  • n_basis (int) – Number of basis functions in dictionary

  • n_features (int) – Number of features in data

  • sparsity_penalty (float, default=0.2) – Sparsity penalty

  • dictionary_lr (float, default=1e-2) – Learning rate of dictionary update

  • device (torch.device, default=torch.device("cpu")) – Which device to utilize

  • check_for_dictionary_nan (bool, default=False) – Flag to check for nans in the dictionary after gradient updates and normalizations. Raises ValueError if nan found

checknan(data=tensor(0), name='data')

Check for nan values in dictinary, or data

Parameters:

  • data (array-like, optional) – Data to check for nans

  • name (str, optional) – Name to add to error, if one is thrown

Raises:

ValueError – If nan is found in data

compute_grad_dict(data, a)

Compute gradient of loss function w.r.t. dictionary elements

Parameters:

  • data (array-like, shape [batch_size, n_features]) – input data

  • a (array-like, shape [batch_size, n_basis]) – already-inferred coefficients

Returns:

dictionary_grad – gradient of dictionary

Return type:

array-like, shape [n_features, n_basis]

compute_loss(data, a)

Compute loss given data and inferred coefficients

Parameters:

  • data (array-like, shape [batch_size, n_features])

  • a (array-like, shape [batch_size, n_basis]) – inferred coefficients

Returns:

loss

Return type:

float

get_numpy_dictionary()

Returns dictionary as numpy array

Returns:

dictionary – numpy array dictionary

Return type:

array-like, shape [n_features,n_basis]

learn_dictionary(dataset, n_epoch, batch_size)

Learn dictionary for n_epoch epochs

Parameters:

  • dataset (torch.utils.data.Dataset or array-like, shape [n_samples, n_features]) – Input dataset

  • n_epoch (int) – Iumber of iterations to learn dictionary

  • batch_size (int) – Batch size to do dictionary updates over

Returns:

losses – Model losses (i.e., energy for the Boltzmann enthusiasts) after each dictionary update

Return type:

array-like, shape [nepoch,]

load_dictionary(filename)

Load dictionary from pkl dump

Parameters:

filename (str) – File to load dictionary from

normalize_dictionary()

Normalize columns of dictionary matrix to unit norm.

save_dictionary(filename)

Save dictionary to pkl dump

Parameters:

filename (str) – File to save current dictionary to

set_dictionary(dictionary)

Set model dictionary to passed dictionary

Parameters:

dictionary (array-like, shape [n_features, n_basis]) – Dictionary to set default dictionary to

update_dictionary(data, a)

Compute gradient of loss function w.r.t. dictionary elements, and update

Parameters:

  • data (array-like, shape [batch_size,n_features]) – Input data

  • a (array-like, shape [batch_size, n_basis]) – Already-inferred coefficients

class sparsecoding.SpikeSlabPrior(dim: int, p_spike: float, scale: float, positive_only: bool = True)

Bases: Prior

Prior where weights are drawn from a “spike-and-slab” distribution.

The “spike” is at 0 and the “slab” is Laplacian.

See:

https://wesselb.github.io/assets/write-ups/Bruinsma,%20Spike%20and%20Slab%20Priors.pdf

for a good review of the spike-and-slab model.

Parameters:

  • dim (int) – Number of weights per sample.

  • p_spike (float) – The probability of the weight being 0.

  • scale (float) – The “scale” of the Laplacian distribution (larger is wider).

  • positive_only (bool) – Ensure that the weights are positive by taking the absolute value of weights sampled from the Laplacian.

property D

Number of weights per sample.

sample(num_samples: int)

Sample weights from the prior.

Parameters:

num_samples (int, default=1) – Number of samples.

Returns:

samples – Sampled weights.

Return type:

Tensor, shape [num_samples, self.D]

class sparsecoding.Vanilla(n_iter=100, coeff_lr=0.001, sparsity_penalty=0.2, stop_early=False, epsilon=0.01, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Gradient descent with Euler’s method on model in Olshausen & Field (1997) with laplace prior over coefficients (corresponding to L1 norm penalty).

Parameters:

  • n_iter (int, default=100) – Number of iterations to run

  • coeff_lr (float, default=1e-3) – Update rate of coefficient dynamics

  • sparsity_penalty (float, default=0.2)

  • stop_early (bool, default=False) – Stops dynamics early based on change in coefficents

  • epsilon (float, default=1e-2) – Only used if stop_early True, specifies criteria to stop dynamics

  • return_all_coefficients (str, default=False) – Returns all coefficients during inference procedure if True User beware: If n_iter is large, setting this parameter to True Can result in large memory usage/potential exhaustion. This function typically used for debugging.

  • solver (default=None)

References

[1] Olshausen, B. A., & Field, D. J. (1997). Sparse coding with an overcomplete basis set: A strategy employed by V1?. Vision research, 37(23), 3311-3325.

grad(residual, dictionary, a)

Compute the gradient step on coefficients

Parameters:

  • residual (array-like, shape [batch_size, n_features]) – Residual between reconstructed image and original

  • dictionary (array-like, shape [n_features,n_coefficients]) – Dictionary

  • a (array-like, shape [batch_size, n_coefficients]) – Coefficients

Returns:

da – Gradient of membrane potentials

Return type:

array-like, shape [batch_size, n_coefficients]

infer(data, dictionary, coeff_0=None, use_checknan=False)

Infer coefficients using provided dictionary

Parameters:

  • dictionary (array-like, shape [n_features, n_basis]) – Dictionary

  • data (array like, shape [n_samples, n_features])

  • coeff_0 (array-like, shape [n_samples, n_basis], optional) – Initial coefficient values

  • use_checknan (bool, default=False) – check for nans in coefficients on each iteration. Setting this to False can speed up inference time

Returns:

coefficients – First case occurs if return_all_coefficients == “none”. If return_all_coefficients != “none”, returned shape is second case. Returned dimension along dim 1 can be less than n_iter when stop_early==True and stopping criteria met.

Return type:

array-like, shape [n_samples, n_basis] OR [n_samples, n_iter+1, n_basis]

sparsecoding.load_bars_dictionary()
sparsecoding.load_dictionary_from_pickle(path)
sparsecoding.load_olshausen_dictionary()
sparsecoding.plot_dictionary(dictionary, color=False, nrow=30, normalize=True, scale_each=True, fig=None, ax=None, title='', size=8)

Plot all elements of dictionary in grid

Parameters:

  • dictionary (array-like, shape [n_features, n_basis]) – Dictionary

  • color (bool, default=False) – Set True if dictionary 3 channel (color)

  • nrow (int, default=30) – Number of dictionary elements in a row

  • normalize (bool, default=True) – Normalize to [0,1] (see https://pytorch.org/vision/main/generated/torchvision.utils.make_grid.html)

  • scale_each (bool, default=True) – Scale each element to [0,1] (see https://pytorch.org/vision/main/generated/torchvision.utils.make_grid.html)

  • fig (matplotlib.pyplot figure handle, optional) – If not provided, new handle created and returned

  • ax (matplotlib.pyplot axes handle, optional) – If not provided, new handle created and returned

  • title (str, optional) – Title of plot

  • size (float, default=8) – Plot size (inches)

Returns:

  • fig (matplotlib.pyplot figure handle)

  • ax (matplotlib.pyplot axes handle)

sparsecoding.plot_patches(patches, color=False, normalize=True, scale_each=True, fig=None, ax=None, title='', size=8)
Parameters:

  • patches (array-like, shape [batch_size, n_pixels]) – Image patches

  • color (bool, default=False) – Set True if dictionary 3 channel (color)

  • nrow (int, default=30) – Number of dictionary elements in a row

  • normalize (bool, default=True) – Normalize to [0,1] (see https://pytorch.org/vision/main/generated/torchvision.utils.make_grid.html)

  • scale_each (bool, default=True) – Scale each element to [0,1] (see https://pytorch.org/vision/main/generated/torchvision.utils.make_grid.html)

  • fig (matplotlib.pyplot figure handle, optional) – If not provided, new handle created and returned

  • ax (matplotlib.pyplot axes handle, optional) – If not provided, new handle created and returned

  • title (str, optional) – Title of plot

  • size (float, default=8) – Plot size (inches)

Returns:

  • fig (matplotlib.pyplot figure handle)

  • ax (matplotlib.pyplot axes handle)

Subpackages

Submodules

sparsecoding.datasets module

class sparsecoding.datasets.BarsDataset(patch_size: int, dataset_size: int, prior: Prior)

Bases: Dataset

Toy dataset where the dictionary elements are horizontal and vertical bars.

Dataset elements are formed by taking linear combinations of the dictionary elements, where the weights are sampled according to the input Prior.

Parameters:

  • patch_size (int) – Side length for elements of the dataset.

  • dataset_size (int) – Number of dataset elements to generate.

  • prior (Prior) – Prior distribution on the weights. Should be sparse.

basis

Dictionary elements (horizontal and vertical bars).

Type:

Tensor, shape [2 * patch_size, patch_size, patch_size]

weights

Weights for each of the dataset elements.

Type:

Tensor, shape [dataset_size, 2 * patch_size]

data

Weighted linear combinations of the basis elements.

Type:

Tensor, shape [dataset_size, patch_size, patch_size]

class sparsecoding.datasets.FieldDataset(root: str, patch_size: int = 8, stride: int | None = None)

Bases: Dataset

Dataset used in Olshausen & Field (1996).

Paper:

https://courses.cs.washington.edu/courses/cse528/11sp/Olshausen-nature-paper.pdf Emergence of simple-cell receptive field properties by learning a sparse code for natural images.

Parameters:

  • root (str) – Location to download the dataset to.

  • patch_size (int) – Side length of patches for sparse dictionary learning.

  • stride (int, optional) – Stride for sampling patches. If not specified, set to patch_size (non-overlapping patches).

B = 10
C = 1
H = 512
W = 512

sparsecoding.dictionaries module

sparsecoding.dictionaries.load_bars_dictionary()
sparsecoding.dictionaries.load_dictionary_from_pickle(path)
sparsecoding.dictionaries.load_olshausen_dictionary()

sparsecoding.models module

class sparsecoding.models.SparseCoding(inference_method, n_basis, n_features, sparsity_penalty=0.2, device=None, check_for_dictionary_nan=False, **kwargs)

Bases: Module

Class for learning a sparse code via dictionary learning

Parameters:

  • inference_method (sparsecoding.InferenceMethod) – Method for inferring coefficients for each image given the dictionary

  • n_basis (int) – Number of basis functions in dictionary

  • n_features (int) – Number of features in data

  • sparsity_penalty (float, default=0.2) – Sparsity penalty

  • dictionary_lr (float, default=1e-2) – Learning rate of dictionary update

  • device (torch.device, default=torch.device("cpu")) – Which device to utilize

  • check_for_dictionary_nan (bool, default=False) – Flag to check for nans in the dictionary after gradient updates and normalizations. Raises ValueError if nan found

checknan(data=tensor(0), name='data')

Check for nan values in dictinary, or data

Parameters:

  • data (array-like, optional) – Data to check for nans

  • name (str, optional) – Name to add to error, if one is thrown

Raises:

ValueError – If nan is found in data

compute_grad_dict(data, a)

Compute gradient of loss function w.r.t. dictionary elements

Parameters:

  • data (array-like, shape [batch_size, n_features]) – input data

  • a (array-like, shape [batch_size, n_basis]) – already-inferred coefficients

Returns:

dictionary_grad – gradient of dictionary

Return type:

array-like, shape [n_features, n_basis]

compute_loss(data, a)

Compute loss given data and inferred coefficients

Parameters:

  • data (array-like, shape [batch_size, n_features])

  • a (array-like, shape [batch_size, n_basis]) – inferred coefficients

Returns:

loss

Return type:

float

get_numpy_dictionary()

Returns dictionary as numpy array

Returns:

dictionary – numpy array dictionary

Return type:

array-like, shape [n_features,n_basis]

learn_dictionary(dataset, n_epoch, batch_size)

Learn dictionary for n_epoch epochs

Parameters:

  • dataset (torch.utils.data.Dataset or array-like, shape [n_samples, n_features]) – Input dataset

  • n_epoch (int) – Iumber of iterations to learn dictionary

  • batch_size (int) – Batch size to do dictionary updates over

Returns:

losses – Model losses (i.e., energy for the Boltzmann enthusiasts) after each dictionary update

Return type:

array-like, shape [nepoch,]

load_dictionary(filename)

Load dictionary from pkl dump

Parameters:

filename (str) – File to load dictionary from

normalize_dictionary()

Normalize columns of dictionary matrix to unit norm.

save_dictionary(filename)

Save dictionary to pkl dump

Parameters:

filename (str) – File to save current dictionary to

set_dictionary(dictionary)

Set model dictionary to passed dictionary

Parameters:

dictionary (array-like, shape [n_features, n_basis]) – Dictionary to set default dictionary to

update_dictionary(data, a)

Compute gradient of loss function w.r.t. dictionary elements, and update

Parameters:

  • data (array-like, shape [batch_size,n_features]) – Input data

  • a (array-like, shape [batch_size, n_basis]) – Already-inferred coefficients

sparsecoding.visualization module

sparsecoding.visualization.plot_dictionary(dictionary, color=False, nrow=30, normalize=True, scale_each=True, fig=None, ax=None, title='', size=8)

Plot all elements of dictionary in grid

Parameters:

  • dictionary (array-like, shape [n_features, n_basis]) – Dictionary

  • color (bool, default=False) – Set True if dictionary 3 channel (color)

  • nrow (int, default=30) – Number of dictionary elements in a row

  • normalize (bool, default=True) – Normalize to [0,1] (see https://pytorch.org/vision/main/generated/torchvision.utils.make_grid.html)

  • scale_each (bool, default=True) – Scale each element to [0,1] (see https://pytorch.org/vision/main/generated/torchvision.utils.make_grid.html)

  • fig (matplotlib.pyplot figure handle, optional) – If not provided, new handle created and returned

  • ax (matplotlib.pyplot axes handle, optional) – If not provided, new handle created and returned

  • title (str, optional) – Title of plot

  • size (float, default=8) – Plot size (inches)

Returns:

  • fig (matplotlib.pyplot figure handle)

  • ax (matplotlib.pyplot axes handle)

sparsecoding.visualization.plot_patches(patches, color=False, normalize=True, scale_each=True, fig=None, ax=None, title='', size=8)
Parameters:

  • patches (array-like, shape [batch_size, n_pixels]) – Image patches

  • color (bool, default=False) – Set True if dictionary 3 channel (color)

  • nrow (int, default=30) – Number of dictionary elements in a row

  • normalize (bool, default=True) – Normalize to [0,1] (see https://pytorch.org/vision/main/generated/torchvision.utils.make_grid.html)

  • scale_each (bool, default=True) – Scale each element to [0,1] (see https://pytorch.org/vision/main/generated/torchvision.utils.make_grid.html)

  • fig (matplotlib.pyplot figure handle, optional) – If not provided, new handle created and returned

  • ax (matplotlib.pyplot axes handle, optional) – If not provided, new handle created and returned

  • title (str, optional) – Title of plot

  • size (float, default=8) – Plot size (inches)

Returns:

  • fig (matplotlib.pyplot figure handle)

  • ax (matplotlib.pyplot axes handle)