sparsecoding.inference package

class sparsecoding.inference.IHT(sparsity, n_iter=10, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Infer coefficients for each image in data using elements dictionary. Method description can be traced to “Iterative Hard Thresholding for Compressed Sensing” (T. Blumensath & M. E. Davies, 2009)

Parameters:

  • sparsity (float) – Sparsity of the solution. The number of active coefficients will be set to ceil(sparsity * data_dim) at the end of each iterative update.

  • n_iter (int, default=10) – number of iterations to run for an inference method

  • return_all_coefficients (str, default=False) – returns all coefficients during inference procedure if True user beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging

  • solver (default=None)

infer(data, dictionary)

Infer coefficients for each image in data using dict elements dictionary using Iterative Hard Thresholding (IHT)

Parameters:

  • data (array-like (batch_size, n_features)) – data to be used in sparse coding

  • dictionary (array-like, (n_features, n_basis)) – dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like (batch_size, n_basis)

class sparsecoding.inference.ISTA(n_iter=100, sparsity_penalty=0.01, stop_early=False, epsilon=0.01, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Iterative shrinkage-thresholding algorithm for solving LASSO problems.

Parameters:

  • n_iter (int, default=100) – Number of iterations to run

  • sparsity_penalty (float, default=0.2)

  • stop_early (bool, default=False) – Stops dynamics early based on change in coefficents

  • epsilon (float, default=1e-2) – Only used if stop_early True, specifies criteria to stop dynamics

  • return_all_coefficients (str, default=False) – Returns all coefficients during inference procedure if True User beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging.

  • solver (default=None)

References

[1] Beck, A., & Teboulle, M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences, 2(1), 183-202.

infer(data, dictionary, coeff_0=None, use_checknan=False)

Infer coefficients for each image in data using dictionary elements. Uses ISTA (Beck & Taboulle 2009), equations 1.4 and 1.5.

Parameters:

  • data (array-like, shape [batch_size, n_features])

  • dictionary (array-like, shape [n_features, n_basis])

  • coeff_0 (array-like, shape [n_samples, n_basis], optional) – Initial coefficient values

  • use_checknan (bool, default=False) – Check for nans in coefficients on each iteration. Setting this to False can speed up inference time.

Returns:

coefficients – First case occurs if return_all_coefficients == “none”. If return_all_coefficients != “none”, returned shape is second case. Returned dimension along dim 1 can be less than n_iter when stop_early==True and stopping criteria met.

Return type:

array-like, shape [n_samples, n_basis] OR [n_samples, n_iter+1, n_basis]

threshold_nonlinearity(u)

Soft threshhold function

Parameters:

u (array-likes, shape [batch_size, n_basis]) – Membrane potentials

Returns:

a – activations

Return type:

array-like, shape [batch_size, n_basis]

class sparsecoding.inference.InferenceMethod(solver)

Bases: object

Base class for inference method.

static checknan(data=tensor(0), name='data')

Check for nan values in data.

Parameters:

  • data (array-like, optional) – Data to check for nans

  • name (str, default="data") – Name to add to error, if one is thrown

Raises:

ValueError – If the nan found in data

grad()

Compute the gradient step.

infer(dictionary, data, coeff_0=None, use_checknan=False)

Infer the coefficients given a dataset and dictionary.

Parameters:

  • dictionary (array-like, shape [n_features,n_basis])

  • data (array-like, shape [n_samples,n_features])

  • coeff_0 (array-like, shape [n_samples,n_basis], optional) – Initial coefficient values.

  • use_checknan (bool, default=False) – Check for nans in coefficients on each iteration

Returns:

coefficients

Return type:

array-like, shape [n_samples,n_basis]

initialize(a)

Define initial coefficients.

class sparsecoding.inference.LCA(n_iter=100, coeff_lr=0.001, threshold=0.1, stop_early=False, epsilon=0.01, solver=None, return_all_coefficients='none', nonnegative=False)

Bases: InferenceMethod

Method implemented according locally competative algorithm (LCA) with the ideal soft thresholding function.

Parameters:

  • n_iter (int, default=100) – Number of iterations to run

  • coeff_lr (float, default=1e-3) – Update rate of coefficient dynamics

  • threshold (float, default=0.1) – Threshold for non-linearity

  • stop_early (bool, default=False) – Stops dynamics early based on change in coefficents

  • epsilon (float, default=1e-2) – Only used if stop_early True, specifies criteria to stop dynamics

  • nonnegative (bool, default=False) – Constrain coefficients to be nonnegative

  • return_all_coefficients (str, {"none", "membrane", "active"}, default="none") – Returns all coefficients during inference procedure if not equal to “none”. If return_all_coefficients==”membrane”, membrane potentials (u) returned. If return_all_coefficients==”active”, active units (a) (output of thresholding function over u) returned. User beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging.

  • solver (default=None)

References

[1] Rozell, C. J., Johnson, D. H., Baraniuk, R. G., & Olshausen, B. A. (2008). Sparse coding via thresholding and local competition in neural circuits. Neural computation, 20(10), 2526-2563.

grad(b, G, u, a)

Compute the gradient step on membrane potentials

Parameters:

  • b (array-like, shape [batch_size, n_coefficients]) – Driver signal for coefficients

  • G (array-like, shape [n_coefficients, n_coefficients]) – Inhibition matrix

  • a (array-like, shape [batch_size, n_coefficients]) – Currently active coefficients

Returns:

du – Gradient of membrane potentials

Return type:

array-like, shape [batch_size, n_coefficients]

infer(data, dictionary, coeff_0=None, use_checknan=False)

Infer coefficients using provided dictionary

Parameters:

  • dictionary (array-like, shape [n_features, n_basis])

  • data (array-like, shape [n_samples, n_features])

  • coeff_0 (array-like, shape [n_samples, n_basis], optional) – Initial coefficient values

  • use_checknan (bool, default=False) – Check for nans in coefficients on each iteration. Setting this to False can speed up inference time.

Returns:

coefficients – First case occurs if return_all_coefficients == “none”. If return_all_coefficients != “none”, returned shape is second case. Returned dimension along dim 1 can be less than n_iter when stop_early==True and stopping criteria met.

Return type:

array-like, shape [n_samples, n_basis] OR [n_samples, n_iter+1, n_basis]

threshold_nonlinearity(u)

Soft threshhold function

Parameters:

u (array-like, shape [batch_size, n_basis]) – Membrane potentials

Returns:

a – Activations

Return type:

array-like, shape [batch_size, n_basis]

class sparsecoding.inference.LSM(n_iter=100, n_iter_LSM=6, beta=0.01, alpha=80.0, sigma=0.005, sparse_threshold=0.01, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Infer latent coefficients generating data given dictionary. Method implemented according to “Group Sparse Coding with a Laplacian Scale Mixture Prior” (P. J. Garrigues & B. A. Olshausen, 2010)

Parameters:

  • n_iter (int, default=100) – Number of iterations to run for an optimizer

  • n_iter_LSM (int, default=6) – Number of iterations to run the outer loop of LSM

  • beta (float, default=0.01) – LSM parameter used to update lambdas

  • alpha (float, default=80.0) – LSM parameter used to update lambdas

  • sigma (float, default=0.005) – LSM parameter used to compute the loss function

  • sparse_threshold (float, default=10**-2) – Threshold used to discard smallest coefficients in the final solution SM parameter used to compute the loss function

  • return_all_coefficients (bool, default=False) – Returns all coefficients during inference procedure if True User beware: If n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging.

  • solver (default=None)

References

[1] Garrigues, P., & Olshausen, B. (2010). Group sparse coding with a laplacian scale mixture prior. Advances in neural information processing systems, 23.

infer(data, dictionary)

Infer coefficients for each image in data using dict elements dictionary using Laplacian Scale Mixture (LSM)

Parameters:

  • data (array-like, shape [batch_size, n_features]) – Data to be used in sparse coding

  • dictionary (array-like, shape [n_features, n_basis]) – Dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like, shape [batch_size, n_basis]

lsm_Loss(data, dictionary, coefficients, lambdas, sigma)

Compute LSM loss according to equation (7) in (P. J. Garrigues & B. A. Olshausen, 2010)

Parameters:

  • data (array-like, shape [batch_size, n_features]) – Data to be used in sparse coding

  • dictionary (array-like, shape [n_features, n_basis]) – Dictionary to be used

  • coefficients (array-like, shape [batch_size, n_basis]) – The current values of coefficients

  • lambdas (array-like, shape [batch_size, n_basis]) – The current values of regularization coefficient for all basis

  • sigma (float, default=0.005) – LSM parameter used to compute the loss functions

Returns:

loss – Loss values for each data sample

Return type:

array-like, shape [batch_size, 1]

class sparsecoding.inference.MP(sparsity, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Infer coefficients for each image in data using elements dictionary. Method description can be traced to “Matching Pursuits with Time-Frequency Dictionaries” (S. G. Mallat & Z. Zhang, 1993)

Parameters:

  • sparsity (scalar (1,)) – sparsity of the solution

  • return_all_coefficients (string (1,) default=False) – returns all coefficients during inference procedure if True user beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging

  • solver (default=None)

infer(data, dictionary)

Infer coefficients for each image in data using dict elements dictionary using Matching Pursuit (MP)

Parameters:

  • data (array-like (batch_size, n_features)) – data to be used in sparse coding

  • dictionary (array-like, (n_features, n_basis)) – dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like (batch_size, n_basis)

class sparsecoding.inference.OMP(sparsity, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Infer coefficients for each image in data using elements dictionary. Method description can be traced to:

“Orthogonal Matching Pursuit: Recursive Function Approximation with Application to Wavelet Decomposition” (Y. Pati & R. Rezaiifar & P. Krishnaprasad, 1993)

Parameters:

  • sparsity (scalar (1,)) – sparsity of the solution

  • return_all_coefficients (string (1,) default=False) – returns all coefficients during inference procedure if True user beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging

  • solver (default=None)

infer(data, dictionary)

Infer coefficients for each image in data using dict elements dictionary using Orthogonal Matching Pursuit (OMP)

Parameters:

  • data (array-like (batch_size, n_features)) – data to be used in sparse coding

  • dictionary (array-like, (n_features, n_basis)) – dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like (batch_size, n_basis)

class sparsecoding.inference.PyTorchOptimizer(optimizer_f, loss_f, n_iter=100, solver=None)

Bases: InferenceMethod

Infer coefficients using provided loss functional and optimizer.

Parameters:

  • optimizer (function handle) –

    Pytorch optimizer handle have single parameter:

    (coefficients)

    where coefficients is of shape [batch_size, n_basis]

  • loss_f (function handle) –

    Must have parameters:

    (data, dictionary, coefficients)

    where data is of shape [batch_size, n_features] and loss_f must return tensor of size [batch_size,]

  • n_iter (int, default=100) – Number of iterations to run for an optimizer

  • solver (default=None)

infer(data, dictionary, coeff_0=None)

Infer coefficients for each image in data using dict elements dictionary by minimizing provided loss function with provided optimizer.

Parameters:

  • data (array-like, shape [batch_size, n_features]) – Data to be used in sparse coding

  • dictionary (array-like, shape [n_features, n_basis]) – Dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like, shape [batch_size, n_basis]

class sparsecoding.inference.Vanilla(n_iter=100, coeff_lr=0.001, sparsity_penalty=0.2, stop_early=False, epsilon=0.01, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Gradient descent with Euler’s method on model in Olshausen & Field (1997) with laplace prior over coefficients (corresponding to L1 norm penalty).

Parameters:

  • n_iter (int, default=100) – Number of iterations to run

  • coeff_lr (float, default=1e-3) – Update rate of coefficient dynamics

  • sparsity_penalty (float, default=0.2)

  • stop_early (bool, default=False) – Stops dynamics early based on change in coefficents

  • epsilon (float, default=1e-2) – Only used if stop_early True, specifies criteria to stop dynamics

  • return_all_coefficients (str, default=False) – Returns all coefficients during inference procedure if True User beware: If n_iter is large, setting this parameter to True Can result in large memory usage/potential exhaustion. This function typically used for debugging.

  • solver (default=None)

References

[1] Olshausen, B. A., & Field, D. J. (1997). Sparse coding with an overcomplete basis set: A strategy employed by V1?. Vision research, 37(23), 3311-3325.

grad(residual, dictionary, a)

Compute the gradient step on coefficients

Parameters:

  • residual (array-like, shape [batch_size, n_features]) – Residual between reconstructed image and original

  • dictionary (array-like, shape [n_features,n_coefficients]) – Dictionary

  • a (array-like, shape [batch_size, n_coefficients]) – Coefficients

Returns:

da – Gradient of membrane potentials

Return type:

array-like, shape [batch_size, n_coefficients]

infer(data, dictionary, coeff_0=None, use_checknan=False)

Infer coefficients using provided dictionary

Parameters:

  • dictionary (array-like, shape [n_features, n_basis]) – Dictionary

  • data (array like, shape [n_samples, n_features])

  • coeff_0 (array-like, shape [n_samples, n_basis], optional) – Initial coefficient values

  • use_checknan (bool, default=False) – check for nans in coefficients on each iteration. Setting this to False can speed up inference time

Returns:

coefficients – First case occurs if return_all_coefficients == “none”. If return_all_coefficients != “none”, returned shape is second case. Returned dimension along dim 1 can be less than n_iter when stop_early==True and stopping criteria met.

Return type:

array-like, shape [n_samples, n_basis] OR [n_samples, n_iter+1, n_basis]

Submodules

sparsecoding.inference.iht module

class sparsecoding.inference.iht.IHT(sparsity, n_iter=10, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Infer coefficients for each image in data using elements dictionary. Method description can be traced to “Iterative Hard Thresholding for Compressed Sensing” (T. Blumensath & M. E. Davies, 2009)

Parameters:

  • sparsity (float) – Sparsity of the solution. The number of active coefficients will be set to ceil(sparsity * data_dim) at the end of each iterative update.

  • n_iter (int, default=10) – number of iterations to run for an inference method

  • return_all_coefficients (str, default=False) – returns all coefficients during inference procedure if True user beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging

  • solver (default=None)

infer(data, dictionary)

Infer coefficients for each image in data using dict elements dictionary using Iterative Hard Thresholding (IHT)

Parameters:

  • data (array-like (batch_size, n_features)) – data to be used in sparse coding

  • dictionary (array-like, (n_features, n_basis)) – dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like (batch_size, n_basis)

sparsecoding.inference.inference_method module

class sparsecoding.inference.inference_method.InferenceMethod(solver)

Bases: object

Base class for inference method.

static checknan(data=tensor(0), name='data')

Check for nan values in data.

Parameters:

  • data (array-like, optional) – Data to check for nans

  • name (str, default="data") – Name to add to error, if one is thrown

Raises:

ValueError – If the nan found in data

grad()

Compute the gradient step.

infer(dictionary, data, coeff_0=None, use_checknan=False)

Infer the coefficients given a dataset and dictionary.

Parameters:

  • dictionary (array-like, shape [n_features,n_basis])

  • data (array-like, shape [n_samples,n_features])

  • coeff_0 (array-like, shape [n_samples,n_basis], optional) – Initial coefficient values.

  • use_checknan (bool, default=False) – Check for nans in coefficients on each iteration

Returns:

coefficients

Return type:

array-like, shape [n_samples,n_basis]

initialize(a)

Define initial coefficients.

sparsecoding.inference.ista module

class sparsecoding.inference.ista.ISTA(n_iter=100, sparsity_penalty=0.01, stop_early=False, epsilon=0.01, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Iterative shrinkage-thresholding algorithm for solving LASSO problems.

Parameters:

  • n_iter (int, default=100) – Number of iterations to run

  • sparsity_penalty (float, default=0.2)

  • stop_early (bool, default=False) – Stops dynamics early based on change in coefficents

  • epsilon (float, default=1e-2) – Only used if stop_early True, specifies criteria to stop dynamics

  • return_all_coefficients (str, default=False) – Returns all coefficients during inference procedure if True User beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging.

  • solver (default=None)

References

[1] Beck, A., & Teboulle, M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences, 2(1), 183-202.

infer(data, dictionary, coeff_0=None, use_checknan=False)

Infer coefficients for each image in data using dictionary elements. Uses ISTA (Beck & Taboulle 2009), equations 1.4 and 1.5.

Parameters:

  • data (array-like, shape [batch_size, n_features])

  • dictionary (array-like, shape [n_features, n_basis])

  • coeff_0 (array-like, shape [n_samples, n_basis], optional) – Initial coefficient values

  • use_checknan (bool, default=False) – Check for nans in coefficients on each iteration. Setting this to False can speed up inference time.

Returns:

coefficients – First case occurs if return_all_coefficients == “none”. If return_all_coefficients != “none”, returned shape is second case. Returned dimension along dim 1 can be less than n_iter when stop_early==True and stopping criteria met.

Return type:

array-like, shape [n_samples, n_basis] OR [n_samples, n_iter+1, n_basis]

threshold_nonlinearity(u)

Soft threshhold function

Parameters:

u (array-likes, shape [batch_size, n_basis]) – Membrane potentials

Returns:

a – activations

Return type:

array-like, shape [batch_size, n_basis]

sparsecoding.inference.ista_test module

sparsecoding.inference.ista_test.test_inference(bars_dictionary_fixture: Tensor, bars_datas_fixture: list[Tensor], bars_datasets_fixture: list[BarsDataset])

Test that ISTA inference recovers the correct weights.

sparsecoding.inference.ista_test.test_shape(patch_size_fixture: int, dataset_size_fixture: int, bars_dictionary_fixture: Tensor, bars_datas_fixture: list[Tensor], bars_datasets_fixture: list[BarsDataset])

Test that ISTA inference returns expected shapes.

sparsecoding.inference.lca module

class sparsecoding.inference.lca.LCA(n_iter=100, coeff_lr=0.001, threshold=0.1, stop_early=False, epsilon=0.01, solver=None, return_all_coefficients='none', nonnegative=False)

Bases: InferenceMethod

Method implemented according locally competative algorithm (LCA) with the ideal soft thresholding function.

Parameters:

  • n_iter (int, default=100) – Number of iterations to run

  • coeff_lr (float, default=1e-3) – Update rate of coefficient dynamics

  • threshold (float, default=0.1) – Threshold for non-linearity

  • stop_early (bool, default=False) – Stops dynamics early based on change in coefficents

  • epsilon (float, default=1e-2) – Only used if stop_early True, specifies criteria to stop dynamics

  • nonnegative (bool, default=False) – Constrain coefficients to be nonnegative

  • return_all_coefficients (str, {"none", "membrane", "active"}, default="none") – Returns all coefficients during inference procedure if not equal to “none”. If return_all_coefficients==”membrane”, membrane potentials (u) returned. If return_all_coefficients==”active”, active units (a) (output of thresholding function over u) returned. User beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging.

  • solver (default=None)

References

[1] Rozell, C. J., Johnson, D. H., Baraniuk, R. G., & Olshausen, B. A. (2008). Sparse coding via thresholding and local competition in neural circuits. Neural computation, 20(10), 2526-2563.

grad(b, G, u, a)

Compute the gradient step on membrane potentials

Parameters:

  • b (array-like, shape [batch_size, n_coefficients]) – Driver signal for coefficients

  • G (array-like, shape [n_coefficients, n_coefficients]) – Inhibition matrix

  • a (array-like, shape [batch_size, n_coefficients]) – Currently active coefficients

Returns:

du – Gradient of membrane potentials

Return type:

array-like, shape [batch_size, n_coefficients]

infer(data, dictionary, coeff_0=None, use_checknan=False)

Infer coefficients using provided dictionary

Parameters:

  • dictionary (array-like, shape [n_features, n_basis])

  • data (array-like, shape [n_samples, n_features])

  • coeff_0 (array-like, shape [n_samples, n_basis], optional) – Initial coefficient values

  • use_checknan (bool, default=False) – Check for nans in coefficients on each iteration. Setting this to False can speed up inference time.

Returns:

coefficients – First case occurs if return_all_coefficients == “none”. If return_all_coefficients != “none”, returned shape is second case. Returned dimension along dim 1 can be less than n_iter when stop_early==True and stopping criteria met.

Return type:

array-like, shape [n_samples, n_basis] OR [n_samples, n_iter+1, n_basis]

threshold_nonlinearity(u)

Soft threshhold function

Parameters:

u (array-like, shape [batch_size, n_basis]) – Membrane potentials

Returns:

a – Activations

Return type:

array-like, shape [batch_size, n_basis]

sparsecoding.inference.lca_test module

sparsecoding.inference.lca_test.test_inference(bars_dictionary_fixture: Tensor, bars_datas_fixture: list[Tensor], bars_datasets_fixture: list[BarsDataset])

Test that LCA inference recovers the correct weights.

sparsecoding.inference.lca_test.test_shape(patch_size_fixture: int, dataset_size_fixture: int, bars_dictionary_fixture: Tensor, bars_datas_fixture: list[Tensor], bars_datasets_fixture: list[BarsDataset])

Test that LCA inference returns expected shapes.

sparsecoding.inference.lsm module

class sparsecoding.inference.lsm.LSM(n_iter=100, n_iter_LSM=6, beta=0.01, alpha=80.0, sigma=0.005, sparse_threshold=0.01, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Infer latent coefficients generating data given dictionary. Method implemented according to “Group Sparse Coding with a Laplacian Scale Mixture Prior” (P. J. Garrigues & B. A. Olshausen, 2010)

Parameters:

  • n_iter (int, default=100) – Number of iterations to run for an optimizer

  • n_iter_LSM (int, default=6) – Number of iterations to run the outer loop of LSM

  • beta (float, default=0.01) – LSM parameter used to update lambdas

  • alpha (float, default=80.0) – LSM parameter used to update lambdas

  • sigma (float, default=0.005) – LSM parameter used to compute the loss function

  • sparse_threshold (float, default=10**-2) – Threshold used to discard smallest coefficients in the final solution SM parameter used to compute the loss function

  • return_all_coefficients (bool, default=False) – Returns all coefficients during inference procedure if True User beware: If n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging.

  • solver (default=None)

References

[1] Garrigues, P., & Olshausen, B. (2010). Group sparse coding with a laplacian scale mixture prior. Advances in neural information processing systems, 23.

infer(data, dictionary)

Infer coefficients for each image in data using dict elements dictionary using Laplacian Scale Mixture (LSM)

Parameters:

  • data (array-like, shape [batch_size, n_features]) – Data to be used in sparse coding

  • dictionary (array-like, shape [n_features, n_basis]) – Dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like, shape [batch_size, n_basis]

lsm_Loss(data, dictionary, coefficients, lambdas, sigma)

Compute LSM loss according to equation (7) in (P. J. Garrigues & B. A. Olshausen, 2010)

Parameters:

  • data (array-like, shape [batch_size, n_features]) – Data to be used in sparse coding

  • dictionary (array-like, shape [n_features, n_basis]) – Dictionary to be used

  • coefficients (array-like, shape [batch_size, n_basis]) – The current values of coefficients

  • lambdas (array-like, shape [batch_size, n_basis]) – The current values of regularization coefficient for all basis

  • sigma (float, default=0.005) – LSM parameter used to compute the loss functions

Returns:

loss – Loss values for each data sample

Return type:

array-like, shape [batch_size, 1]

sparsecoding.inference.lsm_test module

sparsecoding.inference.lsm_test.test_inference(bars_dictionary_fixture: Tensor, bars_datas_fixture: list[Tensor], bars_datasets_fixture: list[BarsDataset])

Test that LSM inference recovers the correct weights.

sparsecoding.inference.lsm_test.test_shape(patch_size_fixture: int, dataset_size_fixture: int, bars_dictionary_fixture: Tensor, bars_datas_fixture: list[Tensor], bars_datasets_fixture: list[BarsDataset])

Test that LSM inference returns expected shapes.

sparsecoding.inference.mp module

class sparsecoding.inference.mp.MP(sparsity, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Infer coefficients for each image in data using elements dictionary. Method description can be traced to “Matching Pursuits with Time-Frequency Dictionaries” (S. G. Mallat & Z. Zhang, 1993)

Parameters:

  • sparsity (scalar (1,)) – sparsity of the solution

  • return_all_coefficients (string (1,) default=False) – returns all coefficients during inference procedure if True user beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging

  • solver (default=None)

infer(data, dictionary)

Infer coefficients for each image in data using dict elements dictionary using Matching Pursuit (MP)

Parameters:

  • data (array-like (batch_size, n_features)) – data to be used in sparse coding

  • dictionary (array-like, (n_features, n_basis)) – dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like (batch_size, n_basis)

sparsecoding.inference.omp module

class sparsecoding.inference.omp.OMP(sparsity, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Infer coefficients for each image in data using elements dictionary. Method description can be traced to:

“Orthogonal Matching Pursuit: Recursive Function Approximation with Application to Wavelet Decomposition” (Y. Pati & R. Rezaiifar & P. Krishnaprasad, 1993)

Parameters:

  • sparsity (scalar (1,)) – sparsity of the solution

  • return_all_coefficients (string (1,) default=False) – returns all coefficients during inference procedure if True user beware: if n_iter is large, setting this parameter to True can result in large memory usage/potential exhaustion. This function typically used for debugging

  • solver (default=None)

infer(data, dictionary)

Infer coefficients for each image in data using dict elements dictionary using Orthogonal Matching Pursuit (OMP)

Parameters:

  • data (array-like (batch_size, n_features)) – data to be used in sparse coding

  • dictionary (array-like, (n_features, n_basis)) – dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like (batch_size, n_basis)

sparsecoding.inference.pytorch_optimizer module

class sparsecoding.inference.pytorch_optimizer.PyTorchOptimizer(optimizer_f, loss_f, n_iter=100, solver=None)

Bases: InferenceMethod

Infer coefficients using provided loss functional and optimizer.

Parameters:

  • optimizer (function handle) –

    Pytorch optimizer handle have single parameter:

    (coefficients)

    where coefficients is of shape [batch_size, n_basis]

  • loss_f (function handle) –

    Must have parameters:

    (data, dictionary, coefficients)

    where data is of shape [batch_size, n_features] and loss_f must return tensor of size [batch_size,]

  • n_iter (int, default=100) – Number of iterations to run for an optimizer

  • solver (default=None)

infer(data, dictionary, coeff_0=None)

Infer coefficients for each image in data using dict elements dictionary by minimizing provided loss function with provided optimizer.

Parameters:

  • data (array-like, shape [batch_size, n_features]) – Data to be used in sparse coding

  • dictionary (array-like, shape [n_features, n_basis]) – Dictionary to be used to get the coefficients

Returns:

coefficients

Return type:

array-like, shape [batch_size, n_basis]

sparsecoding.inference.pytorch_optimizer_test module

sparsecoding.inference.pytorch_optimizer_test.lasso_loss(data, dictionary, coefficients, sparsity_penalty)

Generic MSE + l1-norm loss.

sparsecoding.inference.pytorch_optimizer_test.loss_fn(data, dictionary, coefficients)
sparsecoding.inference.pytorch_optimizer_test.optimizer_fn(coefficients)
sparsecoding.inference.pytorch_optimizer_test.test_inference(bars_dictionary_fixture: Tensor, bars_datas_fixture: list[Tensor], bars_datasets_fixture: list[BarsDataset])

Test that PyTorchOptimizer inference recovers the correct weights.

sparsecoding.inference.pytorch_optimizer_test.test_shape(patch_size_fixture: int, dataset_size_fixture: int, bars_dictionary_fixture: Tensor, bars_datas_fixture: list[Tensor], bars_datasets_fixture: list[BarsDataset])

Test that PyTorchOptimizer inference returns expected shapes.

sparsecoding.inference.vanilla module

class sparsecoding.inference.vanilla.Vanilla(n_iter=100, coeff_lr=0.001, sparsity_penalty=0.2, stop_early=False, epsilon=0.01, solver=None, return_all_coefficients=False)

Bases: InferenceMethod

Gradient descent with Euler’s method on model in Olshausen & Field (1997) with laplace prior over coefficients (corresponding to L1 norm penalty).

Parameters:

  • n_iter (int, default=100) – Number of iterations to run

  • coeff_lr (float, default=1e-3) – Update rate of coefficient dynamics

  • sparsity_penalty (float, default=0.2)

  • stop_early (bool, default=False) – Stops dynamics early based on change in coefficents

  • epsilon (float, default=1e-2) – Only used if stop_early True, specifies criteria to stop dynamics

  • return_all_coefficients (str, default=False) – Returns all coefficients during inference procedure if True User beware: If n_iter is large, setting this parameter to True Can result in large memory usage/potential exhaustion. This function typically used for debugging.

  • solver (default=None)

References

[1] Olshausen, B. A., & Field, D. J. (1997). Sparse coding with an overcomplete basis set: A strategy employed by V1?. Vision research, 37(23), 3311-3325.

grad(residual, dictionary, a)

Compute the gradient step on coefficients

Parameters:

  • residual (array-like, shape [batch_size, n_features]) – Residual between reconstructed image and original

  • dictionary (array-like, shape [n_features,n_coefficients]) – Dictionary

  • a (array-like, shape [batch_size, n_coefficients]) – Coefficients

Returns:

da – Gradient of membrane potentials

Return type:

array-like, shape [batch_size, n_coefficients]

infer(data, dictionary, coeff_0=None, use_checknan=False)

Infer coefficients using provided dictionary

Parameters:

  • dictionary (array-like, shape [n_features, n_basis]) – Dictionary

  • data (array like, shape [n_samples, n_features])

  • coeff_0 (array-like, shape [n_samples, n_basis], optional) – Initial coefficient values

  • use_checknan (bool, default=False) – check for nans in coefficients on each iteration. Setting this to False can speed up inference time

Returns:

coefficients – First case occurs if return_all_coefficients == “none”. If return_all_coefficients != “none”, returned shape is second case. Returned dimension along dim 1 can be less than n_iter when stop_early==True and stopping criteria met.

Return type:

array-like, shape [n_samples, n_basis] OR [n_samples, n_iter+1, n_basis]

sparsecoding.inference.vanilla_test module

sparsecoding.inference.vanilla_test.test_inference(bars_dictionary_fixture: Tensor, bars_datas_fixture: list[Tensor], bars_datasets_fixture: list[BarsDataset])

Test that Vanilla inference recovers the correct weights.

sparsecoding.inference.vanilla_test.test_shape(patch_size_fixture: int, dataset_size_fixture: int, bars_dictionary_fixture: Tensor, bars_datas_fixture: list[Tensor], bars_datasets_fixture: list[BarsDataset])

Test that Vanilla inference returns expected shapes.