sparsecoding.priors package

class sparsecoding.priors.L0Prior(prob_distr: Tensor)

Bases: Prior

Prior with a distribution over the l0-norm of the weights.

A class of priors where the weights are binary; the distribution is over the l0-norm of the weight vector (how many weights are active).

Parameters:: prob_distr (Tensor, shape [D], dtype float32) – Probability distribution over the l0-norm of the weights.

property D: Number of weights per sample.

sample(num_samples: int)

Sample weights from the prior.

Parameters:: num_samples (int, default=1) – Number of samples.
Returns:: samples – Sampled weights.
Return type:: Tensor, shape [num_samples, self.D]

class sparsecoding.priors.Prior

Bases: ABC

A distribution over weights.

Parameters:: weights_dim (int) – Number of weights for each sample.

abstract D(): Number of weights per sample.

abstract sample(num_samples: int = 1)

Sample weights from the prior.

Parameters:: num_samples (int, default=1) – Number of samples.
Returns:: samples – Sampled weights.
Return type:: Tensor, shape [num_samples, self.D]

class sparsecoding.priors.SpikeSlabPrior(dim: int, p_spike: float, scale: float, positive_only: bool = True)

Bases: Prior

Prior where weights are drawn from a “spike-and-slab” distribution.

The “spike” is at 0 and the “slab” is Laplacian.

See:: https://wesselb.github.io/assets/write-ups/Bruinsma,%20Spike%20and%20Slab%20Priors.pdf

for a good review of the spike-and-slab model.

Parameters:

dim (int) – Number of weights per sample.
p_spike (float) – The probability of the weight being 0.
scale (float) – The “scale” of the Laplacian distribution (larger is wider).
positive_only (bool) – Ensure that the weights are positive by taking the absolute value of weights sampled from the Laplacian.

property D: Number of weights per sample.

sample(num_samples: int)

Sample weights from the prior.

Parameters:: num_samples (int, default=1) – Number of samples.
Returns:: samples – Sampled weights.
Return type:: Tensor, shape [num_samples, self.D]

Submodules

sparsecoding.priors.l0_prior module

class sparsecoding.priors.l0_prior.L0Prior(prob_distr: Tensor)

Bases: Prior

Prior with a distribution over the l0-norm of the weights.

A class of priors where the weights are binary; the distribution is over the l0-norm of the weight vector (how many weights are active).

Parameters:: prob_distr (Tensor, shape [D], dtype float32) – Probability distribution over the l0-norm of the weights.

property D: Number of weights per sample.

sample(num_samples: int)

Sample weights from the prior.

Parameters:: num_samples (int, default=1) – Number of samples.
Returns:: samples – Sampled weights.
Return type:: Tensor, shape [num_samples, self.D]

sparsecoding.priors.lo_prior_test module

sparsecoding.priors.lo_prior_test.test_l0_prior()

sparsecoding.priors.prior module

class sparsecoding.priors.prior.Prior

Bases: ABC

A distribution over weights.

Parameters:: weights_dim (int) – Number of weights for each sample.

abstract D(): Number of weights per sample.

abstract sample(num_samples: int = 1)

Sample weights from the prior.

Parameters:: num_samples (int, default=1) – Number of samples.
Returns:: samples – Sampled weights.
Return type:: Tensor, shape [num_samples, self.D]

sparsecoding.priors.spike_slab_prior module

class sparsecoding.priors.spike_slab_prior.SpikeSlabPrior(dim: int, p_spike: float, scale: float, positive_only: bool = True)

Bases: Prior

Prior where weights are drawn from a “spike-and-slab” distribution.

The “spike” is at 0 and the “slab” is Laplacian.

See:: https://wesselb.github.io/assets/write-ups/Bruinsma,%20Spike%20and%20Slab%20Priors.pdf

for a good review of the spike-and-slab model.

Parameters:

dim (int) – Number of weights per sample.
p_spike (float) – The probability of the weight being 0.
scale (float) – The “scale” of the Laplacian distribution (larger is wider).
positive_only (bool) – Ensure that the weights are positive by taking the absolute value of weights sampled from the Laplacian.

property D: Number of weights per sample.

sample(num_samples: int)

Sample weights from the prior.

Parameters:: num_samples (int, default=1) – Number of samples.
Returns:: samples – Sampled weights.
Return type:: Tensor, shape [num_samples, self.D]

sparsecoding.priors.spike_slab_prior_test module

sparsecoding.priors.spike_slab_prior_test.test_spike_slab_prior(positive_only: bool)