Table 1 Observed, hidden and deterministic variables in Celloscope, their corresponding probability distributions and interpretations

Observed variables

    \(C_{gs}\)

Negative binomial

Gene expression for gene g in spot s

    \(B_{gt}\)

-

Indicator whether gene g is a marker for cell type t

Hidden variables

    \(\theta _{st}\)

Gamma

Unnormalized abundance of cell type t in spot s

    \(\lambda _0\)

-

Base gene expression level, shared across all genes and cell types

    \(\Lambda _{gt}\)

-

Overexpression for a marker gene g in its specific cell type t

    \(N_s\)

Truncated normal

Number of cells in spot s

    \(p_g\)

-

Success probability parameter for gene g

    \(Z_{st}\)

Bernoulli

Indicator variable stating presence of cell type t in spot s

    \(\pi _{st}\)

Beta

Prior for the probability of presence of cell type t in a spot s

Deterministic variables

    \(h_{st}\)

Dirichlet

Fraction of cell type t in a spot s

    \(\mu _{gs}\)

-

Average expression of gene g in a spot s