Fig. 1

The schematic view of ZINBMM. With n cells and J genes, let \(X_{ij}\) be the count expression of the \(j^{\text {th}}\) \((j = 1,\dots ,J)\) gene from the \(i^{\text {th}}\) \((i = 1, \dots , n)\) cell; \(X_{ij}\) follows \(f= \sum _{k=1}^{K}p_{k} \cdot f_\text {ZINB}\left( X_{ij};\pi _{jk},\mu _{ijk},\phi _j\right)\) with \(f_{\text {ZINB}}(X_{ij}; \pi _{jk},\mu _{ijk},\phi _j)=\pi _{jk}\text {I}(X_{ij} = 0)+(1-\pi _{jk})f_{\text {NB}}(X_{ij};\mu _{ijk},\phi _j)\) and \(\log (\mu _{ijk}) = \beta _{jk}+\varvec{B}'_i\varvec{\gamma }\). The final output of ZINBMM includes a cluster assignment for each cell and the selected cell type-specific genes