Fig. 6

Initializing ExplaiNN with JASPAR profiles and DanQ models. A Schematic representation of the proposed ExplaiNN model in which the convolutional filters of each unit are initialized with JASPAR [18] profiles. B Performance (average PCC; y-axis) and number of well-predicted sequences (secondary y-axis; black dots) for different models trained on OCRs from the AI-TAC dataset [9]: one ExplaiNN model with 300 units that was trained normally (green), as well as different ExplaiNN models with increasing numbers of units (300, 500, 750, 1000, and 1492) that were initialized with JASPAR profiles (light green), with (pinstripes) and without (no pinstripes) freezing the weights of the filters during training. The performance of AI-TAC is provided as baseline (gray). C Refinement example for a filter whose weights had been initialized with the JASPAR profile of TFAP2C (MA0815.1). Freezing the filter weights during training was detrimental (top): the importance scores of that filter’s unit across all immune lineages were null. However, if filter weight refinement was allowed during training (i.e., no freezing; bottom), that filter was modified until it resembled the motif of EBF1, an important TF for maintaining B cell identity [51], and that same unit became important for predicting accessibility in B cells. D Schematic representation of the proposed transfer learning strategy in which ExplaiNN units are replaced with pretrained DanQ [7] models that can be followed (right) or not (left) by fully connected layers. E, F We used the transfer learning strategy on the left to train one ExplaiNN model with 350 units on the AI-TAC dataset in which the units had been replaced with 350 different pretrained DanQ models, each predicting the binding of a single TF to the mouse genome. During the training process of the ExplaiNN model, the DanQ models were frozen (i.e., their weights were not modified). Unit importance scores of DanQ models belonging to different members of the IRF and PAX TF families are shown. G We repeated the same process but using the transfer learning strategy on the right: each unit was replaced with a pretrained DanQ model but adding two fully connected layers after each model. Then, we applied UMAP [52] to cluster the OCRs based on their unit outputs. (top) UMAP clusters display cell-type specificity (e.g., alpha/beta T and myeloid cells). (bottom) The outputs of the Bcl11b and Cebpa DanQ model units strongly agree with their biologically relevant clusters. OCR, open chromatin region; PCC, Pearson correlation coefficient; TF, transcription factor; UMAP, uniform manifold approximation and projection