Figure 4

Comparison of ChIPOTle with other ChIP-chip analysis approaches. (a) ChIPOTle, the single-array error model (SAEM), median percentile rank, and PeakFinder were used to analyze the same four Rap1p ChIP-chip replicates reported by Lieb and coworkers [13], and judged by their ability to determine enrichment of ribosomal protein gene (RPG) promoters. The binding site for Rap1p is found in most (>90%) RPG promoters [14], which represent approximately half of Rap1p's total in vivo targets. Receiver operating characteristic (ROC) curves summarize the power of each technique and are equivalent to a plot of the true-positive rate (fraction of ribosomal promoters) versus the false-positive rate (fraction of all genomic elements other than ribosomal promoters). Each technique is judged by means of the area under the ROC curve (AUC). An AUC value of 0.5, corresponding to a diagonal ROC curve, is expected by chance, whereas a value of 1.0 indicates a technique that predicts targets perfectly. ChIPOTle (AUC = 0.963) outperformed the other techniques tested here (SAEM: AUC = 0.906; median percentile rank: AUC = 0.897; and PeakFinder: AUC = 0.823). When comparing ChIPOTle with PeakFinder, we used the default settings for smoothing (n = 5 [11-point] smoothing with 7 rounds). In addition, we attempted to optimize the settings by trying varying levels of smoothing, including 7-point and 13-point, which produced similar results. Rap1p's strongest binding sites are located at the telomeres, which are not included with our defined 'true positive' set of RPG promoters. Therefore, the false-positive rate will be somewhat inflated, which will decrease the AUC for all techniques. This is reflected in the ROC curves by the low true-positive rate at the extreme left of the plot. (b) The 95% confidence interval for the AUC for each analysis technique was estimated by bootstrap resampling of RPG occurrence and enrichment value (1,000 iterations) as measured in each technique (P value, percentile rank, or ySmooth). Boostrapping of raw data was not practical because of inability to automate all four analysis methods. (c) ROC curves comparing ChIPOTle, SAEM, and PeakFinder with respect to their ability to identify enrichment of RPG promoters from a single experiment. The average true-positive rate (fraction of ribosomal promoters) versus false-positive rate (fraction of all genomic elements other than ribosomal promoters) for the four individual experiments is plotted. The three techniques performed extremely well, but ChIPOTle (AUC = 0.885) outperformed both SAEM (AUC = 0.835) and PeakFinder (AUC = 0.833).