Replication Study: Discovery and preclinical validation of drug indications using compendia of public gene expression data

Abstract

In 2015, as part of the Reproducibility Project: Cancer Biology, we published a Registered Report (Kandela et al., 2015) that described how we intended to replicate selected experiments from the paper "Discovery and Preclinical Validation of Drug Indications Using Compendia of Public Gene Expression Data" (Sirota et al., 2011). Here we report the results of those experiments. We found that cimetidine treatment in a xenograft model using A549 lung adenocarcinoma cells resulted in decreased tumor volume compared to vehicle control; however, while the effect was in the same direction as the original study (Figure 4C; Sirota et al., 2011), it was not statistically significant. Cimetidine treatment in a xenograft model using ACHN renal cell carcinoma cells did not differ from vehicle control treatment, similar to the original study (Supplemental Figure 1; Sirota et al., 2011). Doxorubicin treatment in a xenograft model using A549 lung adenocarcinoma cells did not result in a statistically significant difference compared to vehicle control despite tumor volume being reduced to levels similar to those reported in the original study (Figure 4C; Sirota et al., 2011). Finally, we report a random effects meta-analysis for each result. These meta-analyses show that the inhibition of A549 derived tumors by cimetidine resulted in a statistically significant effect, as did the inhibition of A549 derived tumors by doxorubicin. The effect of cimetidine on ACHN derived tumors was not statistically significant, as predicted.

Keywords: Reproducibility Project: Cancer Biology; cancer biology; cimetidine; drug retargeting; human; metascience; replication; reproducibility.

Figure 1.. Tumor xenograft experiment testing efficacy of cimetidine in inhibiting the growth of tumors in SCID mice.

Female SCID mice bearing subcutaneous A549 human lung adenocarcinomas (A) or ACHN renal cell carcinomas (B) were randomized into three treatment groups once tumors reached a calculated volume of 100 mm³. Mice were intraperitoneally injected with 100 mg/kg cimetidine, vehicle (PBS), or 2 mg/kg doxorubicin. Tumor volumes were measured by caliper for the duration of the study. For each condition tumor volume was normalized relative to the average Day 1 values. Means reported and error bars represent s.e.m. Number of mice: (A) A549 tumors: n=14 (vehicle), n=13 (cimetidine), n=4 (doxorubicin) (B) ACHN tumors: n=15 (vehicle), n=15 (cimetidine). Two-tailed Welch’s t-test on Day 11 relative volumes between vehicle or doxorubicin treated A549 tumors; t(6.73) = 2.68, uncorrected p=0.0325 with a priori alpha level = 0.0167; (Bonferroni corrected p=0.0976). Two-way ANOVA interaction between cell line (A549 or ACHN) and treatment (vehicle or cimetidine) on natural log transformed Day 11 relative volumes; F(1,53) = 3.88, p=0.054. Pairwise contrast between A549 tumors treated with vehicle or cimetidine; t(53) = 2.16, uncorrected p=0.035 with a priori alpha level = 0.0167; (Bonferroni corrected p=0.105). Pairwise contrast between ACHN tumors treated with vehicle or cimetidine; t(53) = 0.58, uncorrected p=0.562 with a priori alpha level = 0.0167; (Bonferroni corrected p>0.99). Additional details for this experiment can be found at https://osf.io/fh6gn/. DOI: http://dx.doi.org/10.7554/eLife.17044.002

Figure 1—figure supplement 1.. Individual tumor xenografts.

This is the same experiment as in Figure 1, but with the data plotted for each animal rather than averages. Female SCID mice bearing subcutaneous A549 human lung adenocarcinomas (A) or ACHN renal cell carcinomas (B) were intraperitoneally injected with 100 mg/kg cimetidine, vehicle (PBS), or 2 mg/kg doxorubicin. For each condition tumor volume was normalized relative to the average Day 1 values. Number of mice: (A) A549 tumors: n=14 (vehicle), n=13 (cimetidine), n=4 (doxorubicin) (B) ACHN tumors: n=15 (vehicle), n=15 (cimetidine). Additional details for this experiment can be found at https://osf.io/fh6gn/. DOI: http://dx.doi.org/10.7554/eLife.17044.003

Figure 2.. Final tumor weights from xenograft experiment testing efficacy of cimetidine in inhibiting the growth of tumors in SCID mice.

At the end of the predefined study period (Day 12), tumors from the xenograft experiment reported in Figure 1 were excised and weighed. Box and whisker plot with median represented as the line through the box, means represented as the solid red circle, and whiskers representing values within 1.5 IQR of the first and third quartile. Number of mice: A549 tumors: n=14 (vehicle), n=13 (cimetidine), n=4 (doxorubicin). ACHN tumors: n=15 (vehicle), n=15 (cimetidine). Two-tailed Welch’s t-test between vehicle or doxorubicin treated A549 tumors; t(4.078) = 0.771, p=0.483. Two-way ANOVA interaction between cell line (A549 or ACHN) and treatment (vehicle or cimetidine); F(1,53) = 0.00005, p=0.994. Two-way ANOVA main effect of cell line (A549 or ACHN); F(1,53) = 7.56, p=0.00814. Additional details for this experiment can be found at https://osf.io/fh6gn/. DOI: http://dx.doi.org/10.7554/eLife.17044.004

Figure 3.. Meta-analyses of each effect.

Effect size (Cohen’s d) and 95% confidence interval are presented for Sirota et al., 2011, this replication attempt (RP:CB), and a meta-analysis to combine the two effects of day 11 tumor volume comparisons. Sample sizes used in Sirota et al., 2011 and this replication attempt are reported under the study name. Random effects meta-analysis of A549-derived tumors treated with vehicle (PBS) compared to 100 mg/kg cimetidine (meta-analysis p=0.0043), ACHN-derived tumors treated with vehicle (PBS) compared to 100 mg/kg cimetidine (meta-analysis p=0.917), and A549-derived tumors treated with vehicle (PBS) compared to 2 mg/kg doxorubicin (meta-analysis p=0.021). Additional details for these meta-analyses can be found at https://osf.io/jcghv/. DOI: http://dx.doi.org/10.7554/eLife.17044.005