Developing new health technologies for neglected diseases: a pipeline portfolio review and cost model

i) Pipeline portfolio review

Scope. A global review of candidate products for neglected diseases was conducted by Policy Cures Research (authors NC and VC) in 2017, and represents a snapshot of the pipeline as of August 31, 2017. Policy Cures Research, a non-profit global health research and advocacy organization, developed the scope of the R&D pipeline presented in this analysis based on the G-FINDER landscape reports on global funding of R&D for neglected diseases and the reproductive health needs of developing countries. The G-FINDER scope is based on three key principles: the disease or health issue disproportionately affects the developing world; there is a need for new products (i.e., there is no existing product, or improved or additional products are needed); and there is market failure (i.e., there is insufficient commercial market to attract private R&D investment). Accordingly, the R&D pipeline presented here encompasses the 33 neglected diseases included in the G-FINDER report, as well as developing country-specific reproductive health needs, and Ebola, giving a total of 35 neglected diseases or conditions.

Product types included were drugs, vaccines, diagnostics, vector control products, contraceptives, and multipurpose prevention technologies (MPTs); however, in line with the G-FINDER scope, not all product types were included for all areas. Microbicide candidates for HIV have been included under the drug category, while contraceptives and MPTs (which prevent pregnancy and sexually transmitted infections) have been grouped as reproductive health technologies. Medical devices (except for diagnostics and contraceptives), and general or supportive therapies (e.g., oral rehydration or nutritional supplements), were excluded.

We define the pipeline to include product candidates at all stages of development—from discovery through product registration. For drugs, vaccines, and reproductive health products, development stage was broken down into discovery; pre-clinical studies; and clinical trials (further divided into Phase I, Phase II and Phase III trials). Diagnostics and vector control products have different product development and regulatory pathways, and the development stage for these products was broken down into concept and research; feasibility and planning; design and development; and clinical validation and launch readiness. Early stage drug discovery projects that are not linked to a specific, discrete pipeline candidate were excluded. Candidates were no longer considered to be part of the R&D pipeline—and therefore were excluded from this analysis—once granted regulatory approval by a national regulatory authority, or if their development had been placed on hold indefinitely.

In line with the scope of the G-FINDER report, additional restrictions were applied to selected disease and product categories with potential commercial (high-income country market) overlap. For example, drug candidates for HIV/AIDS were only included if they were label-extensions or reformulations specifically intended for developing country use (e.g., paediatric or slow-release formulations; fixed dose combinations; or low-dose formulations for prophylaxis).

Further details on the specific diseases and related product areas within the scope of the pipeline analysis are given in the Policy Cures Research R&D matrix and R&D scope document. Additional details about the search are in the online methodology.

Data sources and validation. Policy Cures Research collected data on the candidates in the global R&D pipeline for 35 neglected diseases. This new 2017 product portfolio review built on Policy Cures Research’s previous pipeline analysis conducted (as Policy Cures) in 2015, pipeline data collected for the 2012 Policy Cures/Global Health Technologies report Saving Lives and Creating Impact: Why Investing in Global Health R&D Works, and the BIO Ventures for Global Health (BVGH) Global Health Primer.

To bring the R&D pipeline candidate data up to date as of August 31, 2017, and to expand on the scope of previous efforts, Policy Cures Research reviewed and cross-referenced all major sources of available data on the R&D pipeline for poverty-related and neglected diseases. Sources included the G-FINDER R&D funding database; the World Health Organization “rainbow tables”; background documents prepared for the WHO Product Development for Vaccines Advisory Committee (see WHO page of Immunization, Vaccines and Biological R&D); WHO Vector Control Advisory Group (VCAG) reports; WHO Pesticide Evaluation Scheme (WHOPES) reports; Unitaid landscape and technical reports; disease-specific pipeline updates prepared by BVGH and the Treatment Action Group; publicly available company and product development partnership R&D portfolios; journal publications; clinical trial registration portals; and university, government, and non-profit organization websites.

Candidates were only included if an authoritative source could confirm they were still in active development. The following sources were considered to be authoritative:

ii) Development of the P2I v.2 costing model

In order to estimate the costs of moving these candidate health products for neglected diseases through the pipeline, and the likely resulting product launches, we made adaptations to an existing, custom-built costing model, P2I (we call the original model P2I version 1, or P2I v.1). We call the adapted model P2I version 2 (P2I v.2).

The P2I v.1 tool is a user-friendly, public Microsoft Excel file (see Supplementary with reference 12) with costing assumptions and formulae built in¹². The tool was developed by TDR. In brief, P2I v.1 is a financial portfolio model that estimates funding needs to move a portfolio of candidate health products through the pipeline from late stage pre-clinical to phase III clinical trials, as well as potential product launches over time (Figure 1). The model, which is deterministic, is based on assumptions for costs, attrition rates (probability of success), and cycle times for four development phases (preclinical to phase III) for eleven different kinds of medical products, called archetypes (Table 1). A detailed description of how these assumptions were developed is given in the accompanying study on development of the P2I v.1 tool¹².

Figure 1. Conceptual overview of P2I model.

In brief, the assumptions on development costs at each phase of product development were initially based on a bottom-up analysis of clinical trial costs from Parexel’s R&D cost sourcebook¹³. The assumptions on attrition rates and cycle times at each phase were initially based on a review of the attrition rates and cycle times of more than 25,000 development candidates. These assumptions were further refined and validated based on (i) academic literature¹⁴, (including literature on the development of antibody therapeutics) (ii) industry publications and databases (such as Pharmaprojects), and (iii) stakeholder interviews with a wide variety of PDPs, biopharmaceutical and diagnostic companies, and major funders of global health R&D. For the stakeholder interviews, a total of 228 stakeholders representing a cross-section of the global R&D landscape were contacted to request an interview and 133 agreed to be interviewed, a response rate of 58% (see reference 11 for further details). Overall, the stakeholder interviews largely confirmed the validity of the assumptions derived from the R&D cost sourcebook, and did not lead to any significant changes. As a final validation step, the P2I model and its assumptions were reviewed by TDR’s Science and Technical Advisory Committee, who provided an additional round of expert inputs.

The P2I v.1 tool allows users to make multiple types of adaptations. For example, (i) users can add archetypes; (ii) users can input disease-specific assumptions (e.g., the attrition rates in developing TB biologics); and (iii) if users have their own additional data on costs, attrition rates, and cycle times across the portfolio, they can enter the tool and modify the assumptions. For this study, we made all three types of adaptations.

First, we wanted to add archetypes that were not in version 1, such as vector control products and unprecedented vaccines (discussed further below). Second, we wanted to refine the model as it applied to a specific set of TB candidate archetypes. Third, we took the opportunity to modify a small number of the assumptions. These three adaptations were made based on data shared by the Bill & Melinda Gates Foundation (Per Liljenberg, personal communication). The Foundation supported the development of the P2I v.1 tool, and has used the tool to estimate its own portfolio development costs, but it has made a number of adaptations derived from three Foundation costing exercises:

Below we summarize the adaptations in the P2I v.2 model, which are based on outputs of the 2016 exercise.

The P2I v.2 model has two additional archetypes, “unprecedented vaccines” and “other products”:

In addition to including these two archetypes, we also modified several assumptions, as follows:

Table 2 shows the assumptions that we used in P2I v.2.

Finally, P2I v.1 allows users to input a portfolio of up to 150 product development projects (the model was developed in the context of potentially launching a pooled fund for R&D that would have had capacity constraints). However, the pipeline of candidates for neglected diseases has far more than 150 projects, so we adapted the tool to be able to input all of these projects.

iii) Classification of product candidates into archetypes

As a starting point, we used the initial archetypes and the descriptions of each archetype from P2I.v1 (Table 1). In order to further “operationalize” each archetype—i.e., to make it easier for the research team to make decisions about how to classify each candidate—we worked with technical experts to further define each archetype. Table 3 shows the original archetype descriptions from P2I v.1, our additional definitions, and examples of candidate classifications.

As shown in Table 3, product candidates were classified into six broad archetypes—repurposed drugs, NCEs, vaccines, biologics, diagnostics, and “other products” (which refers to vector control products). Repurposed drugs, NCEs, and biologics were further sub-classified into simple versus complex; vaccines into simple, complex, or unprecedented; and diagnostics into assay development versus simple technical platform development. For candidates in the pipeline that were contraceptives, microbicides, or MPTs, these were classified according to the constituent drug (e.g., microbicides in the pipeline were classified as repurposed drugs, NCEs, or biologics). If there was more than one active drug ingredient in the MPT, the candidate was classified according to the most complicated component. We did not consider if the polymer or technology itself was innovative in itself as this went beyond the scope of our costing framework.

The classification of candidates was made by different members of the research team, based on their expertise (repurposed drugs, NCEs, and biologics: KS, KC; diagnostics: BR; vaccines: SP, LD, TS; other products: VC). The classification was based on a combination of (a) technical expertise of the researchers, (b) academic literature, (c) relevant publicly available product databases (e.g., for classifying drugs, ChemBL and chemspider), (d) information from international clinical trials registries, including the WHO International Clinical Trials Registry Platform, (e) websites of PDPs, e.g. the Medicines for Malaria Venture website, (e) patent databases, and (f) relevant reports and news releases from bilateral and multilateral funding agencies, companies, PDPs, other product developers, and non-government organizations. In assigning each candidate product to an archetype, we documented any relevant source material that guided the classification (e.g., a published research article on the candidate’s mechanism of action).

Classification of candidate diagnostics into archetypes was conducted by a technical expert in diagnostics R&D (BR). Using the Policy Cures Research list of candidates, these were further classified into six more specific development phases, as required by P2I v.2: concept; feasibility; early development; late development; validation; and commercialization. Table 4 summarizes what these phases mean and how they compare with two other classification systems for technology readiness (the Technology Readiness Level (TRL) and Manufacturing Readiness Level (MRL), developed by the United States Department of Defense). When it came to inputting candidates into the adapted P2I cost model (as shown in Table 2), diagnostic candidates at the concept phase were placed in the category “concept and research”; those in the feasibility stage were placed in “feasibility and planning”; those in either early or late development were placed in “design and development”; those in validation were placed in “clinical validation and launch readiness”; and those in the commercialization phase were excluded from the cost modelling.

iv) Inputting the pipeline of candidates into the adapted P2I model

As described previously, we adapted P2I v.1 into P2I v.2, adding two archetypes (unprecedented vaccines, and other products) and modifying a number of the assumptions. We then used the model “prospectively”—that is, once we had determined which candidates were in the pipeline, and their target disease/condition, archetype and phase, we then inputted these candidates into P2I v.2.

For each disease and archetype, we inputted the number of candidates that were in each phase of development (for repurposed drugs, NCEs, biologics, vaccines, and other products, the phases were preclinical, phase I, phase II, or phase III; for diagnostics, the phases were concept and research, feasibility and planning, design and development, or clinical validation and launch readiness). The analysis was undertaken in 2017, and hence we chose a start date of 2017 (consistent with P2I v.1).

Based on the assumed costs, attrition rates, and time per phase for each archetype, the model estimates the costs and outcomes of moving product candidates through the pipeline from their current phase. When a candidate is put into in a specific phase, the model assumes that it is at the start of that phase and so it includes the costs of moving that candidate through to the end of its current phase.

For each archetype for each disease, the final outputs are (a) the costs of moving the archetype candidates through the pipeline from their current phase, and (b) the estimated product launches at the end of this process. For launches, we rounded only at the very end of the model. For example, for disease X, if there were 3 simple vaccine candidates at Phase II that led to 1.3 expected launches and 3 simple vaccine candidates at Phase III that led to 1.4 expected launches, we rounded the cumulative total—in this case, the total was 2.7. For this paper, we have chosen a conservative approach to presenting the launches—we have considered a launch to be a binary event, i.e., we have always rounded down (in this case, 2.7 rounds down to 2 launches). However, in Supplementary File 1, we also present the results without any rounding (e.g., in this example, 2.7 launches) and with rounding to the nearest integer (2.7 would round to 3). Both of these other approaches give less conservative estimates of the number of launches.

We did not apply a discount rate to our cost estimates. Probability of success, time of phase, cost variables, archetype and complexity classification were assumed to remain constant throughout the lifecycle of the model. We modeled only the current pipeline (i.e., we assumed that no new candidates would enter the pipeline).

(v) Estimating the costs of priority “missing” products

As described in the Results section below, for several diseases and product types (e.g., highly efficacious vaccines for HIV, TB, and malaria), the model suggested that there would be no product launches based on the current status quo (i.e. based on the pipeline of candidates that are in the public domain). In order to estimate the costs to develop those products that are likely to be “missing” but are highly needed, we reviewed the suggested list of “important” or “game changing” diagnostics, drugs, and vaccines prioritized by the Commission on Investing in Health (Table 5). The Commission’s list was developed through expert consensus. We examined the overlap between the Commission’s proposed products and those that our modelling suggested would still be missing (the 18 missing products are noted in Table 5). For each missing product, we used the P2I v.2 model in a retrospective manner to estimate the number of additional candidates that would be needed at preclinical phase—over and above the existing candidates—to lead to one expected launch of that product, and the associated additional cost. For example, as shown in the Results section, we found 41 HIV vaccine candidates in the pipeline and the modelling suggested that these would result in 0.49 launches. Thus, to estimate the additional costs to reach one launch, we estimated the number of additional candidates needed at preclinical phase and the associated additional costs to achieve an additional 0.51 launches (in this case, an additional 125 candidates would be needed at preclinical phase to achieve 0.51 launches, at an additional cost of $2.8 billion).

(vi) Sensitivity analysis

As a final step, we conducted a sensitivity analysis, adopting an approach proposed by Mestre-Ferrandiz et al. at the United Kingdom Office of Health Economics in their study, “The R&D Cost of a New Medicine.” We examined the impact of changing all probabilities of success per phase to 10% higher and 10% lower, and all costs per phase to 10% higher and lower. We also examined the impact of all possible combinations of these changes (e.g., 10% higher probability of success per phase and a 10% higher cost per phase, 10% higher probability of success per phase and a 10% lower cost per phase, etc.). We conducted this sensitivity analysis both for moving current candidates through the pipeline and for the costs of priority “missing” products. We did not conduct a sensitivity analysis varying the length of time per phase, because in the P2I model the length of time is independent of the cost variables (the cost parameters are per phase, not per year). We also conducted a sensitivity analysis in which we used only the assumptions from the P2I v.1 model, which helps to show the impacts of the modifications that we made to P2I v.1 when we developed P2I v.2.

While the approach of changing product development assumptions by 10% higher and lower has been used in previous studies, we recognize that achieving these changes—particularly reducing the attrition rates—may not be realistic. As Paul et al. note: “There is little doubt that reducing the attrition rate of drug candidates in clinical development represents the greatest challenge and opportunity for pharmaceutical R&D, and arguably for sustaining the viability of the entire industry¹⁵.” Reducing costs may be more feasible, e.g. through simpler trials or improved trial management. Nevertheless, we included a sensitivity analysis to show the impact of changing the underlying assumptions on our estimates.

The pipeline of candidates for neglected diseases

In the pipeline portfolio review, we found 685 product candidates for neglected diseases as of August 31, 2017. We excluded 147 of these from the model because (a) there was insufficient information about their development phase, (b) they were already marketed, or in a development phase that is excluded from the P2I.v2 model, or (c) there was insufficient information about the candidate to be able to classify it into an archetype. After exclusion, 538 candidates were included in the model.

Supplementary File 2 gives detailed information on these 538 candidates, showing (a) the candidate’s name, (b) the health area that it targets, (c) the development phase that it was in at the time this study was conducted, (d) the archetype to which we assigned the candidate, (e) relevant source material that guided the classification, and (f) the sponsors and collaborators conducting the study of this candidate.

Figure 2 shows the breakdown of candidates by the archetypes used in P2I v.2 (208 vaccines, 108 NCEs, 101 diagnostics, 90 repurposed drugs, 16 vector control products, and 15 biologics) and Figure 3 shows the breakdown by disease/condition. The pipeline is dominated by three diseases—malaria (109 candidates), HIV/AIDS (99 candidates), and tuberculosis (98 candidates)—which comprise nearly 6 out of 10 (57%) of all candidates. About 1 in 10 candidates (11%) are for reproductive health needs in developing countries. Table 6 shows the breakdown of candidates by disease and archetype for the 10 diseases/conditions that have the most candidates. For several health areas—particularly neglected tropical diseases—there were only one or two candidates (one candidate for cryptococcal meningitis, giardiasis, leptospirosis, multiple diarrheal diseases, multiple vector-borne diseases, trichuriasis, and two candidates for hookworm, leprosy, lymphatic filariasis, meningitis, rheumatic fever, and trachoma).

Figure 2. Number of candidate products for neglected diseases, by archetype.

Figure 3. Number of candidate products for neglected diseases, by disease/condition.

Costs to move candidates through the pipeline

Based on inputting these 538 candidates into the P2I.v2 model, the modelling suggests that it would cost about $16.3B over about 10–12 years to move all of these candidates through the pipeline (Table 7). Supplementary File 3 gives detailed information on the breakdown of these estimated costs. As shown in Table 7, in many cases the overall costs per phase are higher for the simpler archetype (e.g., the costs in all phases are higher for simple vaccines than complex vaccines); this finding simply reflects the fact that there are more candidates of simple complexity in the pipeline (see Figure 2 and Supplementary File 2).

Over half of these estimated costs (55%, $8.9B) would be for Phase III clinical trials, one third (33%, $5.3B) for Phase II trials, just under one tenth (9%, $1.4B) for preclinical development and the remainder (4%, $0.7B) for phase I trials (the percentages do not add up to 100%, since they have each been rounded). About three quarters of the costs (76%, $12.4B) would be incurred in the next 5 years assuming that all candidates are taken through all stages (Figure 4); in reality, of course, there would be a “go/no go review” at each stage gate. The ““front loading” of costs over the first five years reflects the large number of candidates at early phases and the subsequent attrition through each phase.

Figure 4. Costs over time to move candidate products through pipeline, total and by archetype.

As shown in Table 7, if all the current candidates that are in the public domain were taken through all stages, over half of the development costs (55%, $9B) would be for vaccines, about one-fifth (21%, $3.5B) for diagnostics, 12% ($2B) for NCEs, about 7% ($1.2B) on drug repurposing, 4% ($0.6B) on biologics, and 0.4% (under $0.1B) on vector control products (the percentages do not add up to 100%, since they have each been rounded). The four diseases responsible for the highest costs would be TB ($2.6B), HIV ($2.3B), malaria ($2.3B), and Ebola ($1.2B) (Figure 5).

Figure 5. Costs to move candidate products through pipeline, by disease.

Expected product launches

Based on public domain information on the current portfolio of candidates for neglected diseases, and the assumptions of success at each phase for the different archetypes included in the P2I v.2 model, 128 product launches would be expected (this figure is based on rounding down the number of launches for each disease archetype; Supplementary File 1 shows the results without any rounding and also with rounding to the nearest integer). The dominant product type for the anticipated launches would be diagnostics, which would make up almost 6 in 10 expected launches (57%, expected 73 launches), followed by repurposed drugs (13%, 16 expected launches) and simple NCEs (13%, 16 expected launches).

As shown in Figure 6, the model estimates that just over one quarter (27%) of all anticipated launches would be for TB: 35 launches, comprising 27 diagnostics, 7 repurposed drugs and 1 NCE. The diseases that would see the second and third highest number of expected launches, respectively, would be malaria (27 expected launches, comprising 14 diagnostics, 8 vector control products, 3 NCEs and 2 repurposed drugs) and HIV (23 expected launches, comprising 13 diagnostics and 10 NCEs).

Figure 6. Breakdown of the portfolio of expected launches, by disease and archetype.

PRNDs: poverty-related, neglected diseases

Table 8 shows the breakdown of these anticipated 73 diagnostics launches by archetype, complexity (assay development versus simple technical platform development) and disease. For example, of the 27 anticipated TB diagnostics, 16 would be diagnostic assays and 11 would be technological platforms that enhance current technology.

The modelling suggests that there would be very few launches of complex vaccines (only 2). It also suggests that launches of vaccines for HIV, TB, or malaria and launches of complex NCEs would be unlikely.

Supplementary File 3 gives detailed information on the portfolio of anticipated product launches, broken down by disease and archetype.

Sensitivity analysis: moving current candidates through pipeline

Table 9 shows the results of the sensitivity analysis, in which we examined the impact on costs and expected product launches of changing all probabilities of success per phase to 10% higher and 10% lower, all costs per phase to 10% higher and lower, and all possible combinations of these changes. The sensitivity analysis found that the total costs to move the current pipeline through to production range from $13.4B-19.8B and the anticipated launches range from 89–160.

The second sensitivity analysis that we conducted in which we used only the assumptions of P2I v.1 found that the total costs to move the current pipeline through to production would be $13.4B, with 151 anticipated launches.

Estimates of the costs of “missing” products

As shown in Supplementary File 3, based on the current known pipeline (i.e., candidates in the public domain), there are unlikely to be product launches for several diseases and product types. A comparison of these “missing” products with the list of needed priority products proposed by the Commission on Investing in Health highlights 18 missing products. As shown in Table 5, these are: highly efficacious vaccines against HIV, TB, malaria, and hepatitis C; a combined vaccine against multiple diarrheal diseases; a complex NCE for TB; and NCEs for 12 NTDs (based on the WHO list of NTDs: Buruli ulcer, Chagas disease, dengue, human African trypanosomiasis, hookworm, leishmaniasis, leprosy, lymphatic filariasis, onchocerciasis, schistosomiasis, trachoma, and trichuriasis).

Table 10 estimates the number of candidates needed at the preclinical phase, development cost, and development time to launch one product, for each different archetype, assuming an “empty” pipeline (i.e., starting product development from scratch). However, the pipeline for these 18 missing products is not currently empty; Table 11 shows the number of additional candidates that would be needed at preclinical phase—over and above the existing candidates—to lead to one expected launch of that product, and the associated additional cost. Table 11 includes the results of the sensitivity analysis (changing all probabilities of success per phase to 10% higher and 10% lower, all costs per phase to 10% higher and lower, and all possible combinations of these changes).

The total estimated additional costs (over and above the costs to move current candidates through the pipeline) to reach one estimated launch of each of these 18 missing products ranges from $13.6B to $21.8B over 10–12 years, depending on the complexity of the products (Table 11). Around three-quarters (75%) of the costs are likely to be incurred over the first 5 years, i.e., $10.3B-16.6B. The sensitivity analysis found that the total additional costs range from $8.1B-36.6B.

Over the next 5 years, the estimated costs to move all current candidates that are in the public domain through the pipeline plus the additional costs to launch 18 missing products are about $22.7B-$29B or $4.5-5.8B per year.

Strengths of the study

Our study has several strengths, and we highlight two in particular. The first is its novelty. To the best of our knowledge, our study is the first to estimate the costs of global health product development from preclinical to the end of phase III based on the existing portfolio of candidates across multiple neglected diseases, and the first study to use and adapt the P2I v.1 model for this purpose. This approach complements earlier efforts to model a single therapeutic portfolio¹⁹. Our approach of costing a portfolio of candidates using the current pipeline as a basis adds a different dimension to the field of global health R&D costing, and one that aligns with the way in which funders pursue a diversified portfolio of product development projects. The value of the model is in the ability to estimate costs and probable launches based on a portfolio of candidates. The model is much less reliable when it comes to predicting what will happen to any one specific candidate—in other words, it is prone to the ecological fallacy (making inferences about a single candidate based on data from across a portfolio of candidates).

Second, by moving beyond the costing of individual product types for specific diseases (e.g., costing only an HIV vaccine or a TB diagnostic), our study has shown more broadly—across the portfolio of neglected diseases—where the pipeline is most robust, where it is lacking, which product launches are most likely, and which products will probably still be missing based on existing candidates. For global health R&D advocates, this broad picture could potentially help to highlight critical funding and product development gaps.

Strengths of the P2I tool

We highlight two strengths of the tool itself. First, the P2I v.1 tool that we used and adapted is available online, as are our model assumptions, model inputs and outputs, and detailed information on the portfolio review, which means that readers can replicate, improve on, and further adapt our work¹². The P2I v.1 tool was designed for flexibility, as shown in the way in which we adapted it, and we encourage others to refine it further. In particular, we hope that those who have access to updated, high quality data on costs, attrition rates, and cycle times will share and contribute these data to further iterations of the model. All R&D cost modelling exercises are, of course, inherently uncertain, but the steps we have taken will, we believe, allow others to “stress test” our work. In the accompanying study on the development of the P2I tool¹², we describe how the tool can be used to estimate health impact. We note: “The P2I model allows users to estimate the impact of a launched product on both disability, measured in disability-adjusted life years (DALYs) averted, and mortality, measured in deaths averted.” It would thus be possible to estimate the economic value of these health impacts, and therefore to estimate the rates of return on portfolio investment.

Second, most of the model assumptions were based on a large number of data points (e.g., assumptions on success rates and cycle times were based on data from of 25,000 development candidates), and were validated through examining peer-reviewed literature, industry reports/databases, and expert interviews. While no assumptions used in R&D cost models can ever be perfect, we believe that the process for developing the assumptions was “robust enough” to give realistic, real world benchmarks for costs, success rates, and cycle times per phase.

While a detailed discussion of the literature on clinical development success rates for investigational drugs is beyond the scope of this paper, our assumptions appear to be roughly in line with reported industry standards (the amount of variation differs by product types)^20,21. For example, Hay et al. analyzed phase transitions from January 1, 2003 to December 31, 2011 in the proprietary BioMedTracker database, a database of investigational drugs, to estimate success rates per phase²¹. For all drug indications together, they estimate that the success rate for NCEs to advance through phases 1, 2, and 3 is 64.2%, 28.6%, and 53.2%, respectively. In comparison, our assumptions on success rates were 60% (simple NCE) or 57% (complex NCE) for phase 1, 39% (simple NCE) or 20% (complex NCE) for phase 2, and 69% (simple NCE) or 40% (complex NCE) for phase 3.

Limitations of the study

Our study has a number of limitations. First, the pipeline portfolio review only provided a snapshot at a single point in time. The pipeline is constantly changing, and between the end of our pipeline review (August 31, 2017) and today, it has already changed.

Second, our review is probably incomplete, given the lack of publicly available information on some products under development. It is particularly challenging to find information about candidates that are at the pre-clinical research phase, since studies at this phase are not included in clinical trial registries. Unfortunately, proprietary interests and non-disclosure agreements mean that we have no knowledge at all of some candidates under development. While we did our best to gain as full a picture of the pipeline as possible by using a variety of methods—such as searching databases and interviewing product developers—it is likely that we missed some candidates. Using a different search strategy or searching further databases could have identified additional candidates, such as anti-bacterial products under development (though only certain anti-bacterial candidates would qualify for the strict inclusion criteria in our study). For all these reasons, and as mentioned above, our model probably under-estimates the total costs and number of product launches. We call on the global health research community to commit to making its product development research more transparent.

A third, related limitation is that for some candidates, there is very little information on their development phase. We found it particularly challenging to obtain public data, even in trial registries, on the development phase for candidate MPTs, diagnostics, and vector control products. In some cases, we had to make a judgment call based on whatever data we could find.

A fourth limitation relates to trial capacity. We modelled the costs to launch a number of critical “missing” products—but the model does not account for whether there is the actual trial capacity to conduct the additional studies that would be needed should the additional funding be mobilized.

A fifth limitation relates to the modifications that we made to the P2I v.1 tool to create P2I v.2. As described in the Methods section, these modifications were made using a variety of data sources (see Table 2), which introduced some variation in the strength of the evidence underlying the adaptations. For example, the assumptions on probabilities of success for unprecedented vaccines in phases II and III were based on a relatively small number of data points—around 10–25 data points per estimated value. As we note in the methods section, in P2I v.2, the adjustments made to the assumptions in P2I v.1 for biologics were derived from cautious expert judgement based on early industry trends. To assess the impact of the adaptations, we conducted a sensitivity analysis in which we ran the model using only the P2I v.1 assumptions. This resulted in a lower cost estimate for moving existing candidates through the pipeline ($13.4B using P2I v.1 assumptions versus $16.3B using P2I v.2 assumptions) and a higher estimate of expected launches (151 vs. 128).

Limitations of the P2I modelling tool

As with all modelling tools, the P2I tool that we used in this study has several limitations. First, the model is deterministic and static. It does not take into account possible improvements in product development techniques over time, such as reductions in cycle time or lowering of development costs. Historical evidence suggests that factors such as attrition rates and costs per phase do change over time, and the P2I tool does not capture such evolutions over time²⁰. We tried to address this limitation through our sensitivity analysis, but this does not fully capture the uncertainties surrounding the model assumptions. The P2I v.1 and v.2 models also do not account for down selection, in which portfolio managers decide at various stages of the development process to drop certain candidates. By examining only the current, static portfolio of candidates—and costing just these—our study does not account for future candidates that will enter the pipeline. It also does not address the minimum specifications that a product may need to meet. For instance, although the current TB diagnostic pipeline is forecast to result in several new approved diagnostics, there is no guarantee that these products would be fit-for-purpose, or offer a significant improvement over current tools.

Second, the P2I v.2 model requires users to classify every candidate into an archetype, but categorizing candidates based on the archetype definitions was challenging—especially determining a candidate’s complexity. It will be helpful for future iterations of the model to include more fine-grained, detailed descriptions.

Third, the assumptions on costs, success rates, and cycle times per phase for each archetype were based on taking averages from a large number of data points provided by industry and PDPs across a range of diseases. In the model, the averages for each archetype are the same for every disease. For example, the costs, success rates, and cycle times per phase for developing a simple vaccine for schistosomiasis are the same as those for developing a simple vaccine for hepatitis C. Thus the model does not reflect differences that there may be between diseases when it comes to product development cost structures, success rates, and cycle times. In addition, the data that we used to develop the assumptions, provided by industry and PDPs, were not just on neglected diseases—they were on development candidates for multiple infectious and non-communicable diseases, including cancer therapies. Pooling from across a huge number of data points is likely to have led to more robust assumptions, but the inclusion of data from all different types of products (e.g., cancer drugs) may have led to estimates that do not reflect neglected disease product development alone. Furthermore, for some archetypes (e.g. biologics, diagnostics, vector control products) there were few available data on development costs and success rates to inform the model assumptions; greater availability of such data would be helpful for further refining the P2I model.

Fourth, the model does not include all phases of product development R&D—it excludes the costs of early preclinical development (drug discovery, basic research) and of regulatory review and marketing authorization. Mestre-Ferrandiz et al., in The R&D Cost of a New Medicine, and Di Masi et al.⁸ have shown that the pre-clinical costs, including discovery, can be substantial. For example, DiMasi et al.’s estimate of the capitalized cost (i.e. cost including opportunity costs) of $2.6B to the point of marketing approval to develop an NCE comprises $1.1B in pre-clinical and $1.5B in clinical costs. The cost of the regulatory approval stage may represent up to 5.7% of the total R&D cost²⁰. The P2I model does not include other types of critical research that are needed to develop new products for neglected diseases, such as developing appropriate animal models, or to bring new products to poor populations, such as policy and implementation research. In addition, our estimate of costs does not include opportunity costs. There has been a contentious debate about the merits of including these costs—for example, Angell argues that cost estimates should not be capitalized because drug companies “are not investment houses” and they “have no choice but to spend money on R&D if they wish to be in the pharmaceutical business²².” Nevertheless, the literature on R&D costs tends to include opportunity costs, and they can be large (as previously mentioned, Di Masi et al.’s $2.6B estimate includes $1.2B in opportunity costs).

However, we note that these are commercial estimates and many costs—such as the location of production—are strategic in nature and so it is much harder to derive an average cost. In the area of neglected disease R&D, the PDPs challenge these commercial costs and their own cost estimates are much lower. For example, DNDi, the Drugs for Neglected Diseases initiative, a non-profit PDP, estimates that it has spent about $39-52 million per NCE, a figure that adjusts upwards to $130-195 million when risk of failure is taken into account. Most of the supporting basic research and pre-clinical research for neglected disease R&D is publically funded as a public good and may serve many different products. Therefore, trying to develop a meaningful average cost between for-profit and not-for-profit in the later development stages was not feasible and such an average was purposefully excluded in the P2I tool. Given all of these excluded costs, the P2I model was never intended to provide a full price tag; instead, the aim was to offer the first tool to provide an evidence-based method for R&D cost comparisons where there are historical data to draw on. The intention is to inform prioritization and decision making at a global level and not to price individual product development.

It is worth repeating that the P2I v.1 and v.2 models examine the total pipeline of all candidates, with no judgements made on “go/no go” at each stage gate. In reality a portfolio would be managed and ranked with only the most promising candidates likely to be moved on to a subsequent phase. The type of decision required would be based on potential public health impact and feasibility of success. As these are products that are aimed at poor populations, where the market is largely absent, other considerations such as access and affordability will also influence the decisions on a go/no go basis¹¹.

Fifth, the model is “agnostic” when it comes to the public health value of the estimated launches—it cannot judge their clinical utility. For example, the model estimates that there could be 27 TB diagnostics (just over a fifth of all launches). This is the type of scenario that the model was intended to highlight, in order to stimulate debate about what is an appropriate approach moving forwards. Despite this high number of launches, the large unmet need for TB diagnostics may still not be addressed as almost all new developments are of an incremental rather than transformational nature. While some of the incremental developments in the pipeline might be of benefit to address different needs in different epidemiological and geographic settings, others might be redundancies. In addition, the quality and performance of the launched assays may vary widely, and therefore they may not meet public health needs. And, as mentioned, this large estimate may also reflect the underlying assumptions of high success rates in both the design and development phase and the clinical validation and launch readiness phase.