We developed a microsimulation model of cervical carcinogenesis reflecting the best currently available evidence. The model development process included definition of model structure, parameterization, calibration, and evaluation of model performance. Parameter ranges for model inputs relating to natural history were defined from longitudinal cohort studies. Observed epidemiologic data on outcomes such as age-specific HPV prevalence and cervical cancer incidence prior to widespread screening were used to define targets for calibration of model parameters. Parameter values were sampled from the defined ranges, and simulations undertaken for each sampled parameter set. The goodness-of-fit (GOF) of modeled outputs resulting from each candidate set of input parameter values was evaluated using likelihood-based GOF scores calculated based on the full array of calibration targets. We identified a subset of the sampled parameter combinations that had scores that were statistically indistinguishable from that of the best-fitting set. We evaluated model performance in terms of consistency between modeled outcomes and those from large, population-based studies. The types and sources of data used in each of the three steps of model parameterization, calibration, and evaluation of performance are summarized in Table 11420212324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102. To assess the implications of our calibration approach in terms of the uncertainty in modeled outcomes that follows from empirically-calibrated input parameters, we examined the range of modeled cancer incidence reductions that could be expected across the calibrated parameter sets in simulations of screening (following current guidelines), widespread administration of an HPV vaccine, and a combination of vaccination and screening. Additional details of the model type, structure, parameterization, calibration, and evaluation are provided in the Appendix (see Additional File 1).

The stochastic microsimulation model simulates the transitions of individuals between a set of mutually exclusive health states (Figure 1). In the model, HPV infection is stratified into 5 categories: not infected; HPV-16; HPV-18; other high-risk types (category includes types 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, and 68); and low-risk types. CIN status is modeled in three categories: no CIN; CIN 1, and CIN2,3. Individual females enter the model at age 9 prior to sexual debut and remain in the model for the entirety of their lives. Transitions between health states occur at monthly intervals and depend on HPV type, age, history of prior HPV infection, type-specific natural immunity, previous treatment for CIN, and screening patterns. Each month, a woman has an age- and type-specific probability of being infected with HPV. Modeled probabilities of age-related HPV infection act as a proxy for the probability of being sexually active combined with the probability of transmission and distribution of HPV types among sexual partners. The model also has the capability of considering the indirect effects on health outcomes associated with herd immunity by modifying incidence rates based on output from a dynamic model 103. Most women with HPV infections will develop transient abnormalities reflecting productive HPV infection, and some will progress to CIN2,3. Women infected with high-risk HPV types and having persistent high-grade CIN may progress to invasive cancer, and those with invasive cancer can develop symptoms or progress to the next stage of cancer. We assume that symptomatic women with invasive cancer receive stage-specific treatment for their disease and are subject to the corresponding stage-specific survival rates. From every health state and in every month, women face competing mortality risks from all other causes.

We derived initial estimates and ranges for monthly transition probabilities required by the model from published literature (Table 2). Because of uncertainty about the estimated quantities within each study and heterogeneity across studies based on methodological and population differences, the upper and lower bound of each search range were defined to be broadly inclusive of (1) study-reported confidence intervals when available; (2) the highest and lowest estimate reported from different data sources; and (3) expert opinion.

Our approach to model calibration was based on a multi-dimensional random search in which input parameters values were sampled from plausible ranges defined as described above, and then separate simulations of the model were run using each set of sampled parameter values.

We evaluated the performance of the model in terms of its external consistency and face validity by comparing model output to data from several large U.S. screening studies not used in the parameterization or calibration of the natural history model. These included the ASCUS/LSIL Triage Study for Cervical Cancer (ALTS) and Portland Kaiser Permanente studies as well as data from the NCI's Surveillance Epidemiology and End Results (SEER) Program 959697. For all evaluation targets, we derived both point estimates and 95% confidence intervals from the empirical data.

Baseline data from the Portland Kaiser Permanente study were used to calculate the cross-sectional, age-specific prevalence of high-risk HPV and of cervical cytology results that were high-grade squamous intraepithelial lesion or worse (HSIL+). Baseline data from the ALTS study were used to calculate the cross-sectional proportions of HPV-positive persons with CIN1 and CIN2+ (e.g., histology of CIN2 or worse) having HPV-16, HPV-18, or other high-risk HPV types. Data from SEER for 2003 were used to derive incidence of age-specific invasive cervical cancer detected in the presence of cervical cancer screening, and analogous estimates were calculated from IARC data for the U.S. prior to widespread screening. We estimated the incidence reductions due to screening by subtracting SEER incidence rates from IARC rates by age. SEER data from 1996 to 2000 were used to calculate the proportion of invasive cervical cancer cases detected at each SEER historical stage (local, regional, and distant).

To compare consistency of model results to our external evaluation targets, we simulated 5 screening scenarios for each of the 50 parameter sets from the calibrated natural history model. These included no screening and screening using cervical cytology at 4 levels of intensity: every 1, 2, 3, or 5 years from ages 18 to 70. Additional assumptions about screening, diagnosis, and treatment are provided in the Appendix (see Additional file 1). Model outputs for these simulations were based on cohorts of 1,000,000 women.

Because data from these studies were derived from women with different past patterns of screening, we combined modeled outputs from our 5 screening scenarios to produce modeled outputs that would be consistent with a cohort of women whose cervical cancer screening patterns matched nationally observed, age-specific patterns of screening 98102. For the ALTS targets, age-specific modeled outputs were then collapsed across age categories using a weighted average based upon the age structure of the ALTS data. Since SEER stage at cancer detection targets were based on women with detected cervical cancer, matched modeled outputs (e.g., the number of cervical cancer cases detected at each stage for each screening scenario) were combined using screening patterns derived from case-control studies of women with diagnosed cervical cancer 99100101. We assessed model performance in terms of its external consistency and face validity using a benchmark of overlap between the model range and the study's confidence interval or range. Similarity in age patterns between modeled and study outcomes based on visual inspection was used as a further criterion for assessing model performance.

A key motivation of this study was to better reflect parameter uncertainty critical to policy analyses, for example, evaluations of cost-effectiveness of vaccination and screening interventions to prevent cervical cancer. While a full consideration of alternative prevention strategies is beyond the scope of this study, we provide an illustrative example by using the empirically calibrated model to project the reduction in cervical cancer that would be expected with HPV vaccination and cytology-based screening at different frequencies. We assumed the prophylactic HPV vaccine would be given to all women prior to age 12, and assumed complete, lifelong protection against HPV-16 and -18. For each of the 50 resampled good-fitting parameter sets, we estimated cumulative cervical cancer incidence with and without vaccination or screening and examined the distribution across parameter sets, of reductions in cervical cancer incidence, expressed as percentages. The distributions of cancer incidence reduction under alternative prevention strategies illustrate how calibration of model input parameters to epidemiologic data can be used to quantify the uncertainty around results of policy interest.

Across the 50 good-fitting parameter sets, we calculated the pair-wise correlation coefficients for expected cervical cancer reduction due to screening at different frequencies, vaccination without screening, and screening at different frequencies and vaccination used in combination. For this analysis, the unit of observation was the parameter set. The significance (p < 0.05) of the correlation was assessed using a Bonferroni correction for multiple comparisons. The aim of the analysis was to evaluate relationship of benefit derived from screening and vaccination used alone or in combination: specifically, whether parameter sets in which screening produced more benefit also showed more benefit from vaccination and whether the benefit from screening used at different frequencies combined with vaccination was correlated within parameter sets.

The microsimulation was implemented in C++ on high-performance Linux computer clusters. Analyses of results were performed with Stata/SE 9.2 for Windows (StataCorp LP, College Station, TX) and Microsoft Office Excel 2003 SP2 (Microsoft Corporation, Redmond, WA).

For 1,000,000 randomly generated input parameter sets, approximately 1,000 input parameter sets had GOF scores that were statistically indistinguishable from that of the best-fitting set (at p < 0.05). Of these input parameter sets, 183 met the additional criterion of being in the top 1 percent of fits for the high-priority subset of calibration targets. From the parameter sets accepted based on these two criteria, 50 were randomly resampled for further analyses and evaluation. Table 2 shows the results of calibration on the range of model input parameters. Further details on the results on calibration are documented in the Appendix (see Additional file 1).

Figure 2 and Figure 3 compare the calibration targets to the range of model outputs before and after calibration. The process of calibration improved model fit for all calibration targets. The greatest improvement in model fit is noted in age-specific prevalence of CIN2,3 and age-specific incidence of invasive cervical cancer. Observed durations for low-risk HPV in all ages, for HPV-16 in women younger than 30 years, and for HPV-18 in women 30 years or older are overestimated for some parameter sets, but the pattern of the increased duration for higher risk HPV types within age categories is consistent with calibration targets. Age-specific low-risk HPV prevalence is generally low for women under 30, and CIN1 prevalence is low for 25–29 year-olds. HPV type distribution within neoplasia and cancer category, age-specific prevalence of high-risk HPV and CIN2,3, and age-specific incidence of invasive cancer demonstrate consistency between modeled outcomes and calibration targets.

We compared the model-predicted screening outcomes across the spectrum of HPV infection and cervical cancer to selected studies not used in parameterization or calibration of the model. Figure 4A shows the 95% confidence intervals for age-specific high-risk HPV prevalence among women enrolled in the Portland Kaiser Permanente study compared with modeled outputs for women whose screening intensities were derived from national averages. Model output is consistent with the general shape of decline in high-risk HPV prevalence for older ages. For some parameterizations, the model produces lower estimates of age-specific prevalence than those from the Kaiser sample. Figure 4B shows the 95% confidence intervals for age-specific proportion of cytology results that were HSIL or worse among women enrolled in the Portland Kaiser Permanente study compared with modeled outputs for women whose screening intensities were derived from national averages. Modeled outputs fall within their respective confidence intervals for all age ranges.

Figure 4C and 4D show the 95% confidence intervals for the proportions of CIN1 and CIN2+ infected with either HPV-16 or HPV-18, or only with other high-risk HPV types, compared with modeled outputs for women whose screening intensities were derived from national averages. Model output is consistent with the general trend of an increasing proportion of neoplasia being infected with HPV-16 or HPV-18 as the severity of neoplasia increases. Although modeled ranges overlap the 95% confidence intervals of the ALTS study, they are wider and more variable for CIN1. For CIN2+, model output is more consistent with the study data.

Figure 5A shows the range of differences between observed age-specific detected cervical cancer incidence in the presence or absence of screening (from SEER 2003 and IARC 1959–1960, respectively) compared with the modeled outputs for women without screening and those women whose screening intensities were derived from national averages. Ranges from observed and modeled data overlap for all age categories. Even though the modeled ranges overlap for women above 60 years of age, the modeled reduction in age-specific detected cancer incidence tends to be lower than observed values for this age group. Figure 5B shows a comparison of modeled distribution of stage at detection among women with detected invasive cervical cancer to matched SEER data (1996–2000). Model output is consistent with the SEER cancer stage data.

Figure 6 summarizes the distribution across 50 good-fitting parameter sets of cumulative cervical cancer incidence reductions with an HPV-16/18 vaccine, with current screening guidelines carried out at different frequencies, and with combined vaccination and screening. Reductions range from 60 to 88% for HPV vaccination. While the expected reduction in cancer incidence is 75%, parameter uncertainty implies a substantial range of possible benefits: more than 20% of the parameter sets yield reductions in incidence that differ from the mean reduction by more than 10 percentage points. Screening reduces cancer by 76% (range across all 50 parameter sets: 69–82), 69% (60–77), 62% (51–71), and 50% (39–61) for frequencies of every 1, 2, 3, and 5 years, respectively. Model results under frequent screening are less susceptible to parameter uncertainty than less frequent screening or vaccination alone. The range of cancer reduction from screening more frequently than every 3 years exhibits substantial overlap with vaccination. When screening and vaccination are combined cancer incidence reductions are 94% (90–97), 93% (89–97), 91% (86–96), and 89% (83–95) for frequencies of every 1, 2, 3, and 5 years, respectively. Uncertainty is further reduced under combined screening and vaccination programs, and less frequent screening combined with vaccination has expected benefits greater than vaccination alone or yearly screening.

Across parameter sets, estimated cancer reductions associated with screening at different frequencies are significantly positively correlated (Table 3); in other words, those parameter sets yielding high benefits for a given screening frequency also typically indicate high benefits for other screening frequencies as well. This high correlation suggests that factors that influence the effectiveness of screening programs at a variety of screening frequencies (e.g., duration in CIN2,3 prior to progressing to invasive cancer or the proportion of HPV infections of different types leading to CIN) are captured within parameter sets. There is no significant correlation between cancer reductions due to vaccination alone and those due to screening alone at any frequency. While screening strategies achieve benefit by detecting and removing CINs, some of which may have progressed to cancer, vaccination acts to prevent type-specific HPV infection. The pattern of correlation observed is consistent with the observation that uncertainty about HPV infection and clearance is loosely connected to CIN progression and regression. More importantly, vaccination combined with screening provides positive synergy for cancer prevention as evidenced by earlier results that screening alone and vaccination alone both produce substantial cancer reduction, screening alone and vaccination alone are not correlated, and the positive, significant correlation in cancer reduction as screening frequency is increased in the presence of vaccination.