In countries where routine vital registration data are scarce, Demographic Surveillance Sites (DSS: locally defined populations under longitudinal surveillance for vital events and other characteristics) and Demographic and Health Surveys (DHS: periodic national cluster samples responding to cross-sectional surveys) have become standard approaches for gathering at least some data. This paper aims to compare DSS and DHS approaches, seeing how they complement each other in the specific instance of child and adolescent mortality in Ethiopia.
Data from the Butajira DSS 1987–2004 and the Ethiopia DHS rounds for 2000 and 2005 formed the basis of comparative analyses of mortality rates among those aged under 20 years, using Poisson regression models for adjusted rate ratios.
Patterns of mortality over time were broadly comparable using DSS and DHS approaches. DSS data were more susceptible to local epidemic variations, while DHS data tended to smooth out local variation, and be more subject to recall bias.
Both DSS and DHS approaches to mortality surveillance gave similar overall results, but both showed method-dependent advantages and disadvantages. In many settings, this kind of joint-source data analysis could offer significant added value to results.
The acute shortage of reliable population-based health data in the world's poorer countries, where complete and compulsory registration of vital events and other health data is currently impracticable or unaffordable, have led to other strategies for obtaining at least some data. Two major approaches in current use are Demographic Surveillance Sites (DSS) and Demographic and Health Surveys (DHS).
A number of DSSs have been established, mainly across Africa and Asia, the majority of which are affiliated to the Indepth Network . Although there are local and contextual variations in the design of individual DSSs, which generally operate autonomously, the basic concept involves the identification of a geographically defined population, typically of 30 to 100 thousand people. An initial census of the defined population forms the basis for an open cohort that can be followed longitudinally. New individuals can enter the cohort by birth or in-migration, while cohort members can exit by out-migration or death. Alongside surveillance of vital events (birth, migration and death), other health-relevant parameters such as household characteristics, individual characteristics and experiences of health and disease are routinely collected. Established DSSs can also offer unique opportunities as platforms for interventions, such as vaccine trials.
The DHS programme is a global U.S. Government-backed initiative to undertake relatively large and complex cross-sectional surveys of demographic and health parameters on nationally representative samples in poorer and middle-income countries . The underlying methodology for this has been standardised, apart from small details relating to country-specific requirements, and both reports and datasets from particular DHS rounds are made available via the internet. Nationally distributed cluster samples of households are taken as the basis for a DHS round. DHSs are repeated at approximately 5-year intervals in many countries around the world, with each round drawing a fresh cross-sectional sample.
In Ethiopia, the Butajira Rural Health Programme (BRHP) is a DSS that has been in place since 1987 and has accumulated over 700,000 person-years of surveillance data. It is located in the Southern Nations, Nationalities and Peoples Region (SNNPR), some 130 km to the south of Addis Ababa. This district was purposefully chosen as being potentially representative for a DSS in 1986, on the basis of being at least 100 km from any major city, but not in a peripheral border region; combining a mixture of the highland and lowland environments typical of Ethiopia; and containing a mixture of ethnic and religious groups. National DHS sample surveys have been undertaken in Ethiopia in 2000 and 2005.
The aim of this paper is to review child and adolescent mortality outputs from both the BRHP DSS and the Ethiopian DHS data from the 2000 and 2005 rounds, comparing findings within areas of common ground in an attempt to see how the two different approaches complement and speak to each other.
The BRHP DSS dataset started with an initial enumerated cohort of 28,580 persons resident in 10 communities (9 rural, 1 urban) in the Butajira District on 1st January 1987. The sampled communities were selected from the entire district using a probability proportional to size approach and covered approximately 10% of the district. All of the households have since been visited regularly to record vital events (births, deaths, in and out migrations) and the dataset describing this dynamic cohort up to 31st December 2004 was used for this analysis. By this time the resident population had grown to 54,419 .
From this dataset, records relating to all individuals who spent any time between birth and their twentieth birthday as residents in the BRHP cohort were extracted, and compiled into a format that included person-time of exposure according to age group (under 1 year, 1 to 4 years, 5 years and over), sex and urban/rural residence. A total of 68,979 individuals were included, each contributing an average of 6.0 person-years. Deaths were recorded for 6,345 individuals.
The DHS 2000 and 2005 datasets for Ethiopia both included detailed birth histories taken from women in the household sample, which included date of birth for ever-born children and date of death for those no longer alive at the time of interview. In 2000, interviews were completed for 15,367 women aged 15 to 49 years, and in 2005 14,070 women were interviewed [4,5]. The date when women took up residence in their current household was recorded, as well as the interview date. Thus it was possible to construct a pseudo-cohort dataset with a record for each child, covering the same period (1987 to 2004) and same age range (0 to 19 years) as the BRHP dataset. Individual child records from the 2000 and 2005 DHS surveys were allowed to overlap in time, given that the two samples were drawn independently. Person-time for each child started at the latest of: 1st January 1987, date of birth or date of mother's residence starting. Person-time ended at the earliest of: date of death, date of interview, twentieth birthday or 31st December 2004. Variables within the dataset were the same as those extracted in the BRHP DSS dataset. A total of 75,443 children were included, each contributing an average of 7.2 person-years. Deaths were recorded for 10,517 individuals.
Both the DSS and DHS datasets were incorporated into Poisson regression models (using Stata software) in which individual person-time was the exposure variable and death the outcome measure.
The BRHP DSS dataset included 6,345 deaths among 413,965 person-years in the age range 0 to 19 years during the period 1987 to 2004, a crude mortality rate of 15.3 per 1,000 person-years. The corresponding national DHS dataset included 10,517 deaths among 544,598 person-years, a crude rate of 19.3 per 1,000 person-years. Within these national data, the subset for SNNPR amounted to 1,848 deaths in 84,058 person-years, a crude rate of 22.0 per 1,000 person-years. Table 1 shows a breakdown of deaths and person-time by age group, sex, residence, year and region for both DSS and DHS data.
Table 1. Deaths and person-time observed for BRHP DSS (6,345 deaths among 413,965 person-years aged under 20) and Ethiopian DHS data (10,517 deaths among 544,598 person-years aged under 20).
A Poisson regression model of mortality rate ratios was built, including year, age group (under 1 year, 1 to 4 years and 5 to 19 years), sex, urban/rural residence and, for the national DHS data, region. Mortality rate ratios for the BRHP DSS and national DHS data, adjusted for age group, sex, urban/rural residence and, in the case of DHS, for region, are shown in Table 2.
Table 2. Adjusted mortality rate ratios for BRHP DSS (6,345 deaths among 413,965 person-years aged under 20) and Ethiopia DHS data (10,517 deaths among 544,598 person-years aged under 20).
Changes in adjusted mortality rate ratios over time for the BRHP DSS data and the national DHS data are shown in Figure 1, taking DHS data for 1987 as the reference group (mortality rate 33.7 per 1,000 person-years). The overall adjusted mortality rate ratio for the national DHS data in comparison to BRHP DSS was 0.93 (95% CI 0.90 to 0.96) (adjusted for age-group, sex, urban-rural residence and year), and for the DHS data from the SNNPR region 0.95 (95% CI 0.90 to 1.00).
Figure 1. Adjusted mortality rate ratios (aged under 20, adjusted for age group, sex, and urban/rural residence) by year for BRHP DSS data and Ethiopian DHS data, taking DHS data for 1987 (mortality rate 33.7/1,000 person-years) as the reference group. Vertical bars indicate 95% confidence intervals.
Comparisons of the DSS and DHS results for infant and cumulative under-5 year child mortality are shown in Figure 2. DHS figures from the 2000 and 2005 rounds are shown as presented in DHS reports (and thus using standard DHS methods for weighting, etc.) [4-6], compared with adjusted values based on the Poisson regression model described above (combined 2000 and 2005 DHS and the 18-year DSS data). Direct contemporaneous comparisons of DHS mortality shown in Figure 2 between the two survey rounds, with an additional 5 years of recall, suggest that the effects of recall bias may have led to underestimates of infant and under-5 mortality by between 14% and 27% over 5 years.
Figure 2. Cumulative under-5 year child mortality and infant mortality rates from Ethiopian 2000 and 2005 DHS and BRHP DSS data, in 5-year periods from DHS survey rounds. DHS 2000 and 2005 figures are taken from DHS reports [4,5]; DHS adjusted figures from a Poisson regression model of combined 2000 and 2005 DHS data; and BRHP DSS results from the same Poisson regression model, adjusted for age-group, sex and urban-rural residence.
Overall patterns of child and adolescent mortality as measured longitudinally by the BRHP DSS and cross-sectionally in the 2000 and 2005 DHS rounds in Ethiopia are closely comparable. After adjustment for age group, sex, urban/rural residence and year, the overall rates are within 10% of each other, and the effects of age group and sex are similar. One cannot exclude the possibility that these similarities are entirely coincidental, but, given the purposeful initial selection of Butajira as a district that could be nationally representative, such coincidence is unlikely. Nevertheless, Table 2 reveals many small but statistically significant differences, made possible as a result of the relatively large overall samples.
It is clear that urban or rural residence is an important factor for child mortality, although it is not always a concept that is easy to define. In the BRHP DSS, the "urban" population is one of the four administrative units within Butajira town – and, although this is far from being a large, sophisticated city, the benefits of living in the town compared with adjacent villages have been consistently clear in many analyses from the BRHP DSS [3,7]. It is also clear from the DHS data that urban residence is advantageous – the Addis Ababa region has the lowest regional figures, and in the overall sample urban residents experience significantly lower mortality. The DHS sample was based on census enumeration areas (from the 1994 national census) and each of these areas was officially designated as either urban or rural.
The variation with time is a little more complex, however, and reflects important differences between the two approaches. In 1998–1999, the Butajira area experienced a local drought-related food crisis and appreciable epidemics of infectious disease .
Detailed epidemiological evaluations of this are still underway separately, but it seems that abnormal local patterns of rainfall were associated with significant outbreaks of malaria and acute diarrhoea. Lesser cause-specific epidemics were documented in 1998 and 1991 . The geographical extent of these epidemic phenomena beyond the DSS area are unknown, but it is clear that they were not reflected in the national DHS data, nor in the regional figures for SNNPR (data not shown). It remains a moot point as to whether the major epidemics in Butajira were so localised that they should not have been evident in the DHS sample, or whether the retrospective DHS approach just tends to minimise short-term fluctuations. The DSS approach seems to have more scope to detect the extent of local variations in mortality, while the DHS approach has the possible advantage of averaging out local variations across a region, or nationally. These are complementary processes, since both the likely extent of local variation and stable trends over time are both important parameters for health planning. Adequate sample size, in both cases, is important so that confidence intervals can be relatively modest in relation to the extent of variation, as is evident from Figure 1.
Figure 2 illustrates a number of advantages and disadvantages of the DHS and DSS approaches, whilst showing an overall consistency between the two approaches over time. Results from individual DHS rounds appear to under-estimate mortality declines over time, probably due to recall bias effects. Certainly the comparisons of DHS 2000 and DHS 2005 results for the periods 1991–1995 and 1996–2000 show lower figures for the longer-recall sources, and the adjusted analysis of combined DHS data shows a more substantial mortality decline than is seen in either round separately. Since each successive DHS round draws a fresh sample, it is not possible to evaluate the precise source of this bias (for example, missed deaths, missed birth/death pairs, incorrect time estimates for earlier events). This mortality decline is less obvious in the DSS data because of the local epidemic phenomena in 1996–2000.
Variation by region is also an interesting feature of the DHS results, with most inter-regional differences being statistically significant even after adjustment for other factors (Table 1). Ethiopia is a large and diverse country, but it is of interest to see a two-fold range in regional rates, after adjusting for other factors, from a rate ratio of 1.36 (Gambela) to 0.67 (Addis Ababa). If there were more local foci in the sample of sufficient size for separate analysis, one might expect to see a larger range of variation emerging as smaller areas were considered . This is a potential problem in the design of DSS operations, since the inherent need for a DSS often implies that there is little existing data with which to demonstrate that a potential area could be representative. However, these comparisons with national DHS data suggest that the BRHP DSS is nationally representative.
For the purpose of these comparative analyses, the complexities of DSS and DHS sampling design and clustering effects were deliberately ignored, taking individual children as the unit of observation. This had the advantage that exactly the same analytical approach could be applied to the data from both sources, so that the basis of comparison was not confused by methodological differences.
Table 1 reveals some important methodological differences between the DSS and DHS data. The DHS approach tends to over-sample younger children, since younger mothers who are interviewed can only contribute data on younger children. This probably accounts to a large extent for the higher crude mortality observed in the DHS data. In terms of trends over time, the DSS data shows a steady increase in person-time per year, corresponding to the growth of the surveyed population. In contrast, the DHS data yields maximum child person-time at the time of survey rounds, falling off in earlier years. Thus the timing of the survey rounds in 2000 and 2005 is clearly evident from the person-time per year. Trends in numbers of deaths per year are much more stable in the DHS data, not so much because of sample size but due to an averaging over the whole country. The use of multivariate regression modelling was therefore an important part of the comparison process, allowing adjustment for different distributions of key parameters between the DSS and DHS samples.
Although there is now a substantial accumulation of both DSS and DHS datasets from many countries, relatively few comparative analyses have been undertaken. In 1997, an early DHS round for Bangladesh was compared with the Matlab DSS with particular emphasis on fertility rates , which concluded that vital rates were comparable within the Matlab area using both approaches, but with some differences. More recently, comparisons of childhood mortality risk factors in Burkina Faso  and demographic indices in Mozambique  have been published, both concluding that DSS and DHS estimates were reasonably consistent.
Both DSS and DHS operations inevitably suffer from a lack of quality data for validation, since by definition they are implemented in locations where that lack of data is a primary problem. Although this kind of comparison by no means provides a "gold standard" for validating either method, the closely comparable results obtained from two approaches that differ widely conceptually and operationally is encouraging. The extent to which findings from DSSs can reliably and appropriately be extrapolated into wider, surrounding areas is an on-going concern, and an area in which methods are not well established. Conversely, the ability of national sample survey designs as used by DHS to elucidate local patterns and characteristics, as well as trends over time, is dubious. Thus this comparative analysis of the two approaches seems valuable, and could well be undertaken in other countries, and also for parameters other than child mortality, in order to maximise the potential of existing DSS and DHS datasets.
The aim of this paper was not to determine any kind of methodological superiority, but to explore the ways in which two different approaches to measuring child mortality might complement each other. Clearly there are significant and important ways in which the strengths and weaknesses of each method can enhance overall understanding and contribute to essential data for health planning. This kind of joint-source data analysis is to be encouraged.
The author(s) declare that they have no competing interests.
We are grateful to Macro International Inc. for making available the Ethiopian DHS data for these analyses. Butajira DSS has been funded by Sida/SAREC, Sweden, since 1987. This work was undertaken within the Centre for Global Health at Umeå University, with support from FAS, the Swedish Council for Working Life and Social Research (grant no. 2006-1512). We thank the mothers and children who contributed their personal data.
Berhane Y, Wall S, Fantahun M, Emmelin A, Mekonnen W, Högberg U, Worku A, Tesfaye F, Molla M, Deyessa N, Kumie A, Hailemariam D, Enqueselassie F, Byass P: A rural Ethiopian population undergoing epidemiological transition over a generation: Butajira from 1987 to 2004.
Public Health 2002, 116:145-150. PubMed Abstract
Asia Pacific Population Research Abstracts 1997, 11:1-2. PubMed Abstract
Hammer GP, Kouyate B, Ramroth H, Becher H: Risk factors for childhood mortality in sub-Saharan Africa: a comparison of data from a Demographic and Health Survey and from a Demographic Surveillance System.