STRENGTHS

If the NCS is conducted as proposed, its strengths would include:

1. Responsiveness to the Children’s Health Act of 2000 The statedgoals for the NCS, and the design of the NCS for achieving those goals, broadly reflect the stipulations of the Children’s Health Act.
2. The large number of births to be included 100,000 births wouldprovide enough statistical power to examine many hypothesized relations that cannot be investigated with smaller samples.
3. The longitudinal design stretching from before birth until age 21 Adata set that contains data gathered prospectively over the entire course of pregnancy, childhood, adolescence, and early adulthood will enable many new life-cycle relations between exposures and outcomes to be investigated. Data gathered prospectively (or with relatively short retrospective periods) should be more precise than data that are based on long periods of recall. A particularly attractive feature of the study is the effort to recruit births before conception and during very early periods of gestation, when certain environmental exposures may prove to be critically important.
4. The many variables to be generated on both outcomes and exposures The enormous array of social, psychological, biological, chemical, and physical measures that will be generated under present plans will permit investigation of relationships that have not previously been studied. Some of these relationships are included among the study’s hypotheses, but fortuitous and unanticipated findings can also be expected. At the same time, the large number of variables increases the risk of establishing “false positives,” that is, relationships that appear to be statistically significant but have in fact been generated by chance. In addition to the variables generated by the study itself, many ancillary variables on characteristics of participants’ communities from readily available sources, such as the decennial census, surveys, and administrative records, can be appended to the data set to augment the analytical usefulness of the study.
5. The well-designed national probability sample The births selectedfor the NCS will be identified from a probability sample of households chosen with standard and well-justified sampling techniques. The use of established random selection methods at each sampling stage will ensure that the NCS samples of households, eligible women of childbearing age, and births are national probability samples. We endorse the study’s decision to use probability sampling without oversampling any groups. Statistical power could have been increased for any particular investigation by implementing a different sampling design. But modifying the sampling scheme to better address any particular relationship would often have reduced the power for investigating other relationships. The possibility of adding more precisely targeted studies—when resources and respondent burden permit—has been appropriately built into the research design. Nevertheless, it is important to note that the sample size and sampling scheme of the study represent a compromise and are not designed to address any single hypothesis.

WEAKNESSES AND SHORTCOMINGS

1. Absence of an adequate pilot phase A principal shortcoming ofthe NCS, as planned, is the absence of a pilot phase. The study design is extremely complex in terms of identifying subjects, enlisting their enrollment and continued participation, administering the very large number of survey and clinical instruments, and managing huge databases generated by disparate organizations. In addition, we raise many questions about the instruments that have been chosen and about the timing of their application. We think that, if the study is to achieve its promise, experimentation is needed with respect to methods to increase response rates and data instruments. Many of the concerns that we raise about the research plan could be addressed in a pilot phase.
   Data-gathering will begin at the seven centers designated by NICHD as Vanguard Centers a year earlier than elsewhere. The data that will be generated in Vanguard Center sites are expected to become part of the final data set, meaning that experimentation is likely to be kept to a minimum. Moreover, the lead time between the start of data collection at the sites in the Vanguard Centers and elsewhere appears too short to take full advantage of the lessons that will be learned. The absence of a pilot phase is a serious shortcoming that could be at least partially addressed by increasing the delay between data-gathering in the Vanguard sites and elsewhere and by treating the Vanguard sites in a more experimental fashion. A delay for the enrollment phase of the study would also allow time to consider more fully the appropriate conceptual framework and specification of hypotheses and measures for the study.
2. Decentralization of data collection Data collection will be the direct responsibility of 35-40 different study centers. While the study centers will be supervised by government officials and the coordinating center, they will individually contract with outside agencies for data collection. This unusually decentralized data collection strategy reduces the chances that data will be of uniformly high quality over the life of the study and sharply increases the burden of supervision. More centralized and conventional models, such as that employed by the large National Longitudinal Study of Adolescent Health, appear more likely to produce high-quality data. Given that the decentralized approach could not readily be altered at this stage because contracts already have been signed, it will be incumbent upon the government to ensure that staff and other resources are sufficient to closely monitor data collection activities and take prompt remedial steps as necessary. A detailed and specific contractual plan is essential for this purpose.
3. Inadequacy of plans to maximize response rates and retention rates The success of the study will depend critically on the initial surveyresponse rates and the subsequent rates of sample attrition. The NCS research plan does not explicitly address the best methods and procedures for achieving the ambitious baseline response rates that are targeted. The importance of efforts to increase initial response rates by dealing rapidly with underperforming sites cannot be overstated. Maintaining the representativeness of the sample over time is key to the quality of the results. Little is said in the research plan about how the study expects to maximize retention of sample cases. The risk of sample attrition is especially great in a study such as the NCS that targets children and young adults, the most mobile segments of the population. Typically, the largest loss to follow-up occurs in the early stages of a longitudinal study. Ascertaining the best methods to increase initial response rates and to reduce attrition rates is a matter of great urgency for project management.
4. Weakness of conceptual model The research plan does not definethe basic concepts of health and development. While mentioning them, it frequently defaults to a deficit model that focuses on disease and impairment and the risk factors that contribute to them, rather than on the factors that encourage healthy development. Late additions to the set of hypotheses that are addressed to healthy development are not well specified. Consequently, there is an imbalance of hypotheses, with specific hypotheses about disease conditions and vague hypotheses about social environments and children’s intellectual and social development. The same imbalance appears in the measures selected. Little attention has been paid to outcomes in later childhood and adolescence that might have encouraged attention to additional or alternative exposures.
5. Weakness of certain data instruments The success of the studydepends critically on the quality of the data instruments to be employed for assessment of environmental exposures and child outcomes. Our review revealed gaps, uncertainties, and insufficient rationales for a substantial number of instruments. Among the measures that appear most appropriate are those related to asthma and to biological exposures such as allergens and maternal physical activity. The list of proposed variables about which we have raised questions includes: birth defects, pregnancy outcomes (including fetal death), child mental health disorders, maternal depression, brain injuries, reproductive development outcomes, nonpersistent organic chemicals, pesticides, childhood infections, and the social environment in the home. The set of psychosocial measures selected appears particularly problematic. We hope that the issues we have raised about these measures will encourage a reconsideration and fresh articulation of the bases on which they were chosen, including the introduction of additional measures as substitutes or supplements when appropriate. We also suggest that efforts to assess the validity and reliability of the most problematic measures be made by the Vanguard Centers for their sites.
   Even when suitable measures have been selected, the timing of their deployment—as proposed in the research plan—will leave large gaps in the measured trajectories of child health and development. These gaps, particularly with respect to in-person home or clinic visits, will make it difficult to identify critical periods of exposure to various environmental agents. The period that will be most successfully studied is that of pregnancy, birth, and the first year of life because that is the period of most intense observation. At older ages, we urge more frequent measurement of key variables, at least for a subsample of the NCS participants.
6. Insufficient attention to racial, ethnic, and other disparities TheChildren’s Health Act asks the study to “consider health disparities among children,” a phrase that typically directs attention to racial and ethnic disparities and can also include language, socioeconomic, and geographic area disparities. While the study will gather a great deal of information that is relevant to understanding such disparities, the research design was not informed by a concern with understanding their basis. The absence of virtually any hypotheses about racial and ethnic disparities is striking. In particular, there is no attention to generating data on how individuals from different groups may interact with health systems, a factor whose importance has been suggested in many previous studies, nor on psychosocial experiences that differentiate among population groups.
7. Failure to adequately integrate data from medical records The useof health services is an important variable intervening between exposure and outcome. Given a particular exposure to environmental conditions, the quality of the medical services that are employed can make a large difference in outcomes. High-quality information about the use of medical services would also help to address major questions in health policy. These would include, as just noted, the potential sources of racial and ethnic disparities in health outcomes. Data on the use of health services will be derived almost exclusively from retrospective reports of parents, which are often unreliable. While they may be expensive to collect and mobilize and are themselves imperfect, records from physicians and hospitals would provide an extremely valuable and sometimes indispensable supplement to parents’ reports. Medical records could also play an important and often central role in clarifying diagnoses and identifying patterns of child development. We urge that greater attention be paid to incorporating such data.
8. Failure to plan adequately for disclosure of risk to participants As soon as data collection begins, the NCS will face questions about the circumstances under which information about a child’s health and development, as well as his or her exposure to toxic agents, should be conveyed to participants. The study plans to provide information on conditions that are “clinically relevant and actionable,” but this is not a conventional concept and needs to be defined and made operational. There is insufficient detail in the research plan about how decisions will be made about what to disclose. Some of the decisions—for example, regarding transmitting information about fetal defects encountered through ultrasounds—are urgent to make. Clearer plans must also be developed regarding what parents and children need to be told about emerging research findings.
9. Failure to plan for rapid dissemination of data We think that thepresent plan is unwise in reserving a period of time for researchers associated with the data collection phase of the study to have exclusive access to its data for analytic purposes. Such a practice slows the advance of science and violates increasingly widespread norms, including those prominent within the National Institutes of Health itself. We urge wide and rapid dissemination of the data produced by the study. The data will be used for many analytic purposes that cannot presently be anticipated. The NCS is a national study paid for by public dollars, and we think that the data and results should be made accessible to the entire research community (with appropriate protections to preserve confidentiality) as soon as practicable. Achieving rapid data dissemination will require early and elaborate organizational and budgetary planning.

CONCLUDING OBSERVATION

It is clear from our review that the NCS offers not only enormous potential, but also a large number of conceptual, methodological, and administrative challenges. In addition, funding uncertainties make it difficult to plan beyond the relatively short period for which funds have been appropriated. Like the scientists associated with the study itself, we are eager for it to succeed. We present our recommendations in the hope that, as it goes forward, the NCS will achieve its intended objective to examine the effects of environmental influences on the health and development of American children.

RECOMMENDATIONS

The panel presents recommendations in 23 areas. The recommendations are presented in the box grouped according to these areas, and keyed to the chapter in which they appear in the body of the report.