THE NEED FOR ONTOLOGIES IN THE BEHAVIORAL SCIENCES

Ontologies provide a structure for the entities in a domain: they articulate conceptualizations, or descriptions of the nature of the ideas under study, and connections among those concepts, such as a set of relationships. They are a means of reliably specifying and classifying behavioral phenomena; providing a controlled vocabulary for discussion of research; and identifying the inconsistent use of definitions, labels, relations among entities, and measures. Thus, they provide the basis for sharing knowledge about the entities in a domain across diverse approaches and methodologies. In addition, ontologies facilitate the generation, curation, dissemination, and retrieval of knowledge by supporting the codification of research findings in computer-readable formats (Larsen et al., 2017). Thus, ontologies can address problems with research itself and with its application to real-world problems.

A wide variety of stakeholders rely on the knowledge created by the behavioral sciences, including, just in the domain of mental health, scientists who study behavior, social science, psychology, development, and cognition; and clinicians who provide educational, behavioral, social, and psychological interventions; as well as educators, health care practitioners, policy makers, and patients. One in five adults experiences a mental condition in a given year, and the quality of their care depends on both the availability of research on mental disorders and the capacity of clinicians to distill relevant information from the massive volume of research published every year. For new knowledge to benefit patients, clinicians, and communities, it must be tested and reproduced, and the findings must be integrated with other knowledge, synthesized, generalized, disseminated, and applied. The overwhelming cascade of new research makes it difficult to synthesize results, and ontologies offer an infrastructure for systematically organizing and sorting research findings in particular domains so that key developments can easily be discerned by those who rely on them.

The absence of ontologies also undermines research itself; a lack of ontological clarity is at the root of many significant challenges faced by behavioral scientists. Scientists’ work is shaped by their understanding of the concepts and entities they are studying and how they are categorized, decisions about ways to accurately measure the phenomena of interest, and decisions about what is and is not germane to their research investigations. Today, the mapping or documentation of relationships among measures, terms, and entities remains fragmented and scattered across many fields within the behavioral, social, and cognitive sciences. These problems are manifested in poor generalizability of many research findings, which results in behavioral science research findings that do not build cumulatively to the degree that they could. Ontological specifications that are interoperable—useful across applications because they provide formal consistent, computable, and readily re-purposed specifications—can yield cost savings, efficiencies, and opportunities across research needs and applications.

We could not find clear empirical evidence that the absence of publicly available, agreed-upon ontologies has limited progress in the behavioral sciences or that deficiencies in ontologies and common vocabularies by themselves have limited cumulative progress in these fields. It is nonetheless clear from our review that the behavioral sciences are not characterized by robust ontologies, and that the lack of ontologies can hinder scientific progress. To remind the reader of just one example (from Chapter 3), the Behaviour Change Intervention Ontology (BCIO) was designed to address the problem posed by theories that were overlapping and underspecified, often sharing constructs with other theories, using different names for the same constructs, measuring the same constructs using differing items, and inadequately defining constructs and relationships.

We reviewed a number of existing ontological systems in the behavioral sciences, including ones that might fall at various points along a continuum of semantic formality (discussed in Chapter 3) and some that we were not able to clearly characterize in those terms. It may be that some domains of the behavioral sciences have more to gain from a focus on ontology development than others. Nevertheless, while existing ontological systems have served valuable purposes, taken together they have not exploited the large potential for ontologies to accelerate advancement and application of behavioral research. Our scoping review and examination of example ontological systems indicated that there are comparatively few semantically formal behavioral science ontologies. Many of the systems that have been developed to support research are not likely to bring the full benefits that a clear ontology can offer, and the systems that appear to have been most explicitly designed with ontological goals in mind (e.g., BCIO, Cognitive Atlas) are not widely used. A systematic exploration of why this is so was beyond our reach: the available literature provides few generalizable insights, and our investigation of the examples we did look at suggested that the reasons are complex.

CONCLUSION 3-1: Classification systems in the behavioral sciences lie on a continuum of semantic specification. Systems that fall along this continuum serve ontological purposes that are scientifically valuable.

CONCLUSION 3-2: The classification systems that currently are widely used in the behavioral sciences do not have formal semantics, and therefore they do not readily provide opportunities to support automated reasoning and other artificial intelligence applications.

CONCLUSION 3-3: While ontological systems with the most formal semantic specification offer the greatest opportunities for accelerating the behavioral sciences through the use of artificial intelligence, it is not the case that the continuum represents a hierarchy of quality. The most important characteristic of an ontological system is that its level of formal specificity fits its intended purpose.

CONCLUSION 4-1: By establishing a controlled vocabulary of shared terms for the concepts and phenomena of interest in a particular domain and a classification of those entities, ontological systems have three primary benefits:

• They open up opportunities to improve care and services, based on the work of investigators studying disorders who use a common language, shared measures, and the same logical structure for designing their specific studies.
• They provide an infrastructure to support the mechanics and application of contemporary scientific research, helping to ensure that conclusions drawn from the data are justified, the procedures used to create the data are replicable, and new discoveries buried in the data do not go undiscovered; framing communication between people and machines; easing the interpretation of complex datasets; and making scientific data an enduring and available resource for all.
• They create enhanced capacity to expand scientific knowledge, providing a foundation for thought, hypotheses, and understanding of new discoveries.

STRENGTHENING ONTOLOGY USE IN THE BEHAVIORAL SCIENCES

What would it take to achieve the potential benefits we have described? We believe that behavioral science can learn from other fields, such as cancer research and anatomy, where formal ontologies have been developed and currently serve the function of standardizing information and organizing knowledge. The U.S. research community can also learn from the ways ontology development and use are funded and supported in other countries; though reviewing international approaches was beyond our scope, we are aware that researchers outside the United States have been leaders in ontology development.

In attempting to understand what would be needed to engineer ontologies that could better support behavioral science, we examined socio-cognitive practices or functions through which decisions about the terms and relationships the ontology covers are made and the computational tools that can facilitate the intellectual work.

The socio-cognitive practices involved in creating and editing an ontology and adapting it over time require intensive human community engagement, and iteration. Computer tools—including software that supports collaboration and brainstorming, makes it easier to visualize complex relationships, and facilitates sharing and disseminating ontologies—can bring extremely valuable efficiency to the development, maintenance, and editing of ontologies. Statistical methods that identify common factors and hierarchical organizations among correlated behavioral measures can also support ontology development. But these tools can never stand in for the human understanding, ingenuity, establishment of consensus, and leadership that go into the development and use of ontologies. Because of the basic need for human work, ontology development is quite expensive. There is no substitute for the raw people power that is essential for the intellectual work of designing the ontology.

CONCLUSION 5-1: Valuable ontological systems and related tools exist and are supporting research in the behavioral sciences. However, many of these efforts have been isolated, and it appears that their adoption has been constrained; that resources to support them (including training and education) have been limited; and that the developers of ontological systems are largely on their own to identify or develop the models, tools, and approaches that might best advance research and practice.

CONCLUSION 5-2: Ontology engineering rests on two foundations: socio-cognitive functions through which decisions about terms and their relationships are made and computational tools that support the overall process, providing both efficiencies and techniques for working with large bodies of data.

CONCLUSION 5-3: To provide the intended benefits, an ontology should be logically sound, valid, and usable:

• logically sound—contains no contradictions and is technically correct and concisely expressed in formal terms;
• valid—the definitions it provides accurately reflect the domain it covers as completely as possible; and
• usable by a diverse range of stakeholders, depending on its purpose, including scientists, practitioners, and others.

CONCLUSION 5-4: For ontology engineering to progress in the behavioral sciences, sustained resources and specific actions and processes are needed in three areas:

• discovery both foundational and translational research needed to develop and improve effective practices and the next generation of computational tools for ontology engineering in the behavioral sciences.
• capacity to address shortfalls in implementation and to take advantage of the cases when novel research is not required—that is, when what needs to be done is clear, but there is currently no capacity to do it.
• promotion of practices and processes that could support wider use of ontologies in the behavioral sciences, and for which the capacity is already in place, but have not been widely deployed, such as institutional incentives, open data and code, and community-level efforts to bring consensus about ontologies in the behavioral sciences through collaboration.

SUPPORTING AND SUSTAINING BEHAVIORAL ONTOLOGIES

The committee recognizes that some behavioral scientists remain skeptical of the usefulness of ontologies despite their potential benefits. Skeptics argue that ontologies may have the unintended consequence of stifling research creativity or that the imposition of a common ontology could make it more difficult for new conceptualizations to gain a foothold. The committee acknowledges that there may be complicated tradeoffs and that too much emphasis on a common ontological system could hinder originality and punish some of the unorthodox thinking that has led to major scientific advances. In assessing this concern, however, the committee notes that many scientific disciplines rely on ontologies to a far greater extent than do the behavioral sciences, and it would be hard to argue that research in those sciences is less creative than that in the behavioral sciences. What we are proposing is fundamentally consistent with long-standing reliance on constructs, construct validity, and other means of clearly articulating ideas and hypotheses for rigorous study. A greater reliance on ontologies is a way of making those efforts even more rigorous.

Moreover, the committee expects that increased use of ontologies in the behavioral sciences would involve different and sometimes parallel ontologies. The methods for developing ontologies vary, and the creation of categories depends on human judgment. Differences of opinion should be expected. Especially in the near term, “ontologic pluralism”—in which competing or overlapping ontologies exist and are connected to one another through formal mappings—is both inevitable and desirable. Without a doubt, however, developing workable ontologies is difficult, though it is necessary in any scientific domain.

CONCLUSION 6-1: Ontology development and use has the potential to move behavioral science forward from a domain in which research is generally siloed and the data and results are often incompatible to one in which the evidence is searchable and more easily integrated and in which computer technology is leveraged in the discovery of new relationships, the development of novel hypotheses, and the identification of knowledge gaps.

Taking advantage of these opportunities to accelerate the behavioral sciences with the aid of more semantically formal ontologies will require attention to the practical challenges of supporting the needed work. There are only a few examples of behavioral science ontologies that have endured. A primary—perhaps the most important—reason for this situation is that the development and maintenance of ontologies is expensive. Despite the many efficiencies afforded by computer technology, developing an ontology is a painstaking effort. Particularly within the behavioral sciences there has been a lack of sustainable resources: ontology development does not usually lead to a commercial product.

As a result, some ontological systems have been supported by national or international agencies. For example, the International Classification of Diseases has been supported by the World Health Organization because it is essential for public health surveillance. The Diagnostic and Statistical Manual of Mental Disorders has been supported by the American Psychiatric Association because it is the basis for billing for psychiatric and psychological services. The National Institutes of Health (NIH) has devoted intramural funds for ontologies and related resources, including the National Cancer Institute (NCI) Thesaurus, the Medical Subject Headings of the National Library of Medicine (NLM), and the structure that underlies ClinicalTrials.gov.

However, continuing support for nongovernmental efforts has been limited. The most widely used open-source ontology development system, Protégé, has pieced together funding from NLM, NCI, the National Institute of General Medical Sciences, the National Science Foundation, the Defense Advanced Research Projects Agency, and other organizations through limited research grants and contracts. While the project currently has short-term support from its various public and private sponsors, it is not on a sure financial footing for the future. Strategies to supplement long-term agency investments in infrastructure for research exist: see Box 6-1. Such examples are suggestive, but fee-generating structures will not be appropriate for all areas of research, and it is clear that without sustained resources for ontologies, the behavioral sciences are not likely to take advantage of what they can offer.

BOX 6-1

Example of Agency Investment.

CONCLUSION 6-2: Although ontologies are central to the advancement of science, there are no existing funding mechanisms for the development and maintenance of such systems and for the tools that support them. Sustained public and private support for the long-term development, dissemination, and maintenance of ontologies in the behavioral sciences and related tools is needed.

For the behavioral sciences to benefit from the potential advantages of well-designed ontologies, an infrastructure is also needed. In developing its recommendations about this need, the committee weighed competing perspectives about what would be most helpful. It might be reasonable to suppose that an infrastructure that could, in effect, govern the development and use of ontologies in the behavioral sciences to ensure that the effort to substantially expand their use would provide the requisite consistency and support. In that vein, the committee considered recommending a centralized structure that would provide tools for building ontologies, practice principles, training, and other resources. However, in addition to the likely prohibitive scope and cost of such a structure, we recognized that, while some in the field might welcome the clarity it could bring, others would surely reject such a top-down governance model. It is not clear how such an entity would function in a world of ontological pluralism and how such a centralized entity would manage to reconcile differences in understanding of concepts, classifications, and other complex issues in all of the behavioral science disciplines.

At the other end of the spectrum of possibilities, one could simply hope that ontologies will develop organically. The committee believes this would be unwise: that approach has yielded the current situation. Therefore, we have chosen a middle position by focusing on ways to expand available resources and incentives, to stimulate grassroots ontology development, and to coordinate efforts, with the aim of pushing for ontologies to be a higher priority in behavioral science research.

Agencies of the federal government are best positioned to provide the coordination and resources needed for this kind of activity, so we direct two broad recommendations to NIH and other agencies. Ontology development often does not fit within categories commonly supported by research awards. At NIH, the Division of Program Coordination, Planning, and Strategic Initiatives (DPCPSI) oversees cross-institute initiatives and includes offices for behavioral and social science research, prevention research, women’s health, AIDS research, tribal health, diet and nutrition research, and research infrastructure. It also includes programs that use ontologies, such as those on data science and portfolio analysis. Because of its vast reach within NIH, DPCPSI is in a unique position to demonstrate how ontologies can improve the way behavioral science knowledge is created, understood, and used.

RECOMMENDATION 1: The National Institutes of Health (NIH) and the National Science Foundation (NSF) should develop formal agendas for accelerating behavioral science research through the development and use of semantically formal ontologies. These agendas should draw on ideas generated within other scientific domains and the international scientific community and should include a range of activities:

• NIH should use its convening authority to engage experts and to develop a plan for ontology development across NIH institutes and centers. The plan should illustrate how NIH resources might be used to develop ontologies; link them to existing ontologies; and apply them in the interest of higher quality, more replicable behavioral research and improved behavioral health, including through criteria for funding research efforts.
• Within NIH, the Behavioral and Social Science Coordinating Committee should propose a plan for ontology development across NIH institutes and centers.
• The NIH Division of Program Coordination, Planning, and Strategic Initiatives should develop an ontology for classifying intramural and extramural behavioral research at NIH.
• The NSF Social, Behavioral and Economic Science Directorate should coordinate ontology development efforts with the NSF Computer, Information Science, and Engineering Directorate.
• NIH and NSF should collaborate in providing transition grants to allow ontology centers to develop business plans and distribution models that could put them on a sustainable footing.
• The National Library of Medicine should bolster the training it offers in biomedical informatics to strengthen the capacity of the people who will lead the development of the next generation of scientific ontologies.
• To avoid duplication and overlap, NIH and NSF ontology development efforts should be coordinated through the NIH Office of Behavioral and Social Sciences Research and the NSF Social, Behavioral and Economic Sciences Directorate.

RECOMMENDATION 2: The National Institutes of Health, the National Science Foundation, and other agencies that support research should seek and encourage opportunities to fund work that will support continuing progress in the development and use of ontologies in the behavioral sciences, such as research on technological supports for ontology development, the ways scientists develop and use ontologies across diverse fields, and institutional supports and structures that support ontology use in diverse fields.

RECOMMENDATION 3: The Office of Science and Technology Policy should develop a report on how an explicit formal specification of a shared conceptualization for behavioral science can be implemented across federal science agencies, based on review of ontologies developed by other agencies including, but not limited to, the National Science Foundation; the Departments of Health and Human Services, Defense, Transportation, Agriculture, Labor, and Justice; the Environmental Protection Agency; the National Institute of Standards and Technology; the National Oceanic and Atmospheric Administration; and the Defense Advanced Research Projects Agency.

Professional organizations and publishers also have a key role to play, and we direct recommendations to such organizations. The Federation of Associations in Behavioral and Brain Sciences (FABBS) and the Consortium of Social Science Associations (COSSA) represent most professional and scientific organizations in the behavioral and social sciences. FABBS promotes advancing the sciences of mind, brain, and behavior, and its mission includes training and fostering communication among scientists. The COSSA membership includes professional associations, scientific societies, research centers and institutes, colleges and universities, and industry affiliates.

Similarly, there are two major academic publishers in the behavioral sciences. The American Psychological Association is the largest publisher of behavioral science journals, and the Association for Psychological Science is also a leader in scientific publication. In addition, both organizations work with the Council of Graduate Departments of Psychology to coordinate and provide accreditation for educational programs and in psychology. We call out these organizations because they have wide reach, but we hope that similar organizations will also take part in the community building necessary to develop and encourage understanding of what ontologies can offer in the behavioral sciences.

RECOMMENDATION 4: The Federation of Associations in Behavioral and Brain Sciences and the Consortium of Social Science Associations, along with similar organizations, should coordinate ontology development across academic and professional organizations.

RECOMMENDATION 5: The American Psychological Association Council of Editors and the Association for Psychological Science editorial office, along with similar organizations, should develop policies to improve the use of common vocabularies and data-reporting standards in behavioral science journals.

RECOMMENDATION 6: The Council of Graduate Departments of Psychology, the Education Directorate of the American Psychological Association, and the Education Office of the Association for Psychological Science, along with similar organizations, should create strategies to integrate ontology development into graduate-level teaching and practical training.

The goals of our recommendations are to strengthen approaches to categorizing and defining the concepts and phenomena behavioral scientists study and to develop ways to better leverage contemporary technologies in structuring knowledge about human behavior. These ideas build on what has been accomplished through centuries of attempts to synthesize what is known, as well as decades of research on human and animal behavior. The approaches we recommend have the potential to democratize knowledge about human behavior by making that knowledge efficiently retrievable and actionable by the wide diversity of stakeholders in the domain of the behavioral sciences. Ultimately, better communications within the scientific community and between scientists and knowledge consumers will improve the science of behavior, the way it is disseminated, and its capacity to ameliorate and prevent suffering. This report is focused on the behavioral sciences, but most of the issues discussed here would also apply in other domains, and the committee hopes they will be of use in the overall advancement of science.

REFERENCES

• Cella D, Hays R. Commissioned paper prepared for the Committee on Accelerating Behavioral Science through Ontology Development and Use, National Academies of Sciences, Engineering, and Medicine. 2021. Ontological Issues in Patient Reported Outcomes: Conceptual Issues and Challenges Addressed by the Patient-Reported Outcomes Measurement Information System (PROMIS^®) https://nap​.nationalacademies​.org/resource​/26464/Cella-and-Hays-comissioned-paper.pdf .
• Larsen KR, Michie S, Hekler EB, Gibson B, Spruijt-Metz D, Ahern D, Cole-Lewis H, Ellis RJ, Hesse B, Moser RP, Yi J. Behavior change interventions: The potential of ontologies for advancing science and practice. Journal of Behavioral Medicine. 2017;40(1):6–22. https://doi​.org/10.1007​/s10865-016-9768-0 . [PubMed: 27481101]