Interdisciplinary perspectives on the development, integration, and application of cognitive ontologies

We discuss recent progress in the development of cognitive ontologies and summarize three challenges in the coordinated development and application of these resources. Challenge 1 is to adopt a standardized definition for cognitive processes. We describe three possibilities and recommend one that is consistent with the standard view in cognitive and biomedical sciences. Challenge 2 is harmonization. Gaps and conflicts in representation must be resolved so that these resources can be combined for mark-up and interpretation of multi-modal data. Finally, Challenge 3 is to test the utility of these resources for large-scale annotation of data, search and query, and knowledge discovery and integration. As term definitions are tested and revised, harmonization should enable coordinated updates across ontologies. However, the true test of these definitions will be in their community-wide adoption which will test whether they support valid inferences about psychological and neuroscientific data.

Introduction

Mental processes unfold at multiple levels, from the cellular dynamics of memory consolidation to sensory and motor behaviors that engage the entire brain and body. Thus, a comprehensive understanding of the phenomena associated with cognition requires coordinated contributions from at least psychology, biology, and neuroscience (Albus et al., 2007). The last two decades have seen unprecedented steps toward this goal, with accelerated use of neuroimaging (PET and fMRI), neurophysiology (EEG), and neurogenomics, together with experimental paradigms from cognitive, clinical, and neuro-psychology (Jones and Mendell, 1999).

While these methods have produced a wealth of high-quality data about mental functioning¹, major advances will require not just more data, but new ways of integrating, processing, and understanding the data (Decety and Cacioppo, 2010; Akil et al., 2011). Recently, ontologies have emerged as a key tool to support these efforts. An ontology is a formal representation of the types of entities in a given domain, along with their properties and interrelationships. Recent projects, such as the Cognitive Paradigm Ontology (CogPO, Turner and Laird, 2012), Neural Electromagnetic Ontologies (NEMO, Frishkoff et al., 2009, 2011a,b), the Neuroscience Information Framework (NIF, (Gardner et al., 2008; Imam et al., 2012), and the Cognitive Atlas (Poldrack et al., 2011), show that ontologies can be used to link data across neuroscientific databases and other resources (e.g., Imam et al., 2012) and to facilitate meta-analyses of neuroimaging data (e.g., Laird et al., 2009; Frishkoff et al., 2011a; Fox and Friston, 2012; Turner and Laird, 2012).

The present paper summarizes these recent efforts and describes three challenges in the development and integration of cognitive ontologies. Challenge 1 is to adopt standardized definitions for cognitive processes, and integrate cognitive processes within the framework offered by a foundation (upper-level) ontology shared with other biomedical ontologies. We describe three possibilities and recommend one that is consistent with the standard view in cognitive and biomedical sciences.

Challenge 2 is harmonization of terms across ontologies. A search for “memory” in BioPortal (Noy et al., 2009) returns over 150 terms from 30 ontologies². These and other basic terms, such as “perception,” “planning,” and “emotion,” have multiple and sometimes conflicting definitions across ontologies. These conflicts must be resolved so that multiple ontologies can be combined for mark-up and interpretation of multi-modal data. In the present paper, we describe the scope of several cognitive and biomedical ontologies and suggest areas for improved coordination in their development and application.

Challenge 3 is to test the utility of cognitive ontologies for large-scale data annotation, search and query, and ontology-based analysis. While Challenge 1 is concerned with ontology development, Challenges 2 and 3 reflect an interest in ontology harmonization and application. As term definitions are tested and revised, harmonization should enable coordinated updates across ontologies. However, the true test of these definitions will be whether they support valid inferences about psychological and neuroscientific data. We conclude with our view on the future of the development and application of cognitive ontologies.

Challenge #1: Defining and Classifying Mental Processes

Understanding the relationship between mental functioning and brain activity is a unifying aim for biology, neuroscience, psychology, and related fields. Upper-level ontologies, such as the Basic Formal Ontology (BFO; Smith and Grenon, 2004), can provide domain-independent integrating classes and domain-bridging relationships that can serve as a starting point for the annotation of data arising from this difficult search for understanding. Anatomical entities such as the central nervous system are classified as objects within BFO, and BFO also allows for “fiat” object parts for the classification of entities that have been arbitrarily (but usefully) divided into sections, e.g., the nervous system is divided into the central and peripheral nervous systems, although they are physically continuous through the connections of neurons residing in each division. Brain activity is classified as a process in BFO, and the anatomical entities are related via a participation relation to those processes.

Within BFO there are three possible ways to represent the relationship between mental processes and brain processes (Frishkoff, 2012). According to the first view, mental processes are distinct from physical (e.g., brain) processes. This is a folk psychological view and represents a kind of dualism: there are physical entities, and there are nonphysical (mental) entities. In contrast, the second and third views represent mental processes as a subclass of physiological (bodily) processes. The second view characterizes mental processes as a type of bodily process that is distinct from brain processes, while the third view holds that mental processes are a subtype of brain processes.

The first view, dualism, may generate hypotheses that are difficult or impossible to test within a scientific framework and is justifiably scientifically unpopular. In contrast, the second and third views are consistent with what we know about the physiological basis of cognition. Both assert that mental processes are physical processes, while the third view goes one step further in asserting that mental processes unfold solely in the brain, not the autonomic (reproductive, digestive) system or peripheral sensory and motor areas (e.g., hands, feet). This view is consistent with much of what we know about sensorimotor functions, emotion and motivation, and more complex cognitive processes. On the other hand, if we wish to embrace a more embodied view of cognition, we could argue that interactions between the brain and body (visceral and somatic processes) are integral to cognition. Therefore, we may not wish to exclude physiological processes outside the brain in representing mental phenomena. For this reason, we recommend either the second or third view as a reasonable starting point for cognitive ontologies.

Like any complex system, cognition comprises multiple subsystems. Most researchers would agree that cognitive subsystems include at least perception, attention, short- and long-term memory, decision-making, language, and emotion (James, 1890; Poldrack et al., 2011). Consequently, any ontology of mental functioning is likely to include terms for at least these sorts of phenomena. However, many finer-grained distinctions can be drawn. Theories of cognition differ in whether and how they draw these distinctions, and in how the subsystems assumed by the theory are thought to interact and give rise to patterns of behavior.

Mental phenomena cannot be observed in the same way that behavior or brain activity can be observed. A scientific study must therefore “operationalize,” or define, a mental process with respect to a particular experimental framework. To this end, researchers have devised a variety of experimental paradigms (Turner and Laird, 2012). Each paradigm includes a set of explicit instructions for subjects to behave in certain ways (e.g., attend, then push a button) in response to different stimuli (e.g., words or faces). Outcome measures include response time and accuracy, as well as patterns of physiological (bodily or brain) activity. For example, Fliessbach et al. (2010) examined how the strength of a memory (operationalized in terms of response accuracy) depended on the way that people learn, or encode, new information. Some participants were asked to perform an orthographic (letter) judgment, while others were asked to perform a semantic (meaning) judgment. Results showed that memory recall was better for the semantic task, providing support for a familiar construct in psychology of memory, known as “depth of processing.” This study illustrates how experimental variables are used to operationally define a mental phenomenon. At the same time, we note that operational definitions rely on particular measurement methods, and these methods may not be sensitive to all aspects of the phenomenon of interest, or may reflect additional processes, e.g., so-called task demands. Therefore, converging evidence from multiple methods is essential for studies of cognition.

In this context, the BrainMap database (Laird et al., 2005) and the Cognitive Paradigm ontology (Figure 1, Turner and Laird, 2012) have been used to annotate behavioral and brain data in order to test hypotheses about brain-behavior relationships. BrainMap is a large curated repository of cognitive neuroimaging studies that can be used to explore relationships between behavior and patterns of brain activity (mainly from PET and fMRI). It encodes core experiment metadata, including the conditions, tasks, responses and measurement methods. Using this structure, experimental results can be grouped together, based on the objective similarities among the stimuli, task instructions, and measures, regardless of whether the experiment was designed to study a particular subsystem, such as memory, executive function, or attention (Burns and Turner, 2013).

FIGURE 1

FIGURE 1. A partial representation of the Cognitive Paradigm Ontology (CogPO).

The Cognitive Atlas (Poldrack et al., 2011) represents a complementary effort. Using a wikipedia-style (or “bazaar” approach; Raymond, 1999), the Cognitive Atlas captures a wide range of terminology used in cognitive science, relating operational measures to the types of mental functioning that they are designed to measure. For example, the Cognitive Atlas contains statements such as “‘pattern comparison task’³ measures ‘processing capacity”’⁴.

In clinical neuropsychology, behavioral and self-report measures (called “neuropsychological tests”) are widely used for the diagnosis and treatment of patients with psychological and developmental disorders. Neurological assessment rests on the theory that the nature and location of brain damage, either by disease or injury, can be discovered by testing patients’ cognitive functioning. The NeuroPsychological Testing Ontology (NPT⁵) has been used to characterize these tests, which include measures of cognitive abilities, such as short-term vs. long-term memory recall or executive functioning. The Neurological Disease Ontology (ND⁶) classifies entities relevant to neurological diseases, such as Alzheimer’s and multiple sclerosis. Diseases are classified and related to diagnostic criteria and to neuropathology. A key challenge in clinical and cognitive neuroscience is to understand how physiological disorders due to trauma or disease are related to cognitive deficits, such as memory loss or aphasia. We expect that harmonization of the NPT, ND, and similar resources, will support this goal.

Challenge #2: Interoperability of Cognitive Ontologies

Several cognitive ontologies have been mentioned thus far. In recent years, psychology has embraced methods and insights from the biological sciences, including molecular genetics, evolutionary biology, and physiology (Kandel, 2008). These methods show the brain at multiple orders of spatial resolution, from genes to neural maps to interacting systems. Likewise, medical research has increasingly turned to biological evidence for explanations of mental disorders, such as depression and schizophrenia (Weyandt, 2006). As a result, many bio-ontologies now also include terminology for mental processes. For example, the Gene Ontology (GO), widely used for cross-species annotation of the functions of gene products (Ashburner et al., 2000), includes terms for “cognition,” “learning,” and “memory,” among others. This inter-domain diversity of ontologies including terminology for cognitive entities is accompanied by diversity in definitions and classification.

To coordinate work across domains and promote convergence, the Open Biomedical Ontologies (OBO) Foundry (Smith et al., 2007) provides recommendations on best practices in ontology development, such as re-use of existing terms, community involvement (i.e. being “open”), and standardization of meta-data practices. In our experience, these standards have contributed to the quality and interoperability of neuroscientific and cognitive ontologies. For example, these ontologies have largely adopted BFO as a common upper ontology, and there is extensive reuse of mid-level ontologies, such as OBI, IAO, OGMS, and UBERON, as shown in Table 1. This re-use ensures that corresponding terms across ontologies share exactly the same identifier and definition, thus seamlessly harmonizing the resulting data annotations.

TABLE 1

TABLE 1. An overview of ontology efforts for cognitive processes and related domains.

A guiding principle of the OBO Foundry is that of ontological realism, which suggests that ontologies should aim “to identify the sorts of entities that exist... according to the best current scientific understanding” (Smith and Ceusters, 2010). Ontological realism is a foundation for BFO and, in our view, has effectively promoted a community-wide focus on representing scientific knowledge. Interestingly, ontological realism might be seen to be challenged by the need to represent certain mental phenomena, such as hallucination, that can reference entities that do not exist. One solution is to distinguish these processes, which can be viewed as non-canonical, from canonical perception, in which perceptual representations are triggered by sensory stimuli. Perception is relational in the sense that it brings the subject into contact with an external object in the world (Mulligan and Smith, 1986). Just as the Foundational Model of Anatomy (FMA, Rosse and Mejino, 2003) contains terms for arm and leg, but not for amputation stump or shriveled arm, ontologies that represent canonical mental processes would contain terms for visual and aural perception, but not for synaesthesia or tinnitus. Terms like these would belong in extension ontologies relating to specific types of mental or neurological disease (Ceusters and Smith, 2010). By focusing on the canonical case, perceptual processes (for example) can safely be defined as representing objects and involving beliefs about those objects. A next challenge, then, is to capture the relationships between canonical and non-canonical processes, acknowledging that some core sub-processes in normal perception are similar, or even identical, to those in (e.g.) hallucinating.

Challenge #3: Applications of Cognitive Ontologies

In addition to data annotation and integration, ontologies can be used for improved search and retrieval of other resources and for reasoning over data within as well as across experiments.

Information Retrieval

The Neuroscience Information Framework⁷ (NIF) provides for search and retrieval across a wide range of resources, including publications, databases and other information sources. The search engine is backed by the NIF ontology (NIFSTD), which supports better formalization and integration of existing resources. For example, the ontologies include semantically based mappings between related terms, such as “cognition” and “mental process.” Therefore, an ontology-based search that includes only one of these terms still provides excellent coverage.

Data Annotation

Ontologies are increasingly used for data annotation. For example, the behaviors of model organisms are widely used as proxies for the study of mental processes and disorders in humans (e.g. Cryan and Holmes, 2005; Wu and Luo, 2005; Mathur and Guo, 2010). Building on the GO, the NeuroBehaviour Ontology (NBO, Gkoutos et al., 2012) represents behavior and is used for model organism data annotation. Formalization of the relationships between the behavior and assumed homologies in mental functioning represented in ontologies would support better data integration and translation in this domain.

Ontology-Based Analysis

Finally, ontologies can be used to reason across data, e.g., for classification of complex patterns (as supported by NEMO, Bilder et al., 2009; Frishkoff et al., 2009, 2011a,b; Poldrack et al., 2012).

To enable cross-domain applications, Hastings et al. (2012) have proposed a Mental Functioning Ontology (MF) which includes inter-ontology bridging modules. A bridging module assigns semantic relationships between terms in different ontologies, such as identity, parthood, or realization. Bridging modules exist independently of each of the source ontologies but enable applications to safely draw on both of the ontologies, harnessing the knowledge-based relational links between them. Our vision is that MF will support harmonization of search and analysis tools across biochemical, biological, and medical data. In doing so, we wish to achieve stronger links between biological research (“bench sciences”), translational research, and medical records. Due to the explicit relationship between disorder categories and lower-level mental processes, we further anticipate that MF might help to further disambiguate data in diagnostic categories for mental disorders that show high levels of co-morbidity (Hastings and Schulz, 2012).

Outlook: What is Needed for the Future?

In the last few years, multiple ontologies have been created for representation, annotation, and reasoning about cognitive functioning, and for the representation of related behavior and brain data in different experimental contexts. Table 1 gives an overview of these efforts. At present, there is no one project that unites these efforts, although there is considerable cross-pollination, as reflected in the “Related” column, which shows mid-level ontologies that are shared across projects.

For researchers, mapping between different ontologies can present challenges, as independent ontologies may reflect different assumptions, methodological practices and scientific viewpoints (as reflected, for example, in conflicting definitions). It is inefficient and prone to error for each consumer of the combined research data to re-do the mapping independently, and the difficulty in harmonizing hinders the integration of the knowledge contained in their respective annotations into a broader understanding that spans genetic, molecular, cellular and psychological levels of description. As the underlying science progresses, each of these ontologies will evolve in slightly different ways, rendering any mapping potentially out of date in the future.

Rather than post hoc mapping efforts on a project-by-project basis, what is needed is a coordinated international effort towards ontology integration. Ontology integration involves the creation of bridging modules between ontologies that accurately reflect the shared understanding of the semantic relationships between the entities in the different ontologies. This is necessary in order to allow comparison and alignment (and thus maximally effective use) of empirical research data flowing from different sources and methodologies. Since the ontologies and the annotations (data) are separated, ontology-based integration does not change the way researchers in different areas work and in no way forces unwanted theoretical or methodological frameworks onto them. Rather, ontology-based integration flows from the use of a set of common and integrated ontologies to annotate their data. We believe that with some coordinated effort it will be possible to create such common ontologies (in fact, we have already taken some steps in this direction, as described above) in a way that remains neutral as between different methodological approaches and fundamental assumptions.

We believe is the way forward includes (a) continued use of existing ontologies for data annotation by primary data-generating researchers and deposition of those annotated data into shared repositories, alongside (b) ontology-based integration through the creation of shared semantic bridging modules between different ontologies and the increased convergence (as recommended by the OBO Foundry) on re-use of shared components between different ontologies, shared upper and mid-level ontologies, and shared metadata standards.

We are currently building on existing active collaborations among ontology developers, and working towards creating a set of resources that the diverse interdisciplinary community of researchers will be able to stand behind. This will require a large-scale coordinated effort. However, in this era of increasingly data-driven science, as the focus turns to carefully retaining and making data available to drive reproducibility, we believe that there has never been a greater need, and also not a better opportunity.

Conclusion

We have described recent efforts in the development of cognitive ontologies and their use in annotation, integration and analysis of clinical and cognitive neuroscience data. A cross-cutting goal is to identify classes of behavior and brain activity that are related to different types of mental processes. This ambitious aim is consistent with recent calls for large-scale brain-mapping and support for translational neuroscience (e.g., Neuroimaging Informatics Tools and Resources Clearinghouse; Human Brain Project; the Brain Initiative – Devor et al., 2013), projects that are likely to benefit from the development and harmonization of cognitive ontologies. There is a pressing need for coordinated efforts across related disciplines. The NIF plays a key role as an interconnecting hub and primary consumer for many of the ontology efforts described here. There is a real need to provide clear and community-negotiated definitions for core high-level entities within the domain, but for this effort to succeed, wide-scale community participation must be achieved.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

In July 2012, an international interdisciplinary group of scholars met in Graz, Austria to discuss the state of the art in cognitive ontologies across multiple disciplines (http://kr-med.org/icbofois2012/mfo/). The face-to-face meeting was preceded by a series of virtual seminars hosted by the NIF. The authors thank Prof. Stefan Schulz for support in running the workshop in Graz. They also thank all attendees of the workshop and pre-workshop seminar series for contributing to discussions. This work was partially supported by the Swiss Centre for Affective Sciences. Funding for the open access publication was provided by EMBL.

Footnotes

1. ^ For the purposes of this article, the authors agree to use “mental process,” “mental functioning,” and “cognition” synonymously. Distinctions are often made between these, and while there can be good reason to do so in certain cases, clarifying the distinctions here would go beyond our scope.
2. ^ http://bioportal.bioontology.org/search?query=memory&commit=Search
3. ^ http://www.cognitiveatlas.org/task/id/trm_4da86cb034ff6
4. ^ http://www.cognitiveatlas.org/concept/id/trm_4a3fd79d0adf4
5. ^ http://code.google.com/p/neuropsychological-testing-ontology/
6. ^ http://code.google.com/p/neurological-disease-ontology/
7. ^ http://www.neuinfo.org/

References

Akil, H., Martone, M. E., and Van Essen, D. C. (2011). Challenges and opportunities in mining neuroscience data. Science 331, 708–712. doi: 10.1126/science.1199305

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Albus, J. S., Bekey, G. A., Holland, J. H., Kanwisher, N. G., Krichmar, J. L., Mishkin, M, et al. (2007). A proposal for a decade of the mind initiative. Science 317, 1321. doi: 10.1126/science.317.5843.1321b

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ashburner, M., Ball, C. A., Blake, J. A., Botstein, D., Butler, H., Cherry, J. M., et al. (2000). Gene ontology: tool for the unification of biology. Nat. Genet. 25, 25–29. doi: 10.1038/75556

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bilder, R. M., Sabb, F. W., Parker, D. S., Kalar, D., Chu, W. W., Fox, J., et al. (2009). Cognitive ontologies for neuropsychiatric phenomics research. Cogn. Neuropsychiatry 14, 419–450. doi: 10.1080/13546800902787180

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Burns, G. A. P. C., and Turner, J. A. (2013). Modeling functional magnetic resonance imaging (fMRI) experimental variables in the ontology of experimental variables and values (OoEVV). Neuroimage 15, 662–670. doi: 10.1016/j.neuroimage.2013.05.024

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ceusters, W., and Smith, B. (2010). Foundations for a realist ontology of mental disease. J. Biomed. Semantics 1, 10, 1–23. doi: 10.1186/2041-1480-1-10

CrossRef Full Text

Cox, A. P., Jensen, M., Ruttenberg, A., Szigeti, K., and Diehl, A. D. (2013). “Measuring cognitive functions: hurdles in the development of the neuropsychological testing ontology,” in Proceedings of the 4th International Conference on Biomedical Ontology 2013, Montreal, QC.

Cryan, J. F., and Holmes, A. (2005). Model organisms: the ascent of mouse: advances in modelling human depression and anxiety. Nat. Rev. Drug Discov. 4, 775–790. doi: 10.1038/nrd1825

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Decety, J., and Cacioppo, J. (2010). Frontiers in human neuroscience: the golden triangle, and beyond. Perspect. Psychol. Sci. 5, 767–771. doi: 10.1177/1745691610388780

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Devor, A., Bandettini, P. A., Boas, D. A., Bower, J. M., Buxton, R. B., Cohen, L. B., et al. (2013). The challenge of connecting the dots in the B.R.A.I.N. Neuron 80, 270–274. doi: 10.1016/j.neuron.2013.09.008

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Fliessbach, K., Buerger, C., Trautner, P., Elger, C. E., and Weber, B. (2010). Differential effects of semantic processing on memory encoding. Hum. Brain Mapp. 31, 1653–1664. doi: 10.1002/hbm.20969

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Fox, P. T., and Friston, K. J. (2012). Distributed processing; distributed functions? Neuroimage 61, 407–426. doi: 10.1016/j.neuroimage.2011.12.051

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Frishkoff, G. (2012). “Mental functioning is neural functioning: towards a unified ontology of mind, brain, and behavior,” in Proceedings of the Workshop on Mental Functioning Ontologies at the International Conference on Biomedical Ontology (ICBO’12), Graz.

Pubmed Abstract | Pubmed Full Text

Frishkoff, G., Frank, R., and LePendu, P. (2011a). “Ontology-based analysis of event-related potentials,” in Proceedings of the International Conference on Biomedical Ontology 2011, Buffalo, NY.

Frishkoff, G., Sydes, J., Mueller, K., Frank, R., Curran, T., Connolly, J., et al. (2011b). Minimal Information for Neural Electromagnetic Ontologies (MINEMO): a standards-compliant method for analysis and integration of event-related potentials (ERP) data. Stand. Genomic Sci. 5, 211–223. doi: 10.4056/sigs.2025347

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Frishkoff, G., LePendu, P., Frank, R., Liu, H., and Dou, D. (2009) Development of Neural Electromagnetic Ontologies (NEMO): Ontology-based tools for representation and integration of event-related brain potentials. In Proceedings of the International Conference on Biomedical Ontology 2009, Buffalo, NY.

Pubmed Abstract | Pubmed Full Text

Gardner, D., Akil, H., Ascoli, G. A., Bowden, D. M., Bug, W., Donohue, D. E., et al. (2008). The Neuroscience Information Framework: a data and knowledge environment for neuroscience. Neuroinformatics 6, 149–160. doi: 10.1007/s12021-008-9024-z

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gkoutos, G. V., Schofield, P. N., and Hoehndorf, R. (2012). The Neurobehavior Ontology: an ontology for annotation and integration of behavior and behavioral phenotypes. Int. Rev. Neurobiol. 103, 69–87. doi: 10.1016/B978-0-12-388408-4.00004-6

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hastings, J., le Novère, N., Ceusters, W., Mulligan, K., and Smith, B. (2012). “Wanting what we don’t want to want: representing addiction in interoperable bio-ontologies,” in Proceedings of the 3rd International Conference on Biomedical Ontology 2012, Graz, 897.

Hastings, J., and Schulz, S. (2012). Ontologies for human behavior analysis and their application to clinical data. Int. Rev. Neurobiol. 103, 89–107. doi: 10.1016/B978-0-12-388408-4.00005-8

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Imam, F. T., Larson, S. D., Bandrowski, A., Grethe, J. S., Gupta, A., and Martone, M. E. (2012). Development and use of ontologies inside the neuroscience information framework: a practical approach. Front. Gene 3:111. doi: 10.3389/fgene.2012.00111

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

James, W. (1890). The Principles of Psychology, Vol. 2. New York, NY: Dover Publications. doi: 10.1037/10538-000

CrossRef Full Text

Jensen, M., Cox, A. P., Chaudhry, N., Ng, M., Sule, D., Duncan, W., et al. (2013). The neurological disease ontology. J. Biomed. Semantics 4:42. doi: 10.1186/2041-1480-4-42

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Jones, E., and Mendell, L. M. (1999). Assessing the decade of the brain. Science 284, 739. doi: 10.1126/science.284.5415.739

CrossRef Full Text

Kandel, E. R. (2008). Psychiatry, Psychoanalysis, and the New Biology of Mind. Arlington, VA: American Psychiatric Publishing.

Laird, A. R., Lancaster, J. L., and Fox, P. T. (2005). BrainMap: the social evolution of a functional neuroimaging database. Neuroinformatics 3, 65–78. doi: 10.1385/NI:3:1:065

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Laird, A. R., Lancaster, J. L., and Fox, P. T. (2009). Lost in localization? The focus is meta-analysis. Neuroimage 48, 18–20. doi: 10.1016/j.neuroimage.2009.06.047

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mathur, P., and Guo, S. (2010). Use of zebrafish as a model to understand mechanisms of addiction and complex neurobehavioral phenotypes. Neurobiol. Dis. 40, 66–72. doi: 10.1016/j.nbd.2010.05.016

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mulligan, K., and Smith, B. (1986). A Relational Theory of the Act. Topoi 5, 115–130. doi: 10.1007/BF00139225

CrossRef Full Text

Noy, N. F., Shah, N. H., Whetzel, P. L., Dai, B., Dorf, M., Griffith, N., et al. (2009). BioPortal: ontologies and integrated data resources at the click of a mouse. Nucleic Acids Res. 37(Suppl. 2), W170–W173. doi: 10.1093/nar/gkp440

CrossRef Full Text

Poldrack, R. A., Kittur, A., Kalar, D., Miller, E., Seppa, C., Gil, Y., et al. (2011). The Cognitive Atlas: towards a knowledge foundation for cognitive neuroscience. Front. Neuroinform. 5:17. doi: 10.3389/fninf.2011.00017

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Poldrack, R. A., Mumford, J. A., Schonberg, T., Kalar, D., Barman, B., and Yarkoni, T. (2012). Discovering relations between mind, brain, and mental disorders using topic mapping. PLoS Comput. Biol. 8:e1002707. doi: 10.1371/journal.pcbi.1002707

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Raymond, E. S. (1999). The Cathedral and the Bazaar, ed. T. O’Reilly. Sebastopol, CA: O’Reilly & Associates, Inc.

Rosse, C., and Mejino, J. L. (2003). A reference ontology for biomedical informatics: the foundational model of anatomy. J. Biomed. Inform. 36, 478–500. doi: 10.1016/j.jbi.2003.11.007

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Smith, B., Ashburner, M., Rosse, C., Bard, C., Bug, W., Ceusters, W., et al. (2007). The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nat. Biotechnol. 25, 1251–1255. doi: 10.1038/nbt1346

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Smith, B., and Ceusters, W. (2010). Ontological realism: a methodology for coordinated evolution of scientific ontologies. Appl. Ontol. 5, 139–188.

Pubmed Abstract | Pubmed Full Text

Smith, B., and Grenon, P. (2004). The cornucopia of formal-ontological relations. Dialectica 58, 279–296. doi: 10.1111/j.1746-8361.2004.tb00305.x

CrossRef Full Text

Turner, J. A., and Laird, A. R. (2012). The cognitive paradigm ontology: design and application. Neuroinformatics 10, 57–66. doi: 10.1007/s12021-011-9126-x %

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Weyandt, L. L. (2006). The Physiological Bases of Cognitive and Behavioral Disorders. Routledge: Taylor and Francis. %

Wu, Y., and Luo, Y. (2005). Transgenic C. elegans as a model in Alzheimer’s research. Curr. Alzheimer Res. 2, 37–45. doi: 10.2174/1567205052772768

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text