Role of Metallic Nanoparticles in Vaccinology: Implications for Infectious Disease Vaccine Development

Subunit vaccines are safer but less immunogenic than live-attenuated vaccines or whole cell inactivated vaccines. Adjuvants are used to enhance and modulate antigen (Ag) immunogenicity, aiming to induce a protective and long-lasting immune response. Several molecules and formulations have been studied for their adjuvanticity, but only seven have been approved to formulate human vaccines. Metallic nanoparticles (MeNPs), particularly those containing gold and iron oxides, are widely used in medicine for diagnosis and therapy and have been used as carriers for drugs and vaccines. However, little is known about the immune response elicited by MeNPs or about their importance in the development of new vaccines. There is evidence that these particles display adjuvant characteristics, promoting cell recruitment, antigen-presenting cell activation, cytokine production, and inducing a humoral immune response. This review focuses on the characteristics of MeNPs that could facilitate the induction of a cellular immune response, particularly T-helper 1 and T-helper 17, and their potential functions as adjuvants for subunit vaccines.

Introduction

Adjuvant selection for subunit vaccines is a key to increasing immunogenicity and, therefore, guiding stimulation of innate immunity and the development of the appropriate protective response to combat the microorganism of interest. Adjuvants are classified as particulate formulations, immunomodulatory molecules, or a combination of both characteristics. In addition to acting on the diversity of the humoral and cellular immune response, they can act in several different ways: by decreasing the vaccine dose, accelerating the immune response, or prolonging the immune response (1, 2). Among the seven approved vaccine adjuvants for human use, aluminum salts (alum), emulsions (e.g., MF59), and virosomes are particulate formulations. While alum induces efficient antibody (Ab) production and a predominant T-helper 2 (Th2) response, the other two have the capacity to induce T-helper 1 (Th1) and Th2 as well as Ab. Adjuvant system (AS) 01 and 04 used the combination of an immunomodulatory molecule and a particulate formulation composed of a Toll-like receptor 4 (TLR4) agonist, monophosphoryl lipid A that also induces Ab. The incorporation of alum in AS04 improved the humoral response, while the association of saponin (QS-21) and liposome in AS01 favored Th1 responses (3, 4). Imidazoquinolines (TLR7 and TLR8 agonists) and lipid A analogs (TLR4 agonists) are immunomodulatory molecules, capable of generating a Th1 response (5).

There is a demand for safe adjuvants capable of inducing efficient cellular immunity, especially Th1 and Th17, to be used against tuberculosis, leishmaniasis, malaria, and other diseases caused by intracellular microorganisms (1, 6). The majority of molecules with this type of adjuvanticity (Th1 driven) are related toward the response of danger receptors to trigger inflammation, thus safety and tolerance could be major barriers that prevent their use in human vaccines (7). However, comparing Alum and CpG/DNA adjuvants in human trials, only common adverse effects, including local site reaction, flu-like symptoms and headache were observed when CpG/DNA was used (8). Also, Verstraeten et al. (9), analyzing more than 30,000 individuals, who received vaccine-containing AS01, observed that only common side effects occurred.

Nanoparticles (NPs) are classically described as structures smaller than 100 nm and can be classified, based on their composition, as polymeric, inorganic, liposomes, immunostimulating complexes, virus-like particles, emulsions, or self-assembled proteins (10). They are made of different materials and differ in size, shape, and surface properties; interactions with biological systems, therefore, are varied, with several applications in modern medicine. In vaccinology, they are classically thought to have delivery and deposit properties. However, many NPs have been shown to stimulate immune responses, including cell recruitment, activation of antigen (Ag)-presenting cells (APCs), and induction of cytokine and chemokine release. The development of nanostructures and nanoadjuvants may therefore offer alternatives to currently used adjuvants once studies establish ways for them to elicit innate immune response and support the development of adaptive immune response in the context of vaccine formulations (10).

Metallic nanoparticles (MeNPs) are relatively non-biodegradable, have rigid structures, and possess simple synthesis methodology. Many have been studied for their immunological properties (11). However, there are still gaps in understanding the immune response generated by NPs, especially MeNPs. Few studies have compared NPs of different types and there is no standardization among published methodologies, which hampers comparisons of immunostimulatory characteristics. Several important characteristics, therefore, have not been well studied. For example, how chemical and physical properties (including material composition, size, shape, surface charge, and hydrophobicity) impact vaccine immune response (5). This review focuses on the use of MeNPs in formulations against infectious diseases, aiming to assess progress of their use in vaccinology and their possible applications as adjuvant.

The Immune Response Generated by MeNP-Formulated Vaccines

Table 1 summarizes the articles that report the use of MeNPs as part of vaccine formulations against infectious diseases and the immune responses they elicited. A range of immune responses is required to fight a diverse group of microorganisms. The type of protective immune response can be simplistically divided based on the type of microorganism: extracellular bacteria and toxin, intracellular bacteria, viruses, fungi, and protozoa. Among the vaccines targeting extracellular bacteria and toxin, two were formulated with lipopolysaccharide (LPS) in glycopeptide Ag. The use of glycoantigen and LPS can trigger an intense response through TLR4 and B cell receptor activation; the presence of gold NPs (AuNPs) may have minimal influence on this response. However, in the work of Gregory et al. (12) and Torres et al. (13), the use of AuNPs in the formulation generated a different response, improving anti-LPS immunoglobulin G (IgG) response, decreasing bacterial burden, generating a more efficient humoral response, and improving animal survival, showing that AuNPs may influence immune response and protection.

TABLE 1

Table 1. Studies describing immune responses to vaccination with metallic nanoparticles, listed by NPs material and year of publication (n = 18 studies).

Using protein Ag, Barhate et al. (22) formulated a vaccine using AuNPs and toxoid Ag and demonstrated that their formulation could induce a mucosal and systemic IgG and IgA response. When co-administered with Asparagus racemosus extract, a botanically derived adjuvant, the response was further enhanced (22). Dakterzada et al. (24) developed a vaccine against Pseudomonas aeruginosa using the flagellin subunit and AuNPs that elicited an IgG response comparable to that induced by Freund Adjuvant. Flagellin is a TLR5 agonist but the recognition and signaling is structure dependent. This study, however, used only the 1–161aa from flagellin and its ability to activated TLR5 could not be maintained (24). Gregory et al. (12) used an F1 Yersinia pestis Ag conjugated to AuNPs that induced an Ab response with higher IgG2a associated with higher levels of interferon gamma (IFNγ), suggesting activation of Th1 cells.

Among the studies that used MeNPs in vaccine formulation, only one targeted intracellular bacteria (Listeria monocytogenes). The protective immune response against intracellular bacterial infections requires Th1 activation and, therefore, APCs activation and Ag presentation through major histocompatibility complex II (MHC II). To generate a Th1 response, an AuNP and Listeria Ag formulation were used in different strategies. Although the authors tested direct vaccination, when dendritic cells (DC), in vitro loaded with AuNP plus Listeria Ag, were adoptively transferred to a naïve animal, they induced Th1, CD8+, and natural killer (NK) cells that provided better protection against L. monocytogenes than the traditional vaccine approach (23).

In evaluating vaccines developed with MeNPs against viral infections, Niikura et al. (20) used West Nile virus (WNV); Tao et al. (21) used the extracellular portion of Matrix 2 protein (M2) of the influenza virus; Chen et al. (15) conjugated AuNPs with a 28 amino acid VP1-foot-and-mouth virus protein (pFMDV); and Staroverov et al. (17) co-administered AuNPs and partially purified enteropathogenic swine-transmissible gastroenteritis virus. All the above studies evaluated the Ab immune responses and all formulations demonstrated efficient humoral response induction. Tao et al. (21) also evaluated the addition of cytosine and guanine linked by phosphodiester unmethylated (CpG/DNA) and found that it improved Ab levels and animals’ survival rates. Another important feature of studies by Niikura et al. (20) and Chen et al. (15) was the use of various NP sizes and the demonstration that all different NP shapes were capable of inducing a humoral response. The levels of Ab were size dependent, but the results were inconsistent: the first study found that a 40 nm sphere was the most efficient Ab inducer and the second study found that the 8 nm and 12 nm spheres performed best.

A special case of the use of MeNPs was the use of nickel-functionalized nanolipoprotein particles (NiNLPs) by Yan et al. (28) and Wadhwa et al. (27) in combination with HIV Ag. NiNLPs are nanometer-sized nanolipoprotein particles with nickel incorporation into their surface in order to induce polyhistidine tagged proteins adsorption (29). They demonstrated that specific IgG (IgG1 and IgG2a) levels were greater than those obtained when alum was used in the formulation. Fischer et al. (26) used truncated WNV envelope protein Ag and found that a single dose vaccination induced a superior anti-WNV IgG response and improved protection against a WNV challenge (26). These responses were associated with nickel functionalization, described as a hapten, and triggered responses through activation of human TLR4 and intracellular transduction signals through myeloid differentiation primary response (MyD-88), nuclear factor-κB (NF-κB), and mitogen-activated protein kinase (MAPK), inducing pro-inflammatory responses [tumor necrosis factor (TNF)-α and interleukin (IL)-8] (30, 31).

For protozoan infections, Parween et al. (16), using Plasmodium falciparum merozoite surface protein subunit and AuNPs, evaluated the humoral immune response (IgG1, IgG2a, IgG2b, and IgG3) and found an intense IgG1 response compared with the alum formulation (16). Kaba et al. (14), using P. berghei circumsporozoite protein and AuNPs, generated long-lasting protective immunity with Th that produced IL-2 and mixed high avidity IgG1/IgG2a (Th2/Th1) (14). In other studies, these authors replaced Ag with P. falciparum circumsporozoite protein; vaccination was shown to induce protective cytotoxic (CD8+) cells, high avidity Ab titers, and specific effector memory, central memory, and long-term central memory CD8+ T cells in draining lymph nodes, spleen, and liver (18). This response was shown to be generated by DC cross-presentation, which had delayed fusion and interaction of endosomes with lysosomes caused by the AuNP formulation (19). Finally, PfMSP was used with dextran-coated iron oxide NPs (IONPs) and was capable of inducing a humoral response in two animal models (mouse and monkey). This response was also shown to inhibit parasite growth by 55–100% (25).

Most studies evaluated immunogenicity through measurement of the humoral immune response. According to their findings, the use of NPs was efficient in inducing an Ab-based response. Based on heavy chain structure, there are five types of Ab, each with a different role: IgG, IgM, IgA, IgD, and IgE. IgG and IgA can be subdivided as IgG1, IgG2, IgG3, IgG4, IgA1, and IgA2 based on additional small differences in their heavy chain. With regard to vaccination, humoral immunity is especially important in responding to infection by extracellular pathogens, toxins, protozoa, and viruses. Its importance is associated with the biological activities of immunoglobulins, including microorganism opsonization and phagocytosis; complement activation (32); toxins and microorganism neutralization (33); and mast cells and basophil activation (32, 34). In addition, immunoglobulins can help target cytotoxicity against infected cells (Ab-dependent cell cytotoxicity of CD8 T cells and NK). In some cases, however, the pathogens have the ability to evade the humoral system or can even use immunoglobulins as a way to facilitate cell invasion, as in the cases of Mycobacterium tuberculosis and Leishmania spp. (35, 36).

The studies described above clearly show that MeNPs (gold, iron, and nickel) can be used for vaccine development. Different MeNPs were used in conjunction with several Ag for distinct microorganisms and showed the ability to generate humoral and cytotoxic responses. Although the generation of IgG2a and IFN-γ shown in some studies are indicators of Th1 responses using MeNPs as adjuvant, further research is needed to specifically assess the role of different MeNP vaccines in Th1 induction.

Important Physicochemical Characteristics of MeNPS as Activators of Immune Responses

To understand the possible uses of MeNPs as platforms for vaccines against infectious diseases, analysis is needed of the impact of different physicochemical characteristics of NPs on the innate immune response (Figure 1). Several strategies have included MeNPs as vaccine platforms, involving MeNPs of different materials (including gold, iron oxide, and nickel); shapes (including spheres, cubes, rods, and disks); sizes (from 2 nm to over 200 nm); and types of coating [e.g., citrate, chitosan, dextran, or cetyltrimethylammonium bromide/4-styrenesulfonic acid-co-maleic acid (CTAB/PSS-MA)].

FIGURE 1

Figure 1. Important nanoparticle characteristics for adjuvanticity. To be recognized and to stimulate innate immunity, metallic nanoparticles (MeNPs) must have some physicochemical traits that allow for interactions with host cells and lead to the generation of a response. APCs, antigen-presenting cells; MeNPs, metallic nanoparticles; Th, T-helper cell.

The material from which an NP is made has a direct influence on the functions of APCs; gold NPs (AuNPs) have been most commonly used in vaccinology (Table 2). The most recent studies involving AuNPs demonstrate the effects of gold sodium thiomalate on macrophage function, showing lysosomal enzyme inhibition and reducing phagocytosis (37). Similar effects were seen in macrophages of several origins, which, when stimulated with AuNPs, showed diminished bactericidal activity against Staphylococcus aureus (38) and low or absent cytokine production IL-6, IL-10, and TNF-α (39, 40). Moreover, when splenocytes were stimulated with LPS, the addition of AuNP reduced IL-17 and TNF-α release (40). Some of these results raise the concern on the use of AuNPs as adjuvants, since these immunomodulatory properties can act inhibiting the generation of Th1. However, the response to AuNPs is also correlated with other physicochemical characteristics that will be discussed below, which may be tailored to improve immunostimulatory or immunomodulatory capacity.

TABLE 2

Table 2. Studies describing NPs and antigens used as vaccines against infectious diseases, listed by NPs material and year of publication (n = 18 studies).

Iron oxide nanoparticles have also been used as adjuvants. Iron is an important ion in the homeostasis of all cells and in generating immune responses to several microorganisms. The effect of IONPs phagocytosis have been explored in several studies, for example, M2 macrophages after exposure to IONPs induced reactive oxygen species (ROS), but after 24 h induced IL-10 production (41). The use of IONPs in BALB/c mice demonstrated the immunomodulatory capacity of this NP by diminishing splenocyte cytokine production (IL-4 and IFN-γ) (42) as well as suppressing the response to pancreatic Ag in diabetic mice (43). Sindrilaru et al. (44), however, showed that macrophages, under iron overloaded conditions, became unrestrained M1 (with an incomplete switch to M2 macrophages) and produced more TNF-α, which impaired wound healing and had an important role in the immunopathology of chronic venous leg ulcers. Consequently, IONP response seems to have direct correlation with time and dose, once iron overload seems to be a requisite to developed pro-inflammatory response and this aspect must be evaluated to avoid the inhibition of the desired immune response.

Other critical characteristics are the shape and size of NPs, which have a direct impact on vaccine efficiency, Ag load capacity, and interaction with cells (phagocytes and APCs). These characteristics have been studied in different NPs; Shah et al. (45) published a review focusing on the impact of size for alum, oil-in-water, emulsion, polymeric particles, and liposome adjuvanticity, but did not evaluated MeNPs. In the studies reviewed here, NP sizes range from 2 nm nanospheres to 270 nm nanocapsules. Two authors have evaluated the impact of size and shape for MeNPs (Table 2): Chen et al. (15) evaluated differences in immune response based on AuNP sizes (ranging from 2 to 50 nm nanospheres) and found that 8 and 12 nm were the most drained NP (15); Niikura et al. (20) went further and, using four different shapes of NP (20 nm sphere, 40 nm sphere, cube, and rod), showed that Ab responses and TNF-α were directly correlated with the specific NP surface area (the ratio of the total surface area per single NP volume). Furthermore, 40 nm spheres appear to be the most efficient in generating immune responses (IL-6 and IL-12) and granulocyte macrophage colony-stimulating factor production.

Surface charge and hydrophobicity are additional important NP characteristics for immune response induction and are directly influenced by NP functionalization (chemical modification of NPs surface by adding or replacing functional groups) and coating (Ag) (46). Most studies used citrate-coated NPs, but dextran and CTAB/PSS-MA have also been used; all three result in negatively charged (anionic) particles. Only one NP, revised here, used positive charged (cationic) functionalization [(22); Table 2]. The higher hydrophobicity of AuNP was shown to activate the innate immune system (TNF-α secretion) (47). Although the surface charge of other non-metallic NPs has been studied (48), to our knowledge the studies using MeNPs did not address the other characteristics associated with immune response induction. For non-metallic NPs, it appears that a positive charge signified a greater ability to induce immune responses than a negative charge. Interestingly, negatively charged non-metallic NPs were associated with Ag-specific tolerance (48). Further studies are needed to investigate whether or not the charge imputed by NP coating influences the immune response. Though the size and shape of MeNPs had little to no impact on the innate response elicited, coating modifications may improve the capacity of these molecules to influence immune responses. Finally, it is important to note that the majority of adjuvant characteristics were evaluated using non-metallic NPs.

NPs as Adjuvants to Generate Th1 and Th17 Responses

T-helper 1 cells are associated with immunity against intracellular pathogens and the secretion of IFN-γ, which, in turn, is essential for the activation of mononuclear phagocytes, including macrophages, resulting in enhanced phagocytic activity (49). Th17 cells (IL-17A and IL-17F producer cells) are associated mainly with stimulation and chemotaxis of neutrophils to the site of inflammation. However, their function goes beyond this and includes the targeting of various cells types, including non-lymphoid cells and the stimulation of cytokine, chemokine, and prostaglandin production. Another characteristic of these cells is their memory effector subset, which is maintained in mucosal tissues for extended periods. This subset has high plasticity and is able to transform into Th1 or Th2 phenotypes depending on the cytokine milieu at mucosal sites. This diversity of function and actuation make Th17 cells very important in defense against several microorganisms, mainly those acquired through mucosal routes (49, 50).

T-helper 1 and Th17 cells have their own distinct sets of functions and differentiation factors. Both cell types require T cell receptor downstream activation by Ag presentation cells through MHC II and co-stimulatory molecules (6). Consequently, cytokine release during Ag presentation is correlated with the type of adaptive immune response generated. While Th1 differentiation requires stimulation by IL-12, Th17 generation requires transforming growth factor-β and IL-6. However, this generation is influenced by other factors and how MeNP are involved in the possible induction of Th1 or Th17 will be discussed below.

In this review, only one study investigated the development of the direct Th1 (type 1 T helper cell) and Th17 response. Using a Listeria Ag, Rodriguez-Del Rio et al. (23) showed that in contrast to Advax™ adjuvant alone, a combination of 25 nm AuNPs and Advax™ was capable of inducing the highest Th1 response. Pusic et al. (25) immunized mice with IONPs covered with rMSP1, a P. falciparum merozoite Ag, and showed that after immunization (intramuscular, subcutaneous, or intraperitoneal), production of IL-4 was greater than that of IFN-γ, suggesting a predominant Th2 response (although the cellular immune response was not directly evaluated).

The first major determinant in generating Th1 and Th17 populations is the route of vaccine administration, which dictates the cell dynamic and initial response to the vaccine. For example, Mohanan et al. (51), in a cross-sectional study using a liposome plus Ag (OVA) vaccine formulation, compared intradermal (high IgG1; intermediate IgG2; and IFN-γ), intralymphatic (high IgG1, IgG2, and IFN-γ), intramuscular (high IgG1; intermediate IgG2 and IFN-γ), and subcutaneous (high IgG1; low IgG2 and IFN-γ) routes of administration (51). The predominant Th1 response to administration through the intradermal route was most likely due to the cooperation between Langerhans cells, the primary innate immune response cells and keratinocytes that may also be stimulated by the formulation. These elicited the production of cytokines and chemokines that helped in the activation of other APCs (52).

The early phase of vaccination is characterized by recruitment of neutrophils and monocytes to the site of inoculation. Both cell types can also act as APCs, delivering Ag-specific and co-stimulatory signals to T cells. Their collaborative endeavors have been found to modulate (positively or negatively) the activity of different effector T cell subsets (53, 54). Neutrophils are the first cell lineage to migrate to inflammation sites and, when stimulated, they produce cytokines and chemokines that will attract and activate other cell types. For example, neutrophils were shown to be an important inducer of Th1 and Th17 cells (55), but their role in cytokine secretion is much broader (56). Moreover, signals may elicit different function in neutrophils and therefore, influence the quality of T cell responses. For example, AuNPs have been described as capable of inducing neutrophil extracellular traps, which act as damage-associated molecular patterns and stimulate immune system through DNA receptors such as TLR9 (57). Upon stimulation by NPs (TiO2—titanium dioxide—and alum), Duffin et al. (58) demonstrated neutrophil influx to the lungs and also induced production of IL-18. Silver NPs were also shown to be capable of interacting with neutrophils, inducing apoptosis of these cells, and inducing caspase-1\caspase-4 partially dependent IL-1β secretion (59). In another study, cobalt and nickel NPs were shown to induce higher nitric oxide, TNF-α, and CXCL2 chemokine production, by human peripheral blood neutrophils, than titanium NPs (TiO2NP) (60). Nonetheless, TiO2NPs also induced polymorphonuclear cell activation through phosphorylation of several proteins, including p38 MAPK and extracellular signal-regulated kinases-1/2 (Erk-1/2), which were associated with increased neutrophil life-span and production of several cytokines and chemokines (61).

Classically, APCs, macrophages, and DCs act at the site of vaccine inoculation by sensing foreign agents, through TLRs and other receptors, and triggering inflammation. APCs play a key role in the initiation, maintenance, and selectivity of inflammation, through their three major functions: endocytosis, Ag presentation, and production of various cytokines and growth factors (1). The main family of pattern recognition receptors in microbial recognition is the TLRs, part of the family of transmembrane proteins, which affect the transcription of genes involved in inflammatory and immune response-enhancing cellular activities such as phagocytosis, endocytosis, cytotoxic functions, and cytokine production (62, 63).

The adjuvants most frequently used for the induction of Th1 and Th17 responses are TLR agonists, such as AS04, CPG/DNA, and others. MeNPs seems to have capacity to induce the expression of Toll-like receptors, such as TiO2NPs and zirconium oxide NPs that have been described to enhance TLR3, TLR7, and TLR10 expression in macrophages (64) and TLR2 and TLR4 in mouse liver cells (65). Zinc oxide NPs (plus OVA) generated an inflammatory response in BALB/c mice and also improve TLR-2, -4 and -6 expression, followed by activation of Src family kinases (66). Consequently, TiO2NPs and IONPs were shown to induce DC upregulation of co-stimulatory molecules (MHC II, CD80) (25, 67, 68), which can also be related to TLR stimuli pathways. However, none of these works demonstrate the direct interaction of NPs with TLR (using Knock-out mice, agonists, or antagonist molecules) thus, this interaction must be further studied.

The next step in the generation of adaptive responses is the tailoring of cytokine secretion by APCs at immunological synapses, which will guide the development of the response. Several NPs have been reported to trigger cytokine and chemokine production, which may be used as biomarkers for immunotoxicity (69). Among those described, TiO2NPs were used in mimetic systems composed of blood vein endothelial component (including PBMC) and was reported to trigger pro-inflammatory cytokines (IL-6, IFN-γ, and TNFα) (67); Zinc oxide NPs were shown to be preferentially associated with monocytes and, when used in PBMC, induced IFN-γ, TNF-α, and IL-12 cytokine production (70); AuNP-stimulated bone marrow-derived DC produced IL-6, TNF-α, and IFN-γ (20); and IONPs were shown to induce the activation of APCs with an increase of IL-6, TNF-α, IFN-γ, and IL-12, as well as chemokines. The response generated by IONPs, however, was weaker than that generated by the positive control LPS which may be beneficial in controlling possible side effects (25).

The generation of a cellular response associated with protection against intracellular pathogens is the ultimate goal of vaccination. However, the direct effects of NPs on cellular responses have been evaluated in only a few studies. TiO2NPs were shown to activate and induce proliferation of naïve CD4+ T cells and to generate a pronounced Th1 response with IFN-γ and TNF-α production, associated with pro-inflammatory cytokine production (IL-6, IL-1a, IL-1b) and DC maturation (CD86+ and CD83+ expressions increase). Schanen et al. (71) hypothesized that the oxidative capacity of an NP could impact the response and trigger pro-inflammatory (oxidant capacity) or anti-inflammatory (antioxidant capacity) responses. This oxidant effect could control ROS generation and thus control downstream pro-inflammatory effects while antioxidants prevent the initiation of the innate immunity in LPS-stimulated macrophages (71). This study was, however, conducted with mitogens (non-specific stimuli) and not with vaccine stimuli, but nevertheless serves as a warning about the direct action of NPs, not only on the innate immune system but specifically on T cells.

Conclusion

There is enough evidence to suggest that MeNPs are not only particulate formulations but also immunostimulatory molecules with several studies demonstrating their capacity to generate humoral and cytotoxic responses. MeNPs clearly have immunostimulatory capacity and can induce several reactions in all phases of vaccine development. These capabilities correlated with NP physicochemical characteristics such as size, charge, and hydrophobicity, but there are several gaps in our understanding of their mechanism of actions and how they may lead to adjuvanticity, immunomodulation, or tolerance to the Ag formulated with NPs. There are also evidence of MeNP being capable of help in the generation of Th1 and Th17; Figure 2 presents an overview of the generation of these cells subsets and the possible role of MeNP in this induction.

FIGURE 2

Figure 2. Metallic nanoparticles adjuvanticity and its prediction capacity to generate T-helper 1 (Th1) and Th17 responses. To generate a cellular immune response, the NP must be able to be recognized by the host innate immune response and stimulate a sequence of events that will lead to the release of a specific milieu of cytokines and better antigen presentation (bottom arrow). In the top arrow is the immune response elicited by metallic nanoparticles to aid Th1 and Th17 generation. NF-κB, nuclear factor kappa B; CCL, chemokine ligand; CXCL, chemokine (C-X-C motif) ligand; GM-CSF, granulocyte macrophage colony-stimulating factor; IFN, interferon; IL, interleukin; M-CSF, macrophage colony-stimulating factor; MYD, myeloid differentiation factor; TCR, T cell receptor; Th, T-helper cell; TLR, Toll-like receptor; TNF, tumor necrosis factor.

Author Contributions

LN designed the review and wrote the first draft. AJ-K edited the first draft and critically reviewed the manuscript. AK edited the first draft and critically reviewed the manuscript. All authors read and approved the final version of the manuscript and agreed to submission.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This work is part of LN PhD thesis at Biotechnology and Biodiversity Graduate Program from CAPES.

Funding

This work was funded by FAPEG (grant number 20131026700-1143) and CNPq (grant number: 405198/2015-9). LN received a PhD fellow from CAPES, and APJK (#303675/2015-2) and AK (#307186-2013-0) received a productivity research fellow from CNPq.

References

1. Reed SG, Orr MT, Fox CB. Key roles of adjuvants in modern vaccines. Nat Med (2013) 19:1597–608. doi:10.1038/nm.3409

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Agger EM. Novel adjuvant formulations for delivery of anti-tuberculosis vaccine candidates. Adv Drug Deliv Rev (2016) 102:73–82. doi:10.1016/j.addr.2015.11.012

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Morrison C. Landmark green light for Mosquirix malaria vaccine. Nat Biotechnol (2015) 33:1015–6. doi:10.1038/nbt1015-1015

CrossRef Full Text | Google Scholar

4. Didierlaurent AM, Laupeze B, Di Pasquale A, Hergli N, Collignon C, Garcon N. Adjuvant system AS01: helping to overcome the challenges of modern vaccines. Expert Rev Vaccines (2017) 16:55–63. doi:10.1080/14760584.2016.1213632

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Di Pasquale A, Preiss S, Tavares Da Silva F, Garcon N. Vaccine adjuvants: from 1920 to 2015 and beyond. Vaccines (Basel) (2015) 3:320–43. doi:10.3390/vaccines3020320

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Damsker JM, Hansen AM, Caspi RR. Th1 and Th17 cells: adversaries and collaborators. Ann N Y Acad Sci (2010) 1183:211–21. doi:10.1111/j.1749-6632.2009.05133.x

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Knudsen NP, Olsen A, Buonsanti C, Follmann F, Zhang Y, Coler RN, et al. Different human vaccine adjuvants promote distinct antigen-independent immunological signatures tailored to different pathogens. Sci Rep (2016) 6:19570. doi:10.1038/srep19570

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Cooper CL, Davis HL, Morris ML, Efler SM, Adhami MA, Krieg AM, et al. CPG 7909, an immunostimulatory TLR9 agonist oligodeoxynucleotide, as adjuvant to Engerix-B HBV vaccine in healthy adults: a double-blind phase I/II study. J Clin Immunol (2004) 24:693–701. doi:10.1007/s10875-004-6244-3

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Verstraeten T, Descamps D, David MP, Zahaf T, Hardt K, Izurieta P, et al. Analysis of adverse events of potential autoimmune aetiology in a large integrated safety database of AS04 adjuvanted vaccines. Vaccine (2008) 28:6630–8. doi:10.1016/j.vaccine.2008.09.049

CrossRef Full Text | Google Scholar

10. Zhao L, Seth A, Wibowo N, Zhao CX, Mitter N, Yu C, et al. Nanoparticle vaccines. Vaccine (2014) 32:327–37. doi:10.1016/j.vaccine.2013.11.069

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Hofmann-Amtenbrink M, Grainger DW, Hofmann H. Nanoparticles in medicine: current challenges facing inorganic nanoparticle toxicity assessments and standardizations. Nanomedicine (2015) 11:1689–94. doi:10.1016/j.nano.2015.05.005

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Gregory AE, Judy BM, Qazi O, Blumentritt CA, Brown KA, Shaw AM, et al. A gold nanoparticle-linked glycoconjugate vaccine against Burkholderia mallei. Nanomedicine (2015) 11:447–56. doi:10.1016/j.nano.2014.08.005

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Torres AG, Gregory AE, Hatcher CL, Vinet-Oliphant H, Morici LA, Titball RW, et al. Protection of non-human primates against glanders with a gold nanoparticle glycoconjugate vaccine. Vaccine (2015) 33:686–92. doi:10.1016/j.vaccine.2014.11.057

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Kaba SA, Brando C, Guo Q, Mittelholzer C, Raman S, Tropel D, et al. A nonadjuvanted polypeptide nanoparticle vaccine confers long-lasting protection against rodent malaria. J Immunol (2009) 183:7268–77. doi:10.4049/jimmunol.0901957

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Chen YS, Hung YC, Lin WH, Huang GS. Assessment of gold nanoparticles as a size-dependent vaccine carrier for enhancing the antibody response against synthetic foot-and-mouth disease virus peptide. Nanotechnology (2010) 21:195101. doi:10.1088/0957-4484/21/19/195101

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Parween S, Gupta PK, Chauhan VS. Induction of humoral immune response against PfMSP-1(19) and PvMSP-1(19) using gold nanoparticles along with alum. Vaccine (2011) 29:2451–60. doi:10.1016/j.vaccine.2011.01.014

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Staroverov SA, Vidyasheva IV, Gabalov KP, Vasilenko OA, Laskavyi VN, Dykman LA. Immunostimulatory effect of gold nanoparticles conjugated with transmissible gastroenteritis virus. Bull Exp Biol Med (2011) 151:436–9. doi:10.1007/s10517-011-1350-8

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Kaba SA, Mccoy ME, Doll TA, Brando C, Guo Q, Dasgupta D, et al. Protective antibody and CD8+ T-cell responses to the Plasmodium falciparum circumsporozoite protein induced by a nanoparticle vaccine. PLoS One (2012) 7:e48304. doi:10.1371/journal.pone.0048304

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Mccoy ME, Golden HE, Doll TA, Yang Y, Kaba SA, Zou X, et al. Mechanisms of protective immune responses induced by the Plasmodium falciparum circumsporozoite protein-based, self-assembling protein nanoparticle vaccine. Malar J (2013) 12:136. doi:10.1186/1475-2875-12-136

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Niikura K, Matsunaga T, Suzuki T, Kobayashi S, Yamaguchi H, Orba Y, et al. Gold nanoparticles as a vaccine platform: influence of size and shape on immunological responses in vitro and in vivo. ACS Nano (2013) 7:3926–38. doi:10.1021/nn3057005

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Tao W, Ziemer KS, Gill HS. Gold nanoparticle–M2e conjugate coformulated with CpG induces protective immunity against influenza A virus. Nanomedicine (2014) 9(2):237–51. doi:10.2217/nnm.13.58

CrossRef Full Text | Google Scholar

22. Barhate G, Gautam M, Gairola S, Jadhav S, Pokharkar V. Enhanced mucosal immune responses against tetanus toxoid using novel delivery system comprised of chitosan-functionalized gold nanoparticles and botanical adjuvant: characterization, immunogenicity, and stability assessment. J Pharm Sci (2014) 103:3448–56. doi:10.1002/jps.24161

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Rodriguez-Del Rio E, Marradi M, Calderon-Gonzalez R, Frande-Cabanes E, Penades S, Petrovsky N, et al. A gold glyco-nanoparticle carrying a Listeriolysin O peptide and formulated with Advax delta inulin adjuvant induces robust T-cell protection against listeria infection. Vaccine (2015) 33:1465–73. doi:10.1016/j.vaccine.2015.01.062

CrossRef Full Text | Google Scholar

24. Dakterzada F, Mohabati Mobarez A, Habibi Roudkenar M, Mohsenifar A. Induction of humoral immune response against Pseudomonas aeruginosa flagellin(1-161) using gold nanoparticles as an adjuvant. Vaccine (2016) 34:1472–9. doi:10.1016/j.vaccine.2016.01.041

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Pusic K, Aguilar Z, Mcloughlin J, Kobuch S, Xu H, Tsang M, et al. Iron oxide nanoparticles as a clinically acceptable delivery platform for a recombinant blood-stage human malaria vaccine. FASEB J (2013) 27:1153–66. doi:10.1096/fj.12-218362

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Fischer NO, Infante E, Ishikawa T, Blanchette CD, Bourne N, Hoeprich PD, et al. Conjugation to nickel-chelating nanolipoprotein particles increases the potency and efficacy of subunit vaccines to prevent West Nile encephalitis. Bioconjug Chem (2010) 21:1018–22. doi:10.1021/bc100083d

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Wadhwa S, Jain A, Woodward JG, Mumper RJ. Lipid nanocapsule as vaccine carriers for his-tagged proteins: evaluation of antigen-specific immune responses to HIV I His-Gag p41 and systemic inflammatory responses. Eur J Pharm Biopharm (2012) 80:315–22. doi:10.1016/j.ejpb.2011.10.016

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Yan W, Jain A, O’carra R, Woodward JG, Li W, Li G, et al. Lipid nanoparticles with accessible nickel as a vaccine delivery system for single and multiple His-tagged HIV antigens. HIV AIDS (Auckl) (2009) 2009:1–11. doi:10.2147/HIV.S5729

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Fischer NO, Blanchette CD, Chromy BA, Kuhn EA, Segelke BW, Corzett M, et al. Immobilization of His-tagged proteins on nickel-chelating nanolipoprotein particles. Bioconjug Chem (2009) 20:460–5. doi:10.1021/bc8003155

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Schmidt M, Raghavan B, Muller V, Vogl T, Fejer G, Tchaptchet S, et al. Crucial role for human Toll-like receptor 4 in the development of contact allergy to nickel. Nat Immunol (2010) 11:814–9. doi:10.1038/ni.1919

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Schmidt M, Goebeler M. Nickel allergies: paying the toll for innate immunity. J Mol Med (Berl) (2011) 89:961–70. doi:10.1007/s00109-011-0780-0

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Schroeder HW Jr, Cavacini L. Structure and function of immunoglobulins. J Allergy Clin Immunol (2010) 125:S41–52. doi:10.1016/j.jaci.2009.09.046

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Woof JM, Kerr MA. The function of immunoglobulin A in immunity. J Pathol (2006) 208:270–82. doi:10.1002/path.1877

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Kawakami T, Galli SJ. Regulation of mast-cell and basophil function and survival by IgE. Nat Rev Immunol (2002) 2:773–86. doi:10.1038/nri914

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Schorey JS, Carroll MC, Brown EJ. A macrophage invasion mechanism of pathogenic mycobacteria. Science (1997) 277:1091–3. doi:10.1126/science.277.5329.1091

CrossRef Full Text | Google Scholar

36. Dominguez M, Torano A. Immune adherence-mediated opsonophagocytosis: the mechanism of Leishmania infection. J Exp Med (1999) 189:25–35. doi:10.1084/jem.189.1.25

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Turkall RM, Warr GA, Tsan MF. Effect of in vivo administration of gold sodium thiomalate on rat macrophage function. Agents Actions (1982) 12:489–98. doi:10.1007/BF01965932

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Davis P, Johnston C. Effects of gold compounds on function of phagocytic cells. Comparative inhibition of activated polymorphonuclear leukocytes and monocytes from rheumatoid arthritis and control subjects. Inflammation (1986) 10:311–20. doi:10.1007/BF00916126

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Bancos S, Stevens DL, Tyner KM. Effect of silica and gold nanoparticles on macrophage proliferation, activation markers, cytokine production, and phagocytosis in vitro. Int J Nanomedicine (2015) 10:183–206. doi:10.2147/IJN.S72580

CrossRef Full Text | Google Scholar

40. Kingston M, Pfau JC, Gilmer J, Brey R. Selective inhibitory effects of 50-nm gold nanoparticles on mouse macrophage and spleen cells. J Immunotoxicol (2016) 13:198–208. doi:10.3109/1547691X.2015.1035819

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Rojas JM, Sanz-Ortega L, Mulens-Arias V, Gutierrez L, Perez-Yague S, Barber DF. Superparamagnetic iron oxide nanoparticle uptake alters M2 macrophage phenotype, iron metabolism, migration and invasion. Nanomedicine (2016) 12:1127–38. doi:10.1016/j.nano.2015.11.020

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Shen CC, Wang CC, Liao MH, Jan TR. A single exposure to iron oxide nanoparticles attenuates antigen-specific antibody production and T-cell reactivity in ovalbumin-sensitized BALB/c mice. Int J Nanomedicine (2011) 6:1229–35. doi:10.2147/IJN.S21019

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Tsai S, Shameli A, Yamanouchi J, Clemente-Casares X, Wang J, Serra P, et al. Reversal of autoimmunity by boosting memory-like autoregulatory T cells. Immunity (2010) 32:568–80. doi:10.1016/j.immuni.2010.03.015

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Sindrilaru A, Peters T, Wieschalka S, Baican C, Baican A, Peter H, et al. An unrestrained proinflammatory M1 macrophage population induced by iron impairs wound healing in humans and mice. J Clin Invest (2011) 121:985–97. doi:10.1172/JCI44490

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Shah RR, O’hagan DT, Amiji MM, Brito LA. The impact of size on particulate vaccine adjuvants. Nanomedicine (Lond) (2014) 9:2671–81. doi:10.2217/nnm.14.193

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Mout R, Moyano DF, Rana S, Rotello VM. Surface functionalization of nanoparticles for nanomedicine. Chem Soc Rev (2012) 41:2539–44. doi:10.1039/c2cs15294k

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Moyano DF, Goldsmith M, Solfiell DJ, Landesman-Milo D, Miranda OR, Peer D, et al. Nanoparticle hydrophobicity dictates immune response. J Am Chem Soc (2012) 134:3965–7. doi:10.1021/ja2108905

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Fromen CA, Rahhal TB, Robbins GR, Kai MP, Shen TW, Luft JC, et al. Nanoparticle surface charge impacts distribution, uptake and lymph node trafficking by pulmonary antigen-presenting cells. Nanomedicine (2016) 12:677–87. doi:10.1016/j.nano.2015.11.002

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Golubovskaya V, Wu L. Different subsets of T cells, memory, effector functions, and CAR-T immunotherapy. Cancers (Basel) (2016) 8:E36. doi:10.3390/cancers8030036

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Zambrano-Zaragoza JF, Romo-Martinez EJ, Duran-Avelar Mde J, Garcia-Magallanes N, Vibanco-Perez N. Th17 cells in autoimmune and infectious diseases. Int J Inflam (2014) 2014:651503. doi:10.1155/2014/651503

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Mohanan D, Slutter B, Henriksen-Lacey M, Jiskoot W, Bouwstra JA, Perrie Y, et al. Administration routes affect the quality of immune responses: a cross-sectional evaluation of particulate antigen-delivery systems. J Control Release (2010) 147:342–9. doi:10.1016/j.jconrel.2010.08.012

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Kawase A, Isaji K, Yamaoka A, Kobayashi N, Nishikawa M, Takakura Y. Enhanced antigen-specific antibody production following polyplex-based DNA vaccination via the intradermal route in mice. Vaccine (2006) 24:5535–45. doi:10.1016/j.vaccine.2006.04.056

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Didierlaurent AM, Collignon C, Bourguignon P, Wouters S, Fierens K, Fochesato M, et al. Enhancement of adaptive immunity by the human vaccine adjuvant AS01 depends on activated dendritic cells. J Immunol (2014) 193:1920–30. doi:10.4049/jimmunol.1400948

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Iwasaki A, Medzhitov R. Control of adaptive immunity by the innate immune system. Nat Immunol (2015) 16:343–53. doi:10.1038/ni.3123

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Abi Abdallah DS, Egan CE, Butcher BA, Denkers EY. Mouse neutrophils are professional antigen-presenting cells programmed to instruct Th1 and Th17 T-cell differentiation. Int Immunol (2011) 23:317–26. doi:10.1093/intimm/dxr007

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Tecchio C, Micheletti A, Cassatella MA. Neutrophil-derived cytokines: facts beyond expression. Front Immunol (2014) 5:508. doi:10.3389/fimmu.2014.00508

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Schaefer L. Complexity of danger: the diverse nature of damage-associated molecular patterns. J Biol Chem (2014) 289:35237–45. doi:10.1074/jbc.R114.619304

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Duffin R, Tran L, Brown D, Stone V, Donaldson K. Proinflammogenic effects of low-toxicity and metal nanoparticles in vivo and in vitro: highlighting the role of particle surface area and surface reactivity. Inhal Toxicol (2007) 19:849–56. doi:10.1080/08958370701479323

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Liz R, Simard JC, Leonardi LB, Girard D. Silver nanoparticles rapidly induce atypical human neutrophil cell death by a process involving inflammatory caspases and reactive oxygen species and induce neutrophil extracellular traps release upon cell adhesion. Int Immunopharmacol (2015) 28:616–25. doi:10.1016/j.intimp.2015.06.030

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Mo Y, Mo Y, Zhu X, Mo Y, Zhu X, Hu X, et al. Cytokine and NO release from peripheral blood neutrophils after exposure to metal nanoparticles: in vitro and ex vivo studies. Nanotoxicology (2008) 2:79–87. doi:10.1080/17435390802112874

CrossRef Full Text | Google Scholar

61. Gonçalves DM, Chiasson S, Girard D. Activation of human neutrophils by titanium dioxide (TiO2) nanoparticles. Toxicol In Vitro (2010) 24:1002–8. doi:10.1016/j.tiv.2009.12.007

CrossRef Full Text | Google Scholar

62. Platt A, Wetzler L. Innate immunity and vaccines. Curr Top Med Chem (2013) 13:2597–608. doi:10.2174/15680266113136660185

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Junqueira-Kipnis AP, Marques Neto LM, Kipnis A. Role of fused Mycobacterium tuberculosis immunogens and adjuvants in modern tuberculosis vaccines. Front Immunol (2014) 5:188. doi:10.3389/fimmu.2014.00188

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Lucarelli M, Gatti AM, Savarino G, Quattroni P, Martinelli L, Monari E, et al. Innate defence functions of macrophages can be biased by nano-sized ceramic and metallic particles. Eur Cytokine Netw (2004) 15:339–46.

PubMed Abstract | Google Scholar

65. Cui Y, Liu H, Zhou M, Duan Y, Li N, Gong X, et al. Signaling pathway of inflammatory responses in the mouse liver caused by TiO2 nanoparticles. J Biomed Mater Res A (2011) 96:221–9. doi:10.1002/jbm.a.32976

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Roy R, Kumar D, Sharma A, Gupta P, Chaudhari BP, Tripathi A, et al. ZnO nanoparticles induced adjuvant effect via toll-like receptors and Src signaling in Balb/c mice. Toxicol Lett (2014) 230:421–33. doi:10.1016/j.toxlet.2014.08.008

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Schanen BC, Karakoti AS, Seal S, Drake DR III, Warren WL, Self WT. Exposure to titanium dioxide nanomaterials provokes inflammation of an in vitro human immune construct. ACS Nano (2009) 3:2523–32. doi:10.1021/nn900403h

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Winter M, Beer HD, Hornung V, Kramer U, Schins RP, Forster I. Activation of the inflammasome by amorphous silica and TiO2 nanoparticles in murine dendritic cells. Nanotoxicology (2011) 5:326–40. doi:10.3109/17435390.2010.506957

PubMed Abstract | CrossRef Full Text | Google Scholar

69. Elsabahy M, Wooley KL. Cytokines as biomarkers of nanoparticle immunotoxicity. Chem Soc Rev (2013) 42:5552–76. doi:10.1039/c3cs60064e

PubMed Abstract | CrossRef Full Text | Google Scholar

70. Hanley C, Thurber A, Hanna C, Punnoose A, Zhang J, Wingett DG. The influences of cell type and ZnO nanoparticle size on immune cell cytotoxicity and cytokine induction. Nanoscale Res Lett (2009) 4:1409–20. doi:10.1007/s11671-009-9413-8

PubMed Abstract | CrossRef Full Text | Google Scholar

71. Schanen BC, Das S, Reilly CM, Warren WL, Self WT, Seal S, et al. Immunomodulation and T helper TH(1)/TH(2) response polarization by CeO(2) and TiO(2) nanoparticles. PLoS One (2013) 8:e62816. doi:10.1371/journal.pone.0062816

CrossRef Full Text | Google Scholar