Augmenting Human Appearance Through Technological Design Layers

Genç, Çağlar; Raudanjoki, Özge; Colley, Ashley; Häkkilä, Jonna

doi:10.3389/fcomp.2022.755451

ORIGINAL RESEARCH article

Front. Comput. Sci., 03 February 2022

Sec. Mobile and Ubiquitous Computing

Volume 4 - 2022 | https://doi.org/10.3389/fcomp.2022.755451

This article is part of the Research Topic Augmented Humans View all 5 articles

Augmenting Human Appearance Through Technological Design Layers

$\n alar Gen$ Çağlar Genç

Özge Raudanjoki

Ashley Colley^*

Jonna Häkkilä

Faculty of Art and Design, University of Lapland, Rovaniemi, Finland

Augmenting human appearance with the means of technology can focus on different layers attached to or around the body. In this article, we present a categorization of human appearance and expression, with augmenting skin and its appendages, clothing and textile, accessories, body parts, and digital aura around the body. We report a non-systematic review of related works in each category and discuss their means in expressing functional, hedonic, and social aspects. In conclusion, our study contributes design perspectives on augmenting human appearances, as well as reveals challenges and opportunities.

1. Introduction

Augmented humans have lately gained increased attention in ubiquitous computing research. Ubiquitous computing is not anymore merely addressing intelligent environments, but also about embedding the technology close to or integrated in to, people and creating cyber-human systems. As technology integration has advanced and prototyping toolkits developed, creating concepts and prototypes that augment human capabilities has become easier. The visions of augmented humans are not anymore merely futuristic concepts, but through prototyping, they have emerged to proof-of-concept implementations that provide realistic and concrete examples, which can be assessed and studied for their feasibility, usability, and acceptability.

So far, much of the augmented human's research has focused on functional aspects, contributing primarily to technical advances and in validating the proposed solutions. Augmenting the physiological and cognitive capabilities offers possibilities for new types of application that can ease and provide functional advantages for the user. Still, human augmentation does not offer to serve just utilitarian means, but also hedonic and social aspects of our lives. So far, the aspects focusing on expression and human appearance have gained less attention. Approaching the topic from the design background, our research emphasizes the user experience and design aspects. Especially in this article, we focus on augmenting the human appearance through technology. In the contributions, we present the categorization of different layers for augmenting human appearance, and the results of a non-systematical review of related works focusing on different layers and summarize design challenges on the topic.

Following, we first present our wearable categorization and exemplify concepts and prototypes for each category from our research and other related works. Then, we explain the method for our non-systematical review aiming to capture design challenges and opportunities for augmenting human appearance through technological design layers. Finally, we present and discuss the resulting challenges.

2. Technological Design Layers

2.1. The Framework

In order to systematically approach the augmentation of human appearances, we address different design layers how the technology is integrated or attached around the human body. The categorization is illustrated in Figure 1.

FIGURE 1

Figure 1. Categorization of appearance augmentation layers.

While defining our categorization, we base our first three layers on the on-body technology classification of Liu et al. (2016): skin interfaces, clothing, and accessories. Accordingly, we call first and the closest layer to the body artificial skin and appendages to describe a broader framing of skin interfaces by Kao (2021) proposed for the technology directly applied or attached on the body surface including skin and its appendages. Kao argued these technologies are siblings of traditional body crafting practices such as body paintings, tattoos, nail or hair extensions and are used to alter the appearance of the skin and its appendages by adding interactive features to them. Aligned with this view, we consider any application that aims to alter either the appearance or functional capabilities of the skin and its appendages, within this category. The second layer, clothing, includes augmentation through textile-based interactive clothing worn similar to the garments. These applications cover a large part of the body like shirts, dresses, and trousers. Accessories layer, taking a step further, refer to the accessory-like artefacts, such as handbags, jewelry, and watches, that are worn as an addition to garments and are easily removed. Different from the prior study, we framed two additional layers that the advancements in human augmented technology enabled: The fourth layer focuses on artificial body parts attached to the body. The non-interactive additions of artificial body parts are rather uncommon, yet examples exist in prosthetic limbs (i.e., arms and legs) and in some artworks that speculated additional ears and arms attached to the body (Stelarc, 2022). As the outermost layer, we present a digital aura around the body. These applications add visuals around the body that can exist either in the physical or virtual world. The augmentation of this layer is only possible with interactive technologies like AR-enabled devices and projectors.

Each of these layers can be utilized for augmenting human appearances. They provide different types of opportunities expressing both utilitarian and hedonic use cases, and, from the design point of view, different challenges for usable and aesthetic solutions. Some of the layers have been more extensively addressed in research and also occupied by commercial solutions in the area of wearable computing. In this research, many of the current off-the-shelf products fall in the category of accessories, with a vast amount of use cases focusing on wellness, health and sports (Jarusriboonchai and Häkkilä, 2019). Textile and clothing integrated commercial products are also emerging, and so far have been especially popular among researchers experimenting with different solutions. Similarly, research around augmenting skin and its appendages started to exist, reporting diverse approaches to craft and alter the body surface. On the other hand, solutions augmenting the user's appearance with artificial body parts are still somewhat sporadic among HCI, as well as experiments with digital auras, but already they both provide interesting and inspiring examples. In the following sections, we provide examples of different design concepts and prototypes augmenting human appearance for each technology layer.

2.1.1. Artificial Skin and Appendages

In terms of appearance augmentation, applications around artificial skin /& appendages often focus on cosmetic changes on the body surface with interactive features. A number of examples of aesthetic on-skin interfaces have been created (Liu et al., 2016), e.g., with functional tattoos (Kao et al., 2016b) and thermochromic make-up (Kao et al., 2016c). With flexible and transparent materials, attaching on-skin tags has become easier and more comfortable. In our research, we have also explored skin-attached interactive tattoos, Figure 2, created with electrochromic printed electronics (Jensen et al., 2019a). In this research, an electrochromic display allows the design of interactive graphics, where the appearance of the tattoo changes as the pattern switches between two stages when electricity is applied.

FIGURE 2

Figure 2. Two states of a skin-attached interactive tattoo with electrochromic displays.

Moreover, the applications in this layer include technologies applied on the skin's appendages such as hair and nails. In that area, beauty technologies, as presented by Vega and Fuks (2013), offer a special platform focusing specifically on self-expression and appearance exceeding skin interfaces. In this research, Vega et al. have introduced RFID enhanced beauty fingernails, which combine conventional nail make-up with an input functionality. In terms of augmenting the appearance of the hair, Dierk et al. (2018b) presented artificial hair as a material that can alter its color and shape. To provide a dynamically changing beauty appearance, we have also created with VäriWig prototype for artificial hair (Brun and Häkkilä, 2021). In this research, the wig contains fiber optic modules as artificial hair to dynamically change color, Figure 3.

FIGURE 3

Figure 3. The wig prototype for artificial hair displaying different colors.

2.1.2. Clothing and Textile

The integration of various display technologies into clothing and textiles has been a well-researched topic with numerous examples. The use of LEDs has been ubiquitous, e.g., exemplified by Cute Circuit's Galaxy Dress which included 24,000 LEDs¹ but also display technologies such as LCD (Dierk et al., 2018a), thermochromic (Devendorf et al., 2016), and electrochromic (Genç et al., 2020), have been used. To provide a flavor of the domain, we present 3 of our prior studies (Figure 4), the Idle stripes shirt (Harjuniemi et al., 2020), the Decolive jacket (Genç et al., 2020), and the LinnDress (Jarusriboonchai et al., 2019).

FIGURE 4

Figure 4. Examples of the authors' works in smart clothing and textiles. L-R: Linn Dress, Idle Stripes shirt, DecoLive jacket.

In the Idle Stripes shirt (Harjuniemi et al., 2020), optical fibers, integrated in to the chest area of the garment, illuminate to indicate the wearer's sedentary time. The garment was designed holistically, to ensure its wearability during a normal office workday—which is rare in wearables research artefacts. With this garment, our design target was to produce a subtle textile-integrated effect from the light output, avoiding Devendorf et al.'s feeling of “wearing a screen” (Devendorf et al., 2016). Taking a similar direction to Dierk et al.'s Alterwear (Dierk et al., 2018a), the DecoLive jacket (Genç et al., 2020) is a battery-free wearable that harvests energy from an NFC equipped mobile phone. The DecoLive Jacket embodies two electrochromic displays, the patterning of which can be controlled by placing a smartphone in one of the jacket's two pockets, expressing the wearer's willingness to interact with others. The Linn Dress (Jarusriboonchai et al., 2019) presents a neckline, which can be dynamically adjusted by the wearer to suit different contexts. For example, a high neckline may be suitable for a work environment while a lower neckline may suit social evening wear. As with the DecoLive jacket (Genç et al., 2020), the Linn Dress uses the electrochromic technology as the display technology (Jensen et al., 2019a).

2.1.3. Accessories

Interactive accessories such as smart watches, bracelets, and digital jewellery exist in vast numbers as commercial products. Whereas a vast amount of HCI research has addressed smart watches and activity tracking accessories, focusing largely on the utilitarian value, digital jewellery emphasizes the aesthetic appearance of wearable technology. An analysis of commercial products has revealed that bracelets and bands are the most common form factors, and LED displays, data tracking, and vibrotactile feedback represent the usual technologies integrated (Silina and Haddadi, 2015).

In the accessories space, our prior study has explored adding smart functionality to, e.g., handbags, shoes, bracelets, and armbands (refer to Figure 5). For some, handbags and particularly the personal nature of their contents are almost sacred. In our study, we prototyped the addition of a display to the surface of a handbag and explored users, reactions to functionality such as changing color to match an outfit and showing a view of the items contained in the bag (Colley et al., 2016). While much work has explored the integration of sensors to shoes, e.g., sensing gait, few works have considered the potential of footwear as a medium for output. In our prior study, we have presented a design space for shoe displays, identifying aspects such as the scope of visibility, content, visualization style, and usage context (Colley et al., 2018). Based on a prototype and user study, the peripheral visibility of shoe displays, e.g., when running, was identified as a potential design parameter that could be leveraged (Colley et al., 2018). The level of visibility was also an issue in our prior study using a pair of bracelets to communicate between partners in a long distance relationship (LDR) (Jarusriboonchai et al., 2020). In this research, we reported that simple communications through a wearable device could provide an additional channel for communication, with the wearable form factor being critical in creating the feeling of being always connected.

FIGURE 5

Figure 5. Examples of the authors' works on smart accessories. L-R: LDR bracelet, smart running shoe, smart handbag displaying the bag's content on its surface.

2.1.4. Body Parts

Integrating interactive artificial body parts has long been a common practice in the medical field, i.e., for upper and lower limb prostheses (Cordella et al., 2016; Windrich et al., 2016), and also HCI has studied adding robotic limbs for utilitarian purposes such as controlling prosthetic limbs with computer-enabled systems (Duvinage et al., 2011). However, enhancing the human expressions and appearance with extra body parts has still been quite rare. Exploratory examples of attaching artificial body parts such as ears (North, 2018), tail (Xie et al., 2019), and shoulder extensions (Hartman et al., 2020) for augmenting the expressiveness of the human body exist.

2.1.5. Digital Aura

Digital technologies have enabled adding a virtual layer around the human body, creating a digital aura. Using augmented reality (AR) techniques, the aura can be visible to others using digital viewers, such as head-mounted displays (HMD), smartphones, and smartwatches or projected into the physical world (Hirskyj-Douglas et al., 2019). Genç et al. (2018) highlighted “extending the expression of the garments to the environment” as a design opportunity for augmenting human appearances. They exemplified this with a night garment that projects light patterns onto physical surfaces around the body as a reaction to body movements and music.

As an AR-based digital representation to extend the self-expression and personal content related to oneself, we have created two prototypes, where we sought to combine the aesthetic design of the physical artifact with a digital aura. We have presented a necklace (Rantala et al., 2018) and a shirt (Häkkilä et al., 2017), both with AR augmentation visible by using a mobile device. The use of digital self-augmentation is well-known in social media, for instance Instagram filters, whereby, e.g., images of a rabbit's ears or a dog's nose are superimposed in real-time onto the image of the user's face. While such augmentations are purely in the digital realm, in our prior research, we aimed to explore the crossover between physical augmentation, i.e., traditional jewelry, and digital augmentation (Rantala et al., 2018). For example, a physical necklace that complements the wearer's appearance in the physical world carries with it digital augmentations when viewed through an AR viewer, such as a smartphone (refer to Figure 6, right). As well as using jewelry as the carrier for virtual augmentation, our prior study has also explored the use of aesthetic graphical markers integrated in to clothing as carriers of content only visible in the AR domain (Häkkilä et al., 2017). In both of these studies, the need to adjust the physically visible design, e.g., making patterns less symmetrical or increasing contrast, in order to optimize the performance as a marker for virtual content was noted.

FIGURE 6

Figure 6. Examples of the authors' works creating a digital aura. L-R: clothing integrated AR markers, AR jewelry.

Digital auras visible in the physical world can also be achieved through projection techniques. For instance, Winkler et al. (2014) have studied constant personal projection around the user. When considering unobtrusive visualization techniques, shadows offer an interesting opportunity, as they are a natural part of our world. Raudanjoki et al. (2020) illustrate, how manipulated human shadows can carry information as both an extension of the human body and linking it with the surroundings. We have also explored the user perception of shadow-based augmentation with 4 scenarios, including a reminder to drink water, a work task reminder, and a supporting visualization during a presentation (Figure 7).

FIGURE 7

Figure 7. Scenarios and examples of user feedback for shadows as information displays. In each image observe the augmented shadow. (Image source: 1. Photo by ANTONI SHKRABA from Pexels; 2. Photo by Cup of Couple from Pexels; 3. Photo by fauxels from Pexels; 4. Photo by Ricardo Esquivel from Pexels).

3. Non-systematic Review on Augmenting Human Appearance

We performed a non-systematic review, aimed at a preliminary mapping of existing research with the lens of our technological design layers to understand design issues around augmenting human appearance. Non-systematic reviews are common in the human-computer interaction field for providing a preliminary view on the issues around novel technologies, i.e., Xue (2019) and Gaspar et al. (2019). In this direction, we follow the taxonomy of (Cooper (1988) about the knowledge synthesis by using literature reviews to frame our approach: The focus of our review was on research outcomes and applications with a neutral representation perspective, a representative coverage of the existing literature, organized by paying attention to conceptual issues. The aim of the review was to reveal central issues from a design perspective. Finally, we target both practitioners and scholars in the area as the audience.

Being a non-systematic review, our approach did not follow a systematic procedure for the selection, analysis, and organization of the existing literature. However, we followed a process for finding articles: Scientific databases (ACM Digital Library and Google Scholar) were used to find articles by entering keywords under six themes: Generic wearability studies and five technological design layers. For generic wearability, we used keywords such as “wearability” and “social acceptability.” For the individual technological design layers, we used keywords specific to the categories in combination with the keywords such as “wearables,” “human augmentation,” “human-computer interaction,” and “body” to narrow the studies down to technological-oriented ones proposed for the human body. For instance, the works on artificial skin and appendages were found by using keywords specific to the category such as “skin,” “tattoo,” “make-up,” “hair,” and “nails.” The clothing and textile category was reviewed by using keywords of its general focus (“garment,” “textile,” and “clothing,”) as well as some familiar form factors that they usually take, i.e., “shirt” and “dress.” A similar strategy was followed for the clothing category with keywords such as “accessory,” “jewelry,” “necklace,” “bag,” and “shoe.” The articles about body parts were found by using variations of keywords like “prosthesis,” “limbs,” “artificial,” and “additional.” Finally, for the digital aura applications, we mostly used enabling technologies as keywords, i.e., “projection,” “AR,” and “Mixed Reality.” During the review, we also included published articles from our work, and some other related work that authors are aware of due to their years of experience in conducting research in the field of wearables. We did not define any time interval as selection criteria. Yet, the review prioritized relatively recent work (published app. in last 15 years) and considered if the article at hand provides design criteria, concepts, implementations, or user study results regarding augmenting human appearance with technology.

Eventually, 72 scientific articles were identified, alongside one artwork. Among them, 8 of the articles provided information regarding generic wearability and social acceptability factors that apply to all categories, 18 of them were about skin and appendages, 12 of them were related to clothing applications, and 16 of them focused on accessories. A total of 9 articles included design issues around body parts, whereas 9 studies were related to the digital aura. In this research, we note that some of these studies provided information in multiple categories. For instance, Dierk et al. (2018a) presented applications for skin and appendages (nail extensions), accessories (hat and shoe), and clothing (t-shirt).

4. Discussion on Appearance Augmentation Layers

As the result of our non-systematic review, we identify five generic areas of challenge, Blending technology, Privacy, Social and Cultural Differences, Customization, and Comfort and wearability. The following sections, we present and discuss the challenges by highlighting specific issues related to each category.

4.1. Blending Technology to the Augmented Human Applications

In addition to bringing new functions, augmented human technologies open up novel expression possibilities to transform the body from a static state to an interactive, dynamic one, the effect of which can be observable from outside viewers. However, the notion of novelty in social contexts is a tricky issue in wearable design that can easily be interpreted as “unfamiliar” and “weird,” leading to ‘negative attention and judgment from the people around’ (Kelly and Gilbert, 2016). Therefore, one of the challenges mentioned for each layer is to find ways to blend novel expressions on augmented human applications without raising concerns of social acceptability. One approach is to design subtle and calm interactions on the body that blend into the natural appearance (Devendorf et al., 2016). This approach is much cited and found useful in many technological design layers, i.e., clothing (Harjuniemi et al., 2020) and accessories (Harjuniemi and Häkkilä, 2018). Supporting the utilization of calm modalities, Koelle et al. (2020) proposed another strategy, familiarity, for deferring social acceptability concerns in interactive products. Yet, this approach advises only accessory-like implementations, limiting the other expressive possibilities for wearables. Reviewing studies through the lens of our wearable categorization, we observed that the strategies around familiarity are also reflected in skin and appendages, clothing and textiles, and accessory levels that researchers often are influenced by traditional practices around body adornment while designing appearance augmentation technologies. On the other hand, the layers of body parts and digital aura often lack the influence from traditional practices that might influence the designs. In the following sections, we report and discuss challenges and strategies about blending technology in each category (Table 1).

TABLE 1

Table 1. Summary of design strategies for blending technology in each category.

4.1.1. Artificial Skin and Appendages, Clothing and Textile and Accessories

One influence of traditional practices appears to be on the material level in the artificial skin and appendages, clothing and textile, and accessories categories. While the emphasis on materials remains similar in each category, the materials used for interactivity are dictated by the existing practices of body adornment.

Applications on skin and appendages require that materials that can be applied to the skin for enabling interactivity are also appropriate to body decorations such as tattoos, makeup, nail polishing, or hair styling. To cope with this, researchers have utilized powder-like materials such as thermochromic and UV sensitive pigments for output purposes (Kao et al., 2016b,c), as well as using leaves of thin metal for building circuits on the skin (Kao et al., 2016a,b). Another trend has been to imitate or augment the capabilities of existing appendage form-factors, that does not appear unnatural on the body. These include mimicking hair with fiber-optic (Brun and Häkkilä, 2021) and nitinol wires (Dierk et al., 2018b), enabling altering the hair's look dynamically, or augmenting hair extensions with sensing capabilities through chemical processes (Vega et al., 2015).

In the textiles & clothing category, material attention is naturally on textiles, the dominant material of garment design (Loschek, 2009), and how interactive components can be blended with it. Strategies include weaving interactive materials within textiles, achieving composites that can be used for touch detection (Mlakar et al., 2021), or output modalities such as color changes (Sayed et al., 2010; Devendorf et al., 2016). However, this category requires consideration on how the interactive parts relate to the overall aesthetic of the garment. This requires an approach whereby the garment is designed holistically, from scratch, rather than attaching interactive elements to existing clothing (Jarusriboonchai et al., 2019; Genç et al., 2020; Harjuniemi et al., 2020). For example, while designing the IdleStripes shirt (Harjuniemi et al., 2020), we selected a shirt form factor where fiber optic elements were placed as vertical stripes, which are considered for producing a slimming effect.

Within the accessory category, commercial jewelry products are often criticized by the researchers for their gadget-like appearances (Wallace et al., 2007; Silina and Haddadi, 2015). These products typically enclose technology in plastic cases as wrist band or watch form factors. An alternative approach was proposed to focus on the materials of traditional accessories. These approaches vary from precious metals (Tsaknaki et al., 2015) and beads (Arora et al., 2019) in jewelry, to textiles in bags (Harjuniemi and Häkkilä, 2018) and leather in shoes (Jensen et al., 2019b).

The second emphasis, in design strategies of artificial skin and appendages, clothing and textile, and accessories categories, is on seeking existing gestures and materials affordances of existing practices. In this line of research, studies revolve around leveraging explicit ways of interacting with skin, nails, and hair (Weigel et al., 2014; Kao et al., 2015; Vega et al., 2015; Brun and Häkkilä, 2021). For instance, Vega et al. (2015) leveraged touching hair as a secret input, whereas (Weigel et al., 2014) explored on-skin gestures like scratch, squeeze, and twist with the same purpose. As we move away from the body, in the clothing and textile category, researches focus on textile, i.e., the haptic sensation of a fabric's texture (Mlakar et al., 2021), as well as how people interact with other garment parts - such as zippers (Ku et al., 2020)—for exploring gestural affordances of materials. Dunne et al. (2014) speculate a body map highlighting areas on the body—for example, waist, arms, neck—where wearers already touch to interact with garments. Adding onto those, for instance, Genç et al. (2020) exemplified how existing interactions like putting a phone into the pocket can trigger interaction, changing dynamic patterns on a jacket in this case. Further from the body, explorations about affordances of materials are also done in a similar fashion for accessories. For example, Arora et al. (2019) did a systematic exploration of jewelry form factors. They concluded touch and manipulation gestures, such as moving beads of a necklace or touching on an earring surface. We also examined interactions around accessories. An example is a smart handbag that utilizes the gesture of putting keys into the pocket for interaction (Harjuniemi and Häkkilä, 2018).

4.1.2. Body Parts and Digital Aura

Categories of body parts and digital auras lack common references to traditional practices. This may be due to the fact that they are more recent augmentations, enabled by developments in technology. However, tendencies exist toward animalistic influences for appearance augmentations in the body parts layer. For example, Xie et al. (2019) demonstrated a furry cat tale worn by humans to augment their expression and whereas (North, 2018) implemented pair of ears that imitates horse ear movements on human head. Another approach in this category is to attach extra limbs—such as arm (Sasaki et al., 2017), thumb (Meraz et al., 2018), and shoulder extensions (Hartman et al., 2020). Although not mentioned explicitly, these applications align with notions of mimesis, physical body augmentations to resemble something else, that have a long history (Willerslev, 2007). For example, a hunter wears fur and wears the smell of an animal to become like the animal that she is hunting. In HCI, this concept has been examined from the perspective of robots (Dörrenbächer et al., 2020). Further examples of these transformations are apparent in the cosplay subculture, where costumes transform the wearer to become fictional characters from movies or video games (Polvi et al., 2019). Such approaches may be fruitful in the design of artificial body parts and help to blend technology in functional application cases, i.e., exoskeletons and body prosthesis.

In the digital aura category, physical auras are often based on projecting visuals on and around the body. However, these applications lack existing practices for developers to base their design decisions on. In this research, Genç et al. (2018) highlighted the potential for the extension of a garment's design by projecting light marks on the ground. Similar approaches are also proposed for dynamically changing the patterns of the garment by using projection displays to project light on the garment (Kleinberger and Panjwani, 2018). Researchers also used the background of the wearer projected on a garment to make the body invisible (Ahuja et al., 2013). In our study (Raudanjoki et al., 2020), we found it useful to use shadows as a metaphor for designing digital body extensions. However, our user evaluations suggested that maintaining the natural perception of human shadows is critical, with any deviations feeling spooky.

Finally, digital auras with AR technologies often include a physical element attached to the body, such as AR markers or green surfaces to be overlaid, which are also subject to blending challenges similar to the first three categories (Mackey et al., 2017). In this research, our experience also showed that when designing AR markers focusing on the aesthetics of traditional body adornment practices, i.e., jewelry form-factors or clothing badges, designers must provide enough contrast in visual design for them to be readable by AR marker tracking algorithms (Colley et al., 2016; Häkkilä et al., 2017).

4.2. Differences in Social and Cultural Perception of Augmented Humans

4.2.1. Artificial Skin and Appendages, Clothing and Textile, and Accessories

Our review results suggest that the perception of appropriateness in adorning one's body varies in across different cultures and social contexts in skin and appendages, clothing and textile, and accessory layers. In regard to this, some generic wearability studies presented body maps, highlighting cultural annotations, e.g., sexual body parts, hinting that the developers should be precautions while developing body augmentations that require interactivity around the genitals (Dunne et al., 2014; Zeagler, 2017). Similar concerns were raised by participants in our studies on accessories. For instance, while reacting to a wearable display with vibrant color changes indicating busyness of the wearer in a necklace form factor, our female participants were uncomfortable with a display close to their chest (Häkkilä et al., 2020). On the other hand, during the evaluation of the Idle Stripes Shirt (Jarusriboonchai et al., 2019), a shirt with fiber optic fabric as the subtle display on the chest area for women, the participants did not raise any concerns about having a display on the chest. We believe the conflicting results of the two aforementioned studies suggest that the placement of interactive features on the sexual body parts also requires additional attention paid to the subtlety of interaction for not attracting the gazes of others around. Furthermore, related to on-skin displays, two studies (You et al., 2019; Kao et al., 2021) reported cultural differences between participants from Taiwan and the US in the social perception of on-skin applications. For example, Taiwanese participants were more skeptical about placing technology on the skin when compared to participants from the US (You et al., 2019). Associated meanings of on-skin displays have also noted variations between participants from Taiwan and US (Kao et al., 2015).

4.2.2. Body Parts and Digital Aura

In regard to different social and cultural perceptions, the field lacks cross-cultural or social acceptability studies examining digital auras and artificial body parts. From a design perspective, although it might be impossible to satisfy all cultural and social variations with research prototypes, we believe it will be important to examine cross-cultural differences in the perception of augmenting human appearances to understand the cultural constructs affecting the social acceptability of these relatively unusual human augmentation layers in different contexts.

4.3. Privacy and Designing for the Public Appearance

Augmented appearances are not just cosmetic changes on the body, but, with integrated computation, they are capable of modifying appearance based on data sensed from the wearer's body or other devices—, thus, functioning as public displays. In an exploration of perception and motivation of in-situ display of tracker data in wearable form factors, Colley et al. (2020) reported that data source and valance, context of use, presentation form factor, and data representation had a strong influence on the user experience. These parameters can be argued to be important for each category. For instance, data sources such as information related to sleep and biometrics, e.g., heartrate, can be perceived as more personal than displaying information such as environmental temperature, with the valance of displaying these data publicly varying from person to person. Furthermore, use context affects the perception of displaying data in terms of where and to whom the data is displayed. The context influences perception as it forms part of the wearer's public image, but also opens up possibilities for friendly competition or receiving support from others, e.g., displaying performance data in gyms (Genç et al., 2019). For the data that the wearer does not want to share with the others, choosing modality and information encoding are of paramount importance for all categories. In these cases, choosing discreet modalities, such as thermal or haptic, and encoding information as abstracted visualizations that are publicly visible but not understandable without knowing the meaning are suggested (Inget et al., 2019). Examples of different ways of presenting information in the abstracted form are presented in Table 1 for each category.

In addition to the aforementioned strategies, our review revealed that each category presents different affordances, due to their possible locations on the body, as well as possibilities for switching from public to private :

4.3.1. Skin and Appendages, Clothing and Textile, and Accessories

Accessories are usually worn in visible locations, such as the neck, ear, foot, and arm. However, for smart jewelry, if an accessory is designed as a public display, designers select upper and more visible regions of the body, such as the head, neck, and chest; while hands and fingers are considered appropriate locations for private displays (Inget et al., 2019). Moreover, You et al. (2019) examined the public and private perception of on-skin interfaces and reported that placement on the outer forearm and hand are more public, whereas the inner forearm, collarbone, and back of the neck are considered private locations. The same study also highlights accessibility concerns of on-skin displays, due to clothing coverage. This connection between clothing and skin applications could provide possibilities for switching between public and private states of the on-skin interfaces, e.g., by covering and revealing the display with sleeves. Clothing applications can also enable similar cover and reveal interactions, since they are usually worn in layers, e.g., a t-shirt worn under a zippered sweatshirt, allowing users to adjust the public visibility of the t-shirt. Examinations of affordances in switching public and private states of skin and clothing applications could provide interesting possibilities for future study.

4.3.2. Body Parts and Digital Aura

Although augmented body parts are naturally public displays, as abnormal extensions visually protruding from the body, we were unable to find any prior works addressing public and private perceptions of such. Similarly, projection-based aura applications could be argued to be public in nature, e.g., Winkler et al. (2014) demonstrated switching a projected display from public (on the floor) to private state by putting a hand in front of the projector. We believe this could be further explored for other projection-based aura applications. Concepts using AR-based digital aura present an interesting design space for privacy: The content is virtual and only visible when other devices such as mobile phones read the AR marker. This provides a level of control to the wearer over the publicness of the content by camouflaging it. For example, in a work on clothing integrated AR markers (Häkkilä et al., 2017), participants were able to occlude AR markers with clothing such that it could not be read. Similarly, AR markers in a necklace form factor afford similar functionality by placing the jewelry under clothing (Rantala et al., 2018). On the other hand, AR markers placed on the backside of the body caused privacy concerns when the wearer could not observe the tag being read by others (Häkkilä et al., 2017). Similarly, Mackey et al. (2017) presented a garment prototype with large “green screen” areas on its surface to enable overlaying virtual visuals by using an AR application on smartphones. This article reported that the wearer gets annoyed when somebody else modifies their appearance without the wearer's consent.

4.4. Customization: Supporting Individual Expressions

Modifications of the visual appearance of our body require a careful examination of social concerns, such as how they impact personal style (Barnard, 2013; Dunne et al., 2014). Sharing the same design space, augmented human technologies compete with the traditional ways we adjust our appearance and, therefore, are subjected to similar design requirements. The possibility for customization is one approach to address the needs of complementing personal style (Colley et al., 2016; Pateman et al., 2018). In a comprehensive review of the wearables market, it is highlighted that commercial wearables supported the appearance customization to a certain level, e.g., by providing options (i.e., straps, attaching different body locations) for wearers to choose from or letting users modify the appearance digitally (changing watch face, color of the light feedback) (Jarusriboonchai and Häkkilä, 2019). However, free-style customization for personalizing the whole appearance of the wearables was reported as harder to achieve.

In what follows, we report customization strategies observed in each layer:

4.4.1. Skin and Appendages

The skin and appendages category is exception in terms of providing strategies for free-style customization. The augmentations in this layer affect the closest and most private part of the body. Crafting their look is highly personal and wearable design approaches with pre-designed forms would not be acceptable (Kao, 2021). In this category, we can observe that research has often focused on making manufacturing processes accessible, such that users can customize the appearance, rather than solely presenting research artifacts. These approaches include using screen printing with off-the-shelf materials such as thermochromic inks and conductive metal leaves (Kao et al., 2016b; Wang et al., 2017), guiding design processes and design explorations with projected visuals on the skin, and developing new powder-like (Pourjafarian et al., 2021) materials that users could apply by themselves (Kao et al., 2017a). However, these kinds of manufacturing processes still require technical knowledge on how to design and manufacture or circuits for interactivity.

4.4.2. Clothing and Textile and Accessories

Customization strategies in the clothing and textile and accessory categories present pre-designed wearables that can change shape or modify their color and patterns, e.g., Dierk et al. (2018a) enable users to change the visual content on e-ink displays embedded in clothing and accessories. Juhlin et al. (2013) proposed an outfit-centric design strategy where shape and color-changing accessories can fit the visual style of the wearer's outfit. Kao et al. (2017b) demonstrated robots on garments that move and change the patterns of the garment, etch visual marks or become shape-changing jewelry. With an alternative approach, rather than changing the colors of the form-factor of the garment, the Linn dress (Jarusriboonchai et al., 2019) allows wearers to reveal or conceal areas of skin with electrochromic displays, appropriating the garment to different occasions.

4.4.3. Body Parts and Digital Auras

Augmentation through body parts and digital aura reveals customization challenges that are not apparent in other augmented human categories. For example, the movement of added body parts is subjected to personal expression, exemplified by, e.g., North (2018)'s horse ear prototype that enables users to customize the movements of their ears. Since the digital aura category transfers the expressive content to the digital space, Mackey et al. (2017) highlight the need for duration of such personal content.

4.5. Comfort and Wearability

Earlier frameworks addressing wearability and comfort identify parameters such as placement, form language, human movement, proximity, size, attachment, containment, weight, accessibility, sensory interaction, thermal, aesthetic, and long-term use (Gemperle et al., 1998). However, as technology advances and new wearable categories emerge, new dimensions have started to appear. To provide insight in to this, we discuss the wearability and comfort parameters specific to each wearable category.

4.5.1. Skin and Appendages

Implementing technology onto skin and its appendages identified new considerations due to skin conductance: First of all, human skin varies in characteristics such as skin tone (Kao et al., 2021), bodyscape (Pourjafarian et al., 2021) or wrinkles, body hair, moisturization level and the existence of deformations such as eczema and scars (Liu et al., 2016). Also, attaching electronics to skin creates durability challenges, e.g., robust attachment of rigid electronics to withstand daily activities (Kao et al., 2018) and breakages of thin conductive surfaces due to skin flexing (Kao et al., 2016a, 2018; Liu et al., 2016). Skin safety and insulation are other parameters that must be considered when applying unfamiliar materials, such as heating layers for thermochromic applications (Kao et al., 2015, 2016c) or chemically processed powders (Kao et al., 2017a), in close proximity to the skin. Finally, another challenge is the attachment of rigid components directly to the skin. This situation introduces power requirements and requirements for wireless communication as design parameters (Liu et al., 2016). Most of the applications, instead of trying to attach electronics to the skin, demonstrated interactions with external setups (e.g., Kao et al., 2016a; Dierk et al., 2018b). One approach is to omit the electronics completely by focusing on NFC based applications (e.g., Kao et al., 2016b) or applying materials that change based on environmental conditions (Kao et al., 2017a). Alternatively, some have located the electronics in neighboring accessories, such as hair ties (Kao et al., 2016c) or manufactured tiny flexible circuit boards to locations such as fingers (Kao et al., 2018) and nails (Kao et al., 2015).

4.5.2. Clothing and Textile and Accessories

For clothing and textile applications, by interpreting wearability from a functional clothing perspective, washability, sizing, and fit should also be considered (Dunne, 2008). In our studies, we identified challenges of attachment and tightness (Buruk et al., 2021) of the accessories which affect their usability. For instance, for a smart bag design, participants preferred a strap that passes diagonally across the body, providing a tight secure attachment during activities such as cycling (Pakanen et al., 2016).

4.5.3. Body Parts and Digital Auras

Research suggests that extra body parts need specific attention paid to the obtrusiveness of the extension during use (Ding et al., 2021), as well to the possibility of unbalanced weight which can lead to discomfort at the attachment points (Sasaki et al., 2017). Finally, in digital auras, visibility of virtual content under bright lighting conditions have been highlighted as challenging for projection type applications (Winkler et al., 2014).

5. Conclusion

In contrast to the field's emphasis on technical and functional validation of augmented human technologies, in this study, we focused on the design issues about hedonic and social aspects of augmenting the human appearance through technology. To examine this, as a first contribution, our article presented a categorization of and examples for technology layers on augmenting human appearance: Skin and appendages, clothing and textiles, accessories, additional body parts, and digital auras on and around the body. As the second contribution, we reported the results of a non-systematic review of related works through the lens of our categorization to capture common and explicit challenges of different technology layers for augmenting human appearance. This resulted in the detection of broad challenges that apply for all technology layers, namely, Blending technology, Privacy, Social and Cultural Differences, Customization, and Comfort and wearability. We also detailed the specific strategies identified in each category to cope with the mentioned challenges.

In conclusion, our study reported that a vast amount of work and strategies to cope with challenges exist for skin and appendages, clothing and textiles, and accessories. These strategies often make use of the already established traditional practices for designing augmented appearances with technology. On the other hand, one of the aspects evident in this review is that the field needs more comprehensive examinations about digital auras and body parts on how to blend technology on the body. In this direction, the researchers need to create more concepts, implementations, and conduct evaluations concerning the challenges around social acceptability, cultural and social differences in their perception as well as how they relate to the customization needs of their users.

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author Contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

Funding

This research has been supported by a grant from the Academy of Finland as part of the Tech Fashion – Design of Future Wearable Computing Project.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^https://cutecircuit.com/galaxydress/

References

Ahuja, R., Soni, S., Yadav, A., and Thakur, M. (2013). Invisibility cloak: Science fiction transforming into reality. Int. J. Comput. Appl. 82:16–19. doi: 10.5120/14176-2289

PubMed Abstract | CrossRef Full Text | Google Scholar

Arora, J., Mathur, K., Saini, A., and Parnami, A. (2019). “Gehna: exploring the design space of jewelry as an input modality,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (New York, NY), 1–12.