Virtual medical astronaut avatars for future deep space missions

This paper explores the telemedicine possibilities available to astronauts during spaceflight, focusing on the possible utilization of medical avatars for continuous health monitoring. Various types and sources of telemedicine are examined, along with their applications in deep space travel to address the challenges faced by astronauts and facilitate timely intervention in case of injuries and illnesses. The use of medical avatars presents a promising solution to mitigate risks and ensure effective healthcare delivery in the absence of immediate physical medical support. Furthermore, potential terrestrial applications of telemedicine and virtual medical avatars are discussed, highlighting the wider implications and benefits beyond space exploration. By leveraging advanced technologies and innovative approaches, telemedicine in conjunction with medical avatars offers a compelling avenue for enhancing healthcare practices both in space and on Earth.

1 Introduction

1.1 Telemedicine

While telemedicine evokes several ideas and images, for the purpose of this paper we define telemedicine as the remote diagnosis and treatment of patients by means of telecommunications technology (Oxford Languages, 2022). Most often, this includes the use of a telephone or the internet to remotely connect a patient and a healthcare provider when an inperson visit is not possible, for reasons ranging from disabilities to nearby availability of healthcare professionals (National Center for Immunization and Respiratory Diseases Division of Viral Diseases, 2020).

The acceptance and adoption of telemedicine options vary among individuals, with some displaying higher levels of trust while others remain sceptical. This applies to both patients and healthcare providers. Reifegerste, Harst and Otto (2021) have recorded several factors that influence the success of different telemedicine applications. Their main conclusion is that the acceptance and use of telemedicine largely depends on different stakeholders’ “readiness.” Whether or not there is a need and a desire for change is of great impact if telemedicine applications are expected to be taken up successfully. One can follow the implication that, when it comes to space flight, the need and desire for change are both present. The need shows itself as the need to keep healthy and safe while being in an extreme environment, and the desire for change is visible in the space agencies’ and its astronauts interest in innovation and exploration (see, for example, ESA’s Erasmus Innovation Center and NASA’s Center of Excellence for Collaborative Innovation).

1.2 COVID-19 pandemic

Since the COVID-19 pandemic, medicine has had to transform globally as more and more people made use of the options of telemedicine. As social distancing and quarantine became the norm, so did remote healthcare. Telemedicine has moved from radio or simple phone calls to video calls, online planning and exchange of information and virtual support groups (Zahedi et al., 2022; Primholdt Christensen et al., 2021).

One large disadvantage of this globally heterogeneously developed telemedicine, is the fragmentation of sources of information and options for care. Between the video calls with doctors and specialists there is no continuous monitoring or follow up of care, and different sources of health data (such as patient records or data from wearables) are not integrated with each other or interoperable between different applications (Scott et al., 2023; Zhang et al., 2014). The COVID-19 pandemic is similar to the pandemics as described by Reifegerste, Harst and Otto (2021) in their paper on readiness and barriers in telemedicine. These pandemics were demonstrated to temporarily remove some of the main barriers for strategic and technological readiness, with lacking infrastructure and problems regarding data security. In order to improve the uptake of telemedicine, even after a pandemic, any application would need to be ubiquitous, easy to use and support privacy and security of data. Extra care should be applied when it comes to making sure the right parties are ready and willing to invest in the infrastructure and technology needed to make telemedicine work, as even with the best intentions there will be a barrier regarding uptake if the practical implementation isn’t there.

2 Space exploration

In human space exploration there are many situations in which an astronaut’s vital signs and health are important to be monitored, for example, during launch, extravehicular activities or exercise in microgravity conditions (Cermack, 2006). Information such as blood pressure, oxygen levels and heart rate give insights into the wellbeing of the astronauts, and possible medical interventions that need to be taken (Simons et al., 2006; Doarn et al., 1998). Extra care needs to be taken with astronauts, as medical care is more difficult to come by than on earth and new health challenges are being faced where the effect of (long term) spaceflight and microgravity on the human body and mind is not well known – see, for example, cardiovascular deconditioning (Gallo et al., 2020).

We hypothesize that the use of telemedicine, and more specifically virtual medical avatars, could provide benefits to the monitoring and followup of astronauts’ health in long term space travel, where traditional methods of medical care no longer suffice.

The U.S. space program has been interested in telemedicine since the 1960, finding its use especially during the race to the moon where its astronauts were several days away from both regular medical or emergency evacuation (Doarn et al., 1998). More recently, NASA has focused on implantable biomicroelectromechanical systems or bioMEMS (Simons et al., 2004). The advantage of these bioMEMS mainly lies in their size and ease of use, a small implantable sensor is unobtrusive and can be powered by induction, thus not requiring batteries and limiting the risks for an infection, and the wireless receiver can be placed in a garment or other wearable, improving mobility.

Another solution could be akin to the commercially available “Astroskin,” a lightweight garment with builtin biosensors developed by Hexoskin and evaluated by the Canadian Space Agency and NASA ARC (Kumar et al., 2015; Hexoskin, 2024). This garment has the capability to record electrocardiograms, blood pressure and oxygen, temperature, activity level and respiration rate without the need for implants and without being obtrusive, as well as having the capability to monitor over longer amounts of time and thus being useful for spaceflight and exploratory missions.

This information is gathered onto its own platform, which offers data hosting, data science and AI services, meaning it uses proprietary software which makes it easy to use, but difficult to integrate with other applications. Once again fragmentation of services is an issue here.

It seems that research in the area of wearables and telemedicine as a whole is ongoing but lacking a full overview of its design and implementation within the larger context of deep space. During deep space exploration missions more attention has to be given to the mental and physical health of the crew, as problems in these areas will form a larger threat to the mission than in short term spaceflight, with no capability for quick evacuation or resupply. Simultaneously, the focus will have to shift from urgent care and minor issues to preventive and diagnostic, and possibly curative, care on the long term.

The European Space Agency has done its own research in the area of human hibernation during deep space missions, mainly looking into how to monitor astronaut health while in a state of induced torpor (Chouker et al., 2021). They as well mention the importance of Artificial Intelligence units receiving information from different sources such as the life support system and physiological parameters of the crew, through wearable activity trackers, temperature measuring devices and photoelectric sensors.

Another aspect of this telemonitoring has been tested previously with ESA astronaut Thomas Pesquet, using Microsoft’s Hololens Kinect camera and Aexa software to create a 3D model of people on the international space station (Innovative 3D Telemedicine to Help Keep Astronauts Healthy, nasa.gov 2022). It seems that telemedicine is a growing subject, with current events by ESA on biomedicine and medical research such as ESA commercialisation Days explicitly giving time to explore telemedicine and remote diagnosis, treatment and monitoring.

3 Virtual medical avatars

Continuing in the direction of virtualisation and artificial intelligence, virtual medical avatars can be of interest when looking at cuttingedge technology for medical applications, especially in remote environments. Avatars can be defined here as the virtual image which represents the person using a program or application.

There are several different ways in which avatars can work for medical applications. Relevant measurements such as heart rate and blood pressure can be continuously monitored using wearables (such as smart watches) or dedicated sensors. The use of bioMEMS can introduce more specialised physiological data when necessary (Simons et al., 2004). At the same time, this information can be supplemented with context information gathered from external sensors such as oxygen percentage and humidity which are already constantly monitored throughout the missions, Scott et al. (2023) explain the importance of combining different types of devices when looking at how to monitor the large spectrum of flight data for the health of astronauts. These data from different sources can be uploaded to a local server via wireless local transfer (for example, Bluetooth), where it can be stored and analysed for further use. This is similar to the concept of the “sensor web” by Cernack (2006), networks of smart sensors connected to a server that reads, processes and stores the data gathered by the network. Currently this concept is also known as “The Internet of Things” or IoT for short, a network of wirelessly connected devices that can operate without (a lot of) human intervention. The wireless communication can be any kind of near field communication, such as Bluetooth mentioned above or radiofrequency identification (Li et al., 2014; Zhang et al., 2018).

The analysed data from the different wearables and sensors can then be visualised in the form of a virtual medical avatar, a digital representation of the astronaut that shows all relevant physiological information. For those familiar with online gaming, an avatar is not a new concept. From the elaborate world of Second Life to the World of Warcraft, avatars have long been an easy way for people to represent themselves online. A medical avatar would work in much the same way, being a visual representation of the self, but would align closer to the actual physical form and physiological status of the astronaut, sometimes also called a “digital twin”. This concept originates in the engineering world, defined as “a virtual representation of a physical product containing information about said product” (Jones et al., 2020) where a digital twin of a complex system allows for quicker and cheaper testing than doing so on the reallife system. The peculiarity of digital twins is that it aims to match the realworld situation as closely as possible, through using as much information and data as possible. In the medical setting, digital twins integrate large amounts of information relevant to the kind of diagnostics being done (Björnsson et al., 2020). However, creating an actual digital twin of a person is a difficult and labour intensive endeavour, which is why we opt here for a ‘medical avatar’ which still includes a large amount of information on the astronaut in question, but does not quite reach the levels of a digital twin.

Medical avatars would be a graphic representation of the astronaut themselves, showing their basic physiological and vital information at a glance. For example, the gender of the astronauts can be represented, taking the form of male or female representing avatars. Next, their posture at rest in microgravity can be shown in the posture of the avatar, and relevant medical information can be visualised on the relevant body part, such as heart rate and blood pressure at the level of the heart, or respiratory rate at the level of the lungs. The medical background can be included, previous injuries, surgeries or known issues can be shown at the site of said injury.

The virtual medical avatar could be consulted by an onboard medical doctor for information during checkups or medical interventions, and if necessary the medical doctor can input missing or new data straight into the local server which in turn updates the medical avatar. If desired, the medical avatar can also be available to the astronaut themselves, allowing for quick and easy selfmonitoring of their health and updating with the latest information. For example, an old injury that starts bothering the astronaut again might not be necessary to pass by the onboard medic, but can be added to the virtual avatar for extra information (Figure 1).

Figure 1

Figure 1. Example design of virtual avatar system for medical monitoring

Information stored on the local server can be relayed to ground control through the Space Network and other satellites, offering the same visualisation to specialized doctors who can offer their insights if necessary. This is especially crucial during longer missions, where there is no option for quick evacuation or for support during medical interventions that the onboard medic has limited experience with. A secure connection for a videoconference between the astronaut, medic and/or a specialist can be set up for diagnosis or training. However, as missions go further into deep space, the option for direct communication with specialists becomes limited due to time delay, and medical decision making will rely more on onboard options. This is where data analysis and artificial intelligence on the local server can come into play: with access to realtime medical information, analysis through deep learning can be done on the acquired data to provide immediate options for treatment or intervention in urgent situations. NASA has reviewed a similar concept in a workshop on Artificial Intelligence and Modelling for Space Biology, where they’ve placed emphasis on “a maximally automated, autonomous, and intelligent Precision Space Health System, to monitor, aggregate and assess biomedical statuses” (Scott et al., 2023).

3.1 Advantages

A choice for the representation using virtual medical avatars of a data stream is advantageous for several reasons.

3.1.1 Ease of evaluation

As the data of the virtual medical avatars is preanalysed, the onboard medic doesn’t need to do so themselves, thus allowing for quicker and easier evaluation of the situation. For example, a dangerously high blood pressure result would not need to be interpreted based off measurements, but can directly be shown in attention grabbing visuals on the virtual medical avatar.

Garot et al. (2019) found in their study of Visual Patient Technology that the use of avatar visualisations reduced perceived workload as well as improved perception of information in short glances. This method of “direct display of information” and graphical displays allowed for faster detection of changes and a more successful treatment, both of which are of benefit in the extreme environment of deep space where decisions need to be made quickly and effectively, often based on limited data. The inclusion of predictive models and/or artificial intelligence which uses the sensor and historical medical data available can then be added into intuitive visualisations on the medical avatar, where the onboard medic or astronaut themselves do not need to rely on interpretation of numerical or technical results to have quick understanding of the situation.

3.1.2 Ease of use

Graphical, easily interpretable representations of medical data and vital signs bring about another advantage: the ease of use. Garot et al. (2019) used a short introduction video for users to get acquainted with the program, and found that such a short training sessions was enough to improve recall. This also has the advantage that the people interacting with these medical avatars do not need to be experts or specialized medical personnel but can still make informed decisions after a short learning period. In a way, this will also allow the patient to make informed decisions on their own situation, in collaboration with their medical team.

This medium of graphically representing multisensor data also makes it easier to offer continued observation over longer periods of time. Where vitals like blood pressure or heart rate are usually only looked at during specific evaluations, the medical avatar can give a continuous view of vital signs, allowing for constant monitoring that is helpful in cases where the patient has to be left alone for logistical or safety reasons, or when monitoring is otherwise impossible such as during extravehicular activities. On Earth, continuous monitoring using wearables for sepsis cases has been correlated with faster treatment and shorter average length of hospital stay, as well as less readmission in the month after discharge, though the use of these wearables can be rather complex (Downey, Randell, Brown & Jayne, 2018), positive outcomes that, if transferrable to space flight, might be advantageous for astronauts’ health.

3.1.3 Acceptance by user

Zahedi et al. (2022) model of “real” avatars – an avatar that they define as embodying a patient’s vital signs in real time as well as medical history shows that patients possibly form emotional attachment to their avatars when they carry private medical information, which in turn improves the acceptance of virtual medical monitoring. There is also evidence for the use of avatars helping patients track their own health progress and improving satisfaction with their medical followup, especially when an avatar closely resembles the patient (Zahedi et al., 2022).

With the main methods of collection of telemetry data being nonintrusive wearables and sensors, the acceptance rate by users should also be higher. Medical monitoring is only useful when properly executed and accepted by the enduser, as nonuptake or misuse can create missing or false data which, at best, makes the technology useless, and at worst, can cause false information and misguided medical decision making.

This is also where the concept of “readiness” by Reifegerste, Harst and Otto mentioned earlier comes into play. Readiness here means the willingness of users to participate in the telemedicine project. Their analysis showed a higher readiness to use telemedicine when it was the only option to access healthcare, with barriers preventing success. One can reasonably assume that, with some barriers like intrusiveness in this case removed, readiness of the users would improve. So the ease of use and understanding of virtual medical avatars should also help with improving user uptake.

3.1.4 Emergencies

In emergencies, quick decision making based on limited information is paramount. When in an extreme environment such as deep space, where there are limited resources and outside help, this process becomes more complicated. In these cases, the astronauts could rely on artificial intelligencebased pattern recognition and analysis to support them. All of the information stored on the local server, supplemented with the data from relevant sensors can provide enough information for AI to offer deep learning based decision support systems, as has been done with Watson for Oncology (Harish et al., 2020), where AI can learn based on human clinical knowledge but go further to identify patterns not easily recognized by humans. It can then offer different suggestions for intervention to the onboard crew, based on different levels of certainty. In a total of 160 cases, Watson for Oncology’s treatment recommendations followed hospital recommendations in as high as 95% of cases (Yue and Yang, 2017).while in another study conducted by Kris, Gucalp et al. (2015), the results matched in approximately 50% of the cases with those of medical oncologists.

3.1.5 Preventive care and continuity of care

Since deep space explorations missions will be of a larger timeframe, the goal of inflight medical care will have to shift to making sure the astronauts stay healthy over a longer period of time, in more dangerous conditions. It will become important not just to treat illnesses or injuries as they come up, but also to prevent them. Once again, machine learning can be of help in these situations by seeing early patterns and warning the astronauts or medics when there is a change noticeable. In early enough stages, injury or illness may be prevented by selfcorrection. For example, the system can warn the astronauts after several nights of lack of sleep, allowing them to prioritise rest before fatigue starts to impact performance.

After an injury or illness has occurred and treatment has been given, continued monitoring and care will need to be applied. Continuous monitoring will allow for trends to become visible, and allow for changes in treatment or procedure when necessary. As in the previous example, once a lack of sleep has been noticed, one can keep account of the different treatments tried and whether they are getting things back in the right direction, based on the amount and quality of sleep that follows. An evolution of the progress can then also be made visible and monitored over time. Such measures can as well be visualised on the avatar or in more in depth displays for easy interpretation.

3.1.6 Simulations and modelling

A fully developed digital twin of the astronauts would also allow for simulations and tests to be run prior to the actual spaceflight. Several scenarios and emergencies can be simulated, and the effects on the physiology of the astronauts can be made clear and prepared for, without having to put the astronauts through rigorous testing. The longterm effects of spaceflight can be constructed on the digital twin, enabling personalized preventive measures to be created to avoid worst case scenarios. For this purpose, the artificial intelligence and machine learning networks mentioned earlier will also be necessary.

A second advantage comes up when we’re looking at machine learning algorithms being used: while the artificial intelligence learns based on learning data prior to a mission, it can also learn and adapt during the actual space flight. This makes it so that with every use and every subsequent mission, the algorithm will be smarter and eventually be able to prevent and understand most issues that may pop up, making it more useful as it goes along.

3.1.7 Enhanced communication

Avatars provide a visual medium that enhances communication between medical personnel and patients. Especially in telemedicine scenarios, where facetoface interaction is limited, avatars offer a more engaging and personalized way to convey medical information. This visual representation can lead to better understanding and adherence to medical advice, ultimately improving patient outcomes.

3.1.8 Customization and personalization

Imagine a digital twin of a patient, created specifically to match their unique characteristics. This is the concept of a personalized avatar. By tailoring the avatar’s appearance, behaviors, and even preferences to mirror the patient’s reallife counterpart, medical professionals can create a more empathetic and supportive environment. This can lead to increased patient satisfaction and engagement in their own healthcare.

3.1.9 Remote monitoring and intervention

With the advancement of wearable technology, avatars can serve as a central hub for remote monitoring of patients’ health metrics in realtime. This continuous monitoring enables early detection of potential health issues and allows for timely intervention before they escalate into more serious problems. Additionally, avatars can be programmed to provide automated reminders for medication adherence and lifestyle modifications, further promoting proactive management of chronic conditions.

3.2 Difficulties

3.2.1 Privacy

One main concern in this use of medical information is the security and privacy of confidential data. All methods of communication and data transfer would have to be properly secured to ensure that only the relevant parties have access to sensitive information. If third party applications or hardware are being used, they too will have to be assessed on privacy and information security. However, as there has been previous research and real situations with medical followup of astronauts, one can safely assume that secure transfer of highly personal data is something that has already been looked into, and has, at least in part, been established. In fact, data privacy has been at the forefront of ESA’s Telemedicine Alliance as early as 2003 (Data Privacy and its Impact on Implementing Crossborder Healthcare in the ESA Telemedicine alliance, 2003)

Privacy from third parties is not the only concern for astronauts under constant monitoring, as some astronauts might object to being under constant observation in the first place. As this medical and health data that is being gathered can have implications on future fitnesstofly, some astronauts might want to avoid this all together, preferring to shy away from possible medical problems popping up that might exclude them from future space flight. It becomes a question then of whether or not astronauts can optin to using medical avatars and being constantly monitored, or if this will be a requirement for longer duration missions.

This can be compared to airplane crew’s fitnesstofly mandatory evaluations, which are often seen as “inconvenient,” or even create bias against disclosing medical issues (Grout and Leggat, 2021). This can partially be seen as a consequence of the power imbalance present when disclosing medical or psychological issues to an employer.

When looking at the acceptance rate of wearables in terrestrial medical settings, often there is a certain percentage of people who opt out or discontinue their use, for reasons ranging from the sensors being uncomfortable or not giving a reason at all (Downey, Randell, Brown & Jayne, 2018).

3.2.2 Time delay

When used in deep space, certain features of the above proposed system become less feasible. The communication between ground control and the inflight crew is no longer instantaneous, for a Mars mission, for example, the backandforth communication can take anywhere from 10 to 48 min, depending on the orbit. This would of course limit the live follow up of medical data as imagined with virtual avatars. However, for nonemergency situations such follow up can still be of value, and the use of comprehensive machine learning models may assist in quick decision making during emergencies. In any case, realtime communication or manipulation from ground centres to deep space will be close to impossible, requiring a shift toward insitu decision making assisted by onboard medical databases and AI (Cermack, 2006).

3.2.3 Technical issues

The proposed design is not without its technical issues. For starters, all wearables/sensors and bioMEMS would have to transfer their data to the same server, when many currently used health monitoring devices use proprietary applications or software. Think, for example, of the apple or Fitbit watches, or the kino.cardio cardio monitoring app developed by Heartkinetics. For the virtual avatar to be useable, all information needs to be able to be stored and analysed on one local server instead of fragmented over several applications.

Another technical issue will be relaying highly detailed information over long distances. As medical imaging requires large volumes of data (when transferring images like echography’s to ground control units), transferring these reliably from deep space will form a challenge in the future. Smaller files such as blood pressure information or blood oxygen will be easier and quicker to transfer over large distances, but may not offer enough information when specialized consultations are necessary. The European Space Agency has already looked into use of its Deep Space Antennas and optical communications to enhance its communication capabilities (ESA, Deep space communication and navigation, 2022) for future deep space missions.

The general design also relies heavily on the use of machine learning and rather advanced models of artificial intelligence. As promising as this technology has proven to be, in the medical field this is still at its infancy. As was shown in Watson for Oncology, while artificial intelligence can definitely be an added benefit and pick up on things that might be missed due to human error, it can still get things wrong, often in ways unacceptable to medical practitioners. A learning curve is to be expected while it trains on more data and improves its pattern recognition. The ethical aspects of using artificial intelligence for medical purposes is outside the scope of this paper, but has been the subject of previous research (Scott et al., 2023; Keskinbora, 2019; McLennan et al., 2022).

3.2.4 Limited number of subjects

To train an artificial intelligence or deep learning model, a large amount of data is often necessary. However, due to its selective nature, there have only been a limited amount of astronauts, and even fewer that have left Low Earth Orbit. Most of these missions have also been limited in duration, and the gathered health data is as a result not sufficient to train a detailed health model. This is only further made difficult by the lack of knowledge that currently exists about the human body’s adaptation to long duration spaceflight. To fully develop an autonomous space health system or digital twin, it will need to be trained on more accessible and numerous terrestrial data prior to deployment in a mission. The workshop summarized by Scott et al. (2023) suggests for this purpose the use of methods more resilient to operating under different assumptions, such as translation from animals to humans, generalization across domains using risk extrapolation or domain invariant methods and adapting models where the baseline on Earth is different from space.

This comes with an advantage however, where the large amount of computing needed to train the model will not have to be done in space but on Earth, with some in situ operations during the mission, also called transfer learning. This means the energy and resources needed during the actual mission will be minimized, which is important considering the constraints of the space environment (Scott et al., 2023). Such transfer learning has already successfully been implemented in medical imaging, for example, to train for fracture detection as seen with Kim & MacKinnon, (2018).

3.2.5 Regulatory compliance

Implementing avatars for medical monitoring entails navigating complex regulatory frameworks to ensure compliance with data protection laws and medical regulations. Given the sensitive nature of medical data, particularly in the context of deep space missions, ensuring compliance with stringent privacy and security standards poses a significant challenge. Any breaches in data security could not only jeopardize patient confidentiality but also compromise mission integrity and safety.

3.2.6 Limited interoperability

The interoperability of different wearable devices and sensors poses a technical hurdle in aggregating data for avatarbased medical monitoring. As these devices often utilize proprietary software and communication protocols, integrating data from diverse sources onto a unified platform presents challenges in standardization and compatibility. Overcoming interoperability issues requires concerted efforts in developing opensource protocols and fostering collaboration among device manufacturers.

3.2.7 Psychological impacts

Continuous monitoring through avatars may raise concerns regarding psychological privacy and autonomy, particularly in confined and isolated environments such as deep space habitats. The constant surveillance and intrusion into personal health data may lead to feelings of discomfort, surveillance fatigue, and reduced morale among crew members. Addressing these psychological impacts necessitates implementing robust privacy protocols, providing adequate psychological support, and fostering a culture of trust and transparency within the crew.

3.2.8 Data transmission and storage

Transmitting and storing large volumes of medical data in deep space environments pose significant logistical challenges. Limited bandwidth and communication delays hinder realtime transmission of highresolution medical imaging data, potentially compromising timely diagnosis and treatment. Additionally, ensuring the reliability and integrity of data storage systems in space, where resources are scarce and environmental hazards are prevalent, requires innovative solutions in data compression, encryption, and redundancy.

3.2.9 Ethical considerations

The use of avatars in medical monitoring raises ethical questions surrounding consent, autonomy, and equity. Crew members may have varying levels of comfort and willingness to participate in continuous monitoring, necessitating clear protocols for informed consent and optin/optout mechanisms. Moreover, ensuring equitable access to medical resources and interventions, particularly in resourceconstrained environments like deep space, requires careful consideration of distributive justice principles and ethical decisionmaking frameworks. Addressing these ethical considerations requires interdisciplinary collaboration and ongoing dialogue among stakeholders.

Ethical considerations specific to the use of AI are also to be considered. As the model has to train on existing data to make decisions in the future, it is important that this data is of high quality and nondiscriminatory. Due to the small set that would be the cohort of astronauts on which it is trained, the limitation in diversity of training data is something that will need special attention. As AI is known to underperform for minority populations, including women or ethnic minorities, it will be important to also test its performance across different demographics during the whole lifetime of its development (Bruynseels et al., 2018; Tang et al., 2023).

4 Terrestrial applications

The use of avatars extends beyond space exploration and remote locations, finding applications in various terrestrial settings.

4.1 Telemedicine in rural areas

Avatars have been instrumental in extending medical care to rural and underserved communities, where access to healthcare facilities is limited. Telemedicine initiatives, such as the Remote Area Medical (RAM) clinics, utilize avatars to connect patients in remote areas with healthcare professionals located elsewhere. Through realtime virtual consultations, patients can receive medical advice and treatment without the need for physical travel to healthcare facilities. This approach has been particularly impactful in addressing healthcare disparities and improving health outcomes in rural populations. For example, Space Technology Applied to Rural Papago Advanced Health Care (STARPAHC) is a good example of the use of telemedicine for remote locations that don’t have ubiquitous healthcare at their disposal, as a consequence of technology developed for space. A mobile health unit provided care to the Papago Indians on the reservation in Arizona through a link to medical facilities while having medical assistants and diagnostic hardware available in the unit (Doarn et al., 1998).

A similar application can be found in Tempus Pro, a telemedicine device used in the Antarctic bases of Belgrano and Carlini (ESA, From Antarctica to space, 2020). The device gathers different kinds of medical data without the need of medical users, and transmits the information reliably using the Iridium satellite network, towards a medical team and doctors. In this way, vital signs and medical help can securely and quickly be transmitted to remote and harsh locations that otherwise don’t have many options. This specific case has also been used in the followup medical care after landing of ESA astronaut Thomas Pesquet, using a secured satellite link to the Astronaut Centre in Cologne, but as of yet has not been used for gathering of medical information during spaceflight (ESA, Telemedicine via satellite improves care at astronaut landings, 2017).

The research stations on the Antarctic have also been utilising the Tempus Pro as a way for medical followup while in this remote area where medical intervention and contact with the outside world is equally limited (ESA, From Antarctica to Space: telemedicine at the limit, 2020). Besides this, the Italian base Terra Nova Bay includes a “full telemedicine system” which connects to the San Camillo Hospital through a Virtual Private Network (VPN) and includes video camera, Xray monitor, echoduplex scanner, video microscope setups, microphone, electronic stethoscope, etc., with software that is integrated into the Italian healthcare system (Pillon and Todini, 2004).

Another application can be found in Zahedi et al. (2022) concept of a “Wepital”: “a webbased equivalent of a hospital. (…) patients” real avatars digitally represent patients in real time, and the care in a wepital is real, persistent and continuous”. They propose the use of a wepital in which patients receive medical care through their avatars, where medical records and vital information is visualised as an avatar and can be accessed by a virtual doctor or nurse, who can then give recommendations for treatment based on this digital self. These virtual doctors and nurses can be either actual people working through and avatar, or an AI that can make decisions informed on the patient’s virtual information. This process could streamline online and offline medical care, expanding the capacity of hospitals and allowing for persistent care in patients who are home bound for various reasons.

The Portable Telemedicine Workstation (PTW) concept developed by ESA (Vizitiu et al., 2013) is meant to handle telediagnosis, teleconsulting and telemonitoring for remote areas, mainly from Romania, using satellite communication and navigation. The usage of satellite infrastructure allows for use in areas where telephony infrastructures are absent or underdeveloped. The PTW can then autonomously acquire medical data, including imaging, display the medical data locally and transmit the data wirelessly to the remote user. This is also particularly useful in emergency situations.

Many of the proposed visualisations and analyses already exist on a consumer level, with one of the most easily available being smart watches. From tracking your activity level and sleep, some even make predictions on how likely someone is to injure themselves due to overtraining. All of these are neatly packaged into easily understandable visualisations that make extrapolating data easy, and often available in more raw format forms for those interested in playing around with the data themselves. One such example can be found in the sleep tracker application by Apple or Polar.

A question to be asked is how many and which parameters should be monitored. Obvious benefit can be found in things like sleep, activity, temperature, but perhaps some lesser studied aspects could be important to astronauts’ health in space. Reaction time for cognitive ability, calcium in urine to study bone desorption or cortisol levels for stress are only a few of many possibilities.

4.2 Chronic disease management

Avatars play a crucial role in chronic disease management by facilitating remote monitoring and personalized interventions for patients with longterm health conditions. For instance, patients with diabetes can benefit from avatarbased platforms that track blood glucose levels, dietary habits, and physical activity, allowing healthcare providers to remotely monitor their health status and intervene when necessary. By empowering patients to actively participate in their own care and providing continuous support through avatars, chronic disease management becomes more effective and accessible.

4.3 Mental health support

Avatars are increasingly being utilized in mental health settings to provide virtual support and therapy to individuals experiencing mental health challenges. Virtual therapists, represented by avatars, offer counseling sessions, psychoeducation, and selfmanagement tools to individuals struggling with anxiety, depression, and other mental health disorders. These virtual interactions create a safe and nonjudgmental space for individuals to express their emotions, learn coping strategies, and access resources for mental wellbeing.

4.4 Health education and promotion

Avatars are valuable tools for health education and promotion initiatives aimed at raising awareness about various health topics and encouraging positive health behaviors. Virtual health coaches, represented by avatars, deliver tailored health messages, interactive lessons, and behavior change interventions to individuals and communities. By leveraging avatars’ engaging and interactive nature, health educators can effectively communicate health information, promote preventive measures, and empower individuals to make informed decisions about their health.

5 Discussion and conclusion

In conclusion, medical avatars for astronauts can offer a multitude of advantages for the onboard crew, ranging from quick interpretability during emergencies to analysis via machine learning. Such medical monitoring is however not without its challenges, and will require input from many interdisciplinary fields, including doctors, engineers, user interface design and information and communications technology. The proposed design includes a network of sensors, both environmental and physiological, to be connected to an onboard server for analysis and storage of data, representing all information through a digital twin and visualised through an avatar, while at the same time allowing for communication with ground control.

Significant progress has already been made in advancing this technology, drawing from lessons learned in both space and terrestrial applications. Yet, the practical implementation of this concept requires critical analysis and refinement. Insights from astronauts and medical professionals are crucial to tailor the system to meet specific needs and capabilities. Additionally, input from User Experience/User Interface designers is essential to ensure intuitive usability.

However, for the discussed concept to become more practical, the different facets will have to be critically analysed. The unique background of the astronauts themselves and the medical doctors would bring a more exact view of the needs and capabilities for such a system, and User Experience/User Interface designers can offer their expertise to make sure it is intuitive and easy to use.

This paper serves as a foundational framework from which a comprehensive system for virtual medical avatars can evolve. By addressing technical challenges and incorporating feedback from relevant stakeholders, future iterations of this technology have the potential to revolutionize medical monitoring in space exploration and beyond.

6 Future research directions

Although virtual avatars and predictive data analysis for medical purposes show promise for regular use in different settings, practical examples, especially with regards to astronauts, is currently lacking. Further research aimed an investigating the uptake and experience of this kind of technology in astronaut cohorts or analog settings could provide valuable insight into the effectiveness of telemedicine and virtual avatars for representation of medical information in real life settings (Institute of Medicine, and Board on Health Care Services, 2012).

The field of predictive artificial intelligence is in a constant state of improvement and growth, with several examples showing real successes, but still requires plenty of development. The topic of this paper could be further studied and investigated in more depth in collaboration with AI engineers or analysts, focusing more on the specifics of AI programming and its limitations.

Author contributions

CU: Writing–review and editing, Writing–original draft, Conceptualization. BB: Writing–review and editing, Supervision, Funding acquisition, Conceptualization. KT: Writing–review and editing. SB: Writing–review and editing, Supervision, Funding acquisition, Conceptualization.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This work is supported by the ESA/BELSPO/Prodex IMPULSE2 contract (PEA 4000109861).

Acknowledgments

The authors would like to give particular thanks to Dr. Rainee N. Simmons and Aaron Brister from NASA for providing us with the background information on bioMEMS and telemedicine.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Björnsson, B., Borrebaeck, C., Elander, N., Gasslander, T., Gawel, D. R., Gustafsson, M., et al. (2020). Digital twins to personalize medicine. Genome Med. 12 (1), 14. doi:10.1186/s13073-019-0701-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Bruynseels, K., Santoni de Sio, F., and van den Hoven, J. (2018). Digital twins in health care: ethical implications of an emerging engineering paradigm. Front. Genet. 9, 31. doi:10.3389/fgene.2018.00031

PubMed Abstract | CrossRef Full Text | Google Scholar

Cermack, M. (2006). Monitoring and telemedicine support in remote environments and in human space flight. Br. J. Anaesth. 97 (1), 107–114. doi:10.1093/bja/ael132

PubMed Abstract | CrossRef Full Text | Google Scholar

Choukér, A., NgoAnh, T. J., Biesbroek, R., Heldmaier, G., Heppener, M., and BereiterHahn, J. (2021). European space agency's hibernation (torpor) strategy for deep space missions: linking biology to engineering. Neurosci. and Biobehav. Rev. 131, 618–626. doi:10.1016/j.neubiorev.2021.09.054

CrossRef Full Text | Google Scholar

Deep space communication and navigation (2022). European Space Agency. Available at: https://www.esa.int/Enabling_Support/Preparing_for_the_Future/Discovery_and_Preparation/Deep_space_communication_and_navigation.

Doarn, C. R., Nicogossian, A. E., and Merrell, R. C. (1998). Applications of telemedicine in the United States space program. Telemedicine J. 4 (1), 19–30. doi:10.1089/tmj.1.1998.4.19

PubMed Abstract | CrossRef Full Text | Google Scholar

Downey, C., Randell, R., Brown, J., and Jayne, D. G. (2018). Continuous versus intermittent vital signs monitoring using a wearable, wireless patch in patients admitted to surgical wards: pilot cluster randomized controlled trial. J. Med. Internet Res. 20 (12), e10802. doi:10.2196/10802

PubMed Abstract | CrossRef Full Text | Google Scholar

ESA Telemedicine alliance (2003). Data privacy and its impact on implementing crossworder healthcare in the EU and beyond printer version. European Space Agency. Available at: https://www.esa.int/SPECIALS/Telemedicine_Alliance/SEMJKM1P4HD_2.html.

Google Scholar

From Antarctica to space (2020). Telemedicine at the limit. European Space Agency. Available at: https://www.esa.int/Science_Exploration/Human_and_Robotic_Exploration/From_Antarctica_to_space_telemedicine_at_the_limit.

Google Scholar

Gallo, C., Ridolfi, L., and Scarsoglio, S. (2020). Cardiovascular deconditioning during longterm spaceflight through multiscale modeling. npj Microgravity 6 (1), 27. doi:10.1038/s41526020001175

PubMed Abstract | CrossRef Full Text | Google Scholar

Garot, O., Rössler, J., Pfarr, J., Ganter, M. T., Spahn, D. R., Nöthiger, C. B., et al. (2019). Avatarbased versus conventional vital sign display in a central monitor for monitoring multiple patients: a multicenter computerbased laboratory study. doi:10.21203/rs.2.14457/v3

CrossRef Full Text | Google Scholar

Grout, A., and Leggat, P. A. (2021). Cabin crew health and fitnesstofly: opportunities for reevaluation amid COVID19. Travel Med. Infect. Dis. 40, 101973. doi:10.1016/j.tmaid.2021.101973

PubMed Abstract | CrossRef Full Text | Google Scholar

Harish, V., Morgado, F., Stern, A. D., and Das, S. (2020). Artificial intelligence and clinical decision making: the new nature of medical uncertainty. Acad. Med. 96 (1), 31–36. doi:10.1097/acm.0000000000003707

CrossRef Full Text | Google Scholar

Hexoskin (2024). Astroskin. Available at: https://www.hexoskin.com/pages/astroskinvitalsignsmonitoringplatformforadvancedresearch.

Google Scholar

Institute of Medicine, and Board on Health Care Services (2012). The role of Telehealth in an evolving health care environment: workshop summary. Washington, DC: National Academies Press. Available at: https://www.ncbi.nlm.nih.gov/books/NBK207145/pdf/Bookshelf_NBK207145.pdf.

Google Scholar

Jones, D., Snider, C., Nassehi, A., Yon, J., and Hicks, B. (2020). Characterising the Digital Twin: a systematic literature review. CIRP J. Manuf. Sci. Technol. 29, 36–52. doi:10.1016/j.cirpj.2020.02.002

CrossRef Full Text | Google Scholar

Keskinbora, K. H. (2019). Medical ethics considerations on artificial intelligence. J. Clin. Neurosci. 64, 277–282. doi:10.1016/j.jocn.2019.03.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, D., and MacKinnon, T. (2018). Artificial intelligence in fracture detection: transfer learning from deep convolutional neural networks. Clin. Radiol. 73 (5), 439–445. doi:10.1016/j.crad.2017.11.015

PubMed Abstract | CrossRef Full Text | Google Scholar

Kris, M. G., Gucalp, A., Epstein, A. S., Seidman, A. D., Fu, J., Keesing, J., et al. (2015). Assessing the performance of Watson for oncology, a decision support system, using actual contemporary clinical cases. J. Clin. Oncol. 33 (15_Suppl. l), 80238023. doi:10.1200/jco.2015.33.15_suppl.8023

CrossRef Full Text | Google Scholar

Kumar, A., Levin, E., Cowings, P., and Toscano, W. B. (2015). “Evaluation of the accuracy of Astroskin as a behavioral health selfmonitoring system for spaceflight,” in Annual summer STEM internship symposium. (No. ARCEDAATN26234).

Google Scholar

Li, S., Xu, L. D., and Zhao, S. (2014). The internet of things: a survey. Inf. Syst. Front. 17, 243–259. doi:10.1007/s10796-014-9492-7Available at: https://link.springer.com/article/10.1007/s1079601494927.

CrossRef Full Text | Google Scholar

McLennan, S., Fiske, A., Tigard, D., Müller, R., Haddadin, S., and Buyx, A. (2022). Embedded ethics: a proposal for integrating ethics into the development of medical AI. BMC Med. Ethics 23 (1), 6. doi:10.1186/s12910022007463

PubMed Abstract | CrossRef Full Text | Google Scholar

National Center for Immunization and Respiratory Diseases (NCIRD), Division of Viral Diseases (2020). What is Telemedicine in a nonUS Setting. Centers for Disease Control and Prevention. Available at: https://www.cdc.gov/coronavirus/2019ncov/globalcovid19/telemedicine.html.

Google Scholar

Pillon, S., and Todini, A. R. (2004). EHealth in Antarctica: a model ready to be transferred to everyday life. Int. J. Circumpolar Health 63 (4), 436–442. doi:10.3402/ijch.v63i4.17761

PubMed Abstract | CrossRef Full Text | Google Scholar

Primholdt Christensen, N., Skou, K. E., and Boe Danbjørg, D. (2021). Health care professionals’ experiences with the use of video consultation: qualitative study. JMIR Form. Res. 5 (7), e27094. doi:10.2196/27094

PubMed Abstract | CrossRef Full Text | Google Scholar

Reifegerste, D., Harst, L., and Otto, L. (2021). Sauerbruch, STARPAHC, and SARS: historical perspectives on readiness and barriers in telemedicine. J. Public Health 30 (1), 11–20. doi:10.1007/s10389021015131

CrossRef Full Text | Google Scholar

Scott, R. T., Sanders, L. M., Antonsen, E. L., Hastings, J. J. A., Park, S.-M., Mackintosh, G., et al. (2023). Biomonitoring and precision health in deep space supported by artificial intelligence. Nat. Mach. Intell. 5, 196207. doi:10.1038/s42256023006175

CrossRef Full Text | Google Scholar

Simons, R., Hall, D., and Miranda, F. (2004) “RF telemetry system for an implantable bioMEMS sensor,” in 2004 IEEE MTTS international microwave symposium digest. IEEE Cat. No.04CH37535. doi:10.1109/mwsym.2004.1338841

CrossRef Full Text | Google Scholar

Simons, R. N., Miranda, F. A., Wilson, J. D., and Simons, R. E. (2006). “Wearable wireless telemetry system for Implantable BioMEMS sensors,” in 2006 international conference of the IEEE engineering in medicine and Biology society. doi:10.1109/iembs.2006.259598

CrossRef Full Text | Google Scholar

Tang, L., Li, J., and Fantus, S. (2023). Medical artificial intelligence ethics: a systematic review of empirical studies. Digit. health 9, 20552076231186064. doi:10.1177/20552076231186064

PubMed Abstract | CrossRef Full Text | Google Scholar

Vizitiu, C., Valeanu, V., and Tantau, A. D. (2013). “Space based mobile telemedicine assisted by specific systems engineering approach,” in 2013 6th international conference on recent advances in space technologies (RAST). doi:10.1109/rast.2013.6581318

CrossRef Full Text | Google Scholar

Yue, L., and Yang, L. (2017). Clinical experience with IBM Watson for oncology (WFO) for multiple types of cancer patients in China. Ann. Oncol. 28, x162. doi:10.1093/annonc/mdx676.024

CrossRef Full Text | Google Scholar

Zahedi, F. M., Zhao, H., Sanvanson, P., Walia, N., Jain, H., and Shaker, R. (2022). My real avatar has a doctor appointment in the Wepital: a system for persistent, efficient, and ubiquitous medical care. Inf. and Manag. 59 (8), 103706. doi:10.1016/j.im.2022.103706

CrossRef Full Text | Google Scholar

Zhang, W., Kumar, M., Yu, J., and Yang, J. (2018). Medical longdistance monitoring system based on Internet of things. EURASIP J. Wirel. Commun. Netw. 2018 (1), 176. doi:10.1186/s1363801811782

CrossRef Full Text | Google Scholar

Zhang, W., Thurow, K., and Stoll, R. (2014). A knowledgebased Telemonitoring platform for application in remote healthcare. Int. J. Comput. Commun. and Control 9 (5), 644. doi:10.15837/ijccc.2014.5.661

CrossRef Full Text | Google Scholar