Leanne Wang1
Margaret Allman-Farinelli1
Jiue-An Yang2
Jennifer C. Taylor3,4Luke Gemming1
Eric Hekler3,4
Anna Rangan1*- 1Charles Perkins Centre, Faculty of Medicine and Health, University of Sydney, Sydney, NSW, Australia
- 2Department of Population Sciences, Beckman Research Institute, City of Hope, Duarte, CA, United States
- 3The Design Lab, University of California, San Diego, San Diego, CA, United States
- 4Herbert Wertheim School of Public Health and Human Longevity Science, University of California, San Diego, San Diego, CA, United States
As food intake patterns become less structured, different methods of dietary assessment may be required to capture frequently omitted snacks, smaller meals, and the time of day when they are consumed. Incorporating sensors that passively and objectively detect eating behavior may assist in capturing these eating occasions into dietary assessment methods. The aim of this study was to identify and collate sensor-based technologies that are feasible for dietitians to use to assist with performing dietary assessments in real-world practice settings. A scoping review was conducted using the PRISMA extension for scoping reviews (PRISMA-ScR) framework. Studies were included if they were published between January 2016 and December 2021 and evaluated the performance of sensor-based devices for identifying and recording the time of food intake. Devices from included studies were further evaluated against a set of feasibility criteria to determine whether they could potentially be used to assist dietitians in conducting dietary assessments. The feasibility criteria were, in brief, consisting of an accuracy ≥80%; tested in settings where subjects were free to choose their own foods and activities; social acceptability and comfort; a long battery life; and a relatively rapid detection of an eating episode. Fifty-four studies describing 53 unique devices and 4 device combinations worn on the wrist (n = 18), head (n = 16), neck (n = 9), and other locations (n = 14) were included. Whilst none of the devices strictly met all feasibility criteria currently, continuous refinement and testing of device software and hardware are likely given the rapidly changing nature of this emerging field. The main reasons devices failed to meet the feasibility criteria were: an insufficient or lack of reporting on battery life (91%), the use of a limited number of foods and behaviors to evaluate device performance (63%), and the device being socially unacceptable or uncomfortable to wear for long durations (46%). Until sensor-based dietary assessment tools have been designed into more inconspicuous prototypes and are able to detect most food and beverage consumption throughout the day, their use will not be feasible for dietitians in practice settings.
Introduction
In the past decade, there has been a shift away from traditional eating patterns (1). The proportion of adults moving away from the conventional three main meals per day toward smaller but more frequent meals and snacks is increasing (1). These dietary trends are more apparent in young adults whose food intake pattern is less structured (2) and more spontaneous (3). Thus, there are implications for self-reported dietary intakes as snacking occasions are frequently omitted when using traditional dietary assessment methods (4) due to users forgetting or declining to self-report (5). There have been technological advances in the collection and processing of dietary information, with web- and mobile application (app)-based food diaries and computerized adaptations of the 24-h recall such as the National Cancer Institute's Automated Self-Administered 24-h recall (ASA24) (6, 7) and the Measure Your Food on One Day 24-h Recall (myfood24) (8) becoming the norm. However, digital food diaries still rely on user-initiation and self-report (9) and the computerized 24-h recalls have similar limitations to that of traditional dietary assessment methods and have been documented to omit food items (10, 11).
Newer technologies utilizing wearable and non-wearable sensors offer an objective and passive means to measure behaviors related to food and beverage consumption. They provide convenience for users by removing the burden of self-report (12) and may be of higher measurement accuracy, thereby presenting advantages over traditional dietary assessment methods. The capabilities of these technologies are constantly evolving and improving given the relatively new nature of this field (13) and have the potential to advance the Nutrition Care Process (NCP) (14), a framework that guides the Medical Nutrition Therapy (MNT) dietitians provide.
Nutrition Assessment is the first step of the NCP, and it involves the collection and documentation of information, including food-related history (14). In addition to determining what foods are eaten during meals and less structured eating occasions, dietitians may examine how foods are eaten. This may be enhanced via the use of sensors as their ability to detect hand-to-mouth (HTM) actions and biting, chewing, and swallowing of food enables them to provide valuable information about how someone eats, including the timing of consumption, duration of eating episodes, and frequency of eating occasions. Sensors and wearables such as wearable cameras that take images at frequent and regular intervals throughout the day (e.g., every 30 s) (15) may be used to reveal what people are eating and combinations of sensors can be used to determine food type and calorie intake when chewing motions are detected (16). Furthermore, real-time detection of eating gestures allows for the delivery of timely prompts that remind users to record their dietary intake, reducing the chances of inaccurate dietary recalls as a result of memory decay (5). This is a form of event-contingent Ecological Momentary Diet Assessment (EMDA) (17). Beyond their value in rigorous measurement, the use of prompts in tandem with sensing technologies can assess a person's psychological and physical state and social and food environmental surroundings when eating occasions are detected, helping a dietitian understand why a client eats as they do.
The remaining steps of the NCP are Nutrition Diagnosis, Intervention, and Monitoring/Evaluation (14), all of which could also be enhanced with the use of sensor-based devices. Nutrition Diagnosis involves the naming of a specific nutrition problem based on data collected during nutrition assessment. Sensors may be able to detect the signs and symptoms of nutrition problems such as excessive or inadequate energy intake based on the correlation between energy intake and HTM motions (18); swallowing or chewing difficulties using devices that measure these gestures to detect eating (19); and irregular eating patterns based on the detected start time of eating occasions and the consistency of these times across multiple days. This type of information collected by the sensor-based devices may assist dietitians in establishing a nutrition diagnosis more efficiently. During Nutrition Intervention, the dietitian selects an appropriate intervention to be directed toward the root cause of the nutrition problem (14). Again, with the use of sensor-based devices, timely prompts that remind users of this intervention, may be delivered upon the detection of an eating event (20) to enable clients to achieve their dietary goals. This can be in the form of a short message that appears on a smartwatch or a connected smartphone that informs the user of an action or food choice that can be made to motivate them toward achieving their dietary goal. During the Monitoring/Evaluation stage, users' dietary habits in terms of the rate of eating (e.g., bites/chews per minute) and dynamic pattern of food consumption (21, 22) may be monitored and evaluated remotely and in real-time between consultation sessions (23).
Although some authors from dietetic backgrounds have already integrated sensor-based devices into their studies (24, 25), specifically the wrist-worn device known as the Bite Counter that detects HTM motions, there is currently no systematic process in place to evaluate the feasibility of other devices for dietetic practice. For sensor-based devices to be useful to dietitians in real world practice, engineers building the devices need to be cognizant of the practicality of devices. The sensor will need to be discrete and comfortable to wear throughout the day so as not to inhibit client/patient compliance with using the device. Battery life will need to be sufficient to cover waking hours without recharging. Ideally, multiple functions could be incorporated into one device rather than requiring numerous sensors. The field of physical activity demonstrates how passive sensing can move from more obtrusive bulky accelerometers worn on the hip to watch-like devices worn on the wrist (26).
In an area that is rapidly evolving, a scoping review was selected as the most appropriate method to identify both peer-reviewed studies and gray literature between January 2016 and December 2021 that assessed the use of sensor-based devices in detecting eating and/or drinking occasions. These devices were then evaluated with a set of five feasibility criteria to identify and collate those that could potentially be useful for assisting dietitians in performing dietary assessments as part of the NCP in real-world practice settings.
Materials and Methods
Protocol and Registration
Our protocol was drafted using the Open Science Framework (OSF) Preregistration template and registered prospectively with OSF on 11/01/2022 (https://doi.org/10.17605/OSF.IO/EYH5A).
Eligibility Criteria
The selection criteria informed the population, concept, and context relevant to the review's objective. This scoping search was inclusive of any scientific paper published between January 2016 and December 2021 that used sensor-based devices to passively detect and record the initiation of eating in real-time, as shown in Table 1. As this field is rapidly advancing and sensor-based technologies evaluated in earlier papers are likely to have been further developed, we focus on the literature from the past 5 years.
Table 1. Inclusion and exclusion criteria of the scoping review.
Information Sources and Search
For the scoping review, seven databases were searched: ACM digital library, CINAHL (EBSCO), EMBASE (Ovid) (EMBASE, RRID:SCR_001650), IEEE Xplore (IEEE), PubMed, Scopus (Elsevier), and Web of Science (Clarivate Analytics). Sources of gray literature searched included: Google, Trove, MedNar, OpenGrey and official websites of international and government organizations. The search strategy was optimized to extract both peer-reviewed studies and gray literature and was developed and refined with the aid of an experienced librarian with expertise in database searches. The complete search strategy for Ovid MEDLINE can be found in the Appendix A in Supplementary Material. The final search results were exported into EndNote (EndNote, RRID:SCR_014001), and duplicates were removed.
Screening and Selection of Evidence
The titles and abstracts of all studies were reviewed against the eligibility criteria. If the title and/or abstract mentioned a sensor-based device that detected eating, the study was included in the initial screening stage to be assessed in the full-text screening stage.
Studies were further excluded during the full text screening stage if they did not evaluate device performance or if the same research group conducted a more recent study describing a device that superseded previous studies of the same device. Studies evaluating a device that did not have the capacity to detect and record the start time of food intake, did not use sensors, were not applicable for use in free-living settings, or were discontinued at the time of the search were also excluded (Figure 1).
Figure 1. Flow chart of included studies.
Data Charting Process and Data Items
The data extraction table was developed specifically for this review (Supplementary Table 1). In the table, the devices were categorized according to device placement. For each device, the following data items were included:
1. The type(s) of sensors used, and the number of each sensor included in the device.
2. The type(s) of intake (e.g., food, beverages) and eating proxy or gesture (e.g., HTM motions, chewing, swallowing, etc.) measured.
3. Ground truth method, evaluation metric(s), and performance of sensors.
4. Advantages and disadvantages of the study, including both the study design as well as the feasibility of the device for assisting with dietary assessment in real-world settings according to the five feasibility criteria described below.
5. Experiment details including the setting, duration, number of food and/or beverage types tested, and number of participants.
6. The data processing pipeline including the classification algorithm used, sampling rate, and the number of features. The number of papers using each type of classification algorithm was tabulated in a separate table (Supplementary Table 2). This table was developed by a dietitian (LW) and a geospatial data scientist (JY). A short description of each algorithm using plain language is provided.
Synthesis of Results
Devices in the data extraction table were further evaluated against a set of five feasibility criteria to identify those that were suitable for assisting dietitians with performing dietary assessment (Step 1 of the NCP) in real-world settings (Table 2). The five feasibility criteria were: (i) the study's evaluation metric showing an average accuracy or F1-score of ≥80% in detecting individual eating proxies such as HTM movements, biting, chewing, or swallowing; eating bouts (periods of eating within an eating episode); or entire eating episodes (ii) in free-living, semi-free-living, or laboratory settings where participants were free to choose their own foods and activities. The feasibility cut-off score of ≥80% was selected based on the performance of the Bite Counter device employed in dietitian-led trials (24, 25). This device achieved an F1-score of 82% in free-living settings (81). (iii) The device had to be discreet, socially acceptable, and comfortable for the user to wear for long durations. This was determined based on user feedback where available and the appearance and description of the device hardware. Due to the emerging nature of this field, the design of the hardware did not need to match the quality and appearance of off-the-shelf wearables at this stage. However, devices that were large, obtrusive, and not disguisable as everyday accessories such as glasses were considered socially unacceptable. (iv) The battery life of the device on a single charge needed to exceed 12 h or one waking day and (v) the data processing pipeline and algorithm used needed to be able to classify a segment of data in real-time as either eating or not eating within 5 mins. Five minutes was chosen to capture all meals, including snacks.
Table 2. An overview of all identified devices compared with the five feasibility criteria used to determine suitability for use by dietitians in real-world settings.
Table 2 was formatted as a checklist, where each criterion occupied a column, and each device occupied a row. An additional column for the detection of beverages was also included. Like the data extraction table (Supplementary Table 1), devices were categorized by device location (wrist, neck, ear, glasses, other, multi-position).
Results
Study Selection
The search yielded 5,732 results in total: 5,718 studies were identified by searching scientific databases and another 14 were identified by searching the gray literature and the reference lists of identified eligible studies. After removal of duplicates, 3,101 abstracts were screened by title and abstract and then full text for eligibility (Figure 1). A total of 54 studies for sensors were identified in this review. Reasons for full-text exclusions included: (i) the devices not identifying or recording the start of food and/or beverage intake time, (ii) the study not having an experimental section or only evaluating the data-processing pipeline without describing the hardware, (iii) the study not describing a sensor-based device, (iv) the device not being applicable to free-living settings, (v) the study describing a discontinued device, and (vi) the study being superseded by a more recent paper of the same device by the same research.
Study Characteristics
Fifty-three unique devices and four combinations of devices (77–80) identified from 54 studies were included in the data extraction table (Supplementary Table 1). Of the 53 unique devices, 18 devices were placed on the wrist (27–44); nine were worn around the neck (45–52); nine were placed in or around the ears (53–61); seven were glasses-like devices (62–68); and ten were categorized as “Other Devices” (30, 36, 69–76) where devices were worn in less common locations of the body or were non-wearable (69, 72).
Most of the devices were trained and/or tested in laboratory settings (n = 32). This was followed by devices trained and/or tested in free-living settings (n = 22). Two of the devices were trained and tested in both free-living and laboratory settings for different food types (58, 74) and one was tested in semi-free-living settings (60).
Most of the devices (n = 47) were assessed using one or more validation methods as the “ground truth” for comparison with inferred eating and/or drinking occasions. The ground truth was captured by objective methods such as video or image recording, researcher annotation or observation, and electromyography (EMG) waves and voltages (n = 23); subjective methods such as user annotation of recordings, human memory and self-report, activity logging, and the use of a push button to mark chewing or swallowing events (n = 21); or both (n = 3).
Results by Device Type
A summary of the devices' performance, including the advantages and disadvantages of each device category as well as the study design used to evaluate the device(s) are described below. Further details are available in Supplementary Table 1.
Wrist-Worn Devices
Devices worn on the wrist measured intake via the HTM movement. Therefore, a major advantage of wrist devices is the potential for it to detect both eating and drinking as it does not rely on chewing or swallowing activity. Both food and beverage intake were able to be identified in 5/18 wrist devices (27, 33, 37, 40, 44) and three of these could also distinguish between eating and drinking movements (27, 33, 40). The algorithms used by the wrist devices to detect HTM movements could be adapted to different handedness depending on the individual. Most were worn on the dominant wrist whilst one was worn on the utensil-operating hand (32). Wrist devices are assumed to have a high user acceptability especially with the use of commercial smartwatches (28, 29, 32, 42) or models that are compatible with popular commercial health tracking devices (37). However, user acceptability was not formally tested for most wrist devices. Whilst none of the devices could provide an assessment of the actual foods consumed or the energy and nutrient intake, one device could distinguish between the use of different types of cutleries (28), one could distinguish between a limited number of food types (ramen, pasta, bread, onigiri, gyudon, cake) (43), and one could capture (but not provide an assessment of) food type via an embedded camera (34).
The main disadvantages of the studies that used wrist devices included: generating false positives for hand actions close to the face (33, 42), and using a limited number of foods, utensils, and/or behaviors to train and test the model, leading to overly optimistic results (27, 30, 34, 36, 38); and users being required to wear the wrist device on both wrists (35).
Neck-Worn Devices
Devices placed on the neck used a variety of sensors such as microphones, piezoelectric sensors, and proximity sensors to identify chewing and/or swallowing activity. Unlike microphones, the use of piezoelectric sensors (pressure sensor) does not generate privacy concerns (47, 49, 52). Most sensors used in neck-worn devices captured both chewing and swallowing signals and were therefore capable of differentiating between food and liquid intake based on the pattern of chewing and swallowing (46, 47, 49, 50). These four devices could also differentiate between a limited number of food types ranging from three to 17 individual foods.
Like wrist-worn devices, a drawback of the studies that described devices worn on the neck was the strict experimental protocols that did not reflect real-world eating scenarios. This included experiments being conducted in settings with low environmental noise (46, 49) and using a limited number of foods or behaviors (46, 47, 49). Reduced performance was seen in neck devices for users with a higher (51) or lower body mass index (BMI) (46) and for users with dysphagia (50). Both tight-fitting and loose-fitting neck sensors scored modestly for user comfort (47, 50) and it was also common for loose-fitting devices to move out of place during physical activity (45, 48). None of the neck devices could provide an assessment of energy and nutrient intake.
Ear-Worn Devices
Similar to wrist-worn devices, a major strength of using devices in and/or around the ears is the potential user acceptability of an in-ear earphone device (56, 58) or two discrete electrodes behind the ear (59). Seven of the nine ear-worn devices used chewing as a proxy for eating (54, 55, 57–61), two of which deliberately used soft foods such as yogurt, ice cream, puree, and custard to train the devices' algorithms as they are more prone to being missed (55, 60). However, it was unclear how the devices performed in detecting these specific foods as the authors did not report on these foods separately.
Despite being unobtrusive, subjects reported that they would only wear the earbud-like device continuously for 4 h due to discomfort (60). Other ear-worn devices were still in preliminary stages, testing only two food types (57, 58) and requiring manual adjustments by researchers before consuming each food type (57). Some devices were only worn during mealtimes, so it was difficult to determine whether these algorithms were able to classify eating from other activities (53, 58). Studies that used chewing as a proxy for eating saw a lower performance in subjects with bruxism and/or nail biting (59). Movements such as walking (54), talking (56), and strong head movements (59) were commonly confused with chewing. None of these devices were evaluated for drinking detection, could identify or capture food type, or were able to provide an assessment of energy and nutrient intake.
Glasses-Like Devices
Like most ear-worn devices, glasses-like devices also used chewing activity to detect eating. However, rather than detecting chewing via jaw motions, it was detected via temporalis muscle activation located on the side of the forehead. Despite this, one device was still able to detect drinking where the duration between sips was <30 secs (67), although it was not capable of differentiating between eating and drinking occasions. Two devices were designed as a pair of glasses embedded with a small camera and were able to capture but not classify food type (66, 67). One of them could detect chewing within 1.1002 milliseconds (66). Another device with EMG sensors (64) was able to detect eating start time with an error of 24.8 secs. The sensor(s) used in three devices (62, 63, 66) were detachable from the device and could be mounted onto any off-the-shelf glasses, increasing user acceptability in those who regularly wear glasses.
All glasses-like devices identified in the scoping review required the frame of the glasses to be in contact with the temporalis muscle to measure chewing. This meant that the glasses had to be tightly fitted and could be affected by perspiration or hairs between the skin and the glasses. Thus, users who do not wear glasses regularly may experience discomfort (68). Subjects also raised privacy concerns regarding the continuous capture of the cameras attached to the glasses (67). Other disadvantages associated with using chewing activity to identify eating occasions included having a lower performance when detecting soft foods. None of the devices in this category could provide an assessment of energy and nutrient intake.
Other Devices
Devices that were categorized under “other” included wearable (n = 8) and non-wearable apparatuses (n = 2). Non-wearable technology included a WiFi transmitter and a receiver, using interferences in the WiFi signaling between a WiFi access point and a smartphone to deduce human body motion (69). Another non-wearable was a smart fork that detected food pick-up gestures and bite quantity using a load cell embedded inside the fork to measure the weight of each bite (72). This could potentially be used to estimate caloric intake. Other wearable devices included a headband (70, 71), a pin (76), a sensor placed on the chin (73), on the finger (30, 36), on the lower teeth (74), and a hip-worn Global Positioning System (GPS) and accelerometer (75).
Most of the devices in this category were still in preliminary developmental stages and were tested on a limited number of foods and/or behaviors in laboratory settings (30, 36, 68–72). However, devices worn on the chin (73) and teeth (74) were small and discreet and had been tested in free-living settings. One device was able to broadly categorize food by type (moist/dry and hot/cold) using changes in intraoral temperature and jawbone movement (74). Another device that included an accelerometer and GPS worn on the hip (75) was able to predict food purchasing and eating events up to 4 mins ahead of time with 74 and 73% mean accuracy respectively, even in in-home settings. This was the only device that could detect eating prior to the event which is ideal for providing just-in-time dietary feedback or reminders to guide users toward their dietary and health goals. None of the devices in this category were evaluated for drinking detection.
Multi-Position Devices
Devices in this category involved sensors placed on multiple locations of the body. Using a combination of devices has the advantage of being able to counteract each other's drawbacks (79, 80). The study by Mirtchouk and Kleinberg (80) used one earbud and two smartwatches worn on both wrists. Whilst the earbud only detected chewing sounds associated with eating, the smartwatch detected HTM movements for both eating and drinking occasions and also mitigated any false positives that may be picked up by the microphone on the earbud. The sounds picked up by the earbud can also help to reduce false positives generated by the smartwatches when the user performs other hand gestures close to the face. Two devices in this category were able to distinguish between eating and drinking events (78, 79), with one also able to distinguish between six food types, including water (78).
A major drawback of using multiple devices would be user comfort and acceptability, especially when worn for long durations in social settings. Only one device combination, a wrist and neck sensor, evaluated user experience (78) and reported that most participants were comfortable with wearing the two instruments, especially the wrist sensor (smartwatch). This study (78), as well as one other (77), was conducted in laboratory settings with a limited number of activities, subjects, and food types. None of the devices could provide an assessment of energy and nutrient intake.
Summary of Devices Most Promising for Dietetic Practice
Based on the five feasibility criteria, none of the devices identified in this scoping review were deemed feasible for use by dietitians to assist with conducting dietary assessments. The feasibility criteria were, in brief, consisting of an accuracy ≥80%; tested in settings where subjects were free to choose their own foods and activities; social acceptability and comfort; a long battery life; and a relatively rapid detection of an eating episode.
The most common reason for exclusion was an insufficient or lack of reporting on battery life, accounting for 91% of the devices. The wrist-worn category had the highest proportion of devices that did not meet this criterion. This was followed by the use of a limited number of foods and/or behaviors to test device models (63%), where devices from the “other” category had the highest proportion not meeting the criterion. Forty-six percent of devices predominately from the neck-worn category were socially unacceptable or uncomfortable to wear for long durations. Thirty-five percent of devices had not been tested or calculated for real-time applicability. This was mostly seen in the multi-position device category. Twenty-one percent of devices did not meet the accuracy or F1-score of ≥80% criterion. This was also predominately contributed by the neck-worn devices. Note that the percentages do not add up to 100% as studies could have had more than one reason for exclusion. Twenty-three percent of devices, predominately from the neck-worn category, were evaluated for the detection of behaviors related to both eating and drinking. Table 2 provides an overview of the 57 devices and device combinations and the feasibility criteria they fulfilled and did not fulfill.
Select features of devices that fulfilled four out of the five feasibility criteria (34, 54, 66, 67) can be used to highlight their potential application in dietitian-led MNT. These devices included:
A pair of glasses embedded with gyroscope, accelerometer, and proximity sensors (67) (feasibility criterion iii unfulfilled); a wrist-worn device (34) (feasibility criterion ii unfulfilled), a pair of off-the-shelf eyeglasses with an attached sensor system (accelerometer) (66) (feasibility criterion iv unfulfilled), and a headband that housed a contact microphone beside the ear (54) (feasibility criterion iii unfulfilled).
Dietitian-led MNT is guided by the four step NCP. During Nutrition Assessment (Step 1), dietitians will usually conduct a diet history to estimate nutritional adequacy and meal patterns but will sometimes ask patients to keep a diet record in advance of their consultation. However, patients sometimes miss meals and snacks or forget to record in paper- or app-based dietary records (82). The headband device detects eating passively using chewing sounds, with a 3-mins delay between the start of eating and eating detection (54). This type of device may be used to remind users to document food intake retrospectively in the form of an event-triggered EMDA. This may reduce the burden of self-initiation and reduces the number of missed occasions resulting from when users forget to report. The wrist-worn and two glasses-like devices have a relatively quick processing time (60 sec or less) and can be used to passively detect and capture the types of foods and beverages consumed via the embedded camera. This eliminates the need for users to self-report dietary intake altogether.
A Nutrition Diagnosis (Step 2) can be determined via sensor-based devices including the intake domain for excessive or inadequate energy and fluid intake; clinical diagnoses, such as difficulties with swallowing or chewing; and behavioral-environmental aspects, such as eating patterns and physical activity. Hand-to-mouth motions measured via wrist-worn devices have a moderate correlation with energy intake (18) and may be used to estimate over- and under-eating. The two eyeglasses and headband use chewing and/or swallowing motions to detect eating and irregularities in the sensor signal may potentially be developed and used to identify chewing/swallowing difficulties. All four devices are also able to detect the number of eating occasions (food consumption occasions) throughout the day as well as their clock times, revealing the patient's food consumption pattern, including whether they follow a regular eating schedule or engage in more “grazing” styles of eating. This information can help with the establishment of a nutrition diagnosis.
After a Nutrition Intervention (step 3) is agreed upon between the dietitian and the patient, the devices can be used to assist patients with adhering to the planned action(s) of the intervention. A reminder about a pre-determined goal or a strategy delivered a few seconds into a meal can guide patients' food decisions especially in settings where food is self-served throughout the meal. This is potentially feasible with the off-the-shelf glasses device as it can detect eating gestures within a very short duration of 1.1002 milliseconds from the start of the occasion. Although the devices may not be useful in settings where the user prepares or purchases their own food as the food choice would have already been made when the reminder is delivered, it may help to guide their next eating occasion.
As consultation sessions can range from weeks to months apart, sensor-based devices can enable dietitians to Monitor and Evaluate (step 4) patients' progress between sessions. By setting up web- or mobile app-based communication systems such as Healthie (83), Cronometer Pro (84), and Ascend (85), dietitians may be able to view patients' sensor-triggered dietary records in real-time and adjust the dietary intervention accordingly without delay or postponing until the next appointment. This may help to maintain patient motivation and adherence to dietary goals and plans between consultations (86).
Discussion
Currently, traditional dietary assessment methods such as food frequency questionnaires, 24-h recalls, and food records or diaries are still the most commonly used in dietetic practice (87). However, online adaptations of these methods including web-based programs, mobile phone and smartphone applications, and other technologies have become the norm in supporting the delivery of nutrition care (88, 89). Although some authors from both dietetic and engineering backgrounds have already employed sensor-based devices into their studies (21, 24, 25), previous literature reviews in this field have not considered the real-time applicability and feasibility of sensor-based devices when used in dietetic practice settings (13, 90). This review provides an overview of the latest sensors used to detect eating occasions. The sensors were evaluated against a set of five feasibility criteria outlined in this discussion to determine whether they could feasibly be implemented into dietary assessment and potentially improve the accuracy, efficiency, and effectiveness of the NCP. This set of criteria may also be helpful for guiding engineers in the development of future devices to incorporate features that are important and practical for dietitians.
Summary of Evidence
This scoping review identified a total of 57 devices or device sets, spanning across six categories: wrist-worn, neck-worn, ear-worn, glasses, “other” (non-wearable devices and devices worn on less common locations of the body), and a combination of devices. Of the 57, none of the devices met the feasibility criteria as suitable for dietitians to incorporate into practice settings. The most responsive device with the shortest time resolution could detect eating and/or drinking occasions within 1.1002 milliseconds from the start of an eating event, which can be used to provide timely feedback. Devices with longer time resolutions can instead be used to remind users to record their dietary intake as an automated, event-contingent EMDA.
Feasibility Criteria for Sensors in Dietetic Practice Settings
The five criteria for feasibility of use by dietitians in real-world practice settings were: accuracy in detecting all eating episodes; evaluation of device performance in settings where participants had the freedom to select their own food and nutrition behaviors; user comfort and acceptance when worn in social settings and for long durations; detection of episodes in real-time and with little delay; and a sustainable battery life of at least one waking day on a single charge.
An average F1-score or accuracy of ≥80% in detecting eating and/or drinking was selected as the cut-off for a satisfactory performance. Only studies achieving these results in free-living, semi-free-living, or laboratory settings where participants were free to choose their own foods and activities are appropriate for real-world use. Only a small number of devices that met these two criteria had been evaluated to detect both eating and drinking occasions (62, 67, 79, 80) despite beverages contributing to up to one quarter of total daily energy intake (91, 92). This may be because beverages that are consumed continuously throughout the day such as water, tea, and coffee are difficult to detect (67) and distinguish from eating using current sensors (62, 80). Most of the devices that were able to distinguish between eating and drinking were tested on a limited number of foods and beverages (27, 37, 40, 44, 47, 49, 50, 76, 78), using differences in chewing and swallowing patterns (47, 49, 50, 78), sounds (49, 76), or wrist movements (27, 37, 44) to distinguish between the two. Additional evaluation using more food types in settings where subjects are free to choose their own foods and activities is required. The detection of beverages may also require supplementary tools such as the use of sensor-embedded drinking utensils (92) for a more accurate assessment of beverage intake.
Dietary assessments typically involve the collection of dietary data over several days and may require repeated sampling to determine nutrient intakes and food patterns. However, during dietary intervention and monitoring, food consumption may need to be measured at multiple time points over weeks, months, or even seasons to determine the amount of progress made and to continue to elicit behavior change toward nutrition-related goals (93). Therefore, it is important that the devices do not cause wearer burden (94). Few studies addressed user comfort and acceptability (34, 48, 51, 56, 60, 66, 78). Privacy concerns were raised for continuously capturing cameras (34, 67, 95) and air microphones (49). Neck devices (48, 51) and earphone-like devices (60) scored modest ratings for comfort (48, 96), whilst wrist-worn devices were readily accepted by users (78, 96). Further adjustments need to be made to the hardware of the earphone prototype to improve the fit of these devices and camera capture should only be triggered when eating is detected to address privacy concerns. For devices such as neck-worn and glasses devices, a smaller and thinner or a customizable appearance would be more socially acceptable (48, 51, 67).
Unlike traditional event-contingent EMDA, passive sensor-based devices do not require participants to initiate the self-reporting of each eating event (19), reducing the number of missed occasions resulting from when users forget or decline to report (97). However, these devices will only be effective if eating or drinking gestures are detected instantly at the start of a main meal, snack, or beverage or even predicted before the event (74). Although many papers stated that real-time data collection and processing was possible, few tested the entire data processing pipeline in real-time or calculated the anticipated time delay between eating and gesture identification (34, 45, 54, 61, 66, 67). It is important that the time delay is kept to a minimum so that dietitians are able to deliver just-in-time feedback and positively influence the user's food choice before it has been consumed. Devices should continue to be tested in real-time to ensure near instant detection or prediction before the event.
As people may eat at any time during the day, an important requirement for passive dietary monitoring is a battery life that can sustain continuous day-long monitoring. Only five papers evaluated and reported on the battery life of the device, all of which could last for at least 12 h or one waking day on a single charge either during real-time implementation or as estimated using power measurements (34, 48, 51, 54, 67). Further evaluation is required to ensure that the battery life of sensor-based devices can last or exceed the waking hours of one day even whilst operating at maximum performance.
Devices that meet the five feasibility criteria have the potential to assist in all four stages of the NCP. By passively detecting eating gestures, the sensors can be developed to remind users to record their food intake, reducing the reliance on human memory and increasing the accuracy of dietary assessment. A nutrition diagnosis can be established more efficiently via the pattern and frequency of food consumption as well as the estimated energy intake captured by the sensors. Dietary interventions can be made more effective when users are reminded of a predetermined goal or strategy during food intake. By remaining online, dietitians can continuously monitor and adjust the intervention without delaying until the next appointment.
Strengths and Limitations
Our scoping review has some limitations. Studies that used discontinued device models were excluded which may have potentially omitted software and data processing pipelines that were accurate, high speed, and computationally inexpensive. In addition, there were several limitations with the data extraction process. We were not able to compare results across studies due to the lack of standardized eating or drinking outcome and evaluation measure of intake. We also only selected the “best” evaluation metric as indicated by authors to report in Supplementary Table 1. Although this was only as reported in the most realistic scenario. For example, if performance in laboratory and free-living settings were both included in the study, only the free-living performance was reported in our table. The feasibility criteria used to assess the devices' suitability for use in dietetic practice settings may also have been overly restrictive. It is possible that there were devices that were computationally inexpensive with real-time applicability but were excluded because these attributes were not evaluated in the current study identified by the scoping review. The field of wearable and non-wearable sensors is relatively young and most of the devices identified at this point in time are likely to be in preliminary stages, particularly as the review was limited to studies published in 2016 or later. They are likely to be developed further to overcome challenges such as hardware obtrusiveness in the near future.
A major strength of this scoping review is that it was written using plain language and limited technical computer science terminologies by accredited dietitians with experience in dietary assessment and MNT, making it useful for a dietetic audience. Theoreticians, nutritionists, and geospatial data scientists also contributed to the writing and development of this paper as well as the interpretation of studies identified by this review. The systematic process of the PRISMA-ScR framework used to guide this review produced replicable, transparent, and comprehensive results. The wide scope of the search (Appendix A) and inclusion of both published and unpublished literature further maximized the capture of relevant information to synthesize evidence in this emerging field.
Conclusions
Methods of assessing food and beverage consumption that detect food consumption patterns in a more dynamic and objective manner are needed to more robustly capture a broader range of food and nutrient intakes. In this review, we identified all recently published studies describing sensor-based devices that passively detected and recorded the initiation of eating in real-time. A set of feasibility criteria based on high accuracy, user acceptability, real-world and real-time applicability, and a sustainable battery life, was applied to these studies to filter for those that were most promising for use in dietetic practice settings. Whilst most of the identified sensors and devices only embodied some of these attributes, we have demonstrated how sensor-based devices may be useful for assisting with the assessment of dietary intake as well as other stages of the NCP. As sensor-based capture of eating is a rapidly evolving field, further development, refinement, and testing of device hardware and software are likely to occur over the next few years which point toward the potential positive and tangible benefits of utilizing sensor-based devices to assess food consumption patterns.
Author Contributions
MA-F and AR conceptualized the study. AR and LW developed the review protocol. LW conducted the literature search and wrote the first draft of the manuscript. LW and J-AY extracted the data. AR, MA-F, and JT wrote sections of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.
Funding
This project was funded through an International Seed Grant award awarded by the University of Sydney and the University of California San Diego (Allman-Farinelli & Hekler).
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Acknowledgments
The authors would like to thank Phillippa Bourke, an academic liaison librarian at The University of Sydney, for her help in developing and refining the search strategy of this scoping review.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnut.2022.852984/full#supplementary-material
References
1. Kant AK, Graubard BI. 40-year trends in meal and snack eating behaviors of American adults. J Acad Nutr Diet. (2014) 115:50–63. doi: 10.1016/j.jand.2014.06.354
2. Dashti HS, Scheer F, Saxena R, Garaulet M. Timing of food intake: identifying contributing factors to design effective interventions. Adv Nutr. (2019) 10:606–20. doi: 10.1093/advances/nmy131
3. Poobalan AS, Aucott LS, Clarke A, Smith WC. Diet behaviour among young people in transition to adulthood (18–25 year olds): a mixed method study. Health Psychol Behav Med. (2014) 2:909–28. doi: 10.1080/21642850.2014.931232
4. Burke LM. Dietary Assessment Methods for the Athlete: Pros Cons of Different Methods. (2015). Available online at: https://www.gssiweb.org/sports-science-exchange/article/sse-150-dietary-assessment-methods-for-the-athlete-pros-and-cons-of-different-methods (accessed August 16, 2021).
5. Thompson F, Subar A, Loria C, Reedy J, Baranowski T. Need for technological innovation in dietary assessment. J Am Diet Assoc. (2010) 110:48–51. doi: 10.1016/j.jada.2009.10.008
6. Subar A, Thompson F, Potischman N, Forsyth B, Buday R, Richards D, et al. Formative research of a quick list for an automated self-administered 24-h dietary recall. J Am Diet Assoc. (2007) 107:1002–7. doi: 10.1016/j.jada.2007.03.007
7. Subar A, Crafts J, Zimmerman T, Wilson M, Mittl B, Islam N, et al. Assessment of the accuracy of portion size reports using computer-based food photography aids in the development of an automated self-administered 24-h recall (ASA24). J Am Diet Assoc. (2010) 110:55–64. doi: 10.1016/j.jada.2009.10.007
8. Carter MC, Albar SA, Morris MA, Mulla UZ, Hancock N, Evans CE, et al. Development of a UK Online 24-h dietary assessment tool: myfood24. Nutrients (2015) 7:4016–32. doi: 10.3390/nu7064016
9. Jung J, Wellard-Cole L, Cai C, Koprinska I, Yacef K, Allman-Farinelli M, et al. Foundations for systematic evaluation and benchmarking of a mobile food logger in a large-scale nutrition study. Proc ACM Interact Mob Wearable Ubiquitous Technol. (2020) 4:1–25. doi: 10.1145/3397327
10. Chan V, Davies A, Wellard-Cole L, Lu S, Ng HC, Tsoi L, et al. Using wearable cameras to assess foods and beverages omitted in 24 h dietary recalls and a text entry food record app. Nutrients. (2021) 13:1806. doi: 10.3390/nu13061806
11. Wark PA, Hardie LJ, Frost GS, Alwan NA, Carter M, Elliott P, et al. Validity of an online 24-h recall tool (myfood24) for dietary assessment in population studies: comparison with biomarkers and standard interviews. BMC Med. (2018) 16:136. doi: 10.1186/s12916-018-1113-8
12. Vu T, Lin F, Alshurafa N, Xu W. Wearable food intake monitoring technolgoies: a comprehensive review. Computers. (2017) 6:4. doi: 10.3390/computers6010004
13. Hassannejad H, Matrella G, Ciampolini P, De. Munari I, Mordonini M, Cagnoni S. Automatic diet monitoring: a review of computer vision and wearable sensor-based methods. Int J Food Sci Nutr. (2017) 68:656–70. doi: 10.1080/09637486.2017.1283683
14. Nutrition Care Process Research Outcomes (NCPRO) Committee. Electronic Nutrition Care Process Terminology. (2021). Available online at: https://www.ncpro.org/nutrition-care-process (accessed September 6, 2021).
15. Davies A, Chan V, Bauman A, Signal L, Hosking C, Gemming L, et al. Using wearable cameras to monitor eating and drinking behaviours during transport journeys. Eur J Nutr. (2021) 60:1875–85. doi: 10.1007/s00394-020-02380-4
16. Zhang S, Nguyen D, Zhang G, Xu R, Maglaveras N, Alshurafa N. Estimating caloric intake in bedridden hospital patients with audio and neck-worn sensors. 2018 IEEE/ACM Int Conf Connect Health. Appl Syst Eng Technol (CHASE) (2018) 18:1–2. doi: 10.1145/3278576.3278577
17. Schembre SM, Liao Y, O'Connor SG, Hingle MD, Shen S, Hamoy KG, et al. Mobile ecological momentary diet assessment methods for behavioral research: Systematic review. JMIR mHealth uHealth. (2018) 6:e11170. doi: 10.2196/11170
18. Scisco JL. (2012) Source of variance in bite count. [dissertation/master's thesis]. [Clemson (SC)]: Clemson University
19. O'Brien MK, Botonis OK, Larkin E, Carpenter J, Martin-Harris B, Maronati R, et al. Advanced machine learning tools to monitor biomarkers of dysphagia: A wearable sensor proof-of concept study. Digit Biomark. (2021) 5:167–75. doi: 10.1159/000517144
20. De Roos B. Brennan L. Personalised interventions—a precision approach for the next generation of dietary intervention studies. Nutrients. (2017) 9:847. doi: 10.3390/nu9080847
21. Doulah A, Farooq M, Yang X, Parton J, McCrory MA, Higgins JA, et al. Meal microstructure characterization from sensor-based food intake detection. Front Nutr. (2017) 4:31. doi: 10.3389/fnut.2017.00031
22. Ohkuma T, Hirakawa Y, Nakamura U, Kiyohara Y, Kitazono T. Ninomiya T. Association between eating rate and obesity: a systematic review and meta-analysis. Int J Obes. (2015) 39:1589–96. doi: 10.1038/ijo.2015.96
23. Stein K. Remote nutrition counselling: considerations in a new channel for client communication. J Acad Nutr Diet. (2015) 115:10. doi: 10.1016/j.jand.2015.06.375
24. Turner-McGrievy GM, Boutté A, Crimarco A, Wilcox S, Hutto BE, Hoover A, et al. Byte by bite: use of a mobile Bite Counter and weekly behavioral challenges to promote weight loss. Smart Health. (2017) 3:20–6. doi: 10.1016/j.smhl.2017.03.004
25. Beatty JA, Greene GW, Blissmer BJ, Delmonico MJ, Melanson KJ. Effects of a novel bites, steps and eating rate-focused weight loss randomised controlled trial intervention on body weight and eating behaviours. J Hum Nutr Diet. (2020) 33:330–41. doi: 10.1111/jhn.12704
26. Vahid F, Niemelä M, Kangas M, Korpelainen R, Jämsä T. Calibration and validation of accelerometer-based activity monitors: A systematic review of machine-learning approaches. Gait Posture. (2019) 68:285–99. doi: 10.1016/j.gaitpost.2018.12.003
27. Ortega Anderez D, Lotfi A, Pourabdollah A, A. deep learning based wearable system for food and drink intake recognition. J Ambient Intell Human Comput. (2021) 12:9435–47. doi: 10.1007/s12652-020-02684-7
28. Stankoski S, Jordan M, Gjoreski H. Luštrek M. Smartwatch-based eating detection: data selection for machine learning from imbalanced data with imperfect labels. Sensors. (2021) 21:1902. doi: 10.3390/s21051902
29. Solis R, Pakbin A, Akbari A, Mortazavi BJ, Jafari R. A human-centered wearable sensing platform with intelligent automated data annotation capabilities. Proc Int Conf Internet-of-Things Des Implement. (2019) 19:255–60. doi: 10.1145/3302505.3310087
30. Lin B, Hoover A. A comparison of finger and wrist motion tracking to detect bites during food consumption. IEEE Int Conf Wearable Implant Body Sens Netw. (2019) 19:1–4. doi: 10.1109/BSN.2019.8771066
31. Sharma S, Jasper P, Muth E, Hoover A. Automatic detection of periods of eating using wrist motion tracking. 2016 IEEE Connect Health. Appl Syst Eng Technol. (2016) 16:352–63. doi: 10.1109/CHASE.2016.15
32. Kyritsis K, Diou C, Delopoulos A. Detecting meals in the wild using the inertial data of a typical smartwatch. 2019 Conf Proc IEEE Eng Med Biol Soc. (2019) 19:4229-4232. doi: 10.1109/EMBC.2019.8857275
33. Gomes D, Mendes-Moreira J, Sousa I, Silva J. Eating and drinking recognition in free-living conditions for triggering smart reminders. Sensors. (2019) 19:2803. doi: 10.3390/s19122803
34. Sen S, Subbaraju V, Misra A, Balan R, Lee Y. Annapurna: building a real-world smartwatch-based automated food journal. 2018 IEEE I S World Wirel Mobi. (2018) 18:1–6. doi: 10.1109/WoWMoM.2018.8449755
35. Zhang S, Alharbi R, Stogin W, Pourhomayun M, Spring BJ, Alshurafa N. Food watch: detecting and characterizing eating episodes through feeding gestures. BodyNets Int Conf Body Area Netw. (2016) 16:793. doi: 10.4108/eai.15-12-2016.2267793
36. Gan D, Gong J, Lach J. Eating gestures detection by tracking finger motion. IEEE Wirel Health. (2016) 16:1–6. doi: 10.1109/WH.2016.7764549
37. Fortuna C, Giraud-Carrier C, West J. Hand-to-mouth motion tracking in free-living conditions for improved weight control. IEEE Int Conf Health Inform. (2016) 16:341–48. doi: 10.1109/ICHI.2016.62
38. Zhang S, Stogin W, Alshurafa N, I. sense overeating: motif-based machine learning framework to detect overeating using wrist-worn sensing. Inf Fusion. (2018) 51:37–47. doi: 10.1016/j.inffus.2017.08.003
39. Kim H, Shin J, Kim S, Ko Y, Lee K, Cha H, et al. Collaborative classification for daily activity recognition with a smartwatch. IEEE Int Conf Syst Man Cybern. (2016) 16:003707–12. doi: 10.1109/SMC.2016.7844810
40. Siddhartha Varma ALSV, Kumar Sinha V, Das D. Health assessment for office workers by tracking the wrist motion centric activity. 2019 IEEE India Counc Int Conf. (2019) 19:1–4. doi: 10.1109/INDICON47234.2019.9030309
41. Lee J, Paudyal P, Banerjee A, Gupta SKS. IDEA: Instant detection of eating action using wrist-worn sensors in absence of user-specific model. User Model User-adapt Interact. (2018) 18:371–2. doi: 10.1145/3209219.3209265
42. Navarathna P, Bequette BW, Cameron F. Wearable device based activity recognition and prediction for improved feedforward control. Am Control Conf . (2018) 18:3571–6. doi: 10.23919/ACC.2018.8430775
43. Kamachi H, Hossain T, Tokuyama F, Yokokubo A, Lopez G. Prediction of eating activity using smartwatch. Ubicomp/ISWC. (2021) 2021:304–9. doi: 10.1145/3460418.3479348
44. Hnoohom N, Jitpattanakul A, You I, Mekruksavanich S. Deep learning approach for complex activity recognition using heterogeneous sensors from wearable device. RI2C2021. (2021) 21:60-65. doi: 10.1109/RI2C51727.2021.9559773
45. Shin J, Lee S, Lee S. Accurate eating detection on a daily wearable necklace (demo). MobiSys 2019: Proc Annu Int Conf Mob Syst Appl Serv. (2019) 19:649–50. doi: 10.1145/3307334.3328565
46. Bi Y, Lv M, Song C, Xu W, Guan N, Yi W. AutoDietary: A wearable acoustic sensor system for food inake recognition in daily life. IEEE Sens J. (2016) 16:808–16. doi: 10.1109/JSEN.2015.2469095
47. Hussain G, Javed K, Cho J, Yi J. Food intake detection and classification using a necklace-type piezoelectric wearable sensor system. IEICE Trans Inf Syst. (2018) E101:2795–807. doi: 10.1587/transinf.2018EDP7076
48. Chun KS, Bhattacharya S, Thomaz E. Detecting eating episodes by tracking jawbone movements with a non-contact wearable sensor. Proc ACM Interact Mob Wearable Ubiquitous Technol. (2018) 2:1–21. doi: 10.1145/3191736
49. Kalantarian H, Mortazavi B, Alshurafa N, Sideris C, Le T, Sarrafzadeh M, et al. comparison of piezoelectric-based inertial sensing and audio-based detection of swallows. Obes Med. (2016) 1:6–14. doi: 10.1016/j.obmed.2016.01.003
50. Lee K. Food intake detection using ultrasonic doppler sonar. IEEE Sens J. (2017) 17:6056–68. doi: 10.1109/JSEN.2017.2734688
51. Zhang S, Zhao Y, Nguyen D, Xu R, Sen S, Hester J, et al. NeckSense: A multi-sensor necklace for detecting eating activities in free-living conditions. Proc ACM Interact Mob Wearable Ubiquitous Technol. (2020) 4:1–26. doi: 10.1145/3397313
52. Nguyen DT, Cohen E, Pourhomayoun M, Alshurafa N. SwallowNet: Recurrent neural network detects and characterizes eating patterns. IEEE Int Conf Pervasive Comput Commun. (2017) 17:401–6. doi: 10.1109/PERCOMW.2017.7917596
53. Kondo T, Shiro H, Yokokubo A, Lopez G. Optimized classification model for efficient recognition of meal-related activities in daily life meal environment. Int Conf Inform Electron Vis 3rd Int Conf Imagine Vis Pattern Recognit. (2019) 19:146–51. doi: 10.1109/ICIEV.2019.8858526
54. Bi S, Wang T, Tobias N, Nordrum J, Wang S, Halvorsen G, et al. Auracle: detecting eating episodes with an ear-mounted sensor. Proc ACM Interact Mob Wearable Ubiquitous Technol. (2018) 2:1–27. doi: 10.1145/3264902
55. Bi S, Wang T, Davenport E, Peterson R, Halter R, Sorber J, et al. Toward a wearable sensor for eating detection. Proc 2017 Workshop Wearable Syst Appl. (2017) 17:17–22. doi: 10.1145/3089351.3089355
56. Islam MS, Hossain T, Ahad MAR, Inoue S. “Exploring human activities using eSense earable device”. In: editors Ahad MAR, Inoue S, Roggen D, Fujinami K, Activity and Behavior Computing. (Singapore: Springer) p. 169-185 (2021).
57. Taniguchi K, Kondo H, Tanaka T, Nishikawa A. Earable RCC: development of an earphone-type reliable chewing-count measurement device. J Healthc Eng. (2018) 2018:1–8. doi: 10.1155/2018/1838563
58. Taniguchi K, Chiaki H, Kurosawa M, Nishikawa A, A. novel earphone type sensor for measuring mealtime: consideration of the method to distinguish between running and meals. Sensors. (2017) 17:252. doi: 10.3390/s17020252
59. Blechert J, Liedlgruber M, Lender A, Reichenberger J, Wilhelm FH. Unobtrusive electromyography-based eating detection in daily life: A new tool to address underreporting? Appetite. (2017) 118:168–73. doi: 10.1016/j.appet.2017.08.008
60. Papapanagiotou V, Diou C, Zhou L, van den Boer J, Mars M, Delopoulos A, et al. novel chewing detection system based on PPG, audio, and accelerometry. IEEE J Biomed Health Inform. (2017) 21:607–18. doi: 10.1109/JBHI.2016.2625271
61. Bedri A, Li R, Haynes M, Kosaraju RP, Grover I, Prioleau T. EarBit: Using wearable sensors to detect eating episodes in unconstrained environments. Proc ACM Interact Mob Wearable Ubiquitous Technol. (2017) 1:1–20. doi: 10.1145/3130902
62. Farooq M, Sazonov E. Accelerometer-based detection of food intake in free-living individuals. IEEE Sens J. (2018) 18:3752–8. doi: 10.1109/JSEN.2018.2813996
63. Farooq M, Sazonov E. Real time monitoring and recognition of eating and physical activity with a wearable device connected to the eyeglass. Int Conf Sens Tech. (2017) 17:1–6. doi: 10.1109/ICSensT.2017.8304420
64. Zhang R, Amft O. Free-living eating event spotting using EMG-monitoring eyeglasses. 2018 IEEE EMBS Int Conf Biomed Health Inform. (2018) 18:128–132. doi: 10.1109/BHI.2018.8333386
65. Chung J, Chung J, Oh W, Yoo Y, Lee WG, Bang H, et al. glasses-type wearable device for monitoring the patterns of food intake and facial activity. Sci Rep. (2017) 7:41690. doi: 10.1038/srep41690
66. Ghosh T, Hossain D, Imtiaz M, McCrory MA, Sazonov E. Implementing real-time food intake detection in a wearable system using accelerometer. 2020 IEEE-EMBS Conf Biomed Eng Sci. (2021) 21:439–43. doi: 10.1109/IECBES48179.2021.9398760
67. Bedri A, Li D, Khurana R, Bhuwalka K, Goel M. Fitbyte: Automatic diet monitoring in unconstrained situations using multimodal sensing on eyeglasses. Proc 2020 CHI Conf Hum Factors Comput Syst. (2020) 20:1–12. doi: 10.1145/3313831.3376869
68. Selamat NA, Md Ali SH. “Analysis of chewing signals based on chewing detection using proximity sensor for diet monitoring”. In: Editors A Del Bimbo et al. Pattern Recognition, (New York, NY: Springer International Publishing), p. 599–616 (2021).
69. Lin Z, Xie Y, Guo X, Wang C, Ren Y, Chen Y. “Wi-fi-enabled automatic eating moment monitoring using smartphones”. In: Editors Garcia N, Pires I, Goleva R, IoT Technologies for HealthCare, (New York, NY: Springer), p. 77–91 (2020).
70. Nyamukuru MT, Odame KF. Tiny eats: eating detection on a microcontroller. 2020 IEEE 2nd Workshop Mach Learn Edge Sensor Syst. (2020) 20:19-23. doi: 10.1109/SenSysML50931.2020.00011
71. Wang S, Zhou G, Ma Y, Hu L, Chen Z, Chen Y, et al. Eating detection and chews counting through sensing mastication muscle contraction. Smart Health. (2018) 9:179–91. doi: 10.1016/j.smhl.2018.07.004
72. Zhang Z, Zheng H, Rempel S, Hong K, Han T, Sakamoto Y, et al. A smart utensil for detecting food pick-up gesture and amount while eating. Proc 11th Augment Hum Int Conf. (2020) 20:1–8. doi: 10.1145/3396339.3396361
73. Chun KS, Jeong H, Adaimi R, Thomaz E. Eating episode detection with jawbone-mounted inertial sensing. 2020 42nd Conf Proc IEEE Eng Med Biol Soc. (2020) 20:4361–63. doi: 10.1109/EMBC44109.2020.9175949
74. Chun KS, Bhattacharya S, Dolbear C, Kashanchi J, Thomaz E. Intraoral temperature and inertial sensing in automated dietary assessment: a feasibility study. Proc Int Symp Wearable Comput. (2020) 20:27–31. doi: 10.1145/3410531.3414309
75. Yang J, Wang J, Nakandala S, Kumar A, Jankowska MM. Predicting eating events in free living individuals. Int Conf eSci. (2019) 19:627–29. doi: 10.1109/eScience.2019.00090
76. Chen Y, Cui Z, Yen CC. Demo: Chewpin: A wearable acoustic device for chewing detection. UbiComp/ISWC. (2021) 2021:11–2. doi: 10.1145/3460418.3479290
77. Johnson G, Wang Y, Rajamani R. Real-time detection of food consumption activities using wearable wireless sensors. 2019 Am Control Conf. (2019) 19:3450–55. doi: 10.23919/ACC.2019.8814983
78. Hussain G, Maheshwari MK, Memon ML, Jabbar MS, Javed K, A CNN. based automated activity and food recognition using wearable sensor for preventive healthcare. Electronics. (2019) 8:1425. doi: 10.3390/electronics8121425
79. Farooq M, Doulah A, Parton J, McCrory MA, Higgins JA, Sazonov E. Validation of sensor-based food intake detection by multicamera video observation in an unconstrained environment. Nutrients. (2019) 11:609. doi: 10.3390/nu11030609
80. Mirtchouk M, Kleinberg S. Detecting granular eating behaviors from body-worn audio and motion sensors. 2021 IEEE EMBS Int Conf Biomed Health Inform. (2021) 21:1–4. doi: 10.1109/BHI50953.2021.9508519
81. Jasper PW, James MT, Hoover AW, Muth ER. Effects of bite count feedback from a wearable device and goal-setting on consumption in young adults. J Acad Nutr Diet. (2016) 116:1785–93. doi: 10.1016/j.jand.2016.05.004
82. Chen J, Berkman W, Bardouh M, Ng CYK, Allman-Farinelli M. The use of a food logging app in the naturalistic setting fails to provide accurate measurements of nutrients and poses usability challenges. Nutrition. (2019) 57:208–16. doi: 10.1016/j.nut.2018.05.003
83. Healthie, [Mobile application software]. (Version 7.4.0.). Healthie Inc. Available online at: https://apps.apple.com/us/app/healthie/id1112029170 (accessed October 1, 2021).
84. Cronometer, Nutrition Tracker [Mobile application software]. (Version 3.11.2.). Cronometer Software Inc. Available online at: https://apps.apple.com/us/app/cronometer-nutrition-tracker/id1145935738 (accessed October 1, 2021).
85. Ascend for Lose It! [Web application software]. FitNow, Inc. Available online at: from https://www.ascendapp.com/ (accessed October 27, 2021).
86. Chen J, Gemming L, Hanning R, Allman-Farinelli M. Smartphone apps and the nutrition care process: Current perspectives and future considerations. Patient Educ Couns. (2018) 101:750–7. doi: 10.1016/j.pec.2017.11.011
87. Dao MC, Subar A, Warthon-Medina M, Cade J, Burrows T, Golley R, et al. Dietary assessment toolkits: an overview. Public Health Nutr. (2019) 22:404–18. doi: 10.1017/S1368980018002951
88. Rollo ME, Aguiar EJ, Williams RL, Wynne K, Kriss M, Callister R, et al. eHealth technologies to support nutrition and physical activity behaviors in diabetes self-management. Diabetes Metab Syndr Obes. (2016) 9:381–90. doi: 10.2147/DMSO.S95247
89. Rollo ME, Williams RL, Burrows T, Kirkpatrick SI, Bucher T, Collins CE. What are they really eating? a review on new approaches to dietary intake assessment and validation. Curr Nutr Rep. (2016) 5:307–14. doi: 10.1007/s13668-016-0182-6
90. Bell B, Alam R, Alshurafa N, Thomaz E, Mondol A., de la Haye K, et al. Automatic, wearable-based, in-field eating detection approaches for public health research: a scoping review. NPJ Digit Med. (2020) 3:38. doi: 10.1038/s41746-020-0246-2
91. Riley MD, Hendrie GA, Baird DL. Drink Choice is Important: Beverages make a substantial contribution to energy, sugar, calcium and vitamin C intake among Australians. Nutrients. (2019) 11:1389. doi: 10.3390/nu11061389
92. Kreutzer JF, Deist J, Hein CM, Lueth TC. Sensor systems for monitoring fluid intake indirectly and directly. 2016 IEEE Int Conf Wearable Implant Body Sens Netw. (2016) 16:1–6. doi: 10.1109/BSN.2016.7516223
93. Becker MH, Maiman LA. Strategies for enhancing patient compliance. J Community Health. (1980) 6:113–35. doi: 10.1007/BF01318980
94. Zhang R, Amft O. Monitoring chewing and eating in free-living using smart eyeglasses. IEEE J Biomed Health Inform. (2018) 22:23–32. doi: 10.1109/JBHI.2017.2698523
95. Doulah A, Ghosh T, Imtiaz MH, Sazonov E. “Automatic Ingestion Monitor Version 2” – A novel wearable device for automatic food intake detection and passive capture of food images. IEEE J Biomed Health Inform. (2021) 25:568–76. doi: 10.1109/JBHI.2020.2995473
96. Lymberis A, de Rossi D. Wearable ehealth systems for personalised health management: state of the art and future challenges. Amsterdam: IOS Press. (2004).
Keywords: wearable sensors, dietary assessment, dietary intake, food timing, food intake detection, nutrition care, sensors, scoping review
Citation: Wang L, Allman-Farinelli M, Yang J-A, Taylor JC, Gemming L, Hekler E and Rangan A (2022) Enhancing Nutrition Care Through Real-Time, Sensor-Based Capture of Eating Occasions: A Scoping Review. Front. Nutr. 9:852984. doi: 10.3389/fnut.2022.852984
Received: 11 January 2022; Accepted: 11 April 2022;
Published: 02 May 2022.
Edited by:
Jennifer Ann Nasser, Drexel University, United StatesReviewed by:
Juan Manuel Fontana, National University of Río Cuarto, ArgentinaMegan A. McCrory, Boston University, United States
Copyright © 2022 Wang, Allman-Farinelli, Yang, Taylor, Gemming, Hekler and Rangan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Anna Rangan, anna.rangan@sydney.edu.au