Karthik K. Kulkarni1
Suren Jayasuriya- 1School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, AZ, United States
- 2School of Sustainability, College of Global Futures, Arizona State University, Tempe, AZ, United States
- 3School of Arts, Media and Engineering, Arizona State University, Tempe, AZ, United States
- 4School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ, United States
Extreme heat puts tremendous stress on human health and limits people’s ability to work, travel, and socialize outdoors. To mitigate heat in public spaces, thermal conditions must be assessed in the context of human exposure and space use. Mean Radiant Temperature (MRT) is an integrated radiation metric that quantifies the total heat load on the human body and is a driving parameter in many thermal comfort indices. Current sensor systems to measure MRT are expensive and bulky (6-directional setup) or slow and inaccurate (globe thermometers) and do not sense space use. This engineering systems paper introduces the hardware and software setup of a novel, low-cost thermal and visual sensing device (MaRTiny). The system collects meteorological data, concurrently counts the number of people in the shade and sun, and streams the results to an Amazon Web Services (AWS) server. MaRTiny integrates various micro-controllers to collect weather data relevant to human thermal exposure: air temperature, humidity, wind speed, globe temperature, and UV radiation. To detect people in the shade and Sun, we implemented state of the art object detection and shade detection models on an NVIDIA Jetson Nano. The system was tested in the field, showing that meteorological observations compared reasonably well to MaRTy observations (high-end human-biometeorological station) when both sensor systems were fully sun-exposed. To overcome potential sensing errors due to different exposure levels, we estimated MRT from MaRTiny weather observations using machine learning (SVM), which improved RMSE. This paper focuses on the development of the MaRTiny system and lays the foundation for fundamental research in urban climate science to investigate how people use public spaces under extreme heat to inform active shade management and urban design in cities.
1 Introduction
The year 2020 marks the Earth’s warmest 10-year period with an average increase in global temperature of 1.3 °C above pre-industrial levels. Extreme heat and related heat waves put tremendous stress on individuals’ health and well-being and limits their ability to work, travel, and socialize in outdoor settings. Globally, extreme heat and associated heat wave events are occurring more frequently and longer (Masson-Delmotte et al., 2021). Future trends of urban warming indicate the need for adaption measures to promote resilience in the population. The outdoor urban environment is a complex arrangement of urban forms and materials that impact how heat is experienced by pedestrians at the microscale. In hot, dry cities pedestrian comfort is strongly dictated by the availability of shade (Middel et al., 2014; Colter et al., 2019). Pedestrian may respond to microscale outdoor conditions by changing their walking path from Sun to shade or vice versa based on their heat exposure.
The most common way to report urban heat is air temperature, which has been shown to be insufficient to quantify personal heat exposure (Harlan et al., 2006; Kuras et al., 2017). A more human-centric metric that emphasizes the heat load on the human body is the Mean Radiant Temperature (MRT). MRT objectively quantifies the total short- and longwave radiation the human body is exposed to at a given location and time (Kántor and Unger, 2011). This includes longwave radiation emitted from surrounding surfaces, such as asphalt parking lots or concrete walls, and shortwave radiation from the Sun. MRT roughly equals air temperature in the shade but can be 30 °C higher in the Sun, making a person feel much less comfortable when it is hot. In warm, dry climates such as the desert city of Phoenix, Arizona in the USA, MRT is the heat metric that best describes how people experience heat (Middel et al., 2016). MRT is also a crucial input parameter for calculating outdoor human thermal comfort indices such as PET (Höppe, 1999) and UTCI (Jendritzky et al., 2012).
MRT has been successfully used in urban climate and human biometeorology research to predict heat-related mortality and outperformed air temperature as predictor (Thorsson et al., 2014). Using computer simulations, MRT was estimated to assess the impact of tree planting strategies on human thermal exposure under climate change in Vancouver, Canada (Aminipouri et al., 2019) and to perform thermal comfort routing in Tempe, Arizona, USA (Middel et al., 2017). Observational studies have quantified the benefit of shade for thermal comfort of different shade types including trees, engineered structures, and urban form (Lee et al., 2018; Middel et al., 2021). Accurate, high resolution MRT measurements require expensive equipment, such as the biometeorological instrument platform MaRTy (Middel and Krayenhoff, 2019), but lower-cost alternatives such as the gray 38 mm globe thermometers and cylindrical thermometers have been developed (Thorsson et al., 2006; Brown, 2019; Vanos et al., 2021).
Active shade management in cities is important, especially in the Southwestern US, to provide shade where people work, travel, and socialize outdoors, because cooling benefits are hyperlocal. While a large body of literature has investigated shade and microclimate in hot regions (Ali-Toudert and Mayer, 2007; Emmanuel et al., 2007; Shashua-Bar et al., 2009; Coutts et al., 2016), little information exists on how people use public spaces and when and where they are exposed to outdoor heat. We close this gap by developing a novel low-cost, portable, smart IoT weather station (MaRTiny) that can measure passively the local meteorological conditions, the heat exposure at the given location and count people in the shade and Sun. Connecting hyperlocal meteorological conditions with space use data captured by a camera reveals behavioral patterns of shade and Sun preferences that vary by time of day, location, and ambient conditions. MaRTiny, as a passive sensor package, designed for hot, dry climates, can provide local heat exposure, such as MRT, and space use data without using external database.
Our contributions can be summarized as follows:
1. MaRTiny Weather Station—A low-cost and compact IoT weather station that records air temperature, relative humidity, globe temperature, and wind speed at 1-min intervals. Globe temperature was converted to MRT using an empirical model in the literature and compared with high-end MRT measurements, resulting in a root mean square error (RMSE) of 10.0°C for observed 6-directional measurements vs globe temperature measurements.
2. MaRTiny Vision System—A low-cost, low-powered, compact and smart vision system driven by state-of-the-art AI algorithms. This system counts pedestrian and is also capable of identifying if a pedestrian is under the cooling effect of shade. From our observations, we calculate a precision of 95% for pedestrian detection and an accuracy of 80% for shade detection.
3. Machine Learning based MRT—We developed a novel machine learning model that relies only on a few meteorological parameters and is robust to changes in its surrounding environment. This model corrects errors introduced by the low-cost IoT sensor, such as slow response time, shape, color and material inconsistencies, etc., and predicts MRT with an accuracy of RMSE = 4°C.
This research paper is focused on the engineering and development of a low-cost, portable IoT weather stations for MRT measurement. The study does not focus on a large-scale scientific study of human exposure in outdoor spaces, but rather the engineering and data science challenges of estimating MRT with a combined hardware-software system.
2 Background and Related Work
2.1 Mean Radiant Temperature Sensing
MRT is typically determined with integral radiation measurements using the so-called 6-directional method (Höppe, 1992). Three net radiometers are orthogonally setup to measure the longwave and shortwave radiation in six directions. The radiative fluxes are then summarized into a temperature value using the Stefan-Boltzmann Law:
where Ki and Li are the directional shortwave and longwave radiation fluxes, respectively; ak and al are absorption coefficients for short- and long wave radiation fluxes, respectively; σ is the Stefan-Boltzmann constant; and Wi are factors that weigh the directional fluxes to match the cylindrical shape of the human standing body (0.06 is used for sensors pointing up and down, 0.22 for lateral sensors). This method is limited by cost with three net radiometers that cost $5k each.
A more affordable but less accurate method to estimate MRT is using a black globe thermometer. Globe thermometers such as the Kestrel Heat Stress meter ($500) have been used to quantify the heat load of pedestrians, athletes, and outdoor workers in various studies (Johansson et al., 2014). Thorsson et al. (2006) developed a low-cost globe thermometer using a thermocouple in a gray ping pong ball (
Various convection coefficients have been developed for globe thermometers to improve MRT estimations from globe temperature (Oliveira et al., 2019; Manavvi and Rajasekar, 2020; Acero et al., 2021; Alfano et al., 2021). Those coefficients are usually derived under specific outdoor conditions and cannot be generalized easily. Here, we will use an empirical model for acrylic gray globe temperature Tg developed by Vanos et al. (2021) in Phoenix, AZ based on air temperature Ta, wind speed Va, globe thermometer diameter D = 38mm, and emissivity ϵ = 0.97 of the globe:
2.2 MRT Modeling
Due to limited sensing resources, MRT measurements across space and time are usually sparse. To address this gap, microclimate and radiation models calculate MRT using information on the built form and meteorological data. For example, RayMan (Matzarakis et al., 2010) is a point-based, single location model that requires hemispherical fisheye photos as input and calculates MRT based on the horizon limitation and standard weather information. ENVI-met (Bruse and Fleer, 1998) is a 3D gridded computational fluid dynamics (CFD) model in urban climates studies to assess heat at the neighborhood level. ENVI-met and RayMan calculate MRT based on Sun position to calculate the direct solar radiation and other radiative fluxes. However, Crank et al. (2020) found that both models do not perform well in extreme heat cases and struggle with complex urban forms. Acero and Herranz-Pascual (2015) also report deviations in MRT simulations from globe thermometer readings, especially under clear sky conditions, and Krüger et al. (2014) found that all approaches discussed above (ENVI-met, RayMan, and globe thermometer observations) overestimate MRT when compared to ISO calculations. Currently, no model can accurately estimate MRT in the absence of detailed urban form parameters.
2.3 Pedestrian Counting
Much research has been developed for pedestrian counting and crowd estimation. Sensor-based techniques (Zappi et al., 2010; Wahl et al., 2012; Raykov et al., 2016; Lau et al., 2018) use passive infrared (PIR) and proximity sensors to monitor moving pedestrians. Although these setups are compact and low-cost, they have a low accuracy and misclassify often, and work best only under certain environmental conditions. Alternatively, network-based techniques (Kjærgaard et al., 2012; Weppner and Lukowicz, 2013; Depatla et al., 2015) use Bluetooth and WiFi networks for crowd sensing.
Recently, machine learning techniques low-level image feature extraction methods (Chen et al., 2013, 2012), such as Haar cascade (Viola and Jones, 2001) and HOG (Histogram of Oriented Gradient) (Dalal and Triggs, 2005; Yao et al., 2020) combined with regression models like SVM (Support Vector Machine) (Yao et al., 2020) or detectors like AdaBoost (Viola and Snow, 2003). State-of-the-art methods leverage deep convolutional neural networks for crowd estimation using individual detection (Wu and Nevatia, 2005; Brostow and Cipolla, 2006; Wang and Wang, 2011; Stewart et al., 2016; Liu et al., 2019) and using perspective maps (Chan et al., 2008; Lempitsky and Zisserman, 2010; Zhang et al., 2015).
Further, there are works revolving around analysis of crowd behaviour in urban areas (Hoogendoorn and Bovy, 2004; Hashimoto et al., 2016; Lee, 2020) and their relation with thermal comfort (Arens and Bosselmann, 1989; Givoni et al., 2003; Eliasson et al., 2007; Eom and Nishihori, 2021). We do not aim to outperform any existing pedestrian counting techniques, but to combine them with a weather station as a single setup.
2.4 Research Gaps
In summary, thermal exposure measurements in tandem with public space use assessments are crucial for active shade management in cities, but accurate MRT measurement setups are expensive and bulky. Low-cost systems such as gray globe thermometers have been developed but are not connected to the cloud for easy data storage and analysis. In addition, such low-cost sensors can suffer from over- and under-estimation of MRT at various times of the day as noted in previous literature. None of the existing MRT sensing platforms have vision capabilities, and space use is often assessed through time-consuming manual observations. Finally, physics-based MRT models require detailed 3D data of the urban environment to model radiation flux densities and sun-exposure. Our MaRTiny system aims to address all these gaps.
3 System Overview
The MaRTiny system is a compact, Internet-of-Things (IoT), low-cost sensing and vision/recording/surveying platform (see Figure 1). Its primary functionality is to measure MRT for a given sun-exposed location using off-the-shelf meteorological sensors and a custom-made globe thermometer. MaRTiny measures air temperature, relative humidity, wind speed, and globe temperature, which are used to calculate MRT (see Eq. (2)). In addition, MaRTiny is outfitted with a camera to detect and count people in the shade and Sun. This data helps analyze pedestrian behavior in public spaces (e.g. identifying the number of people who utilize shade, umbrella, bicycles and transportation etc). Privacy is preserved by only storing quantitative metrics (e.g. pedestrian count) and discarding the captured images after analysis.
FIGURE 1. Top view of MaRTiny setup. Jetson Nano is attached with cooling Fan along with camera and WiFi module. Arduino boards are connected to different meteorological sensors.
The entire system transmits data to a cloud database via WiFi. It is powered by a single power source of 20 W which is split among different components according to their power ratings. MaRTiny was built under $200 using different micro-controllers and AI edge devices. MaRTiny is envisioned to be a useful scientific platform to capture MRT data and correlate it with pedestrian behavior in outdoor settings at a fraction of the size/cost of existing solutions. No active human labor is needed for data collection which helps save funds, time, and heat exposure for researchers.
3.1 MaRTiny Weather Station
MaRTiny has four types of sensors to collect meteorological data every minute—multiple temperature probes/thermometers, UV sensor, humidity sensor, and anemometer (wind speed sensor) (see Figure 2 and Table 1 and Table 2 for details). Two temperature probes are utilized for globe and air temperature respectively. Globe temperature is measured using a gray ping-pong ball attached on top of its probe. The globe’s gray color almost matches the albedo of the human skin. The globe thermometer and the derived MRT emulate the omnidirectional thermal exposure for a human body as a function of radiation, air temperature, and velocity, and thus are an accurate low-cost solution to net radiometers (Thorsson et al., 2006). Air temperature is measured using a downward hanging white cup that shades the attached temperature probe. The white cup reflects most of the solar radiation instead of absorbing it to provide an air temperature “free” from the influence of solar radiations. The UV sensor is used to measure the UV intensity and train the machine learning model to estimate MRT based on all measured parameters. MaRTiny is powered by a DC adapter of 5V/4A, which is shared by both systems (weather station and vision system). The anemometer is supplied with 9 V power by stepping up the primary voltage source. This setup can be easily scaled with more sensors without compromising on space and power. In practice, low-cost sensors are subject to noise and variation, which can yield errors in MRT estimation using Eq. (2) as we show later in Section 5. To solve this problem, we introduce a machine learning model to robustly estimate MRT despite these inaccuracies.
FIGURE 2. Block Diagram of the MaRTiny setup along with communication protocols. Five sensors are configured to Arduino Uno and the collected data is transmitted to NodeMCU using Serial Communication which is further sent to AWS database using MQTT Protocol.
TABLE 1. List of meteorological parameters measured by MaRTiny.
TABLE 2. List of electrical MaRTiny parts, costs, and meteorological sensor accuracies.
3.2 MaRTiny Vision
Along with meteorological parameters, MaRTiny requires vision capabilities including object detection and identification as well as shade detection in outdoor areas. We leverage the NVIDIA Jetson Nano, a low-cost and low-powered edge device to run state-of-the-art deep learning models. The Jetson Nano features an ARM-based micro-processor built with a Nvidia V100 GPU programmed through Nvidia’s low level API TensorRT engine. It has configurable power consumption modes of 5W and 10 W. As we perform computationally heavy tasks, we have configured the Jetson Nano to 10 W mode. To capture video, we utilize a compact MIPI (Mobile Industry Processor Interface) camera and stream the data to the Jetson Nano using a gstreamer pipeline. Vision data is sent to AWS via an external USB WiFi on-board. In the next section, we describe in detail our deep learning networks to detect pedestrians in shade.
3.3 Data Logging and Communication
To read meteorological sensor data, we use an Arduino Uno microcontroller. The Uno board communicates with a NodeMCU micro-controller featuring an ESP8266 architecture that has an inbuilt WiFi module, flash memory, and supports the PEM (Privacy Enhanced Mail) file system (see Figure 2). Sensor data are continuously read in a loop by the Uno with a small 1 ms delay to avoid overheating. Data are collected in a buffer, and an average is calculated for every minute, which is then transmitted to the NodeMCU board. The Uno acquires around 80 readings per minute. Both boards communicate via the serial communication protocol UART (universal Asynchronus Receiver/Transmitter).
The NodeMCU communicates securely with the online database. We utilize AWS DynamoDB, a NoSQL flexible database that can handle unstructured data. All the necessary security PEM files are stored in the NodeMCU’s flash memory for authentication of MaRTiny. Using these files, NodeMCU establishes a communication path with AWS through the MQTT protocol, an extremely lightweight publish/subscribe messaging protocol designed for IoT. Once the communication is established, Node MCU waits for bytes of data to be received from the Uno board. Sensor data collected by Uno is sent to NodeMCU via serial communication every minute, which is then transmitted to DynamoDB using the MQTT protocol.
4 Machine Learning Algorithm Development
4.1 Machine Learning for Accurate MRT Estimation
As MaRTiny is a low-cost, compact alternative to the MaRTy sensing platform (Middel and Krayenhoff, 2019; Middel et al., 2020, 2021), the replacement of highly accurate sensors has drawbacks including less accuracy and sensor lag (Häb et al., 2015). We noticed these inaccuracies caused serious errors in the calculated MRT values (Figure 7). In particular, MRT was sensitive to the positioning and orientation of the MaRTiny relative to MaRTy (e.g the MaRTiny was shaded in one of the test, which resulted in lower MRT values, while MaRTy’s net radiometers were partially sun-exposed).
To overcome this limitation, we formulate MRT estimation as a supervised learning problem. This requires labeled ground-truth MRT values to be provided in correspondence with our less robust meteorological sensor data. In Section 5, we discuss data collection consisting of paired MaRTy and MaRTiny measurements to create this labeled data. This allows us to train a machine learning model to estimate MRT accurately from MaRTiny sensor data. We explored both traditional machine learning methods using a support vector machine (SVM) as well as a deep learning-based artifical neural network. These two algorithms are versatile and well-known in machine learning as they satisfy universal approximation theorems (Cybenko, 1989; Hammer and Gersmann, 2002). In particular, we observed an SVM with RBF (Radial Basis Function) kernel achieved the highest accuracy on our evaluation dataset in Section 5. This method is also computationally lightweight and can be easily deployed on the Jetson Nano for performing inference, i.e. the process of using a trained machine learning algorithm to make a prediction.
4.2 People and Shade Detection
4.2.1 Shadow Detection
To perform shadow detection in an image, we use the deep learning model Bi-directional Feature Pyramid with Recurrent Attention Residual Modules (BDRAR) (Zhu et al., 2018), visualized in the upper branch of Figure 3. BDRAR network takes a single image as input and outputs a binary shadow map as output in an end-to-end manner. First, it leverages a convolutional neural network (CNN) to extract feature maps at different spatial resolutions. It then employs two series of recurrent attention residual modules to fully exploit global and local context for these feature maps. The features captured by shallow layers exploit shadow details in the local regions and the features captured by deep layers understands the overall shadow region of the image. Figure 4 provides an example of shadow maps produced by the network.
FIGURE 3. System Overview of the MaRTiny Vision using different types of Neural Networks. The top network represents BDRAR network, responsible for shade detection and the bottom network represents Yolov3. Shade map and bounding box of pedestrian is fed into the pedestrian algorithm to check number of people in shade and sun.
FIGURE 4. Examples of shadow maps produced by BDRAR network. Note how the network works well for dark pixels without mistaking them for shade effect.
4.2.2 Object Detection
For object detection, we utilize the state-of-the-art YOLOv3 network (AlexyAB, 2016) visualized in the lower branch of Figure 3. The model is trained on 80 different classes of the Microsoft COCO dataset. The YOLOv3 algorithm can be built using two different frameworks - DarkNet and MobileNet (Redmon et al., 2016). The MobileNet framework is computationally light but has low accuracy, hence we decided to use the Darknet framework. YOLOv3 has a mAP (mean Average purpose) of around 57 (Redmon and Farhadi, 2018) and has been proven to be efficient in crowd places (Hsu et al., 2020). Since the YOLOv3-darknet model is large and computationally expensive to run on the NVIDIA Jetson Nano, we converted it into a simple neural graph using Nvidia’s TensorRT. This allowed the model to run successfully on the Nano with a application-sufficient frame-rate of 4fps.
4.2.3 Pedestrian in Shade Detection
An image is a 2D representation of the 3D world, so it is difficult to determine the exact location of a pedestrian on the ground and their distance from the camera. We introduce a simple approach to identify pedestrians in shade without determining their position in 3D space. First, a binary shadow map from BDRAR indicating the presence of shade per pixel is computed periodically (in our case, every 15 min as shade does not vary significantly). For every MIPI camera frame, YOLOv3 outputs objects with their bounding boxes consisting of pixel coordinates for the corners. Our algorithm calculates the IOU (Intersection over Union) of the bounding box with the shade map. We consider a person to be in shade if 40% of the bounding box region is inside the shade map (i.e. IOU = 0.4).
Calculating IOU without considering the pedestrian’s position with respect to shade can lead to errors. For example, in Figure 5, one person is sun-exposed and the other person is in the shade. The IOU of the bounding box with the shadow map equals 60% in the first case and 40% in the second case. The IOU for the first case is high due to background shade and shadow cast by the person’s body. This is the most common type of error that occurs at different orientations of a person; therefore, it is necessary to distinguish between shade from a person and shade from the surroundings. Our algorithm first checks if the bounding box edge is in the shade. A person does not have to be completely in shade to feel the cooling effect, hence we consider only the bottom half i.e. 50% of the bounding box as a ROI (Region of Interest). We then calculate the IOU between this region and the shadow map. An IOU of 80% (which implies an IOU of 40% of the complete bounding box) is considered as the optimum value for a person to experience the cooling effects of shade. The ROI and IOU can change based on the environment and application. Pedestrian count under Sun and shade along with other relevant counts (umbrellas, pets, and bicycles) are reported to the online database, and the frame with identifying features is discarded. This allows our device to preserve the privacy of the individuals being observed which is necessary for public deployment.
FIGURE 5. Images of a person with bounding box and shadow map of the surrounding. Using our algorithm, we can detect that the person in the first image is sun-exposed and the person in the second scenario is in shade.
5 System Evaluation
5.1 Data Collection
For evaluation, we collected a custom dataset of ground truth MRT values for two sun-exposed outdoor locations for 3 days in Tempe, Arizona, United States. For validation purposes, the MaRTy human-biometeorological platform (Middel and Krayenhoff, 2019) was paired with the MaRTiny system for simultaneous data logging. Figure 6 illustrates the paired setup, the top box corresponding to MaRTiny and the bottom setup corresponds to MaRTy. We can clearly see the difference in scale between both the setups. In addition, an image dataset was collected for evaluating object and shade detection. Images from the MIPI camera were stored at random intervals along with the bounding boxes of the interested objects. Ground truth bounding boxes were drawn manually using tools such as AlexyAB, (2016); Tosmonav, (2020) for 30 images consisting of around 50 different objects. Precision and Recall for each object were calculated and then used to calculate mAP (mean Average Precision). The same images were used to evaluate shade detection using IOU (Intersection Over Union) metrics. Small video snippets were stored at random intervals which helped to cross-verify the number of people in a given time frame. All the images and videos were stored in an AWS S3 bucket and were deleted after testing.
FIGURE 6. MaRTy and MaRTiny setup. The top white box corresponds to MaRTiny and the entire bottom setup corresponds to MaRTy.
5.2 MRT Estimation
We first evaluated the performance of MaRTiny in estimating MRT values. We utilize Eq. 2 with the sensor data on-board to calculate MRT. MaRTy logs data every 2 s, and MaRTiny stores data every minute, hence we calculated 1-min averages for comparison. Ground truth MRT was calculated using Eq. 1. Figure 7 shows MaRTiny MRT results in green and MaRTy’s ground truth calculation in red. A significant error in MaRTiny’s estimation of MRT was found in the mornings with an MSE of around 10°C. The error is due to the spatial offset between the two devices, which caused the gray globe thermometer of the MaRTiny sensor to be partially shaded by a nearby palm tree in the mornings while MaRTy’s net radiometers were sun-exposed. A palm tree has a narrow shadow pattern covering only portions of the whole MaRTy and MaRTiny setup (see Figure 6). For a detailed discussion of limitations, please see section 6 (Discussion and Limitations).
FIGURE 7. Part I: Performance of MaRTiny weather station along with trained ML model tested between 05-21-2021 to 05-23-2021 in Tempe, Arizona. Comparison of calculated MRT values with SVM model. Note the dips in the calculation due to the shading effects of the environment experienced by the MaRTiny system, which is corrected by the SVM Model.
To overcome these issues, we utilized our supervised learning approach using both SVM and ANN. Machine learning models were trained on selected meteorological parameters - air temperature, globe temperature, humidity and UV intensity, which were comparatively more accurate and less prone to noise. We used around 12,000 data points for training and 3,000 for testing from a range of dates, times, and locations in the sensing period. These training points were fed as vectors into the scikit-learn package in Python for training SVMs and ANNs. 5-fold cross-validation was used to tune model hyperparameters such as learning rate. A separate dataset for evaluation consisted of around 700 data points from a single location collected in a day as is the usual application for this algorithm.
Since there is a non-linear relation of globe temperature and air temperature with MRT given in Eq. 2, machine learning models need to understand complex non-linear relations between these parameters. A SVM with RBF kernel and a neural network with ReLU (Rectified Linear Unit) activation function are example of such models. In Table 3, we present a comparison of SVMs with three different kernels and a traditional artificial neural network (ANN). We report the Root Mean Square Error (RMSE) for both the testing and evaluation datasets. Note that the results of linear and polynomial SVM kernels justify our earlier assumption and the results of SVM with RBF kernel as well as the ANN achieved the best performance in quantitative metrics. From Figure 8 we can see the performance of SVM with RBF kernel, which is almost linear with the ground truth.
TABLE 3. Performance of different supervised machine learning models for MRT estimation.
FIGURE 8. Part II: Performance of MaRTiny weather station along with trained ML model tested between 05-21-2021 to 05-23-2021 in Tempe, Arizona. Performance of SVM Model on test dataset, which is almost linear with ground truth.
We trained our ANN on a i7 CPU. We set our learning rate α to 0.001, which took around 5 min and 300 epochs to converge. Although this model performs slightly better on the test dataset than the SVM-RBF model, performance is identical on the evaluation dataset. The SVM model is also computationally lighter and can be easily trained and deployed on edge devices such as Jetson Nano.
5.3 Shade and Object Detection
For object detection, we leverage the YOLOv3 architecture (Redmon and Farhadi, 2018). While not a state-of-the-art object detector, this model is computationally lightweight in comparison to more modern object detection models. Further, the object detector needed to be compatible with both the TensorRT engine which we utilize on the NVIDIA Jetson board as well as the Python dependencies and packages necessary to run BDRAR as well as itself. Future research could investigate the optimal choice of object detector with shade detection (or a joint-model) for enhanced application performance. Although the model is out-of-box, we wanted to evaluate its performance in the environment suitable for the MaRTiny device and hence, we collected a small custom dataset and evaluated performance on these images. This evaluation on custom dataset should only be considered as a secondary evaluation while we still refer the reader to the main evaluation mentioned in the original study (Redmon and Farhadi, 2018) for the full performance of the object detector.
The standard evaluation metric used for any object detection is mAP (mean Average Precision). Bounding boxes were manually drawn using the tool for the dataset consisting of 30 images and IOU was calculated with the bounding boxes predicted by our model. Precision and recall is calculated for a series of different IOU thresholds ranging from 0.5 to 0.95. A precision-recall graph is constructed and the area under this graph provides us the mAP value of around 55%, which is close to the value reported in their study (Redmon and Farhadi, 2018). For our application IOU threshold of 0.5 gives us the optimal results. We also achieved an Average Precision of more than 85% for the class of Pedestrian, which is important for our application (Figure 9).
FIGURE 9. The graph shows the performance of object detection on different classes of images collected during evaluation.
Evaluation of shadow detection is done on a per pixel basis, which is a binary evaluation method. A dataset consisting of 30 shade images was collected from different location and time. We manually annotated these images using the tool Tosmonav, (2020). We use the pre-trained BDRAR model to evaluate these images and calculated IOU of the shadow map with the ground truth and found a precision of 90%. This is not the most effective method to calculate model accuracy due to the irregular shapes, human error in annotation and small dataset and hence we also refer readers to the evaluation metrics of the original paper (Zhu et al., 2018). We evaluate our pedestrian in shade detection algorithm on a custom dataset of 50 images collected using MaRTiny. We have manually compared the detected values from our algorithm with the ground truth and plotted a confusion matrix to obtain an accuracy of around 80%. Test data consisted of different shading effects and relative positions of pedestrian. Since this kind of testing has not been carried out before, our result can act as a baseline for future tests. The accuracy can be improved on edge cases where pedestrian is partially exposed to Sun at different orientations. Figure 10 and Figure 11 provides examples of MaRTiny Vision where YOLOv3 detects different objects and works along with BDRAR to determine if a pedestrian is in shade or sun.
FIGURE 10. Example of Pedestrian detection along with masks using Yolov3 algorithm (left) and MaRTiny performing pedestrian detection in shade and sun (right).
FIGURE 11. Sample images of Shade Detection and Pedestrian Counting. The top two rows are example of shade map estimation carried at different time and environments (Trees, Buildings, Empty Spaces) and last row has sample images captured by MaRTiny for Object Detection (Pedestrian and Bike).
6 Discussion and Limitations
This systems engineering study introduced a novel low-cost device that combines meteorological sensing with computer vision to estimate MRT and space use. While previous work has mainly focused on assessing the accuracy and precision of various sensors and on advancing MRT simulation tools, our contribution focuses on developing a low-cost hardware and software setup that can be used by non-experts such as city staff and citizen scientists. We also explored the use of state-of-the-art machine learning techniques to improve MRT estimation from low-cost sensors.
This article introduces the setup of the novel MaRTiny system to monitor biometeorological conditions and people’s use of public spaces with changing weather conditions. An empirical study must follow to collect robust data over a long period of time to systematically analyze the relationship between thermal conditions and space usage. In addition, the MaRTiny biometeorological setup must be fully calibrated against NIST certified sensors before deployment, as it is built using off-the-shelf sensors with low accuracy (see Table 2).
While an RMSE of 10 °C between 6-directional MRT observations and globe temperature derived MRT may seem large, it is on the order of magnitude of errors reported by other authors and quite common for outdoor MRT measurements in heterogeneous built environments. Acero et al. (2021) found an RMSE of 7.4 °C for the standard ISO7726 coefficient between the 6-directional setup and a standard black globe. Vanos et al. (2021) found an average difference of -1.6 ± 7.2 °C between an acrylic gray globe and integral radiation measurements on a solar roof that was not subject to shading from the surrounding built environment. Most recently, Lee et al. (2022) reported a large mean difference of 13.2–21.6°C on sunny days between globe thermometer MRT and traditional MRT measurements.
Globe thermometers have various shortcomings, mostly related to the indirect measurement of incident radiative fluxes, which is highly sensitive to globe size, shape, material properties/assumptions, color, and wind speed (Vanos et al., 2021). Guo et al. (2018) and Chen et al. (2014) found significant impacts of wind speed on MRT obtained from globe thermometers, and Teitelbaum et al. (2020) point to errors from free convection. Globe thermometers also have a long response time (Nikolopoulou et al., 1999) that grows with globe diameter. MaRTinies are operated in stationary settings, which reduces the error, but they will not be able to respond quickly to changing cloud conditions. Lastly, globe thermometers are known to overestimate MRT during high incoming solar radiation periods and an underestimate at low solar elevation (Thorsson et al., 2006; Acero and Herranz-Pascual, 2015; Vanos et al., 2021).
The presented machine learning model is a proof-of-concept and is not a reliable MRT predictor in its current state. Our work demonstrates the potential of SVM models for MRT estimation but requires future data collection with identical micrometeorological conditions for all sensors, ideally for a full year, to build a robust model. The RMSE we calculated for our two testing days is high due to a palm tree that shaded the globe thermometer during the morning hours. RMSE is
With respect to the vision system, the BDRAR network exhibits minor inaccuracies in the shadow map estimation and Yolov3. Although it performs well when detecting pedestrians in crowds, it struggles when an individual person occludes another person in the video feed. The shade detection works well but can misclassify a person as shaded or sun-exposed when intersecting the position of a person with the shadow mask, because the algorithm does not take into account the 3-dimensional nature of the shade and person in the scene.
7 Conclusion
Advancements in sensor technology have led to smaller, more portable, and more affordable sensors that facilitate low-cost sensing for many applications. In the domain of urban climate, low-cost sensing has gained popularity for crowdsourcing and citizen science studies, but is also increasingly used to build IoT sensor networks, for example, to monitor air pollution (Xiaojun et al., 2015) or thermal conditions in occupational settings (Sulzer et al., 2022).
MaRTiny leverages edge devices that are low-cost, low-powered, and yet computationally capable of running state-of-the-art machine learning algorithms. Integrating a vision system and people detection into the biometeorological sensing system enables in-depth analyses of how weather and microclimate conditions impact people’s walking behavior in public spaces, including the use of shaded and sun-exposed areas. Once calibrated, the system will be deployed in City of Tempe parks and at playgrounds to inform municipal decision-making on targeted investments for cooling infrastructure in public spaces. The MaRTiny system is an example of how the emerging field of Urban Climate Informatics can support heat mitigation efforts through non-traditional observational methodologies.
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Author Contributions
KK designed and built the system, designed the algorithms and software, analyzed data from experiments, and helped write and edit the manuscript. FS assisted with hardware prototyping and data collection for both the MaRTiny and MaRTy setups as well as writing/revising the paper. TG helped with initial project discussions/conception. SJ helped supervise the project especially with machine learning and computer vision algorithms and helped write and edit the paper. AM conceived of the project idea, supervised the project, and helped write and edit the paper.
Funding
This research was funded by the National Science Foundation (NSF), grant number CMMI-1942805 (CAREER: Human Thermal Exposure in Cities - Novel Sensing and Modeling to Build Heat-Resilience), NSF CNS-1951928 (Understanding Heat Resiliency via Physiological, Mental, and Behavioral Health Factors for Indoor and Outdoor Urban Environments), and DEB-1832016 (Central Arizona-Phoenix Long-Term Ecological Research Program CAP LTER). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the sponsoring organizations.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Acero, J. A., Dissegna, A., Tan, Y. S., Tan, A., and Norford, L. K. (2021). Outdoor Performance of the Black Globe Temperature Sensor on a Hot and Humid Tropical Region. Environ. Technol., 1–13. doi:10.1080/09593330.2021.1989057
Acero, J. A., and Herranz-Pascual, K. (2015). A Comparison of Thermal Comfort Conditions in Four Urban Spaces by Means of Measurements and Modelling Techniques. Build. Environ. 93, 245–257. doi:10.1016/j.buildenv.2015.06.028
[Dataset] AlexyAB (2016). YOLO_mark. Available at: https://github.com/AlexeyAB/Yolo_mark.
Alfano, F. R. D., Dell'isola, M., Ficco, G., Palella, B. I., and Riccio, G. (2021). On the Measurement of the Mean Radiant Temperature by Means of Globes: An Experimental Investigation under Black Enclosure Conditions. Build. Environ. 193, 107655. doi:10.1016/j.buildenv.2021.107655
Ali-Toudert, F., and Mayer, H. (2007). Thermal Comfort in an East–West Oriented Street Canyon in Freiburg (Germany) Under Hot Summer Conditions. Theor. Appl. Climatol. 87, 223–237. doi:10.1007/s00704-005-0194-4
Aminipouri, M., Rayner, D., Lindberg, F., Thorsson, S., Knudby, A. J., Zickfeld, K., et al. (2019). Urban Tree Planting to Maintain Outdoor Thermal Comfort Under Climate Change: The Case of Vancouver's Local Climate Zones. Build. Environ. 158, 226–236. doi:10.1016/j.buildenv.2019.05.022
Arens, E., and Bosselmann, P. (1989). Wind, Sun and Temperature-Predicting the Thermal Comforf of People in Outdoor Spaces. Build. Environ. 24, 315–320. doi:10.1016/0360-1323(89)90025-5
Brostow, G. J., and Cipolla, R. (2006). Unsupervised Bayesian Detection of Independent Motion in Crowds. USA: IEEE Computer Society.
Brown, R. D. (2019). Correcting the Error in Measuring Radiation Received by a Person: Introducing Cylindrical Radiometers. Sensors 19, 5085. doi:10.3390/s19235085
Bruse, M., and Fleer, H. (1998). Simulating Surface-Plant-Air Interactions inside Urban Environments with a Three Dimensional Numerical Model. Environ. Model. Softw. 13, 373–384. doi:10.1016/S1364-8152(98)00042-5
Chan, A. B., Zhang-Sheng John Liang, J., and Vasconcelos, N. (2008). “Privacy Preserving Crowd Monitoring: Counting People without People Models or Tracking,” in Computer Vision and Pattern Recognition. doi:10.1109/CVPR.2008.4587569
Chen, K., Gong, S., Xiang, T., and Change Loy, C. (2013). “Cumulative Attribute Space for Age and Crowd Density Estimation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/cvpr.2013.319
Chen, K., Loy, C. C., Gong, S., and Xiang, T. (2012). “Feature Mining for Localised Crowd Counting,” in Proceedings of the British Machine Vision Conference (BMVA Press), 21. doi:10.5244/C.26.21
Chen, Y.-C., Lin, T.-P., and Matzarakis, A. (2014). Comparison of Mean Radiant Temperature from Field Experiment and Modelling: a Case Study in Freiburg, germany. Theor. Appl. Climatol. 118, 535–551. doi:10.1007/s00704-013-1081-z
Colter, K. R., Middel, A. C., and Martin, C. A. (2019). Effects of Natural and Artificial Shade on Human Thermal Comfort in Residential Neighborhood Parks of Phoenix, arizona, usa. Urban For. Urban Green. 44, 126429. doi:10.1016/j.ufug.2019.126429
Coutts, A. M., White, E. C., Tapper, N. J., Beringer, J., and Livesley, S. J. (2016). Temperature and Human Thermal Comfort Effects of Street Trees across Three Contrasting Street Canyon Environments. Theor. Appl. Climatol. 124, 55–68. doi:10.1007/s00704-015-1409-y
Crank, P. J., Middel, A., Wagner, M., Hoots, D., Smith, M., and Brazel, A. (2020). Validation of Seasonal Mean Radiant Temperature Simulations in Hot Arid Urban Climates. Sci. Total Environ. 749, 141392. doi:10.1016/j.scitotenv.2020.141392
Cybenko, G. (1989). Approximation by Superpositions of a Sigmoidal Function. Math. Control Signal Syst. 2, 303–314. doi:10.1007/BF02551274
Dalal, N., and Triggs, B. (2005). “Histograms of Oriented Gradients for Human Detection,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), 886–893. doi:10.1109/CVPR.2005.177
Depatla, S., Muralidharan, A., and Mostofi, Y. (2015). Occupancy Estimation Using Only Wifi Power Measurements. IEEE J. Sel. Areas Commun. 33, 1381–1393. doi:10.1109/JSAC.2015.2430272
Eliasson, I., Knez, I., Westerberg, U., Thorsson, S., and Lindberg, F. (2007). Climate and Behaviour in a Nordic City. Landsc. Urban Plan. 82, 72–84. doi:10.1016/j.landurbplan.2007.01.020
Emmanuel, R., Rosenlund, H., and Johansson, E. (2007). Urban Shading-A Design Option for the Tropics? A Study in Colombo, Sri Lanka. Int. J. Climatol. 27, 1995–2004. doi:10.1002/joc.1609
Eom, S., and Nishihori, Y. (2021). How Weather and Special Events Affect Pedestrian Activities: Volume, Space, and Time. Int. J. Sustain. Transp. 0, 1–31. doi:10.1080/15568318.2021.1897907
Givoni, B., Noguchi, M., Saaroni, H., Pochter, O., Yaacov, Y., Feller, N., et al. (2003). Outdoor Comfort Research Issues. Energy Build. 35, 77–86. doi:10.1016/S0378-7788(02)00082-8
Guo, H., Teitelbaum, E., Houchois, N., Bozlar, M., and Meggers, F. (2018). Revisiting the Use of Globe Thermometers to Estimate Radiant Temperature in Studies of Heating and Ventilation. Energy Build. 180, 83–94. doi:10.1016/j.enbuild.2018.08.029
Häb, K., Ruddell, B. L., and Middel, A. (2015). Sensor Lag Correction for Mobile Urban Microclimate Measurements. Urban Clim. 14, 622–635. doi:10.1016/j.uclim.2015.10.003
Hammer, B., and Gersmann, K. (2003). A Note on the Universal Approximation Capability of Support Vector Machines. Neural Process. Lett. 17, 43–53. doi:10.1023/A:1022936519097
Harlan, S. L., Brazel, A. J., Prashad, L., Stefanov, W. L., and Larsen, L. (2006). Neighborhood Microclimates and Vulnerability to Heat Stress. Soc. Sci. Med. 63, 2847–2863. doi:10.1016/j.socscimed.2006.07.030
Hashimoto, Y., Gu, Y., Hsu, L.-T., Iryo-Asano, M., and Kamijo, S. (2016). A Probabilistic Model of Pedestrian Crossing Behavior at Signalized Intersections for Connected Vehicles. Transp. Res. Part C Emerg. Technol. 71, 164–181. doi:10.1016/j.trc.2016.07.011
Hoogendoorn, S. P., and Bovy, P. H. L. (2004). Pedestrian Route-Choice and Activity Scheduling Theory and Models. Transp. Res. Part B Methodol. 38, 169–190. doi:10.1016/S0191-2615(03)00007-9
Höppe, P. (1992). A New Procedure to Determine the Mean Radiant Temperature Outdoors. Wetter und Leben 44, 147–151.
Höppe, P. (1999). The Physiological Equivalent Temperature - a Universal Index for the Biometeorological Assessment of the Thermal Environment. Int. J. Biometeorology 43, 71–75. doi:10.1007/s004840050118
Hsu, Y. W., Chen, Y. W., and Perng, J. W. (2020). Estimation of the Number of Passengers in a Bus Using Deep Learning. Sensors (Basel) 20, 2178. doi:10.3390/s20082178
Jendritzky, G., de Dear, R., and Havenith, G. (2012). UTCI-why Another Thermal Index? Int. J. Biometeorol. 56, 421–428. doi:10.1007/s00484-011-0513-7
Johansson, E., Thorsson, S., Emmanuel, R., and Krüger, E. (2014). Instruments and Methods in Outdoor Thermal Comfort Studies - the Need for Standardization. Urban Clim. 10, 346–366. doi:10.1016/j.uclim.2013.12.002
Kántor, N., and Unger, J. (2011). The Most Problematic Variable in the Course of Human-Biometeorological Comfort Assessment - the Mean Radiant Temperature. Central Eur. J. Geosciences 3, 90–100. doi:10.2478/s13533-011-0010-x
Kjaergaard, M. B., Wirz, M., Roggen, D., and Troster, G. (2012). “Mobile Sensing of Pedestrian Flocks in Indoor Environments Using WiFi Signals,” in 2012 IEEE International Conference on Pervasive Computing and Communications, PerCom. doi:10.1109/PerCom.2012.6199854
Krüger, E. L., Minella, F. O., and Matzarakis, A. (2014). Comparison of Different Methods of Estimating the Mean Radiant Temperature in Outdoor Thermal Comfort Studies. Int. J. Biometeorol. 58, 1727–1737. doi:10.1007/s00484-013-0777-1
Kuras, E. R., Richardson, M. B., Calkins, M. M., Ebi, K. L., Hess, J. J., Kintziger, K. W., et al. (2017). Opportunities and Challenges for Personal Heat Exposure Research. Environ. Health Perspect. 125, 085001. doi:10.1289/EHP556
Lau, B. P. L., Wijerathne, N., Ng, B. K. K., and Yuen, C. (2018). Sensor Fusion for Public Space Utilization Monitoring in a Smart City. IEEE Internet Things J. 5, 473–481. doi:10.1109/JIOT.2017.2748987
Lee, H., Jo, S., and Park, S. (2022). A Simple Technique for the Traditional Method to Estimate Mean Radiant Temperature. Int. J. Biometeorol. 66, 521–533. doi:10.1007/s00484-021-02213-x
Lee, I., Voogt, J., and Gillespie, T. (2018). Analysis and Comparison of Shading Strategies to Increase Human Thermal Comfort in Urban Areas. Atmosphere 9, 91. doi:10.3390/atmos9030091
Lee, J. M. (2020). Exploring Walking Behavior in the Streets of new york City Using Hourly Pedestrian Count Data. Sustainability 12, 7863. doi:10.3390/su12197863
Lempitsky, V., and Zisserman, A. (2010). “Learning to Count Objects in Images,”. Advances in Neural Information Processing Systems. Editors J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta (Curran Associates, Inc.), 23.
Liu, W., Salzmann, M., and Fua, P. (2019). “Context-aware Crowd Counting,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/cvpr.2019.00524
Manavvi, S., and Rajasekar, E. (2020). Estimating Outdoor Mean Radiant Temperature in a Humid Subtropical Climate. Build. Environ. 171, 106658. doi:10.1016/j.buildenv.2020.106658
Masson-Delmotte, V., Zhai, P., Pirani, A., Connors, S., Péan, C., Berger, S., et al. (2021). Ipcc, 2021: Climate Change 2021: The Physical Science Basis. Contribution of Working Group I to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press. Available at : https://www.ipcc.ch/report/sixth-assessment-report-working-group-i/
Matzarakis, A., Rutz, F., and Mayer, H. (2010). Modelling Radiation Fluxes in Simple and Complex Environments: Basics of the RayMan Model. Int. J. Biometeorol. 54, 131–139. doi:10.1007/s00484-009-0261-0
Middel, A., AlKhaled, S., Schneider, F. A., Hagen, B., and Coseo, P. (2021). 50 Grades of Shade. Bull. Am. Meteorological Soc. 1, 1–35. doi:10.1175/bams-d-20-0193.1
Middel, A., Häb, K., Brazel, A. J., Martin, C. A., and Guhathakurta, S. (2014). Impact of Urban Form and Design on Mid-afternoon Microclimate in Phoenix Local Climate Zones. Landsc. Urban Plan. 122, 16–28. doi:10.1016/j.landurbplan.2013.11.004
Middel, A., and Krayenhoff, E. S. (2019). Micrometeorological Determinants of Pedestrian Thermal Exposure during Record-Breaking Heat in Tempe, arizona: Introducing the Marty Observational Platform. Sci. Total Environ. 687, 137–151. doi:10.1016/j.scitotenv.2019.06.085
Middel, A., Lukasczyk, J., and Maciejewski, R. (2017). Sky View Factors from Synthetic Fisheye Photos for Thermal Comfort Routing-A Case Study in Phoenix, Arizona. Urban Planning 2, 19–30. doi:10.17645/up.v2i1.855
Middel, A., Selover, N., Hagen, B., and Chhetri, N. (2016). Impact of Shade on Outdoor Thermal Comfort-A Seasonal Field Study in Tempe, Arizona. Int. J. Biometeorol. 60, 1849–1861. doi:10.1007/s00484-016-1172-5
Middel, A., Turner, V. K., Schneider, F. A., Zhang, Y., and Stiller, M. (2020). Solar Reflective Pavements-A Policy Panacea to Heat Mitigation? Environ. Res. Lett. 15, 064016. doi:10.1088/1748-9326/ab87d4
Nikolopoulou, M., Baker, N., and Steemers, K. (1999). Improvements to the Globe Thermometer for Outdoor Use. Archit. Sci. Rev. 42, 27–34. doi:10.1080/00038628.1999.9696845
Olesen, B., Rosendahl, J., Kalisperis, L., Summers, L. H., and Steinman, M. (1989). Methods for Measuring and Evaluating the Thermal Radiation in a Room. Ashrae Trans. 95, 1028–1044.
Oliveira, A. V. M., Raimundo, A. M., Gaspar, A. R., and Quintela, D. A. (2019). Globe Temperature and its Measurement: Requirements and Limitations. Ann. Work Expo. Health 63, 743–758. doi:10.1093/annweh/wxz042
Raykov, Y., Ozer, E., Dasika, G., Boukouvalas, A., and Little, M. (2016). “Predicting Room Occupancy with a Single Passive Infrared (Pir) Sensor through Behavior Extraction,” in UbiComp 2016 - Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (United States: ACM), 1016–1027. 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp 2016 ; Conference date: 12-09-2016 Through 16-09-2016.
Redmon, J., Divvala, S. K., Girshick, R. B., and Farhadi, A. (2016). “You Only Look once: Unified, Real-Time Object Detection,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 779. doi:10.1109/cvpr.2016.91
Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv, ArXiv abs/1804.02767. doi:10.48550/ARXIV.1804.02767
Shashua-Bar, L., Pearlmutter, D., and Erell, E. (2009). The Cooling Efficiency of Urban Landscape Strategies in a Hot Dry Climate. Landsc. urban Plan. 92, 179–186. doi:10.1016/j.landurbplan.2009.04.005
Stewart, R., Andriluka, M., and Ng, A. Y. (2016). “End-to-End People Detection in Crowded Scenes,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/cvpr.2016.255
Sulzer, M., Christen, A., and Matzarakis, A. (2022). A Low-Cost Sensor Network for Real-Time Thermal Stress Monitoring and Communication in Occupational Contexts. Sensors 22, 1828. doi:10.3390/s22051828
Teitelbaum, E., Chen, K. W., Meggers, F., Guo, H., Houchois, N., Pantelic, J., et al. (2020). Globe Thermometer Free Convection Error Potentials. Sci. Rep. 10, 2652–2713. doi:10.1038/s41598-020-59441-1
Thorsson, S., Honjo, T., Lindberg, F., Eliasson, I., and Lim, E.-M. (2007). Thermal Comfort and Outdoor Activity in Japanese Urban Public Places. Environ. Behav. 39, 660–684. doi:10.1177/0013916506294937
Thorsson, S., Rocklöv, J., Konarska, J., Lindberg, F., Holmer, B., Dousset, B., et al. (2014). Mean Radiant Temperature - A Predictor of Heat Related Mortality. Urban Clim. 10, 332–345. doi:10.1016/j.uclim.2014.01.004
[Dataset] Tosmonav (2020). Cvat. Available at: https://github.com/openvinotoolkit/cvat.
Vanos, J. K., Rykaczewski, K., Middel, A., Vecellio, D. J., Brown, R. D., and Gillespie, T. J. (2021). Improved Methods for Estimating Mean Radiant Temperature in Hot and Sunny Outdoor Settings. Int. J. Biometeorol. 65, 967–983. doi:10.1007/s00484-021-02131-y
Viola, P., Jones, M. J., and Snow, D. (2003). “Detecting Pedestrians Using Patterns of Motion and Appearance,” in Proceedings Ninth IEEE International Conference on Computer Vision, 734. doi:10.1109/ICCV.2003.1238422
Viola, P., and Jones, M. (2001). “Rapid Object Detection Using a Boosted Cascade of Simple Features,” in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR. doi:10.1109/CVPR.2001.990517
Wahl, F., Milenkovic, M., and Amft, O. (2012). “A Distributed Pir-Based Approach for Estimating People Count in Office Environments,” in 2012 IEEE 15th International Conference on Computational Science and Engineering, 640–647. doi:10.1109/ICCSE.2012.92
Wang, M., and Wang, X. (2011). Automatic Adaptation of a Generic Pedestrian Detector to a Specific Traffic Scene. CVPR, 3401–3408. doi:10.1109/CVPR.2011.5995698
Weppner, J., and Lukowicz, P. (2013). “Bluetooth Based Collaborative Crowd Density Estimation with Mobile Phones,” in 2013 IEEE International Conference on Pervasive Computing and Communications, PerCom, 193–200. doi:10.1109/PerCom.2013.6526732
Wu, B., and Nevatia, R. (2005). Detection of Multiple, Partially Occluded Humans in a Single Image by Bayesian Combination of Edgelet Part Detectors. Tenth IEEE Int. Conf. Comput. Vis. 1, 90–97. doi:10.1109/iccv.2005.74
Xiaojun, C., Xianpeng, L., and Peng, X. (2015). “Iot-based Air Pollution Monitoring and Forecasting System,” in 2015 International Conference on Computer and Computational Sciences (ICCCS), 257–260. doi:10.1109/iccacs.2015.7361361
Yao, Y., Zhang, X., Liang, Y., Zhang, X., Shen, F., and Zhao, J. (2020). “A Real-Time Pedestrian Counting System Based on Rgb-D,” in 2020 12th International Conference on Advanced Computational Intelligence (ICACI), 110. doi:10.1109/ICACI49185.2020.9177816
Zappi, P., Farella, E., and Benini, L. (2010). Tracking Motion Direction and Distance with Pyroelectric Ir Sensors. IEEE Sensors J. 10, 1486–1494. doi:10.1109/JSEN.2009.2039792
Zhang, C., Li, H., Wang, X., and Yang, X. (2015). “Cross-scene Crowd Counting via Deep Convolutional Neural Networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/cvpr.2015.7298684
Keywords: mean radiant temperature (MRT), thermal sensing, shade detection, urban microclimate, urban heat
Citation: Kulkarni KK, Schneider FA, Gowda T, Jayasuriya S and Middel A (2022) MaRTiny—A Low-Cost Biometeorological Sensing Device With Embedded Computer Vision for Urban Climate Research. Front. Environ. Sci. 10:866240. doi: 10.3389/fenvs.2022.866240
Received: 31 January 2022; Accepted: 15 April 2022;
Published: 12 May 2022.
Edited by:
Adam Schlosser, Massachusetts Institute of Technology, United StatesReviewed by:
Giuseppe Riccio, University of Naples Federico II, ItalyJuan Angel Acero Alejandro, Singapore-MIT Alliance for Research and Technology (SMART), Singapore
Copyright © 2022 Kulkarni, Schneider, Gowda, Jayasuriya and Middel. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Suren Jayasuriya, c2pheWFzdXJAYXN1LmVkdQ==