Systematic Review of Deep Learning and Machine Learning for Building Energy

Ardabili, Sina; Abdolalizadeh, Leila; Mako, Csaba; Torok, Bernat; Mosavi, Amir

doi:10.3389/fenrg.2022.786027

REVIEW article

Front. Energy Res., 18 March 2022

Sec. Process and Energy Systems Engineering

Volume 10 - 2022 | https://doi.org/10.3389/fenrg.2022.786027

Systematic Review of Deep Learning and Machine Learning for Building Energy

Sina Ardabili¹

Leila Abdolalizadeh¹

Csaba Mako²

Bernat Torok²

Amir Mosavi^3,4,5*

¹Department of Informatics, J. Selye University, Komarom, Slovakia
²Institute of the Information Society, University of Public Service, Budapest, Hungary
³Faculty of Civil Engineering, Technische Universitat Dresden, Dresden, Germany
⁴Institute of Information Engineering, Automation and Mathematics, Slovak University of Technology in Bratislava, Bratislava, Slovakia
⁵John von Neumann Faculty of Informatics, Obuda University, Budapest, Hungary

The building energy (BE) management plays an essential role in urban sustainability and smart cities. Recently, the novel data science and data-driven technologies have shown significant progress in analyzing the energy consumption and energy demand datasets for a smarter energy management. The machine learning (ML) and deep learning (DL) methods and applications, in particular, have been promising for the advancement of accurate and high-performance energy models. The present study provides a comprehensive review of ML- and DL-based techniques applied for handling BE systems, and it further evaluates the performance of these techniques. Through a systematic review and a comprehensive taxonomy, the advances of ML and DL-based techniques are carefully investigated, and the promising models are introduced. According to the results obtained for energy demand forecasting, the hybrid and ensemble methods are located in the high-robustness range, SVM-based methods are located in good robustness limitation, ANN-based methods are located in medium-robustness limitation, and linear regression models are located in low-robustness limitations. On the other hand, for energy consumption forecasting, DL-based, hybrid, and ensemble-based models provided the highest robustness score. ANN, SVM, and single ML models provided good and medium robustness, and LR-based models provided a lower robustness score. In addition, for energy load forecasting, LR-based models provided the lower robustness score. The hybrid and ensemble-based models provided a higher robustness score. The DL-based and SVM-based techniques provided a good robustness score, and ANN-based techniques provided a medium robustness score.

Introduction

One of the essential aspects of smart buildings is to provide optimum living conditions following numerous standards and energy parameters (Al Dakheel and Tabet Aoul, 2017; Li et al., 2017). There is a need for energy consumption management to achieve the optimum comfort, cost, and tranquility in the buildings (Vázquez-Canteli et al., 2019). Therefore, several smart facilities and equipment are installed in the building to regulate the living condition. The major parts of energy consumption in buildings provide thermal and refrigeration comfort, i.e., cooling and heating systems, water supply facilities, sanitary spas, lighting-related facilities, and so on (Faizollahzadeh Ardabili et al., 2016; Ma et al., 2017). In addition, different pieces of equipment are installed in each building depending on the type of building, each of which in turn consumes energy. Therefore, in each building, energy is used in different ways to face the needs of residents. Buildings are candidates for about forty percent of the total energy consumption (Vázquez-Canteli et al., 2019). The management of the energy in buildings can be considered as one of the important aspects of smart cities. The sustainability index in urban development can be carefully considered as a social function of energy generation and local consumption in each developing city. The sustainability index of urban development efficiently is a function of energy generation and direct consumption of each developing city. The energy consumption of buildings is responsible for a considerable value of energy consumption in an urban settlement (Ardabili et al., 2019a; Nosratabadi et al., 2019). There are several techniques aiming to accurately estimate and predict the energy production and consumption in the building sector (Mocanu et al., 2016; Li et al., 2017; Singaravel et al., 2018). In overall, at the building level, two types of necessary actions in this direction can be helpful. A series of models are based on physical principles to justify thermal dynamics and energy behavior in mathematical language. These models are the basic models and are characterized based on the type of building and effective parameters. Models developed based on statistical methods are other types of models that are used to estimate the energy consumption based on variables affecting climate and energy costs. From this perspective, the forecasting of demand and consumption is important in development of smart cities. Machine learning (ML)-based techniques, as a subset of artificial intelligence (AI), can provide a practical platform for modeling by considering a wide range of parameters (Fan et al., 2017). ML-based techniques have recently contributed significantly in implementing the reliable estimation models (Amasyali et al., 2018; Zou et al., 2018; Cai et al., 2019; Guiqing Zhang et al., 2020). Several researchers have employed ML-based techniques in different fields of area (Safdari Shadloo et al., 2021). Yahya et al. (2021) employed multilayer perceptron (MLP), radial basis function (RBF), cascade feedforward (CFF), and generalized regression neural networks (GRNN) for estimation of the thermal conductivity of water–alumina nanofluids (Yahya et al., 2021). Zhang et al. (2021) employed an AI-based optimization procedure to obtain the highest efficiency of an off-grid hybrid renewable energy scheme composed of wind, fuel cell, and hydrogen storage schemes (Weiping Zhang et al., 2021). Therefore, these techniques cover a wide range of scientific fields. Figure 1 claims the exponential trend in implementing the ML-based techniques in this realm during the past decade. The main contribution of the present study is to peruse the use of novel ML-based techniques in forming the applications of smart cities in terms of energy. The scope covers the building energy (BE) demand and consumption and energy load prediction which are the key energy concerns in the building sector (Chou and Ngo, 2016; Fan et al., 2019; Chaobo Zhang et al., 2020).

FIGURE 1

FIGURE 1. Number of documents.

In this way, there are several survey articles which have been developed with a little overlap in terms of the purpose of the study. Martinez et al. (2021) developed a survey article for analyzing the AI-based approach employed in distribution networks (Barja-Martinez et al., 2021). Hasan and Roy (2021) developed a review work by emphasizing on the application of DL-based techniques, transfer learning, active learning, and reinforcement learning for handling the cyber-physical building environment energy consumption (Hasan et al., 2021). In a review work by Mohapatra et al. (2021), ML-based and DL-based techniques are evaluated only for predicting the BE consumption (Mohapatra et al., 2021). Uzum et al. (2021), in a review work, evaluated the possible solutions with AI-, DL-, and ML-based optimization techniques for analyzing the effects of rooftop PV on distribution networks (Uzum et al., 2021). Vázquez-Canteli et al. (2019) developed a survey study to analyze the reinforcement learning techniques applied for demand response applications in the smart grid (Vázquez-Canteli et al., 2019). Panchalingam and Chan (2019) developed a survey study for describing and analyzing the energy use in smart buildings using the applications of ML- and DL-based techniques (Panchalingam and Chan, 2021). A number of recent surveys studied the application of machine learning in building energy. However, there is still a gap in the presence of a standard review article where the PRISMA statement is correctly adapted. In addition, an in-depth investigation on the accuracy of the model accuracy is still missing. Other differences are the taxonomy content, study concept, and the perspective of the study, as well as the main findings of the study. The limitations of the concepts used, as well as the study of a limited community of applications of ML-based techniques in previous studies, led to a more comprehensive study in the present survey to cover the weaknesses of previous studies. One of the main weaknesses in previous studies was the evaluation of ML-based and DL-based techniques in energy applications in building sectors. On the other hand, according to studies, the existence of a systematic review article based on a standard method that can extract all the strengths and weaknesses of using machine learning methods and deep learning in energy applications in the building, is missing. The PRISMA method is a method of preparing review articles that have not been discussed in this field. Accordingly, the present study presents a comprehensive systematic review based on the standard PRISMA method. The present study has taken steps to cover these weaknesses by providing evaluation methods and comparing different models in different applications. Accordingly, the present work has four main steps. The first step is to describe the main procedure of the searching and methodology for developing the base of the study. The second step is to analyze the studies with different ML- and DL-based techniques separately in different applications. The third step is to analyze the results and main findings including advantages and dis-advantages of each method in each application, and the final step is to summarize and conclude the main findings and suggestions of the study.

Methodology

This section qualifies how to study and the original taxonomy of the work. Given that this study is a review study, it is necessary to collect similar and related studies and categorize them according to the taxonomy of the work. Therefore, before explaining how to collect information and similar studies, we need to explain the main taxonomy of the article. This article is structured in eight main parts. The first part is the introduction so that we can have an initial introduction of the work and present the problem statement, the existing gaps in this field, and the purpose and justify the work. The second part is the methodology section, which explains how to do the study. The main mission of this article begins with the third section. The third section describes the studies conducted in the field of forecasting energy demand in buildings. This section has six sub-sections. First, the structure and description of artificial neural network (ANN) models, support vector machines (SVMs), hybrid models, and ensemble techniques are examined. In each subset, studies in this field are discussed in parallel. The fifth subdivision is related to the introduction of evaluation parameters, and the sixth subdivision is related to the presentation of the results of studies conducted in the field of forecasting energy demand in buildings. It should be noted that the description of the structure of the models is done once and in the next sections, only the studies are reviewed. The fourth section presents studies on energy consumption forecasting in buildings. This section has five subsets to express the studies performed with ANN models, SVM, hybrid models, and ensemble models and the results obtained. Therefore, the results of each section are reviewed and discussed within the same section. The fifth section presents studies on energy load forecasting in buildings. This section has five subsets to express the studies performed with ANN models, SVM, hybrid models, and ensemble models and the results obtained. The sixth section examines studies conducted by deep learning (DL) models in the BE sector. Due to the high importance of DL models, these models were separated from other machine learning models to be examined separately. This section also includes four subsets to express and describe the structure of recurrent neural networks (RNNs), long–short term memory (LSTM), and convolutional neural networks (CNNs) and the results obtained from these studies. The seventh section expresses the discussion, and the eighth section expresses the conclusion.

The procedure of data collection for review process adopts the PRISMA standard. According to Mosavi et al., (2020), the PRISMA method defines four main levels including: 1) identification, 2) screening, 3) eligibility, and 4) inclusion for developing a systematic review. According to the identification stage, an initial search was performed among the databases. Using Thomson Reuters Web-of-Science (WoS) and Elsevier Scopus, 350 of the most relevant articles are identified. The next level is screening the duplicate articles and choosing the relevant articles according to the title and abstract section. In this level, 70 articles have been identified. The next step is eligibility, to study the full text of articles by authors and to select the relevant articles by considering the eligibility for the final review process. In this level, 52 articles have been selected for the required evaluations. The final level of the PRISMA technique is to build the database of the study for qualitative and quantitative comparisons. The database of the present study includes 52 articles, for performing the required analyses. Figure 2 presents the flowchart of forming the database of the current study using the PRISMA technique.

FIGURE 2

FIGURE 2. Procedure of the PRISMA technique for the reviewing process.

Demand Prediction in the BE Sector

Demand prediction in the BE sector is vital for planning and management purposes in the energy systems. This section presents studies developed for demand prediction using different ML methods in the building sector. Table 1 proposes the top studies in the BE demand prediction using ML-based methods.

TABLE 1

TABLE 1. Studies developed for BE demand prediction.

Ayoub et al. (2018) employed the ANN for forecasting microlevel energy supply and demand for buildings (Ayoub et al., 2018). Marmaras et al. (2017) developed an ANN-based technique for the estimation of the power demand of the building (Marmaras et al., 2017). Buratti et al. (2014) presented an ANN-based predictive model for the reduction of cooling energy demand in buildings (Buratti et al., 2014). Avni et al. (2014) presented a forecasting of Turkey energy demand using the ANN method (Es et al., 2014). Djenouri et al. (2019) presented a survey for the use of ANN-based methods for considering demand problems in smart buildings (Djenouri et al., 2020). Attanasio et al. (2019) developed the SVM method for the prediction of the building primary energy demand in comparison with ANN and DT methods (Attanasio et al., 2019). Paudel et al. (2015) developed the SVM method for forecasting the BE demand in the presence of the pseudo-dynamic approach (Paudel et al., 2015). Luo et al. (2019) developed a hybrid ML-based method for day-ahead prediction of BE demands based on the IoT-based big data platform (Luo et al., 2019). Martina et al. (2019) developed a hybrid prediction model for the estimation of daily global solar radiation to cope with the BE demand (Christy Martina and Amudha, 2019). Kokkinos et al. (2017) (query) developed hybrid ML-based methods for the prediction of energy demand in the building sector (Kokkinos et al., 2017). Popoola et al. (2015) developed a hybrid ML-based method for the prediction of energy savings associated with energy efficient projects in the building sector (Popoola et al., 2015). Reza et al. (2017) provided a platform for the estimation of load demand using a hybrid ensemble method in the BE sector (Raza et al., 2017). Huang et al. (2019) employed a novel ensemble technique for the estimation of energy demand in the building sector (Yao Huang et al., 2019). Consequently, the major share and allocation of the studies conducted for the evaluation of the ML-based techniques for the BE sector’s applications are related for forecasting and optimization purposes. Results claimed that the ML-based methods could successfully cope with the task.

ANN-Based Studies

The ANN can be considered as the practical and frequently used technique among other computational intelligence techniques. ANN-based methods can be successfully applied in forecasting, classification, modeling, clustering, error filtering, and optimization purposes. Artificial neurons make up the center unit of ANNs. Components of the ANN contain neurons, connections, weights and biases, and propagation function. Figure 3 presents a simple form of an ANN method in the presence of the related compounds. In general, there are three steps for developing an ANN method: training, testing, and validating. Training is the most important step because it generates the ANN network.

FIGURE 3

FIGURE 3. Architecture of a simple ANN method.

The ANN contains connections which transport information from the previous node (which is called the output of a neuron) to the next node (which is called the input of a neuron). Equation (1) presents the total simple function of the ANN application in the presence of weights and biases.

I_{j} (t) = \sum_{i} O_{i} (t) w_{i j} + w_{0 j}, (1)

where I_j is the input value from the neuron i to neuron j in the presence of O_i as the output value of the neuron i. w_ij is the weight value, and w_0j is the related bias for the neuron j.

Recently, ANN-based techniques have been successfully used in the BE information sector for controlling, prediction, and optimization tasks. Ayoub et al. (2018) developed the ANN for the forecasting task in the energy demand sector for buildings for considering the building demand in the presence of a hybrid supply system. Accordingly, results claimed that the ANN could cope with the forecasting task. But there was a need for a robust model for preventing overfitting issues. Marmaras et al. (2017) presented an ANN-predictive model for the estimation of power demand in buildings. Six commercial buildings’ electricity demand data were employed in order to train the ANN method. Data were collected from a business park during 1 year. Results claimed that the ANN provided higher accuracy and lower error values in comparing the model outputs and target values. Avni et al. (Es et al., 2014) developed the ANN method for the estimation of energy demand in Turkey using the population and area of the building. The developed ANN technique has been compared with linear regression in terms of accuracy and the robustness of methods. ANN provided higher accuracy than that for the LR technique. Buratti et al. (2014) performed a study for the optimization of the cooling energy demand using the ANN method in the presence of indoor thermal comfort. All of these studies have successfully employed the ANN to obtain the desired results. However, in further studies, researchers found that ANN-based methods can be successful only under certain conditions so that researchers have compared different methods with the ANN to reach an accurate model. The most common of these methods was the support vector-based methods.

Support Vector-Based Studies

Support vector-based machine learning methods are considered as supervised learning models which analyze data required for classification and regression purposes. Support vector-based machine learning builds a model to assign new examples for the training step in the presence of one or more categories. This makes support vector-based methods to be a non-probabilistic binary linear classifier. In the following, the mathematical model of a linear algorithm of the support vector-based machine learning can be found by considering a training dataset to be.

$(\vec{x_{1}}), \dots, (\vec{x_{n}})$ . Each $\vec{x_{i}}$ indicates a p-dimensional vector. The target is to propose the maximum-margin hyperplane. Eq. 2 indicates any hyperplane as the set of the desired points.

\vec{w} . \vec{x} - b = 0, (2)

where $\vec{w}$ is the normal vector to the hyperplane. $\frac{b}{\vec{w}}$ demonstrates the offset value of the hyperplane from the origin along the normal vector.

Originally, there are two types of margins, hard and soft margins. In a hard margin, the optimization problem is to minimize the $\vec{w}$ along $y_{i} (\vec{w} . \vec{x} - b) \geq 1$ for i = 1 … n. But in a soft margin, the optimization problem is a little different. In a soft margin, Eq. 3 has to be minimized.

[\frac{1}{n} \sum_{i = 1}^{n} max (0,1 - y_{i} (\vec{w} . {\vec{x}}_{i} - b))] + λ | | {\vec{w}}^{2} | |, (3)

where λ indicates the trade-off between enhancing the margin size and ensuring that the ${\vec{x}}_{i}$ lie on the correct side of the margin.

Figure 4 presents the maximum margin hyperplane for a support vector-based machine trained with samples which are called the support vectors.

FIGURE 4

FIGURE 4. Maximum margin hyperplane for a support vector.

Djenouri et al. (2020) developed a review work on the use of the SVM method in comparison with ANN and GA techniques for forecasting energy demands in buildings. The SVM method has been proposed for solving issues related to occupants and energy demands. This method has been introduced as a multi-disciplinary solution for energy demand problems in buildings. The SVM provided a higher performance than the ANN, but it can be an accurate model in energy demands by developing novel procedures. Ahmad et al. (2018) presented a review article in the application of support vector-based estimation techniques for forecasting the BE demand. Support vector-based techniques have been compared with ANN-based techniques, as the frequently employed techniques, in terms of accuracy and robustness. The study emphasizes on the highest robustness of support vector-based techniques compared with ANN-based methods in the BE demand sector. Paudel et al. (2015) developed a forecasting model for the BE demand using the SVM. The SVM has been introduced as a sensitive model to the selection of training data which made the author to employ dynamic time warping for training the SVM model.

In another study, Attanasio et al. (2019) developed the SVM method for the estimation of the building primary energy demand in comparison with ANN and DT methods for finding an accurate and robust method. The comparison factors were RMSE and accuracy factors. Based on results, the SVM could successfully do the task with a high reliability and performance compared with the ANN method, but it performed weak compared with the DT method. As a general note, it can be claimed that, in all cases, the SVM performs better than the ANN method. But, as is reported in the study by Attanasio et al. (2019), there is another method which can perform better than the SVM method for forecasting tasks. RF-based methods can be considered with a higher performance than the ANN and SVM. RF is an ensemble-based learning technique. The following sections present a brief description about hybrid and ensemble-based methods.

Hybrid-Based Studies

Hybrid methods are appeared for generating robust techniques by combining single methods for different purposes such as prediction, classification, and optimization purposes. The main aim is to collect the advantages of different methods and eliminate the disadvantage of methods. In most cases, hybrid methods compose one prediction method as the base method and one optimization method (as the complementary or second method) for increasing the accuracy of the prediction method. The most popular hybrid method is the ANFIS. The ANFIS employs fuzzy rules and ANN architecture for obtaining a sustainable prediction model. Figure 5 presents a simple algorithm of developing a hybrid method. Recently, these methods have been frequently used in the prediction of energy demand in the building sector.

FIGURE 5

FIGURE 5. Simple algorithm of developing a hybrid technique.

Luo et al. (2019) developed a hybrid k-means-ANN method for the estimation of the BE demand using a platform based on IoT-based big data. The proposed method could perform the task in the presence of IoT sensors for training the ANN method. Martina et al. (Christy Martina and Amudha, 2019) developed an ANFIS method for the prediction of daily global solar radiation in line with a proper solution for BE demand problems in the presence of meteorological parameters. The proposed ANFIS model has been compared with the statistical models in terms of accuracy and correlation values. In another study, Kokkinos et al. (2017) developed the ANFIS method for the prediction of the future energy demand of buildings for obtaining a future perspective form the BE balance. The proposed ANFIS method has been compared with the ANN and fuzzy cognitive map method in terms of RMSE, MAPE, and MAE. The performance of the ANFIS model was significantly higher than other methods. In another study, Popoola et al. (2015) developed an ANFIS method in comparison with the ANN technique for the prediction of energy savings in the building sector. The evaluation performance was determination coefficient (R2). The ANFIS method provided the best performance due to its hybrid advantages.

Ensemble-Based Studies

Ensemble-based methods, or in other words multiple classifiers, are supervised learning algorithms which have been employed in order to improve the classification accuracies. Developing an efficient ensemble method requires determination of an effective combination of classifiers for elimination of their weakness. These methods have an important difference compared with hybrid methods. Ensemble methods benefit different training algorithms for enhancing the training accuracy for generating a higher performance in the testing phase. Ensemble methods in fact allow for different training algorithms for generating flexible training (ref. Advances in machine learning modeling, Hybrids, and Ensembles). Boosting and bagging are most frequently used ensemble methods. This technique is also frequently employed in energy demand prediction in the building sector. Raza et al. (2017) developed an ensemble method including the neural ensemble, Bayesian model, and wavelet transform for the estimation of photo voltaic application in the energy demand for the building sector. This technique has been compared in the term of the nRMSE with different single and hybrid machine learning techniques. The selected ensemble method could successfully forecast the demand parameters and enhance the reliability of the method significantly. Huang et al. (Yao Huang et al., 2019) developed a novel ensemble method by employing extreme gradient boosting, MLR and ELM for developing an SVR method in the presence of a historical energy comprehensive variable. Outputs have been evaluated using RMSE and MAE parameters. According to the findings, the proposed ensemble method significantly provided a higher performance than single methods.

Criteria for Evaluations

The success of the discussed methods has been evaluated based on how capable the developed techniques are in generating most accurate predictions, detection, optimization, and monitoring of the process in terms of their statistical performance accuracy. Table 2 presents the most common evaluating factors used for comparing the efficiency of the discussed techniques.

TABLE 2

TABLE 2. Model evaluation criteria.

Results

Table 3 gives a summarized comparison about the accuracy, reliability, and sustainability of techniques developed for the BE demand prediction. The accuracy index has been obtained through the performance indexes related to the training phase, and reliability has been obtained from the performance indexes related to the testing phase. But, the sustainability index was a little different and has been obtained by comparing reliability, accuracy, processing time, and other factors which have been considered by outputs and findings of the reviewed articles.

TABLE 3

TABLE 3. The comparison findings of techniques for BE demand.

As is clear from Table 3, ANN-based methods provide the lowest sustainability, and hybrid and ensemble-based methods provide the highest sustainability. In order to better understand and discuss the power of each method, it has been employed an index called robustness. This index has been provided as a novel index for describing the strength of each method based on their accuracy, reliability, and sustainability values. Figure 6 presents a concluded graph for each technique, according to their robustness. Figure 6 has been categorized into four limitations including high, good, medium, and low robustness score to describe the capability and strength of each method based on our own observations and understandings from conclusion and results of each study.

FIGURE 6

FIGURE 6. Robustness score for energy demand forecasting methods.

As is clear from Figure 6, for energy demand forecasting, hybrid and ensemble methods are located in the high robustness range, SVM-based methods are located in good robustness limitation, ANN-based methods are located in medium robustness limitation, and linear regression models are located in low robustness limitations.

BE Consumption Prediction

BE consumption is a factor indexed by watt-hours which is considered as a functional unit of energy, consumed in a specific period of time (e.g., daily, monthly, etc.). This factor can be calculated by multiplying BE load values by the number of consumption hours (Badea et al., 2014). BE consumption is considered as one of the most effective factors for designing sustainable instruments in the building sector and can help policy makers in taking a future perspective. Therefore, forecasting tools can play an effective role in this field. Table 4 collects and presents the most important studies developed with energy consumption prediction purposes in the building sector using ML techniques.

TABLE 4

TABLE 4. Studies developed by ML techniques for forecasting BE consumption.