Skip to main content

Application of Reinforcement Learning in HVAC systems. Part 1


How Machine Learning and Reinforcement Learning applied for physical systems like building heating?

Some intro ...

Many civil engineering and computer science researchers are pushing forward the idea of applying new intelligent methods of control of energy consuming systems such as heating, ventilation and air conditioning(HVAC) in non-residential or commercial buildings. The motivation behind improvement of performance of these systems is evident, they account for almost 40% out of total building energy demand.
Since this was a question of my university study I have already done some literature review on this topic and outline the main research directions. Even though some research directions may not be seem directly related, but they are have potential to used for our purpose. Thus, the following are main research topics related to the improvement of energy and comfort performance of HVAC systems:

  1. Statistical methods. Record previous loads, building statistical models trying to predict future loads
  2. Classical types of control methods, mainly applied to air conditioning, heating. Rule based control (if ... else ...),PID controls preferable used locally on thermostats, model predictive control is a finite-horizon optimization of a plant model
  3. Modelling of building occupancy. People study the methods of accurate prediction of the number of occupants, setting up occupancy patterns, finding relations to other systems (like HVAC)
  4. Machine learning. ML methods actually used to develop controllers that are capable to substitute for example PID ones and outperform
  5. Finally Reinforcement Learning(RL). Idea is simple - learning a better controller through observing actual controlled behavior, kind of unsupervised version of ML controller
In the next part (part 2) I will be applying RL methods straightforward to simulated model of three office one floor building simulated in EnergyPlus with reward function simply aimed to keep power consumed low and stay within temperature limits. P.S. Stay tuned and keep environment safe :)

Comments

Post a Comment

Popular posts from this blog

Machine Learning. Part II.

Non-residential building occupancy modeling. Part II. Occupancy classification So dataset was taken from this place . The dataset comprised of different sources: surveys, data logging from sensors, weather, environment variables. Total feature list consist of 118 features and can be grouped as general (occupancy, time), environment (indoor, outdoor), personal characteristics (age, office type, accepted sensation range etc), comfort/productivity/satisfaction, behavior (clothing, window, interaction with thermostat etc ), personal values (choices on different set points). It contains data on 24 occupants whether it private office or joint one, the first task is to implement binary classification of each occupant using some input data from sensors and time. For rapid protoyping I will use python Tensor Flow wrapper Keras along Anaconda framework. First, loading all required libraries from keras.models import Sequential from keras.layers import Dense import numpy ...
2 comments
Next >>

Machine Learning

Machine Learning. Non-residential building occupancy modeling. Part I. Idea : providing occupancy driven energy efficiency model. Since about 40% of building energy goes to HVAC(heating, ventilation and air conditioning) it is quite good idea to use these equipment when it is really required,e.g. switch on the air conditioner or ventilation when people in a room The study discusses approaches towards optimizing energy consumption of commercial buildings. Such task is considered to be a part of more broad topics, e.g. smart buildings, green buildings. During the past years, the number of papers dedicated to energy efficiency optimization in buildings has been growing which confirms societal concern about finding the most efficient methods of improving energy usage by buildings. To maximize building’s energy efficiency various methods are known and can be split into re-organizational advances and strengthening currently employed management systems [1], [2]. This methods incl...
Post a Comment
Next >>

Application of Reinforcement Learning in HVAC systems. Part 2

So, how to model an office building to simulate the work of our controller? In short, I have used the following list of programs: Matlab, EnergyPlus and MLE+ in tandem. First things first, EnergyPlus - is an building simulation engine that will allow you to modulate maybe not all but most of the physical phenomena running inside real physical structures, including heat transfer and temperature spread. Even though it is a quite a hard to understand how to use EnergyPlus if you are not expert (maybe even for civil engineers), you can always download already designed models of a buildings like I did), which can be found on energy.gov site. Therefore, I took one floor three office medium building EnergyPlus model that comes with and .idf and weather files. Basically, the building has three rooms with electric radiant heating floors and one window and some ventilation system. Simple schematics shown on a picture below: As you may guess, I want to be able to test some controllers on th...
2 comments
Next >>