Skip to main content

Application of Reinforcement Learning in HVAC systems. Part 3


SARSA controller setup

After we figured out the environment, it is important to understand how to apply RL concept on it. RL works in continuous as well as in discrete time state-space environments. MLE+ dictates the conditions in which we will operate: we will construct discrete state-space using variation of three parameters – temperature set point, actual temperature of a zone and electric heater. I did it in straightforward way – having the vectors of values that can be possible for each variable (e.g. heater can have loads between 0 and 12000KW); made combinations of all three vectors to define all state-spaces within which our building model can operate (more importantly we are trying to not define state-spaces that not possible for a real building systems). Along creation of states it creates Q matrix for three actions meaning increase, decrease and don't change temperature set point using reward function described bellow: As you may notice reward function appreciates minimum power consumption and penalizes for out of range temperatures. As i mentioned I used SARSA lambda algorithm for that (you can find pseudo code bellow): The following code bellow is control file for MLE+ and contains implementation of SARSA lambda algorithm. P.S. Any comments are highly appreciated!!!

Comments

Post a Comment

Popular posts from this blog

Machine Learning. Part II.

Non-residential building occupancy modeling. Part II. Occupancy classification So dataset was taken from this place . The dataset comprised of different sources: surveys, data logging from sensors, weather, environment variables. Total feature list consist of 118 features and can be grouped as general (occupancy, time), environment (indoor, outdoor), personal characteristics (age, office type, accepted sensation range etc), comfort/productivity/satisfaction, behavior (clothing, window, interaction with thermostat etc ), personal values (choices on different set points). It contains data on 24 occupants whether it private office or joint one, the first task is to implement binary classification of each occupant using some input data from sensors and time. For rapid protoyping I will use python Tensor Flow wrapper Keras along Anaconda framework. First, loading all required libraries from keras.models import Sequential from keras.layers import Dense import numpy ...
2 comments
Next >>

Machine Learning

Machine Learning. Non-residential building occupancy modeling. Part I. Idea : providing occupancy driven energy efficiency model. Since about 40% of building energy goes to HVAC(heating, ventilation and air conditioning) it is quite good idea to use these equipment when it is really required,e.g. switch on the air conditioner or ventilation when people in a room The study discusses approaches towards optimizing energy consumption of commercial buildings. Such task is considered to be a part of more broad topics, e.g. smart buildings, green buildings. During the past years, the number of papers dedicated to energy efficiency optimization in buildings has been growing which confirms societal concern about finding the most efficient methods of improving energy usage by buildings. To maximize building’s energy efficiency various methods are known and can be split into re-organizational advances and strengthening currently employed management systems [1], [2]. This methods incl...
Post a Comment
Next >>

Application of Reinforcement Learning in HVAC systems. Part 2

So, how to model an office building to simulate the work of our controller? In short, I have used the following list of programs: Matlab, EnergyPlus and MLE+ in tandem. First things first, EnergyPlus - is an building simulation engine that will allow you to modulate maybe not all but most of the physical phenomena running inside real physical structures, including heat transfer and temperature spread. Even though it is a quite a hard to understand how to use EnergyPlus if you are not expert (maybe even for civil engineers), you can always download already designed models of a buildings like I did), which can be found on energy.gov site. Therefore, I took one floor three office medium building EnergyPlus model that comes with and .idf and weather files. Basically, the building has three rooms with electric radiant heating floors and one window and some ventilation system. Simple schematics shown on a picture below: As you may guess, I want to be able to test some controllers on th...
2 comments
Next >>