Skip to main content

Application of Reinforcement Learning in HVAC systems. Part 3


SARSA controller setup

After we figured out the environment, it is important to understand how to apply RL concept on it. RL works in continuous as well as in discrete time state-space environments. MLE+ dictates the conditions in which we will operate: we will construct discrete state-space using variation of three parameters – temperature set point, actual temperature of a zone and electric heater. I did it in straightforward way – having the vectors of values that can be possible for each variable (e.g. heater can have loads between 0 and 12000KW); made combinations of all three vectors to define all state-spaces within which our building model can operate (more importantly we are trying to not define state-spaces that not possible for a real building systems). Along creation of states it creates Q matrix for three actions meaning increase, decrease and don't change temperature set point using reward function described bellow: As you may notice reward function appreciates minimum power consumption and penalizes for out of range temperatures. As i mentioned I used SARSA lambda algorithm for that (you can find pseudo code bellow): The following code bellow is control file for MLE+ and contains implementation of SARSA lambda algorithm. P.S. Any comments are highly appreciated!!!

Comments

Post a Comment

Popular posts from this blog

Machine Learning. Part II.

Non-residential building occupancy modeling. Part II. Occupancy classification So dataset was taken from this place . The dataset comprised of different sources: surveys, data logging from sensors, weather, environment variables. Total feature list consist of 118 features and can be grouped as general (occupancy, time), environment (indoor, outdoor), personal characteristics (age, office type, accepted sensation range etc), comfort/productivity/satisfaction, behavior (clothing, window, interaction with thermostat etc ), personal values (choices on different set points). It contains data on 24 occupants whether it private office or joint one, the first task is to implement binary classification of each occupant using some input data from sensors and time. For rapid protoyping I will use python Tensor Flow wrapper Keras along Anaconda framework. First, loading all required libraries from keras.models import Sequential from keras.layers import Dense import numpy ...
2 comments
Next >>

Machine Learning

Machine Learning. Non-residential building occupancy modeling. Part I. Idea : providing occupancy driven energy efficiency model. Since about 40% of building energy goes to HVAC(heating, ventilation and air conditioning) it is quite good idea to use these equipment when it is really required,e.g. switch on the air conditioner or ventilation when people in a room The study discusses approaches towards optimizing energy consumption of commercial buildings. Such task is considered to be a part of more broad topics, e.g. smart buildings, green buildings. During the past years, the number of papers dedicated to energy efficiency optimization in buildings has been growing which confirms societal concern about finding the most efficient methods of improving energy usage by buildings. To maximize building’s energy efficiency various methods are known and can be split into re-organizational advances and strengthening currently employed management systems [1], [2]. This methods incl...
Post a Comment
Next >>

Machine Learning. Part III

Non-residential building occupancy modeling. Part III. Defining occupancy patterns. Recalling from the last part II we have got building occupancy data-set. In this part I will apply k-means cluster algorithm in order to identify typical occupancy patterns with daily resolution. To better visualize input data we have, see the following table below: Month Day Weekday Minutes Temperature Relative humidity Air Velocity Occupancy out of 24 7 31 3 779 23 89 1.3411 0 8 1 4 823 24 88 3.578 2 ... ... ... ... ... ... ... ... So I changed initial data by transforming Matlab absolute time readings into month, day, weekday(where 1 - Sunday,2 - Monday etc.), day time in minutes (with 15 min time step); temperature,velocity and humidity is general outdoor data; and finally total building occupancy. Original...
Post a Comment
Next >>