Application of Reinforcement Learning in HVAC systems. Part 2

So, how to model an office building to simulate the work of our controller?

In short, I have used the following list of programs: Matlab, EnergyPlus and MLE+ in tandem. First things first, EnergyPlus - is an building simulation engine that will allow you to modulate maybe not all but most of the physical phenomena running inside real physical structures, including heat transfer and temperature spread. Even though it is a quite a hard to understand how to use EnergyPlus if you are not expert (maybe even for civil engineers), you can always download already designed models of a buildings like I did), which can be found on energy.gov site. Therefore, I took one floor three office medium building EnergyPlus model that comes with and .idf and weather files. Basically, the building has three rooms with electric radiant heating floors and one window and some ventilation system. Simple schematics shown on a picture below:

As you may guess, I want to be able to test some controllers on this electric radiant system installed in each office room of our model. Class of radiant heater outlined in picture below for three thermal zones:

Ok, we do have our non-real building, what next? Next, we need to know how we can control that thing (radiant heater). Even though, this system can be controlled within EnergyPlus model body (e.g. via thermostats) we want to impose more sophisticated control on it, right?  Now MLE+, steps into the game. MLE+ is sort of middleware written on Matlab language that serves as intermediate between our building and controller that will be written on Matlab. Here are the screenshot of MLE+ front side:

After you've loaded your .idf and weather files you may want check control section #3 where our future control file will be loaded.

Also, you need to choose variables coming from EnergyPlus model that you will operate with via control file, as depicted on figure below:

Finally, in order to demonstrate the whole concept the last schematic picture explains how the process runs:

Iteratively, EnergyPlus model generates some outputs (we can actually choose which outputs to generate in our .idf file). Out of this output variables via EnergyPlus feature(class) called External Interface allows MLE+ to capture required outputs and process it in the control file that will be written in Matlab. On the next post (part 3) I will explain how I applied Q-learning and SARSA to dvelop controller for the electric radiant heating system in our building. More detailed info on EnergyPlus and MLE+ can be found here https://energyplus.net/ https://github.com/mlab-upenn/mlep_v1.1 Thanks for reading, and leave comments I will be glad answering them!