Experimental implementation in the environmental chamber of the ICE lab
Experimental implementation in the environmental chamber of the ICE lab

Occupant behavior is known as a key driver of building energy use. It is a highly stochastic and complicated phenomenon, driven by a wide variety of factors, and is unique in each building. Therefore, it cannot be addressed using analytical approaches traditionally used to describe physics-based aspects of buildings. Consequently, current controls which rely on hard-coded expert knowledge have a limited potential for integrating occupant behavior. An alternative approach is to program a human-like learning mechanism and develop a controller that is capable of learning the control policy by itself through interacting with the environment and learning from experience. Reinforcement Learning (RL), a machine learning algorithm inspired by neuroscience, can be used to develop such a self-learning controller. Given the learning ability, these controllers are able to learn optimal control policy without prior knowledge or a detailed system model, and can continuously adapt to the stochastic variations in the environment to ensure an optimal operation.

The main question that the study “BehaveLearn: Reinforcement Learning for occupant-centric operation of building energy systems” lead by the Ph.D. student Amirreza Heidari  deals with is how to  develop  a controller that can perceive and adapt to the occupant behavior to minimize energy use without compromising user needs?

In this context, three occupant-centric control frameworks are developed in this study:

  • DeepHot: focused on hot water production in residential buildings
  • DeepSolar: focused on solar-assisted space heating and hot water production in
    residential buildings;
  • DeepValve: focused on zone-level space heating in offices

DeepHot and DeepSolar frameworks are evaluated using real-world hot water use behavior of occupants, and DeepValve is experimentally implemented in the environmental chamber of the ICE lab. Results show promising energy saving potentials of each control framework.


  • Occupant behavior is highly stochastic, and is unique in each building
  • Current controls rely on expert knowledge and cannot easily integrate occupant behavior
  • Learning controls can autonomously learn and adapt to occupant behavior to reduce energy use in buildings


  • Collecting data on hot water use behavior in 3 Swiss houses
  • Develop and evaluate DeepHot and DeepSolar frameworks in simulation using real-world data
  • Develop and evaluate DeepValve framework in simulation using real-world data from other studies
  • Experimentally implement DeepValve in the ICE environmental chamber


  • Develop self-learning control frameworks to integrate occupant behavior in building controls
  • Test the frameworks in simulation using real-world data of occupant behavior
  • Test the frameworks experimentally in the environmental chamber of the ICE lab


  • A significant energy reduction can be achieved by integrating occupant behavior
  • Energy reduction is achieved without compromising health and comfort of occupants
  • Self-learning control can also coordinate stochastic renewable energy use and occupant behavior

Journal Publications

Conference Publications