<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Open Sans";
panose-1:2 11 6 6 3 5 4 2 2 4;}
@font-face
{font-family:"Times New Roman \(Body CS\)";
panose-1:2 11 6 4 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Open Sans",sans-serif;
color:windowtext;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black">Dear all,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black">We are now opening the period to <a href="https://docs.google.com/forms/d/e/1FAIpQLSf8PeqKqw9lzI7xSmjXqdTzzqbYdl3GrgOb7hpPtXETjQVlSg/viewform" title="https://docs.google.com/forms/d/e/1FAIpQLSf8PeqKqw9lzI7xSmjXqdTzzqbYdl3GrgOb7hpPtXETjQVlSg/viewform"><span style="color:#0563C1">sign
up</span></a> for the CityLearn Challenge. Which will begin with the release of the first dataset on <b>January 15<sup>th</sup></b> and is endorsed by International Building Performance Simulation Association (IBPSA). For more information, please visit the <a href="https://sites.google.com/view/citylearnchallenge/home" target="_blank"><span style="color:#0563C1">website </span></a>of
the challenge or the <a href="https://github.com/intelligent-environments-lab/CityLearn" target="_blank" title="https://github.com/intelligent-environments-lab/CityLearn"><span style="color:#0563C1">Github</span></a> repository.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="caret-color: rgb(0, 0, 0);font-variant-caps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-size-adjust: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="caret-color: rgb(0, 0, 0);font-variant-caps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-size-adjust: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black">Reinforcement learning (RL) has gained popularity in the research community as a model-free and adaptive control algorithm for the built-environment. Despite its potential, there
are still open questions regarding its plug-and-play capabilities, performance, safety of operation, and learning speed. Yet, a lack of standardization in previous research has made it difficult to compare different RL algorithms with each other, as different
publications aimed at solving different problems.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="caret-color: rgb(0, 0, 0);font-variant-caps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-size-adjust: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="caret-color: rgb(0, 0, 0);font-variant-caps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-size-adjust: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black">In an attempt to tackle these problems, we have organized this challenge using CityLearn, an OpenAI Gym Environment for the implementation of RL agents for demand response at the
urban level. The environment allows the implementation of single-agent and multi-agent RL controllers. The RL agents can control domestic hot water (DHW) and chilled water storage devices in different sets of 9 simulated buildings in 4 different climate zones
within the US. The RL agents must minimize a multi-objective cost function of 5 equally weighted metrics related to demand response and load shaping in an entire district of buildings.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="caret-color: rgb(0, 0, 0);font-variant-caps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-size-adjust: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="caret-color: rgb(0, 0, 0);font-variant-caps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-size-adjust: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black">The RL agents will be designed, tuned and pre-trained using a design dataset. There will be an evaluation and a challenge dataset with different buildings in different locations (but
within the same climate zones as the design dataset). The agents will be provided with basic information about the buildings, their climate zone, and the similarities between their demand profiles. Participants will submit their agents to us for their evaluation
on the evaluation dataset, and we will update a leaderboard with their scores. At the end of the challenge, on March 31st, participants will submit their agents for the final run on the challenge dataset, which will determine the winners. Although the design
dataset will allow the teams to pre-train their controllers, some action exploration and not just exploitation is encouraged, as the evaluation and challenge datasets will be different than the design dataset. Only the design dataset will be public during
the challenge. </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="caret-color: rgb(0, 0, 0);font-variant-caps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-size-adjust: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="caret-color: rgb(0, 0, 0);font-variant-caps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-size-adjust: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black">Teams can sign up any time (max. 3 people per team). On January 15th, we will upload the design dataset to the Github repository. Teams can submit their agent(s) and get feedback
from the evaluation dataset until <b>March 15th</b>, and they must submit their final agent(s) to be run on the challenge dataset by<span class="apple-converted-space"> </span><b>March 31st</b>. The teams with the top performing agent(s) will be encouraged
to submit their findings to a special issue in a reputed journal (in preparation), more details will be disclosed as soon as possible.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="caret-color: rgb(0, 0, 0);font-variant-caps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-size-adjust: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="caret-color: rgb(0, 0, 0);font-variant-caps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-size-adjust: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black">We appreciate your interest, and for any questions about the challenge please don’t hesitate to contact us on<span class="apple-converted-space"> </span><a href="mailto:citylearn@utexas.edu"><span style="color:#0563C1">citylearn@utexas.edu</span></a><span class="apple-converted-space"> </span>We
are looking forward to seeing you sign up and obtaining state-of-the-art results!</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="caret-color: rgb(0, 0, 0);font-variant-caps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-size-adjust: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black">Here are all the links again:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black">SignUp:
<a href="https://docs.google.com/forms/d/e/1FAIpQLSf8PeqKqw9lzI7xSmjXqdTzzqbYdl3GrgOb7hpPtXETjQVlSg/viewform">
https://docs.google.com/forms/d/e/1FAIpQLSf8PeqKqw9lzI7xSmjXqdTzzqbYdl3GrgOb7hpPtXETjQVlSg/viewform</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black">Website:
<a href="http://www.citylearn.net/">http://www.citylearn.net/</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black">GitHub:
<a href="https://github.com/intelligent-environments-lab/CityLearn">https://github.com/intelligent-environments-lab/CityLearn</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black">eMail:
<a href="mailto:citylearn@utexas.edu">citylearn@utexas.edu</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="caret-color: rgb(0, 0, 0);font-variant-caps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-size-adjust: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black">Best,</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="caret-color: rgb(0, 0, 0);font-variant-caps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-size-adjust: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style="font-size:10.5pt;font-family:"Open Sans",sans-serif;color:black">Zoltan</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Open Sans",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Open Sans",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Open Sans",sans-serif;color:black">-- </span><span style="font-size:11.0pt;font-family:"Open Sans",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Open Sans",sans-serif;color:black">Prof. Dr. Zoltan Nagy</span><span style="font-size:11.0pt;font-family:"Open Sans",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Open Sans",sans-serif;color:black">Intelligent Environments Laboratory: <a href="http://nagy.caee.utexas.edu"><span style="color:#0563C1">http://nagy.caee.utexas.edu</span></a></span><span style="font-size:11.0pt;font-family:"Open Sans",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Open Sans",sans-serif;color:black">Department of Civil, Architectural & Environmental Engineering</span><span style="font-size:11.0pt;font-family:"Open Sans",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Open Sans",sans-serif;color:black">The University of Texas at Austin</span><span style="font-size:11.0pt;font-family:"Open Sans",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Open Sans",sans-serif;color:black">Twitter: <a href="https://twitter.com/Z0ltanNagy"><span style="color:#0563C1">@Z0ltanNagy</span></a></span><span style="font-size:11.0pt;font-family:"Open Sans",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Open Sans",sans-serif;color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Open Sans",sans-serif;color:black">The CityLearn Challenge — Multi-Agent Reinforcement Learning for energy management in cities — <a href="http://www.citylearn.net"><span style="color:#0563C1">http://www.citylearn.net</span></a></span><span style="font-family:"Open Sans",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Open Sans",sans-serif;color:black">Organizing Member — UT Grand Challenge — Whole Communities Whole Health:<a href="https://bridgingbarriers.utexas.edu/whole-communities-whole-health/"><span style="color:#0563C1">https://bridgingbarriers.utexas.edu/whole-communities-whole-health/</span></a></span><span style="font-size:11.0pt;font-family:"Open Sans",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Open Sans",sans-serif;color:black">Organizing Member — BuildSys 2019: <a href="http://buildsys.acm.org/2019/"><span style="color:#0563C1">http://buildsys.acm.org/2019/</span></a></span><span style="font-size:11.0pt;font-family:"Open Sans",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Open Sans",sans-serif;color:black">Guest Editor — Sustainability SI: Sustainable Technologies for Net Zero Energy Buildings (NZEB) <a href="https://www.mdpi.com/journal/sustainability/special_issues/Net_Zero_Energy_Buildings"><span style="color:#0563C1">https://www.mdpi.com/journal/sustainability/special_issues/Net_Zero_Energy_Buildings</span></a></span><span style="font-size:11.0pt;font-family:"Open Sans",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Open Sans",sans-serif;color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Open Sans",sans-serif;color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Open Sans",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Open Sans",sans-serif"><o:p> </o:p></span></p>
</div>
</body>
</html>