<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>As a side note, Dru was probably quoting George Box:</p>
<p><a class="moz-txt-link-freetext" href="https://en.wikipedia.org/wiki/All_models_are_wrong">https://en.wikipedia.org/wiki/All_models_are_wrong</a></p>
<p>Jason<br>
</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 3/14/2023 2:12 PM, Jim Dirkes via
Bldg-sim wrote:<br>
</div>
<blockquote type="cite"
cite="mid:E-XNMoS0jj8ky02e4JeYkLrVPKmnWHsE4XsP6_lgleYRRnAK3RjlSdxHlURK9zixfHhc8WPrwODDQ_9JfCHSkeXBZnCWjXt2vgqh90MbHkY=@protonmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div style="font-family: Arial, sans-serif; font-size: 14px;">I
think Dru Crawley coined the phrase, "All models are wrong. Some
are useful."</div>
<div style="font-family: Arial, sans-serif; font-size: 14px;"><br>
</div>
<div style="font-family: Arial, sans-serif; font-size: 14px;">Chris,
you highlighted an assortment of variables which are
omnipresent, inconsistent and uncontrollable - so what, exactly,
does your client expect? Is it a realistic expectation? For
example, are they going to nail you to the wall when,
inevitably, you are "wrong" next year?</div>
<div style="font-family: Arial, sans-serif; font-size: 14px;"><br>
</div>
<div style="font-family: Arial, sans-serif; font-size: 14px;">I
have not calibrated many models, but have been made more
appreciative of all the uncontrollables by the ones I calibrated
:(. All the statisticians know that there is always more than
one solution which will result in a high R2 value or low CVRSME,
so which is correct?</div>
<div style="font-family: Arial, sans-serif; font-size: 14px;"><br>
</div>
<div style="font-family: Arial, sans-serif; font-size: 14px;">Rather
than a calibrated model, lately I've been encouraging clients to
consider one of the FDD platforms on top of their Building
Automation System. Spending time and money to evaluate whether
things are working properly makes more sense to me - it's "real
life" vs a prediction. (Can't forget to mention thoughtful and
thorough commissioning here; that's essential.)</div>
<div style="font-family: Arial, sans-serif; font-size: 14px;"><br>
</div>
<div style="font-family: Arial, sans-serif; font-size: 14px;">ps,
I love your thoughtful approach. You're setting a great example!
One aspect of that is to reach out to the wider modeling
community to gather input and feedback.</div>
<div style="font-family: Arial, sans-serif; font-size: 14px;"><br>
</div>
<div class="protonmail_signature_block" style="font-family: Arial,
sans-serif; font-size: 14px;">
<div class="protonmail_signature_block-user">
<p style="margin-bottom:6pt;" class="MsoNormal"><span
style="background-color:rgb(255,255,255);"
class="highlight"><span style="color:rgb(34,34,34);"
class="colour"><span style="font-family:Arial,
Helvetica, sans-serif;" class="font">Jim Dirkes 1631
Acacia Drive NW Grand Rapids, MI 49504 - 616 450 8653</span></span></span><br>
</p>
<div><span style="background-color:rgb(255,255,255);"
class="highlight"><span style="color:rgb(34,34,34);"
class="colour"><span style="font-family:Arial,
Helvetica, sans-serif;" class="font"><b>Coffee
Conversation:</b></span></span></span></div>
<div><span style="background-color:rgb(255,255,255);"
class="highlight"><span style="color:rgb(34,34,34);"
class="colour"><span style="font-family:Arial,
Helvetica, sans-serif;" class="font"><b><span
style="color:rgb(0,0,0);font-family:Calibri,
sans-serif;font-size:14.6667px;font-weight:400;background-color:rgb(255,255,255);display:inline;">The "individual"
is an impossible concept, conceived by the
Enlightenment philosophers. It makes no sense to
the Christian. In marriages, and families, in
associations and friendships and religious orders,
we are not individuals, but a communion of
persons.</span><br>
</b></span></span></span></div>
</div>
<div class="protonmail_signature_block-proton
protonmail_signature_block-empty"> </div>
</div>
<div style="font-family: Arial, sans-serif; font-size: 14px;"><br>
</div>
<div class="protonmail_quote"> ------- Original Message -------<br>
On Tuesday, March 14th, 2023 at 2:10 PM,
<a class="moz-txt-link-abbreviated" href="mailto:chris.malcolm.yates@gmail.com">chris.malcolm.yates@gmail.com</a>
<a class="moz-txt-link-rfc2396E" href="mailto:chris.malcolm.yates@gmail.com"><chris.malcolm.yates@gmail.com></a> wrote:<br>
<br>
<blockquote class="protonmail_quote" type="cite">
<div class="WordSection1">
<p class="MsoNormal"><span
style="mso-fareast-language:EN-US">I think I need to
qualify this: informed by G14, but definitely not
compliant with it! There is some allowance for repairing
or “healing” data, but when the data has a lot of holes
or modes/ category variables then forget it. This is my
case, but the client still wants some kind of
representative simulation.</span></p>
<p class="MsoNormal"><span
style="mso-fareast-language:EN-US"> </span></p>
<p class="MsoNormal"><span
style="mso-fareast-language:EN-US">Monthly models can be
garbage. School holidays cut across months at different
times, combined heat and power is popular which
complicates gas usage especially when metering is
limited… did the heat by-product of electricity
generation go to the building, or was it rejected? They
can work for heating in our temperate climate, but not
for cooling.</span></p>
<p class="MsoNormal"><span
style="mso-fareast-language:EN-US"> </span></p>
<p class="MsoNormal"><span
style="mso-fareast-language:EN-US">Nevertheless, we need
some kind of representative simulation model. We can’t
make any ECM qualifying claims but we can do something
useful. This is where these methods can give you a lot
of insight before you start modelling.</span></p>
<p class="MsoNormal"><span
style="mso-fareast-language:EN-US"> </span></p>
<p class="MsoNormal"><span
style="mso-fareast-language:EN-US">I hadn’t tried the
IMT previously. We tend to have limited our regression
analysis to monthly “degree day” methods (your 2p model,
I think). I plugged some project specific daily
electricity data into the MVR example (multi variate
regression) and it seemed to give decent CVRMSE (~2-3%)
but low R2 (~0.7). However, it appeared to provide some
insight on cooling usage (monthly 2p models are
meaningless for this in the UK’s temperate climate).</span></p>
<p class="MsoNormal"><span
style="mso-fareast-language:EN-US"> </span></p>
<p class="MsoNormal"><span
style="mso-fareast-language:EN-US">I also made some 2p
monthly models of gas. I thought these were good until I
compared successive years. <b>I guess this is were
understanding a range of statistical indices is
helpful.</b></span></p>
<p class="MsoNormal"><span
style="mso-fareast-language:EN-US"> </span></p>
<p class="MsoNormal"><span
style="mso-fareast-language:EN-US">Here’s the final rub,
because the underlying data has so many inconsistencies
that can only be made sense of with some regression
models, it’s easier to “calibrate” the simulation model
to the regression models than the original data. But I
may use 2p monthly for gas, daily MVR for electric…</span></p>
<p class="MsoNormal"><span
style="mso-fareast-language:EN-US"> </span></p>
<p class="MsoNormal"><span
style="mso-fareast-language:EN-US">So I need to ask if I
have wondered completely off-piste with this!</span></p>
<p class="MsoNormal"><span
style="mso-fareast-language:EN-US"> </span></p>
<p class="MsoNormal"><span
style="mso-fareast-language:EN-US">Chris</span></p>
<p class="MsoNormal"><span
style="mso-fareast-language:EN-US"> </span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1
1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span
lang="EN-US"> David Eldridge
<a class="moz-txt-link-rfc2396E" href="mailto:dancingdavide@hotmail.com"><dancingdavide@hotmail.com></a> <br>
<b>Sent:</b> Tuesday, March 14, 2023 12:53 PM<br>
<b>To:</b> Jim Dirkes
<a class="moz-txt-link-rfc2396E" href="mailto:jvdirkes2@protonmail.com"><jvdirkes2@protonmail.com></a><br>
<b>Cc:</b> <a class="moz-txt-link-abbreviated" href="mailto:chris.malcolm.yates@gmail.com">chris.malcolm.yates@gmail.com</a>;
<a class="moz-txt-link-abbreviated" href="mailto:bldg-sim@onebuilding.org">bldg-sim@onebuilding.org</a><br>
<b>Subject:</b> Re: [Bldg-sim] Lies, darn lies, and
statistics</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Indirectly there probably isn’t a daily
set of metrics in the Guideline since the simulation
programs aren’t usually outputting daily results, but
there’s no reason there couldn’t be one statistically. </p>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">You could make one if you had only
daily utility data and had to aggregate the simulation
results to daily totals, there isn’t a published target
metric but you could still show that you calculated one
and why you think it was a good or bad result.</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">DSE Mobile</p>
</div>
<div>
<div>
<p class="MsoNormal"><br>
<br>
</p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p style="margin-bottom:12.0pt" class="MsoNormal">On
Mar 14, 2023, at 6:10 AM, Jim Dirkes via Bldg-sim
<<a href="mailto:bldg-sim@lists.onebuilding.org"
rel="noreferrer nofollow noopener" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">bldg-sim@lists.onebuilding.org</a>>
wrote:</p>
</blockquote>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;font-family:"Arial",sans-serif">Dear
Chris,</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;font-family:"Arial",sans-serif">Kudos
for appreciating a gap in your understanding.
(I'm in your camp)</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;font-family:"Arial",sans-serif"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;font-family:"Arial",sans-serif">On
the other hand, there are SO many variables in
building operation that, short of a highly
instrumented (and carefully calibrated) building
for everything from lights to people to plug
loads to HVAC - calibration is a fiction (and
I'm confident that no such building exists).
Daily calibration is a complete fiction, perhaps
even a deception. On top of that, a "calibrated"
model is just a moment in time; everything going
forward is guaranteed to be different than
during the calibration time period.</span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;font-family:"Arial",sans-serif"> </span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;font-family:"Arial",sans-serif">I
think of "calibration" as more like a
sensitivity analysis - determine which variables
matter more and which matter less. GenOpt works
nicely for that purpose <a target="_blank"
href="https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Flbl-srg%2FGenOpt&data=05%7C01%7C%7C0d63c1ff284443bc766608db247cc033%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638143890442545264%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=hMX7ISkjm6aByq3ZaUXf5p2QKTf3nlOBYbgP7ALodWQ%3D&reserved=0"
rel="noreferrer nofollow noopener"
moz-do-not-send="true">https://github.com/lbl-srg/GenOpt</a></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;font-family:"Arial",sans-serif"> </span></p>
</div>
<div>
<div>
<p
style="mso-margin-top-alt:auto;margin-bottom:6.0pt"
class="MsoNormal"><span class="font"><span
style="font-size:10.5pt;font-family:"Arial",sans-serif;color:#222222;background:white">Jim
Dirkes 1631 Acacia Drive NW Grand Rapids,
MI 49504 - 616 450 8653</span></span><span
style="font-size:10.5pt;font-family:"Arial",sans-serif"></span></p>
<div>
<p class="MsoNormal"><span class="font"><b><span
style="font-size:10.5pt;font-family:"Arial",sans-serif;color:#222222;background:white">Coffee
Conversation:</span></b></span><span
style="font-size:10.5pt;font-family:"Arial",sans-serif"></span></p>
</div>
<div>
<p class="MsoNormal"><span class="font"><span
style="color:black;background:white">The "individual"
is an impossible concept, conceived by the
Enlightenment philosophers. It makes no
sense to the Christian. In marriages, and
families, in associations and friendships
and religious orders, we are not
individuals, but a communion of persons.</span></span><span
style="font-size:10.5pt;font-family:"Arial",sans-serif"></span></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:10.5pt;font-family:"Arial",sans-serif"> </span></p>
</div>
<div>
<p class="MsoNormal">------- Original Message
-------<br>
On Tuesday, March 14th, 2023 at 6:52 AM, Chris
Yates via Bldg-sim <<a
href="mailto:bldg-sim@lists.onebuilding.org"
rel="noreferrer nofollow noopener"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">bldg-sim@lists.onebuilding.org</a>>
wrote:<br>
<br>
<br>
</p>
<blockquote
style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"
class="MsoNormal">Hi All</p>
<p
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"
class="MsoNormal"> </p>
<p
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"
class="MsoNormal">I do find ASHRAE Guideline
14 a little too hardcore for my basic
understanding of statistics. I can plug any of
the equations into Excel, but I’ve realised my
statistics understanding is very limited! (I’m
outed!)</p>
<p
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"
class="MsoNormal"> </p>
<p
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"
class="MsoNormal">We don’t actually have to
work to G14 in the UK (probably good because
my copy is a bit old). I finally realised I
didn’t know enough after I’d been (lazily)
using R2 in Excel on some monthly data. I
thought that R2 > 0.9 was generally ok…
yeah, it wasn’t.</p>
<p
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"
class="MsoNormal"> </p>
<p
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"
class="MsoNormal">So, are there any easy to
understand resources available?</p>
<p
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"
class="MsoNormal"> </p>
<p
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"
class="MsoNormal">I’ve been messing around
with the IMT as well. It’s been fun going back
to DOS <span style="font-family:"Segoe
UI Emoji",sans-serif">😊</span>. This
got me into daily methods, which leads to my
next question. Is there any reason why there
isn’t a daily calibration option specified in
G14?</p>
<p
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"
class="MsoNormal"> </p>
<p
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"
class="MsoNormal">Many thanks!</p>
<p
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"
class="MsoNormal"> </p>
<p
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"
class="MsoNormal">Chris Yates</p>
<p
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"
class="MsoNormal"> </p>
</div>
</blockquote>
<p class="MsoNormal"> </p>
</div>
<p class="MsoNormal">_______________________________________________<br>
Bldg-sim mailing list<br>
<a
href="http://lists.onebuilding.org/listinfo.cgi/bldg-sim-onebuilding.org"
rel="noreferrer nofollow noopener" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">http://lists.onebuilding.org/listinfo.cgi/bldg-sim-onebuilding.org</a><br>
To unsubscribe from this mailing list send a blank
message to <a
href="mailto:BLDG-SIM-UNSUBSCRIBE@ONEBUILDING.ORG"
rel="noreferrer nofollow noopener" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">BLDG-SIM-UNSUBSCRIBE@ONEBUILDING.ORG</a></p>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
Bldg-sim mailing list
<a class="moz-txt-link-freetext" href="http://lists.onebuilding.org/listinfo.cgi/bldg-sim-onebuilding.org">http://lists.onebuilding.org/listinfo.cgi/bldg-sim-onebuilding.org</a>
To unsubscribe from this mailing list send a blank message to <a class="moz-txt-link-abbreviated" href="mailto:BLDG-SIM-UNSUBSCRIBE@ONEBUILDING.ORG">BLDG-SIM-UNSUBSCRIBE@ONEBUILDING.ORG</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="60">--
Jason Glazer, P.E., BEMP, GARD Analytics, 90.1 ECB chair
Admin for onebuilding.org building performance mailing lists
</pre>
</body>
</html>