[Bldg-sim] Lies, darn lies, and statistics

Tue Mar 14 16:39:02 PDT 2023

Yep, not me. But I use it a lot.

On Tue, Mar 14, 2023, 4:19 PM Vikram Sami via Bldg-sim <
bldg-sim at lists.onebuilding.org> wrote:

> George Box coined that one I think
>
>
>
> *From:* Bldg-sim <bldg-sim-bounces at lists.onebuilding.org> *On Behalf Of *Jim
> Dirkes via Bldg-sim
> *Sent:* Tuesday, March 14, 2023 12:13 PM
> *To:* chris.malcolm.yates at gmail.com
> *Cc:* bldg-sim at onebuilding.org
> *Subject:* Re: [Bldg-sim] Lies, darn lies, and statistics
>
>
>
> I think Dru Crawley coined the phrase, "All models are wrong. Some are
> useful."
>
>
>
> Chris, you highlighted an assortment of variables which are omnipresent,
> inconsistent and uncontrollable - so what, exactly, does your client
> expect? Is it a realistic expectation? For example, are they going to nail
> you to the wall when, inevitably, you are "wrong" next year?
>
>
>
> I have not calibrated many models, but have been made more appreciative of
> all the uncontrollables by the ones I calibrated :(. All the statisticians
> know that there is always more than one solution which will result in a
> high R2 value or low CVRSME, so which is correct?
>
>
>
> Rather than a calibrated model, lately I've been encouraging clients to
> consider one of the FDD platforms on top of their Building Automation
> System. Spending time and money to evaluate whether things are working
> properly makes more sense to me - it's "real life" vs a prediction. (Can't
> forget to mention thoughtful and thorough commissioning here; that's
> essential.)
>
>
>
> ps, I love your thoughtful approach. You're setting a great example! One
> aspect of that is to reach out to the wider modeling community to gather
> input and feedback.
>
>
>
> Jim Dirkes  1631 Acacia Drive NW Grand Rapids, MI 49504 -  616 450 8653
>
> *Coffee Conversation:*
>
> The "individual" is an impossible concept, conceived by the Enlightenment
> philosophers. It makes no sense to the Christian. In marriages, and
> families, in associations and friendships and religious orders, we are not
> individuals, but a communion of persons.
>
>
>
> ------- Original Message -------
> On Tuesday, March 14th, 2023 at 2:10 PM, chris.malcolm.yates at gmail.com <
> chris.malcolm.yates at gmail.com> wrote:
>
>
> I think I need to qualify this: informed by G14, but definitely not
> compliant with it! There is some allowance for repairing or “healing” data,
> but when the data has a lot of holes or modes/ category variables then
> forget it. This is my case, but the client still wants some kind of
> representative simulation.
>
>
>
> Monthly models can be garbage. School holidays cut across months at
> different times, combined heat and power is popular which complicates gas
> usage especially when metering is limited… did the heat by-product of
> electricity generation go to the building, or was it rejected? They can
> work for heating in our temperate climate, but not for cooling.
>
>
>
> Nevertheless, we need some kind of representative simulation model. We
> can’t make any ECM qualifying claims but we can do something useful. This
> is where these methods can give you a lot of insight before you start
> modelling.
>
>
>
> I hadn’t tried the IMT previously. We tend to have limited our regression
> analysis to monthly “degree day” methods (your 2p model, I think). I
> plugged some project specific daily electricity data into the MVR example
> (multi variate regression) and it seemed to give decent CVRMSE (~2-3%) but
> low R2 (~0.7). However, it appeared to provide some insight on cooling
> usage (monthly 2p models are meaningless for this in the UK’s temperate
> climate).
>
>
>
> I also made some 2p monthly models of gas. I thought these were good until
> I compared successive years. *I guess this is were understanding a range
> of statistical indices is helpful.*
>
>
>
> Here’s the final rub, because the underlying data has so many
> inconsistencies that can only be made sense of with some regression models,
> it’s easier to “calibrate” the simulation model to the regression models
> than the original data. But I may use 2p monthly for gas, daily MVR for
> electric…
>
>
>
> So I need to ask if I have wondered completely off-piste with this!
>
>
>
> Chris
>
>
>
> *From:* David Eldridge <dancingdavide at hotmail.com>
> *Sent:* Tuesday, March 14, 2023 12:53 PM
> *To:* Jim Dirkes <jvdirkes2 at protonmail.com>
> *Cc:* chris.malcolm.yates at gmail.com; bldg-sim at onebuilding.org
> *Subject:* Re: [Bldg-sim] Lies, darn lies, and statistics
>
>
>
> Indirectly there probably isn’t a daily set of metrics in the Guideline
> since the simulation programs aren’t usually outputting daily results, but
> there’s no reason there couldn’t be one statistically.
>
>
>
> You could make one if you had only daily utility data and had to aggregate
> the simulation results to daily totals, there isn’t a published target
> metric but you could still show that you calculated one and why you think
> it was a good or bad result.
>
>
>
> DSE Mobile
>
>
>
> On Mar 14, 2023, at 6:10 AM, Jim Dirkes via Bldg-sim <
> bldg-sim at lists.onebuilding.org> wrote:
>
> 
>
> Dear Chris,
>
> Kudos for appreciating a gap in your understanding. (I'm in your camp)
>
>
>
> On the other hand, there are SO many variables in building operation that,
> short of a highly instrumented (and carefully calibrated) building for
> everything from lights to people to plug loads to HVAC - calibration is a
> fiction (and I'm confident that no such building exists). Daily calibration
> is a complete fiction, perhaps even a deception. On top of that, a
> "calibrated" model is just a moment in time; everything going forward is
> guaranteed to be different than during the calibration time period.
>
>
>
> I think of "calibration" as more like a sensitivity analysis - determine
> which variables matter more and which matter less. GenOpt works nicely for
> that purpose https://github.com/lbl-srg/GenOpt
> <https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Flbl-srg%2FGenOpt&data=05%7C01%7Cvikram%40olsonkundig.com%7C23d28a6b405f47b5562808db24c09278%7Cd5b3df51a48046ba8f92ef77dd2c7d72%7C0%7C0%7C638144181738291281%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dqFeAoOGreAjFSPjUPx7jYjnk6VDzoAgDtyGVo0z4DQ%3D&reserved=0>
>
>
>
> Jim Dirkes  1631 Acacia Drive NW Grand Rapids, MI 49504 -  616 450 8653
>
> *Coffee Conversation:*
>
> The "individual" is an impossible concept, conceived by the Enlightenment
> philosophers. It makes no sense to the Christian. In marriages, and
> families, in associations and friendships and religious orders, we are not
> individuals, but a communion of persons.
>
>
>
> ------- Original Message -------
> On Tuesday, March 14th, 2023 at 6:52 AM, Chris Yates via Bldg-sim <
> bldg-sim at lists.onebuilding.org> wrote:
>
> Hi All
>
>
>
> I do find ASHRAE Guideline 14 a little too hardcore for my basic
> understanding of statistics. I can plug any of the equations into Excel,
> but I’ve realised my statistics understanding is very limited! (I’m outed!)
>
>
>
> We don’t actually have to work to G14 in the UK (probably good because my
> copy is a bit old). I finally realised I didn’t know enough after I’d been
> (lazily) using R2 in Excel on some monthly data. I thought that R2 > 0.9
> was generally ok… yeah, it wasn’t.
>
>
>
> So, are there any easy to understand resources available?
>
>
>
> I’ve been messing around with the IMT as well. It’s been fun going back to
> DOS 😊. This got me into daily methods, which leads to my next question.
> Is there any reason why there isn’t a daily calibration option specified in
> G14?
>
>
>
> Many thanks!
>
>
>
> Chris Yates
>
>
>
>
>
> _______________________________________________
> Bldg-sim mailing list
> http://lists.onebuilding.org/listinfo.cgi/bldg-sim-onebuilding.org
> <https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.onebuilding.org%2Flistinfo.cgi%2Fbldg-sim-onebuilding.org&data=05%7C01%7Cvikram%40olsonkundig.com%7C23d28a6b405f47b5562808db24c09278%7Cd5b3df51a48046ba8f92ef77dd2c7d72%7C0%7C0%7C638144181738291281%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3GlZl01UE3n9OolRQ9aaVR9PpXGKfw%2BUnhMR2WKvBsw%3D&reserved=0>
> To unsubscribe from this mailing list send  a blank message to
> BLDG-SIM-UNSUBSCRIBE at ONEBUILDING.ORG
>
>
> _______________________________________________
> Bldg-sim mailing list
> http://lists.onebuilding.org/listinfo.cgi/bldg-sim-onebuilding.org
> To unsubscribe from this mailing list send  a blank message to
> BLDG-SIM-UNSUBSCRIBE at ONEBUILDING.ORG
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.onebuilding.org/pipermail/bldg-sim-onebuilding.org/attachments/20230314/031c5098/attachment.htm>