[Bldg-sim] Lies, darn lies, and statistics

[Julien Marrec]

Hello,

This looks like a very interesting conversation. Unfortunately we once again get into a problem where not everyone is able to follow because the information is behind a paywall.

It's not crystal clear to me what is "hardcore" about it. It is about using notmalized mean bias (NMBE) + CV(RMSE) instead of the more familiar Rsquared?

If my limited memory serves, guideline 14 and the IPMVP protocol define, and explain, them properly.

The first critical piece is to use actual weather data and not try to calibrate to a tmy3 file.

I agree that daily (or hourly/subhourly) calibration is probably too optimistic, though again if memory serves the protocol has relaxed thresholds for that case.

My two cents is that you should first Downsample the utility data to monthly, unless you have less than 12 months of data without any major changes (ECMs or unusual activities).

If you have no period of 12 months without major external changes, massage the data. What's needed there depends on whether it's an ECM or unusual activity. If an ECM probably try to use 12 months before it If you can. If unusual, do a regression on the regular data, extrapolate the rest.

As it's been said before, the key point is to document whatever assumptions you've taken clearly.

Even when you have perfect data for 36 months of "regular operation" (whatever does that mean), and calibrate your model correctly,  you'll still be off in the next 12 months if nothing happened.

Never forget that you're dealing with 5000+ independent variables and trying to calibrate to 12month of one, or two, dependent variable(s) (utility data). It's a **grossly** overfit model. And that's why a careful audit is needed with existing buildings, so you can try to 1) get the history for the calibration period and 2) tune every variable you can so only the unknowns remain (typically nail the loads etc, leaving stuff that's not easily measured and very influential like infiltration when a blower door test is not practical)


Disclaimer: writing on my phone so zero external resources available and probably a lot of typos.

Best,

Julien

Get BlueMail for Android



On Mar 14, 2023, 11:54, at 11:54, Chris Yates via Bldg-sim <bldg-sim at lists.onebuilding.org> wrote:
>Hi All
>
> 
>
>I do find ASHRAE Guideline 14 a little too hardcore for my basic
>understanding of statistics. I can plug any of the equations into
>Excel, but I’ve realised my statistics understanding is very limited!
>(I’m outed!)
>
> 
>
>We don’t actually have to work to G14 in the UK (probably good because
>my copy is a bit old). I finally realised I didn’t know enough after
>I’d been (lazily) using R2 in Excel on some monthly data. I thought
>that R2 > 0.9 was generally ok… yeah, it wasn’t.
>
> 
>
>So, are there any easy to understand resources available?
>
> 
>
>I’ve been messing around with the IMT as well. It’s been fun going back
>to DOS 😊. This got me into daily methods, which leads to my next
>question. Is there any reason why there isn’t a daily calibration
>option specified in G14?
>
> 
>
>Many thanks!
>
> 
>
>Chris Yates
>
> 
>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Bldg-sim mailing list
>http://lists.onebuilding.org/listinfo.cgi/bldg-sim-onebuilding.org
>To unsubscribe from this mailing list send  a blank message to
>BLDG-SIM-UNSUBSCRIBE at ONEBUILDING.ORG
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.onebuilding.org/pipermail/bldg-sim-onebuilding.org/attachments/20230314/f7f2f575/attachment.htm>