[Bldg-sim] Energy model calibration - normalizing the utility bills to month start-end

Jim Dirkes jim at buildingperformanceteam.com
Thu Jun 25 01:48:57 PDT 2015


Excellent insights in this thread!
Our "two cents":

   - We measure key variables such as larger motor kW and outdoor airflow.
   The idea is to eliminate uncertainty for energy aspects that we know are
   "big".  Because it's easy to obtain, we use the actual weather data.  It's
   not always a big impact, but eliminates needless uncertainty.
   - We don't trust people's memory about almost anything; it's amazing how
   different the story can be when told by two different people, both of whom
   should be knowledgeable.
   - We also don't trust sensor calibration.  Not all sensors matter as
   much, however; discharge sensors on reheat systems, for example, matter
   more than room temp sensors.
   - More data is better.  Smart meters are very helpful, as is trend data
   if the client has taken time to set them up. Not many do :(
   - We have not yet tinkered with sensitivity analysis for uncertain
   variables such as boiler efficiency or infiltration, but we do run GenOpt
   for the best R-squared fit to each (exact) billing period and review /
   revise the output for reasonableness.
   - On the one hand, it bothers me that calibrated models are "grossly
   incorrect" in Maria's experience.  My bias is that those models may have
   too little measured data.  On the other hand, this is a brave new world and
   we probably all have a lot to learn.


On Tue, Jun 23, 2015 at 4:51 PM, Justin Spencer <jspencer17 at gmail.com>
wrote:

> I'll second that this has turned into a really good thread. My personal
> rule of thumb in doing calibration is, "don't mess with things that relate
> directly to an ECM." If you have a retrofit ECM going on, you should have
> detailed data on the pre-installation conditions associated with that ECM.
> If that isn't documented, your model is never going to provide you with
> accurate savings results. It pays to think about these things at the
> fundamental level. And also apply a reasonableness range to those values.
> What am I most unsure about? What's the possible range of this value?
> Personally, I twist my dials all in one direction and then use setpoint as
> my fine calibration factor at the end. We need to remember that we're using
> models as tools to help guide decision-making. When we make decisions in
> calibration, we should think about how it will impact the decision being
> made -- does it impose a bias (like the 35% assumed pre-retrofit boiler
> efficiency for a boiler retrofit project)? These models are just tools for
> extrapolating from known energy consumption data to unknown energy
> consumption data.
>
> An earlier post commented that hourly data might make calibration
> easier... It doesn't make it easier in the sense of less work, but it does
> make your model a lot better if you do it right. Calibrating models to real
> hourly consumption data tells you so much about what is likely wrong with
> your model and so much about how your real building performs. If you have
> hourly end use data, or close, then you are really in business. At the
> aggregate level, this kind of data can tell you that you didn't really know
> much to start with. For one project for Con Edison, we had access to hourly
> whole-premise consumption data for several thousand NYC buildings by
> building type. We were able to use this data to uncover that our models
> vastly underestimated consumption at nights and on weekends. We didn't know
> why, but we altered schedules accordingly.
>
> As for the monthly billing data question at the beginnning of this thread,
> if you're dealing with a building and climate where monthly variability in
> occupancy and weather are driving changes in energy consumption in smooth,
> continuous fashion, you can make some other approximations. We've used a
> simplified slope method, where you calculate the slope in usage between the
> two adjacent billing months, i.e. if period 1 is 1000 kWh/day, period 2 is
> 1100 kWh/day, and period 3 is 1200 kWh/day, and all are 30 days long, you
> wind up with a slope of 3.33 kWh/day. We then assign daily kWh for each
> month using this slope and then reslice the data to match our calendar
> months. This is important when you are calibrating an aggregate model,
> which is a common need for us when we're estimating program-level savings
> for an ECM. For example, we're using a building simulation model to
> extrapolate measured consumption at a large sample of sites to usage in a
> typical year.
>
> Alternatively, for individual buildings, we don't worry about it and just
> calibrate to the billing periods instead. We're not generally working in
> eQuest, but rather with the hourly output of whatever engine we're using.
>
> One thing I've been wondering about recently is how to avoid
> over-calibration. The econometrics/statty/mathy folks I occasionally
> consort with talk a lot about overfitting of regressions and the same
> problem applies here. Has anybody ever tried reserving part of their
> calibration data set to use as a test set? I've gotten some things that
> looked like really great calibrations in the past, but I'm wondering how
> folks have sought to prove whether they were getting things right or
> overfitting. When I teach junior staff about this sort of thing, I always
> include something about not getting too cute. I point them to the John Von
> Neumann quote:
>
>    - *With four parameters I can fit an elephant
>    <https://en.wikiquote.org/wiki/Elephant>, and with five I can make him
>    wiggle his trunk.*
>
>
> On Tue, Jun 23, 2015 at 1:04 PM, Maria Karpman <
> maria.karpman at karpmanconsulting.net> wrote:
>
>> Jim and all,
>>
>>
>>
>> We have incentive programs in NY (Multifamily Performance Program) and NJ
>> (Pay for Performance Program for C&I buildings, P4P) that require
>> developing calibrated models to estimate ECM savings. Both programs rely on
>> spreadsheet-based tools to facilitate model calibration, and require that
>> the proposed ECM package reduces overall energy consumption by at least
>> 15%. The programs have been around for 5+ years, and have hundreds of
>> participating projects. Part of the incentive is awarded based on the
>> projected (i.e. modeled) savings, and the rest (as much as 50% for P4P)
>> based on the actual achieved savings established using Whole Building
>> approach by comparing pre/post utility bills.
>>
>>
>>
>> My biggest take away from the involvement with these programs was that a
>> calibrated model that meets MBE,  CVRMSE, and uncertainty requirements of
>> Guideline 14 may produce grossly incorrect ECM savings. It is not feasible
>> to create a calibrated model that can be reliably used to project savings
>> from any conceivable ECM in a commercial non-research setting because,
>> aside from the modeling effort, it would require a lot of very detailed
>> field work. On the other hand, developing a calibrated simulation to
>> estimate savings from a particular set of measures considered for a given
>> project is a much more manageable task. So the bulk of the recent updates
>> to the technical requirements of our incentive programs were focused on
>> itemizing parameters that should be tweaked to achieve calibration
>> depending on the ECMs included in the project scope. For example, if
>> project involves boiler replacement, efficiency of existing boilers that
>> are being replaced must be measured. (We had a project which modeled
>> existing boiler as 35% efficient because that produced calibrated
>> simulation. Of course such model would very likely exaggerate savings from
>> installing a new boiler.)
>>
>>
>>
>> Do any of you know references that outline calibration techniques
>> depending on the ECMs being modeled, beyond the general advice included in
>> IPM&VP?
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Maria
>>
>>
>>
>> --
>>
>> *Maria Karpman *LEED AP, BEMP, CEM
>>
>> ________________
>>
>> Karpman Consulting
>>
>> www.karpmanconsulting.net
>>
>> Phone 860.430.1909
>>
>> 41C New London Turnpike
>>
>> Glastonbury, CT 06033
>>
>>
>>
>> *From:* Bldg-sim [mailto:bldg-sim-bounces at lists.onebuilding.org] *On
>> Behalf Of *Jim Dirkes
>> *Sent:* Tuesday, June 23, 2015 1:31 PM
>>
>> *To:* bldg-sim at lists.onebuilding.org
>> *Subject:* Re: [Bldg-sim] Energy model calibration - normalizing the
>> utility bills to month start-end
>>
>>
>>
>> I'm encouraged to see so many people addressing this topic because it
>> means you are modeling existing buildings; a lot of work is needed in this
>> arena.  Keep it up!
>>
>>
>>
>> We, as usual, have a spreadsheet solution.  In this case, the spreadsheet
>> is happy to use billing periods of any length, such as is normal for
>> day-of-reading variations, but also to combine "estimated" readings into a
>> period that has an actual reading at each end.
>>
>> It requires that you tell EnergyPlus to report hourly meter data for each
>> fuel (e.g., electricity and natural gas).  A macro totals that data into
>> the billing periods for your site, displays the predicted vs actual energy
>> and calculates an R-squared value for each fuel and the total. An example
>> is shown below.
>>
>>
>>
>> [image: Inline image 1]
>>
>>
>>
>> On Tue, Jun 23, 2015 at 12:02 PM, Maria Karpman <
>> maria.karpman at karpmanconsulting.net> wrote:
>>
>> Hello all,
>>
>>
>>
>> We usually do the following to calibrate model to monthly utility bills:
>>
>> 1)      Create or purchase weather file corresponding to pre-retrofit
>> period for which we have billing data. Lately we’ve been using
>> WeatherAnalytics files, which we found to be more cost effective than
>> creating our own (they charge $40 for an annual file).
>>
>> 2)      Run simulation using this weather file instead of TMY.
>>
>> 3)      Standard simulation reports (we typically use eQUEST) show usage
>> by calendar month (e.g. January, February, etc.) which is usually not
>> aligned with dates of utility bills, as noted in the question that started
>> this thread. As Brian mentioned in one of the earlier posts, this may be
>> circumvented by entering the actual meter read dates into eQUEST as shown
>> in the screenshot below. This will align usages shown in eQUEST’s “E*”
>> reports such as ES-E with the actual utility bills.  The approach does not
>> allow entering more than one read date per month (e.g. we can’t capture
>> April 3 – 28 bill). For projects where this limitation is an issue we
>> generate hourly reports that show consumption by end use for each meter in
>> the project, and aggregate it into periods that are aligned with utility
>> bills.
>>
>>
>>
>> 4)      We then copy simulation outputs (either from ES-E or hourly
>> reports, depending on the method used) into a standard spreadsheet with
>> utility data. The spreadsheet is set up to plot side by side monthly
>> utility bills and simulated usage, and also calculates normalized mean bias
>> error (NMBE) and variance CV(RMSE).
>>
>> 5)      If we did not to where we want to be with NMBE and CV(RMSE) we
>> adjust and re-run the model, and re-paste results into the same
>> spreadsheet.
>>
>>
>>
>> In my experience regression analysis using weather as independent
>> variable (i.e. running model with TMY file and normalizing for difference
>> in weather) or relying on HDD to allocate usage to billing periods can be
>> very misleading, mainly because on many projects weather is not the main
>> driver of consumption. For example energy usage of a school during a given
>> time period depends much more on vacation schedule than outdoor dry bulb
>> temperatures.
>>
>>
>>
>> Thanks,
>>
>>
>>
>> --
>>
>> *Maria Karpman *LEED AP, BEMP, CEM
>>
>> ________________
>>
>> Karpman Consulting
>>
>> www.karpmanconsulting.net
>>
>> Phone 860.430.1909
>>
>> 41C New London Turnpike
>>
>> Glastonbury, CT 06033
>>
>>
>>
>> *From:* Bldg-sim [mailto:bldg-sim-bounces at lists.onebuilding.org] *On
>> Behalf Of *Jeff Haberl
>> *Sent:* Tuesday, June 23, 2015 10:16 AM
>> *To:* Joe Huang; bldg-sim at lists.onebuilding.org
>>
>>
>> *Subject:* Re: [Bldg-sim] Energy model calibration - normalizing the
>> utility bills to month start-end
>>
>>
>>
>> Hello Joe,
>>
>>
>>
>> Yes, you can count the degree days and regress against that to show a
>> correlation. However, one will get a better "fit" to the weather data if
>> you regress to the degree day that is calculated for the balance point
>> temperature of the building -- hence the inverse model toolkit or the
>> variable based degree day method.
>>
>>
>>
>> PRISM actually calculates the degree days to a variety of change points
>> and actually provides a table for each location that you use as a look up.
>> The IMT will actually perform a variable based degree day calculation that
>> agrees well with PRISM. IMT will also provide you with the average daily
>> temperature for the billing period.
>>
>>
>>
>> When using DOE-2 for actual billing periods, one will have to extract the
>> appropriate hourly variable, sum it to daily and then regroup to align with
>> the billing periods. Here's a chunk of code that will create a dummy plant,
>> display PV-A, PS-A, PS-E and BEPS, and extract the relevant hourly
>> variables to normalize the BEPS to the utility bills:
>>
>>
>>
>> INPUT PLANT ..
>>
>>
>>
>> PLANT-REPORT VERIFICATION = (PV-A)
>>
>> $ PV-A, EQUIPMENT SIZES
>>
>>
>>
>> SUMMARY = (PS-A,PS-E,BEPS)
>>
>>
>>
>> $ PS-A, PLANT ENERGY UTILIZATION SUMMARY
>>
>> $ PS-E, MONTHLY ENERGY END USE SUMMARY
>>
>> $ BEPS, BUILDING ENERGY PERFORMANCE SUMMARY
>>
>>
>>
>> HVAC=PLANT-ASSIGNMENT ..
>>
>>
>>
>> $ EQUIPMENT DESCRIPTION
>>
>> $ ELECTRIC DOMESTIC WATER HEATER
>>
>>
>>
>> BOIL-1 =PLANT-EQUIPMENT TYPE=ELEC-DHW-HEATER SIZE=-999 ..
>>
>>
>>
>> $ ELECTRIC HOT-WATER BOILER
>>
>>
>>
>> BOIL-2 =PLANT-EQUIPMENT TYPE=ELEC-HW-BOILER SIZE=-999 ..
>>
>>
>>
>> $ HERMETICALLY SEALED CENT CHILLER
>>
>>
>>
>> CHIL-1 =PLANT-EQUIPMENT TYPE=HERM-CENT-CHLR SIZE=-999 ..
>>
>>
>>
>> $ Graphics block for Data Processing ***
>>
>>
>>
>> RP-3 = SCHEDULE THRU DEC 31 (ALL) (1,24) (1) ..
>>
>>
>>
>> $ 8 = Total PLANT heating load (Btu/h)
>>
>> $ 9 = Total PLANT cooling load (Btu/h)
>>
>> $ 10 = Total PLANT electric load (Btu/h)
>>
>>
>>
>> BLOCK-3-1 = REPORT-BLOCK
>>
>> VARIABLE-TYPE = PLANT
>>
>> VARIABLE-LIST = (8,9,10) ..
>>
>> BLOCK-3-2 = REPORT-BLOCK
>>
>> VARIABLE-TYPE = GLOBAL
>>
>> VARIABLE-LIST = (1) ..
>>
>> HR-3 = HOURLY-REPORT
>>
>> REPORT-SCHEDULE = RP-3
>>
>> REPORT-BLOCK = (BLOCK-3-1,BLOCK-3-2) ..
>>
>>
>>
>> END ..
>>
>>
>>
>> COMPUTE PLANT ..
>>
>>
>>
>> STOP ..
>>
>>
>>
>> 8=!  8=)  :=)  8=)  ;=)  8=)  8=(  8=)  8=()  8=)  8=|  8=)  :=')  8=) 8=?
>> Jeff S. Haberl, Ph.D.,P.E.inactive,FASHRAE,FIBPSA,......jhaberl at tamu.edu
>> <........jhaberl at tamu.edu>
>> Professor........................................................................Office
>> Ph: 979-845-6507
>> Department of Architecture............................................Lab
>> Ph:979-845-6065
>> Energy Systems Laboratory...........................................FAX:
>> 979-862-2457
>> Texas A&M
>> University...................................................77843-3581
>> College Station, Texas, USA, 77843.............................
>> http://esl.tamu.edu
>> 8=/  8=)  :=)  8=)  ;=)  8=)  8=()  8=)  :=)  8=)  8=!  8=)  8=? 8=) 8=0
>> ------------------------------
>>
>> *From:* Bldg-sim [bldg-sim-bounces at lists.onebuilding.org] on behalf of
>> Joe Huang [yjhuang at whiteboxtechnologies.com]
>> *Sent:* Monday, June 22, 2015 9:17 PM
>> *To:* bldg-sim at lists.onebuilding.org
>> *Subject:* Re: [Bldg-sim] Energy model calibration - normalizing the
>> utility bills to month start-end
>>
>> Maybe I'm missing something here, but why can't you just count up the
>> degree days for the utility period?
>> I hope you're not working with average or "typical year" degree days, but
>> the degree days from the same time period.
>>
>> I also recall that the old Princeton Scorekeeping Method (PRISM) back in
>> the 1980's allows the user to enter the degree days for that time period,
>> so it's not a new problem.
>>
>> Joe
>>
>> Joe Huang
>>
>> White Box Technologies, Inc.
>>
>> 346 Rheem Blvd., Suite 205A
>>
>> Moraga CA 94556
>>
>> yjhuang at whiteboxtechnologies.com
>>
>> http://weather.whiteboxtechnologies.com for simulation-ready weather data
>>
>> (o) (925)388-0265
>>
>> (c) (510)928-2683
>>
>> "building energy simulations at your fingertips"
>>
>> On 6/22/2015 6:09 AM, Jones, Christopher wrote:
>>
>> When calibrating an energy model to utility bills the utility bills often
>> don’t align with the month start and end.  I have reviewed a couple methods
>> to calendar normalize the utility bills but find them somewhat
>> unsatisfactory.
>>
>>
>>
>> For example the method I am looking at does the following:
>>
>> The April gas bill runs from March 25 – April 24.  The algorithm takes
>> the average number of m3 per day from that bill, applies it to the days in
>> April.  Then it takes the average number of days from the May bill which
>> runs from April 24 – May 25 and applies that average to the remaining days
>> in April.
>>
>>
>>
>> The issue is that the March-April period has much higher HDD than the
>> April-May period and the “normalized” gas usage is significantly lower than
>> the simulation data for April.
>>
>>
>>
>> I am wondering if there are any papers or other sources of information as
>> to how others approach this problem.
>>
>>
>>
>>
>>
>> [image: cid:image003.png at 01D09C46.E75BA0D0]
>>
>> *Christopher Jones,*P.Eng.
>> Senior Engineer
>>
>>
>>
>> *WSP Canada Inc.*
>>
>> 2300 Yonge Street, Suite 2300
>>
>> Toronto, ON M4P 1E4
>> T +1 416-644-4226
>>
>> F +1 416-487-9766
>>
>> C +1 416-697-0065
>>
>>
>>
>> www.wspgroup.com
>>
>>
>> ------------------------------
>>
>>
>> You are receiving this communication because you are listed as a current
>> WSP contact. Should you have any questions regarding WSP’s electronic
>> communications policy, please consult our Anti-Spam Commitment
>> www.wspgroup.com/casl
>> <https://teesmail.tees.tamus.edu/owa/UrlBlockedError.aspx>. For any
>> concern or if you believe you should not be receiving this message, please
>> forward this message to us at caslcompliance at wspgroup.com so that we can
>> promptly address your request. This message is intended only for the use of
>> the individual or entity to which it is addressed, and may contain
>> information which is privileged, confidential, proprietary or exempt from
>> disclosure under applicable law. If you are not the intended recipient or
>> the person responsible for delivering the message to the intended
>> recipient, you are strictly prohibited from disclosing, distributing,
>> copying or in any way using this message. If you have received this
>> communication in error, please notify the sender, and destroy and delete
>> any copies you may have received.
>>
>> WSP provides professional land surveying services through the following
>> entities: WSP Surveys (AB) Limited Partnership and WSP Surveys (BC) Limited
>> Partnership
>>
>>
>>
>> _______________________________________________
>>
>> Bldg-sim mailing list
>>
>> http://lists.onebuilding.org/listinfo.cgi/bldg-sim-onebuilding.org
>>
>> To unsubscribe from this mailing list send  a blank message to BLDG-SIM-UNSUBSCRIBE at ONEBUILDING.ORG
>>
>>
>> ------------------------------
>>
>> No virus found in this message.
>> Checked by AVG - www.avg.com
>> Version: 2014.0.4800 / Virus Database: 4365/10055 - Release Date: 06/19/15
>>
>>
>> _______________________________________________
>> Bldg-sim mailing list
>> http://lists.onebuilding.org/listinfo.cgi/bldg-sim-onebuilding.org
>> To unsubscribe from this mailing list send  a blank message to
>> BLDG-SIM-UNSUBSCRIBE at ONEBUILDING.ORG
>>
>>
>>
>>
>>
>> --
>>
>> James V Dirkes II, PE, BEMP, LEED AP
>> CEO/President
>> The Building Performance Team Inc.
>> 1631 Acacia Dr, GR, Mi 49504
>>
>> Direct: 616.450.8653
>> jim at buildingperformanceteam.com
>>
>> Website <http://buildingperformanceteamcom>l  LinkedIn
>> <https://www.linkedin.com/pub/jim-dirkes/7/444/413>
>>
>> Studies show that four out of every three people have a hard time with
>> math.
>> ------------------------------
>>
>> No virus found in this message.
>> Checked by AVG - www.avg.com
>> Version: 2014.0.4800 / Virus Database: 4365/10055 - Release Date: 06/19/15
>>
>> _______________________________________________
>> Bldg-sim mailing list
>> http://lists.onebuilding.org/listinfo.cgi/bldg-sim-onebuilding.org
>> To unsubscribe from this mailing list send  a blank message to
>> BLDG-SIM-UNSUBSCRIBE at ONEBUILDING.ORG
>>
>>
>


-- 

James V Dirkes II, PE, BEMP, LEED AP
CEO/President
The Building Performance Team Inc.
1631 Acacia Dr, GR, Mi 49504

Direct: 616.450.8653
jim at buildingperformanceteam.com

Website <http://buildingperformanceteamcom>l  LinkedIn
<https://www.linkedin.com/pub/jim-dirkes/7/444/413>

 Studies show that four out of every three people have a hard time with
math.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.onebuilding.org/pipermail/bldg-sim-onebuilding.org/attachments/20150625/37923b09/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.png
Type: image/png
Size: 58565 bytes
Desc: not available
URL: <http://lists.onebuilding.org/pipermail/bldg-sim-onebuilding.org/attachments/20150625/37923b09/attachment-0009.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.png
Type: image/png
Size: 6574 bytes
Desc: not available
URL: <http://lists.onebuilding.org/pipermail/bldg-sim-onebuilding.org/attachments/20150625/37923b09/attachment-0010.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.png
Type: image/png
Size: 139044 bytes
Desc: not available
URL: <http://lists.onebuilding.org/pipermail/bldg-sim-onebuilding.org/attachments/20150625/37923b09/attachment-0011.png>


More information about the Bldg-sim mailing list