[Equest-users] Actual Building Energy Cost

Thu Jun 17 07:50:59 PDT 2010

Eric and everyone,

I think we're all in general agreement here - I've been tasked with
developing existing models to match historical consumptions to varying
deadbands of accuracy - but never with a prescriptive mandate.  The
targeted degree of calibration for any of my projects is both selected
by myself and varies, and for a variety of reasons - primarily (1) time
allowed to develop the model,  (2) the information available/obtainable,
(3) the perceived accuracy of that information.  

I don't disagree:  Tight levels of matched historical data may not
reflect a more sophisticated and accurate model, especially for
"predicitive" purposes, but until we get a time travel feature added to
the Wizards (come on dev team, get moving! ^_^) - what is the
alternative metric for modeling accuracy?

My statement that you quoted wasn't properly elaborated on my end, and I
apologize for writing in haste (end of the workday) - I don't mean to
suggest 1-2% is "good" and anything higher is "bad," or anything of that
nature.  More directly: I simply mean to state that setting a target of
monthly utility consumption accuracy (either for yourself as the modeler
or for another), is entirely reasonable if the modeler is given the time
and resources to continue learning and investigating the building until
such "historical accuracy" is obtained.  I personally, given no other
direction, consider +/-10% (for each utility and each modeled month) to
be a minimum before results are share-able.

I also want to emphasize: "Predictive modeling" as a concept is a
potential minefield, both for design teams and building owners if and
when it is (often) misunderstood.  I'm no professional orator/writer, so
I don't know if there is better existing vocabulary for this, but I
forcefully insist on keeping the distinction between "predictive" and
"historical" accuracy understood by all parties and at all stages
whenever I am interacting with others over modeling projects.

To respond to your hypothetical - If I am in a project and I find the
monthly consumptions are generally ~6% off, where a target of 2% has
been chosen by myself or others, my first reaction would not be to
arbitrarily tweak unknown assumptions to make the number "work."  When I
do this sort of project, I understand a myriad of assumptions have to be
made.  In my mind, some assumptions are more of a "leap" than others, so
I keep a running list of those larger assumptions in a notepad at my
desk for future reference.  Perhaps I never gathered information
regarding a specific roof retrofit's construction, or I haven't
investigated whether there are a separate set of setback conditions for
a given school season.  When the time comes to calibrate the model,
after ensuring everything is entered as I intended, I then proceed to
investigate further (ask questions, make another site visit if
necessary) as required to resolve the items in my list to the best of my
ability.  Nothing is altered purely for the purpose of "matching the
numbers," only in the name of improving the model's historical accuracy.

If a deadline should arrive someday where the targeted level of
calibration is not met (hasn't happened yet - but then I've set my own
targets...), I'll rest easy knowing I've developed the most accurate
model I can with the time and resources available to me.  I also be
ready to discuss at length the major assumptions made and actions taken
to address those unknowns.

Off the record, I've approached +/- 2% within each monthly utility
record consumption one (1) time, and I had to become intimately familiar
with the project over months to get there.  Much discussion with the
building maintenance crew, site visits, and poring over record documents
was required.  I fully agree this is not a level of calibration that
ought to be required except when an exceptional level of time and
resources are dedicated to investigating the building (at least for a
modeler with only a few years experience).  If you were to look at a
"predictive accuracy gained : time/money spent developing the model"
ratio, I'm sure for most projects you would find sharply diminishing
returns trying to approach such a deadband.  If there is a metric to
measure "predictive accuracy" for any model better than calibrating to
historical data, I'd love to learn about it, but to my knowledge it's
all we've got to work with.

That I'm writing such lengthy responses hopefully conveys - this is a
kinda tough topic to discuss!  I'm not trying to make any legally
binding statements here, so please don't read between the lines - if
anything I'm writing is clear as mud, I'll be happy to elaborate, time
permitting =).

Thanks,

~Nick

NICK CATON, E.I.T.

PROJECT ENGINEER

25501 west valley parkway

olathe ks 66061

direct 913 344.0036

fax 913 345.0617

Check out our new web-site @ www.smithboucher.com 

NICK CATON, E.I.T.

PROJECT ENGINEER

25501 west valley parkway

olathe ks 66061

direct 913 344.0036

fax 913 345.0617

Check out our new web-site @ www.smithboucher.com 

From: Eric O'Neill [mailto:elo at MichaelsEngineering.com] 
Sent: Thursday, June 17, 2010 9:32 AM
To: Li, Lan; Nick Caton; David Bastow; aazhari at jainconsultants.com;
equest-users at lists.onebuilding.org
Subject: RE: [Equest-users] Actual Building Energy Cost

Lan,

That's a good example. I'm always amazed at the number of outside air
dampers that are broken/disconnected, have broken actuators, or crummy
seals. A significant percentage of existing buildings (and new ones for
that matter!) have that problem. It would be great if we had the time
and budget to test all the systems to find problems like this, but then
we're doing retrocommissioning on top of energy modeling. And even then,
our data is only as good as the meters and the people placing them (is
your mixed air temperature logger - with an error of 2-3% - just a
single logger ziptied right in the middle of the airstream, potentially
missing stratification?).

Sorry, I'm done ranting J. Just some pent up frustration from being
asked to do things that really don't make sense to me. I get the
heebie-jeebies even thinking about calibrating a model to an existing
building, let alone to within 1-2%. I'm just hoping someone will tell me
that I'm being too academic and that these things work themselves out in
the real world. The DOE must have good reason to request calibration
within 1-2% other than a warm fuzzy feeling when they get results,
right?

Eric

From: Li, Lan [mailto:lli at sbmce.com] 
Sent: Thursday, June 17, 2010 8:53 AM
To: Eric O'Neill; Nick Caton; David Bastow; aazhari at jainconsultants.com;
equest-users at lists.onebuilding.org
Subject: RE: [Equest-users] Actual Building Energy Cost

In one of my learning course, the instructor mentioned that after they
used eQuest simulated an existing campus, they were not able to have the
simulation results to match utility bills close enough. It turned out
that the economizer damper was on all the time and the some meters were
set up wrong which leads the historical data was inaccurate.  

Lan 

Lan Li, PE

Mechanical Engineer

Scheeser Buckley Mayfield LLC                                     

1540 Corporate Woods Parkway

Uniontown, OH 44685

Phone: (330) 896-4664 ext. 123

Cell: (330) 904-6292

Fax: (330) 896-9180

lli at sbmce.com <mailto:lli at sbmce.com> 

www.sbmce.com <http://www.sbmce.com/> 

Columbus Branch Office (614) 448-1498

 Please consider the environment before printing this email.

This e-mail (including any attachments) is confidential and may be
legally privileged. If you are not an intended recipient or an
authorized representative of an intended recipient, you are prohibited
from using, copying or distributing the information in this e-mail or
its attachments. If you have received this e-mail in error, please
notify the sender immediately by return e-mail and delete all copies of
this message and any attachments. 

Thank you

________________________________

From: Eric O'Neill [mailto:elo at MichaelsEngineering.com] 
Sent: Thursday, June 17, 2010 8:11 AM
To: Nick Caton; David Bastow; aazhari at jainconsultants.com;
equest-users at lists.onebuilding.org
Subject: Re: [Equest-users] Actual Building Energy Cost

Nick,

I should make it clear that my initial questions weren't meant to drill
David. I'm genuinely interested if the DOE projects required an
extremely high level of technical accuracy on the input side. (Although
they were also somewhat of a reaction to those requirements.) We have
this discussion in our office every so often, and I'm usually somewhat
on my own, but I'm still not convinced.

Would you mind elaborating a little more about why you feel this to be
true:

"HOWEVER:  The concept of building a model for an existing building,
whose modeled energy consumption/costs fall within a 1-2% of historical
utility records, is entirely feasible - and a reasonable requirement if
the goal is to generate a model for predictive purposes." 

So let me ask it this way. Let's say you've got a model, and it's coming
in about 6% low (both heating and electric), but to the best of your
knowledge everything is set up properly. You've got to start making some
educated guesses to close the gap. Do you assume the boiler needs a
tuning (drop your efficiency a few percent) and the AHU coils need to be
cleaned (bump up your static)? Or do you bump up your infiltration
slightly (because it's probably not a very tight building). Or did some
maintenance guy came along and switched on both the boiler pump manually
on when he got a cold complaint, and no one returned them to auto? Or
maybe you have data from a few years with abnormally warm summers and
cold winters. I could go on practically ad nauseum. There's just so many
things that can be wrong with an existing building, as all of you
probably know.

Whatever you choose, that guess you make may interact (or lack an
interaction) with some measures you're predicting. So, predicatively,
what makes a 1-2% off model better than a 5-6% off model unless you
really spend the time and find out whether your inputs are accurate? 

Cheers,

Eric

From: Nick Caton [mailto:ncaton at smithboucher.com] 
Sent: Wednesday, June 16, 2010 5:43 PM
To: Eric O'Neill; David Bastow; aazhari at jainconsultants.com;
equest-users at lists.onebuilding.org
Subject: RE: [Equest-users] Actual Building Energy Cost

While I (and I am sure many others) have books to write regarding my
thoughts on the topic, I think I might suggest a simple step back:  The
concept of predictively modeling to within a few percentage points of
accuracy is ridiculous.  There are simply way to many things that could
not be known.  HOWEVER:  The concept of building a model for an existing
building, whose modeled energy consumption/costs fall within a 1-2% of
historical utility records, is entirely feasible - and a reasonable
requirement if the goal is to generate a model for predictive purposes.
How often such predictive models' accuracy is misinterpreted is
something I get depressed to think about.

I haven't done the DOE-modeling work that's being referenced with such
requirements, but I have done work for educational clients
(physics/building science departments) who wish to get these models set
up for ongoing study/tweaking purposes.  The exercise is challenging,
and as close as energy modeling gets to "fun," when you can rest at ease
knowing the client is fully on board with what the model is and isn't.  

It's fully rational to squirm and cringe when you have to make models
that you know will be mis-used and mis-understood, despite your best
efforts.  Fortunate is the practicing energy modeler who gets to work
for fully educated clients and design teams all the time =).

Rather than drill David, I think it would be safe to assume those DOE
requirements exist to calibrate a model to a given degree to whatever
historical records are available, NOT to mandate a level of accuracy for
predictive purposes.  If I'm wrong, I hope that work never crosses my
desk!

~Nick

NICK CATON, E.I.T.

PROJECT ENGINEER

25501 west valley parkway

olathe ks 66061

direct 913 344.0036

fax 913 345.0617

Check out our new web-site @ www.smithboucher.com 

From: equest-users-bounces at lists.onebuilding.org
[mailto:equest-users-bounces at lists.onebuilding.org] On Behalf Of Eric
O'Neill
Sent: Wednesday, June 16, 2010 4:52 PM
To: David Bastow; aazhari at jainconsultants.com;
equest-users at lists.onebuilding.org
Subject: Re: [Equest-users] Actual Building Energy Cost

How much effort was put into minimizing the error of your inputs to
justify that level of accuracy in your output? For example, did you
convert real weather data for the two years and use an average of the
simulations? Were you modeling small buildings so you could get a fairly
reasonable infiltration rate empirically? Were occupancy schedules
trended, and for how long? Were all the systems and controls working
correctly, with all sensors calibrated regularly? 

1 to 2% seems to me to be fairly unreasonable. Unless you do an amazing
job verifying your inputs, in my opinion that level of precision doesn't
get you a better model. If you set everything up as best you can and it
comes in there, great! But a correctly set up model can be off by over
2% because of a couple "El Nino" years, a facilities guy locking a
humidity high limit to 50% for a summer, or any number of operational
factors. On the other side of it, we've tried to match a simple DOE2.2
model to a DOE2.1e model with limited success (I don't think we got
within 3%, although we didn't spend too much time with it). Who knows
what the difference would be with E+.

I know I always squirm when I'm asked to do existing building models,
and it may be irrational. Heck, if the DOE is asking for 1-2%, I
probably am being irrational. But it seems to me that the error from
assumptions could easily swing a model 1-2% (what would a 30% error on
your infiltration do to an otherwise correct building?). What are other
people's thoughts? 

Eric

From: equest-users-bounces at lists.onebuilding.org
[mailto:equest-users-bounces at lists.onebuilding.org] On Behalf Of David
Bastow
Sent: Wednesday, June 16, 2010 3:48 PM
To: aazhari at jainconsultants.com; equest-users at lists.onebuilding.org
Subject: Re: [Equest-users] Actual Building Energy Cost

On all the DOE modeling projects that we have done, of an existing
building, they have required the model to be within 1 to 2% or less of
the actual energy usage, based on an average two year history.  Is that
what you are talking about?

David A. Bastow 

McClure Engineering, Inc.  

________________________________

From: equest-users-bounces at lists.onebuilding.org
[mailto:equest-users-bounces at lists.onebuilding.org] On Behalf Of Ahmed
Azhari
Sent: Wednesday, June 16, 2010 12:10 PM
To: equest-users at lists.onebuilding.org
Subject: [Equest-users] Actual Building Energy Cost

Hi all, 

Does anyone have a ballpark percentage of the actual annual energy cost
versus the modeled annual energy cost for a building?  

Thanks,

________________________________________________________________________
__

Ahmed Azhari, B.Eng., LEED(r) AP

Energy Analyst

Jain Sustainability Consultants Inc.

2260 Argentia Road, 2nd Floor

Mississauga, Ontario L5N 6H7, CANADA

Tel:  (905) 542 7211 Ext 234

Fax: (905) 542 7622

Email: aazhari at jainconsultants.com <mailto:aazhari at jainconsultants.com> 

Web: www.jainassoc.com <http://www.jainassoc.com/> 

NOTICE OF CONFIDENTIALITY
This communication, including any attachments, is intended only for the
use of the addressees and is confidential. 

If you received this communication in error, please permanently delete
the entire communication from any computer or other storage medium.

 Please consider your environmental responsibility before printing this
e-mail & any documents

__________ Information from ESET Smart Security, version of virus
signature database 5202 (20100616) __________

The message was checked by ESET Smart Security.

http://www.eset.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.onebuilding.org/pipermail/equest-users-onebuilding.org/attachments/20100617/bb1e5b5b/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 1459 bytes
Desc: image001.jpg
URL: <http://lists.onebuilding.org/pipermail/equest-users-onebuilding.org/attachments/20100617/bb1e5b5b/attachment.jpeg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 1257 bytes
Desc: image002.gif
URL: <http://lists.onebuilding.org/pipermail/equest-users-onebuilding.org/attachments/20100617/bb1e5b5b/attachment.gif>