[Bldg-sim] Updated matrix of simulation tool timings - followup

[Jon Hand]

In response to Justin's question (below) about whether I noticed differences
in predictions across the computer and operating system matrix.

Looking first at ESP-r we carry out a regime of testing before
we release which involves running through ~300 models/scenarios
to check the impact of proposed code or model changes. These
test runs are done on different computers. 

Factors such as compiler switches tend to introduce noise in the order
of a Watt/hr or a fraction of a Watt/hr for a zone or surface energy
balance. There are often rounding errors between different compiler 
versions and different operating systems. Across the matrix I
will be using the GCC/G++/GFORTRAN compilers but often there
are differences in the compiler version. Again these tend to result
only in a bit of noise in the predictions rather than the kinds of
differences that might lead users to made different design decisions.

Exceptions are models that have some stochastic in nature
or require iteration - CFD runs, mass flow predictions 
and the like occasionally diverge because, for example a control
switches a time step earlier or later. I would need to look more
closely at the outputs to provide a more definitive view.

In the case of EnergyPlus I was primarily looking at the run times
rather than checking for differences in the predictions. I just 
randomly looked at a few files for the medium complexity model
between a legacy laptop and a current Dell desktop (3-4 chip
generations apart) and find the *.csv (~48MB) and *.mtd files 
are identical the *.eso (~60MB) is identical (except for date stamp)
and *.bnd *.dxf *eio *.end *.err *.rvaudit (except for the time taken) 
are essentially the same for the CFT solver and for the Conduction
Finite Difference solver.  
Between the ARM and Intel there seem to be some differences but 
I need to rerun the ARM to generate
*.csv and *.eso files. There might have been a point difference
in the EPP version. Also the Intel implementations sometime use
the downloaded version rather than a local compile while I must
always build the ARM executables from scratch.

Regards Jon Hand

________________________________________
From: Justin Spencer [jspencer17 at gmail.com]
Sent: 01 August 2019 19:00
To: Jon Hand
Cc: bldg-sim at lists.onebuilding.org
Subject: Re: [Bldg-sim] Updated matrix of simulation tool timings

Thanks for sharing this, Jon. I'm curious if you observed any differences in results with the different hardware you tried. This is not my area of expertise, exactly, but I'm curious.

Years ago, when I was in graduate school, I observed getting significantly different results with the same model and same software version running EnergyPlus on my personal computer vs the computers at my office and we pinned the cause down to the way some of the fundamental calculation engines worked deep in the hardware (one had an AMD processor vs the other with Intel). It was a bigger deal when EnergyPlus had a harder time solving the plant loop.

It would make me happy to hear you say there was nothing of the sort observed.

On Thu, Aug 1, 2019 at 1:25 AM Jon Hand via Bldg-sim <bldg-sim at lists.onebuilding.org<mailto:bldg-sim at lists.onebuilding.org>> wrote:
After about a year I have revisited the simulation tool timings matrix. This
matrix of tests has revised the list of hardware to include a Dell gaming laptop,
a Chromebook, an 8-core Odroid ARM computer as well as an ARM based Laptop.
We also explore the option of Windows Subsystem for Linux (WSL) deployments
of ESP-r/EnergyPlus/Radiance as well as recent improvements in run-times
for simulation tool use in Virtual computer environments.

Didn't know you could run simulation tools on Chromebook? And before Microsoft
roles out their 2nd generation of WSL find out how their initial offering
compares to 'running native'.

Here is link to timings web page:
http://contrasting.no-ip.org/ESP-r_tour/timings.html

The link is to the full-matrix version.  These will feed into the tool-specific web
pages on the Contrasting web site (http://contrasting.no-ip.org/Contrast/Index.html)
in the near future.

Regards, Jon Hand, Glasgow Scotland


_______________________________________________
Bldg-sim mailing list
http://lists.onebuilding.org/listinfo.cgi/bldg-sim-onebuilding.org
To unsubscribe from this mailing list send  a blank message to BLDG-SIM-UNSUBSCRIBE at ONEBUILDING.ORG<mailto:BLDG-SIM-UNSUBSCRIBE at ONEBUILDING.ORG>