[Bldg-sim] Sometimes things are easier than they seem re: merging raw data from two weather stations

Jones, Christopher Christopher.r.Jones at wspgroup.com
Fri Sep 4 14:22:53 PDT 2015 

Joe’s the greatest!



[cid:image003.png at 01D09C46.E75BA0D0]
Christopher Jones, P.Eng.
Senior Engineer

WSP Canada Inc.
2300 Yonge Street, Suite 2300
Toronto, ON M4P 1E4
T +1 416-644-4226
F +1 416-487-9766
C +1 416-697-0065

www.wspgroup.com<http://www.wspgroup.com/>

From: Bldg-sim [mailto:bldg-sim-bounces at lists.onebuilding.org] On Behalf Of Jim Dirkes
Sent: Friday, September 04, 2015 4:31 PM
To: Joe Huang
Cc: BLDG-SIM
Subject: Re: [Bldg-sim] Sometimes things are easier than they seem re: merging raw data from two weather stations

Joe,
You're solving problems that most of the world has never even thought about.  Good work.

On Fri, Sep 4, 2015 at 4:24 PM, Joe Huang <yjhuang at whiteboxtechnologies.com<mailto:yjhuang at whiteboxtechnologies.com>> wrote:
I've just had a mini "Eureka" moment that I wanted to share.

As I've been processing more weather data from the ISD, I've been countering more cases for smaller locations (mostly in the US) where there are two weather stations where the WMO numbers are the same for the first part but not for the smaller part. For example, for Socorro New Mexico there are two files:

    2014  NM_SOCORRO-MUNI-AP             723620 93040
    2014  NM_SOCORRO-MUNI                   723620 99999

I generally interpret these cases as where a station may be switching their instrumentation.  However, in this case the two stations seem to be reporting at alternating times (table below shows the number of observations by month for the 4 key climate parameters that I need):

NM_SOCORRO-MUNI-AP            723620         34.067 -106.900 1485 4  -7.0 NAM BSk
2014 dbt   464  302  151  538  697  155    0*   0* 511  710  244    0*   419
2014 dpt   462  297  148  535  696  154    0*   0* 505  710  244    0*   416
2014 cld   451  245  108* 477  646  153    0*   0* 413  652  217    0*   373
2014 wsp   464  302  151  538  697  155    0*   0* 510  710  244    0*   419
2014 all  1841 1146  558 2088 2736  617    0    0 1939 2782  949    0

NM_SOCORRO-MUNI               723620         34.022 -106.903 1485 4  -7.0 NAM BSk
2014 dbt   259  389  610  161   41* 549  689  698  224    0* 444  684    431
2014 dpt   256  385  607  159   38* 545  687  698  224    0* 443  684    429
2014 cld   259  312  550  144   41* 549  598  583  182    0* 435  625    388
2014 wsp   259  389  610  161   41* 549  689  698  224    0* 444  683    431
2014 all  1033 1475 2377  625  161 2192 2663 2677  854    0 1766 2676

I have software to merge data from two files but only at defined switchover dates, so this situation seemed at first quite hard to tackle.
Then, I thought, why not just take concatenate the two raw data files, and then sort the combined file to get the data in the right
chronological order?  That took me just a few seconds, and voila' ,  out comes a full weather file with quite high data completeness:

NM_SOCORRO-MUNI-AP2           723620         34.067 -106.900 1485 4  -7.0 NAM BSk
2014 dbt   713  643  714  690  710  690  689  698  688  710  687  691    693
2014 dpt   713  643  714  690  710  690  687  698  688  710  687  690    693
2014 cld   700  531  635  613  671  688  598  583  565  652  651  625    626
2014 wsp   713  643  714  690  710  690  689  698  687  710  687  690    693
2014 all  2839 2460 2777 2683 2801 2758 2663 2677 2628 2782 2712 2696

A quick look at the plots also showed a weather file of seemingly good quality:
[cid:image002.png at 01D0E736.549733F0]
[cid:image003.png at 01D0E736.549733F0]

Problem solved!

Joe


--

Joe Huang

White Box Technologies, Inc.

346 Rheem Blvd., Suite 205A

Moraga CA 94556

yjhuang at whiteboxtechnologies.com<mailto:yjhuang at whiteboxtechnologies.com>

http://weather.whiteboxtechnologies.com for simulation-ready weather data

(o) (925)388-0265<tel:%28925%29388-0265>

(c) (510)928-2683<tel:%28510%29928-2683>

"building energy simulations at your fingertips"

_______________________________________________
Bldg-sim mailing list
http://lists.onebuilding.org/listinfo.cgi/bldg-sim-onebuilding.org
To unsubscribe from this mailing list send  a blank message to BLDG-SIM-UNSUBSCRIBE at ONEBUILDING.ORG<mailto:BLDG-SIM-UNSUBSCRIBE at ONEBUILDING.ORG>



--
James V Dirkes II, PE, BEMP, LEED AP
CEO/President
The Building Performance Team Inc.
1631 Acacia Dr, GR, Mi 49504

Direct: 616.450.8653
jim at buildingperformanceteam.com<mailto:jim at buildingperformanceteam.com>

Website <http://buildingperformanceteamcom> l  LinkedIn<https://www.linkedin.com/pub/jim-dirkes/7/444/413>

The body must be given a little less than it needs. Otherwise it will turn traitor.

________________________________

You are receiving this communication because you are listed as a current WSP contact. Should you have any questions regarding WSP’s electronic communications policy, please consult our Anti-Spam Commitment www.wspgroup.com/casl. For any concern or if you believe you should not be receiving this message, please forward this message to us at caslcompliance at wspgroup.com<mailto:caslcompliance at wspgroup.com> so that we can promptly address your request. This message is intended only for the use of the individual or entity to which it is addressed, and may contain information which is privileged, confidential, proprietary or exempt from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to the intended recipient, you are strictly prohibited from disclosing, distributing, copying or in any way using this message. If you have received this communication in error, please notify the sender, and destroy and delete any copies you may have received.

WSP provides professional land surveying services through the following entities: WSP Surveys (AB) Limited Partnership and WSP Surveys (BC) Limited Partnership
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.onebuilding.org/pipermail/bldg-sim-onebuilding.org/attachments/20150904/8daf38d5/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 6574 bytes
Desc: image001.png
URL: <http://lists.onebuilding.org/pipermail/bldg-sim-onebuilding.org/attachments/20150904/8daf38d5/attachment-0009.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 12123 bytes
Desc: image002.png
URL: <http://lists.onebuilding.org/pipermail/bldg-sim-onebuilding.org/attachments/20150904/8daf38d5/attachment-0010.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 9910 bytes
Desc: image003.png
URL: <http://lists.onebuilding.org/pipermail/bldg-sim-onebuilding.org/attachments/20150904/8daf38d5/attachment-0011.png>

More information about the Bldg-sim mailing list