[Bldg-sim] Sometimes things are easier than they seem re: merging raw data from two weather stations
Joe Huang
yjhuang at whiteboxtechnologies.com
Tue Sep 8 13:14:18 PDT 2015
Stephen,
I was aware of the difference in coordinates, although I have some doubts because the
center of town station is listed in the ISD-History.txt file as SOCORRO MUNICIPAL AP,
while the airport station is listed at SOCORRO MUNI. I'm more surprised, though, that the
two stations seem to alternate several times over the year in reporting their data. In
90+% of the other 50 or so stations with duplicate WMO#s, there was a single switch-over
date, mostly at the end of July.
I really don't know what's going on here, but I was amused that the ISD reporting format
(WMO# followed by time stamp) makes it trivial to merge multiple files.
Joe
Joe Huang
White Box Technologies, Inc.
346 Rheem Blvd., Suite 205A
Moraga CA 94556
yjhuang at whiteboxtechnologies.com
http://weather.whiteboxtechnologies.com for simulation-ready weather data
(o) (925)388-0265
(c) (510)928-2683
"building energy simulations at your fingertips"
On 9/8/2015 7:48 AM, Mayer, Stephen wrote:
>
> Joe,
>
> I believe the reason for the two files is that there are two weather stations.
>
> As you shared, the files have the following file name and location:
>
> File Name GPS Coordinates
>
> 2014 NM_SOCORRO-MUNI-AP 723620 93040 34.067 -106.900
>
> 2014 NM_SOCORRO-MUNI 723620 99999 34.022 -106.903
>
> Just putting them quickly in Google Maps (see image below), the one is the center of
> town and the other is the airport (about 3.5 miles apart).
>
> Therefore it would make sense that these are two separate stations.
>
> This being said, the two stations should have similar data and therefore combining them
> into one file shouldn’t affect the model that much.
>
> Best Regards,
>
> Stephen Mayer, EIT, LEED AP BD+C
>
> Mechanical Discipline
>
> ————————————————————————————————————
> *M+W U.S., Inc. *– A Company of the M+W Group
>
> 380 Stonebreak Road Extension, P.O. Box 2318, Malta, NY 12020, USA
>
> Phone +1 518-305-1614
>
> mailto:stephen.mayer at mwgroup.net, www.mwgroup.net <http://www.mwgroup.net/>
>
> ————————————————————————————————————
>
> P *Please consider the environment before printing this email *
>
> This electronic message transmission contains information from the firm of M+W Group and
> is confidential or privileged. The information is intended to be for the sole use of the
> individual or entity named herein. If you are not the intended recipient, be aware that
> any disclosure, copying, distribution or use of the contents of this information is
> prohibited. If you have received this electronic transmission in error, please notify us
> by telephone (972-535-7300) immediately and please then delete this electronic transmission.
>
> *From:*Jones, Christopher [mailto:Christopher.r.Jones at wspgroup.com]
> *Sent:* Friday, September 04, 2015 5:23 PM
> *To:* 'Jim Dirkes'; Joe Huang
> *Cc:* BLDG-SIM
> *Subject:* Re: [Bldg-sim] Sometimes things are easier than they seem re: merging raw
> data from two weather stations
>
> Joe’s the greatest!
>
> cid:image003.png at 01D09C46.E75BA0D0**
>
> *Christopher Jones,***P.Eng./
> /Senior Engineer**
>
> **
>
> *WSP Canada Inc.*
>
> 2300 Yonge Street, Suite 2300
>
> Toronto, ON M4P 1E4
> T +1 416-644-4226
>
> F +1 416-487-9766
>
> C +1 416-697-0065
>
> www.wspgroup.com <http://www.wspgroup.com/> //
>
> *From:*Bldg-sim [mailto:bldg-sim-bounces at lists.onebuilding.org] *On Behalf Of *Jim Dirkes
> *Sent:* Friday, September 04, 2015 4:31 PM
> *To:* Joe Huang
> *Cc:* BLDG-SIM
> *Subject:* Re: [Bldg-sim] Sometimes things are easier than they seem re: merging raw
> data from two weather stations
>
> Joe,
>
> You're solving problems that most of the world has never even thought about. Good work.
>
> On Fri, Sep 4, 2015 at 4:24 PM, Joe Huang <yjhuang at whiteboxtechnologies.com
> <mailto:yjhuang at whiteboxtechnologies.com>> wrote:
>
> I've just had a mini "Eureka" moment that I wanted to share.
>
> As I've been processing more weather data from the ISD, I've been countering more cases
> for smaller locations (mostly in the US) where there are two weather stations where the
> WMO numbers are the same for the first part but not for the smaller part. For example,
> for Socorro New Mexico there are two files:
>
> 2014 NM_SOCORRO-MUNI-AP 723620 93040
> 2014 NM_SOCORRO-MUNI 723620 99999
>
> I generally interpret these cases as where a station may be switching their
> instrumentation. However, in this case the two stations seem to be reporting at
> alternating times (table below shows the number of observations by month for the 4 key
> climate parameters that I need):
>
> NM_SOCORRO-MUNI-AP 723620 34.067 -106.900 1485 4 -7.0 NAM BSk
> 2014 dbt 464 302 151 538 697 155 0* 0* 511 710 244 0* 419
> 2014 dpt 462 297 148 535 696 154 0* 0* 505 710 244 0* 416
> 2014 cld 451 245 108* 477 646 153 0* 0* 413 652 217 0* 373
> 2014 wsp 464 302 151 538 697 155 0* 0* 510 710 244 0* 419
> 2014 all 1841 1146 558 2088 2736 617 0 0 1939 2782 949 0
>
> NM_SOCORRO-MUNI 723620 34.022 -106.903 1485 4 -7.0 NAM BSk
> 2014 dbt 259 389 610 161 41* 549 689 698 224 0* 444 684 431
> 2014 dpt 256 385 607 159 38* 545 687 698 224 0* 443 684 429
> 2014 cld 259 312 550 144 41* 549 598 583 182 0* 435 625 388
> 2014 wsp 259 389 610 161 41* 549 689 698 224 0* 444 683 431
> 2014 all 1033 1475 2377 625 161 2192 2663 2677 854 0 1766 2676
>
> I have software to merge data from two files but only at defined switchover dates, so
> this situation seemed at first quite hard to tackle.
> Then, I thought, why not just take concatenate the two raw data files, and then sort the
> combined file to get the data in the right
> chronological order? That took me just a few seconds, and voila' , out comes a full
> weather file with quite high data completeness:
>
> NM_SOCORRO-MUNI-AP2 723620 34.067 -106.900 1485 4 -7.0 NAM BSk
> 2014 dbt 713 643 714 690 710 690 689 698 688 710 687 691 693
> 2014 dpt 713 643 714 690 710 690 687 698 688 710 687 690 693
> 2014 cld 700 531 635 613 671 688 598 583 565 652 651 625 626
> 2014 wsp 713 643 714 690 710 690 689 698 687 710 687 690 693
> 2014 all 2839 2460 2777 2683 2801 2758 2663 2677 2628 2782 2712 2696
>
> A quick look at the plots also showed a weather file of seemingly good quality:
>
>
>
> Problem solved!
>
> Joe
>
> --
> Joe Huang
> White Box Technologies, Inc.
> 346 Rheem Blvd., Suite 205A
> Moraga CA 94556
> yjhuang at whiteboxtechnologies.com <mailto:yjhuang at whiteboxtechnologies.com>
> http://weather.whiteboxtechnologies.com for simulation-ready weather data
> (o) (925)388-0265 <tel:%28925%29388-0265>
> (c) (510)928-2683 <tel:%28510%29928-2683>
> "building energy simulations at your fingertips"
>
>
> _______________________________________________
> Bldg-sim mailing list
> http://lists.onebuilding.org/listinfo.cgi/bldg-sim-onebuilding.org
> To unsubscribe from this mailing list send a blank message to
> BLDG-SIM-UNSUBSCRIBE at ONEBUILDING.ORG <mailto:BLDG-SIM-UNSUBSCRIBE at ONEBUILDING.ORG>
>
>
>
> --
>
> James V Dirkes II, PE, BEMP, LEED AP
> CEO/President
> The Building Performance Team Inc.
> 1631 Acacia Dr, GR, Mi 49504
>
> Direct: 616.450.8653
> jim at buildingperformanceteam.com <mailto:jim at buildingperformanceteam.com>
>
> Website <http://buildingperformanceteamcom>l LinkedIn
> <https://www.linkedin.com/pub/jim-dirkes/7/444/413>
>
> The body must be given a little less than it needs. Otherwise it will turn traitor.
>
> ------------------------------------------------------------------------------------------
>
>
> You are receiving this communication because you are listed as a current WSP contact.
> Should you have any questions regarding WSP’s electronic communications policy, please
> consult our Anti-Spam Commitment www.wspgroup.com/casl. For any concern or if you
> believe you should not be receiving this message, please forward this message to us at
> caslcompliance at wspgroup.com <mailto:caslcompliance at wspgroup.com> so that we can promptly
> address your request. This message is intended only for the use of the individual or
> entity to which it is addressed, and may contain information which is privileged,
> confidential, proprietary or exempt from disclosure under applicable law. If you are not
> the intended recipient or the person responsible for delivering the message to the
> intended recipient, you are strictly prohibited from disclosing, distributing, copying
> or in any way using this message. If you have received this communication in error,
> please notify the sender, and destroy and delete any copies you may have received.
>
> WSP provides professional land surveying services through the following entities: WSP
> Surveys (AB) Limited Partnership and WSP Surveys (BC) Limited Partnership
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.onebuilding.org/pipermail/bldg-sim-onebuilding.org/attachments/20150908/dfe020cf/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 36401 bytes
Desc: not available
URL: <http://lists.onebuilding.org/pipermail/bldg-sim-onebuilding.org/attachments/20150908/dfe020cf/attachment-0001.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 6574 bytes
Desc: not available
URL: <http://lists.onebuilding.org/pipermail/bldg-sim-onebuilding.org/attachments/20150908/dfe020cf/attachment-0009.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 12123 bytes
Desc: not available
URL: <http://lists.onebuilding.org/pipermail/bldg-sim-onebuilding.org/attachments/20150908/dfe020cf/attachment-0010.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 9910 bytes
Desc: not available
URL: <http://lists.onebuilding.org/pipermail/bldg-sim-onebuilding.org/attachments/20150908/dfe020cf/attachment-0011.png>
More information about the Bldg-sim
mailing list