Data mining techniques for analysing the weather patterns

Data mining techniques for analysing the weather patterns in Kumeu multi-sensor data Subana Shanmuganathan Geoinformatics Research Centre (GRC) Auckland University of Technology(AUT) 24 May 2013

overview • • • Background Literature Methodology Data Results Conclusions

background

live web display

literature 1. Sallis, P. and Hernandez. (2011). A precision agronomic state-space estimation method for event anticipation using dynamic multivariate continuous data. to be published in the Journal of Computer Science and Computational Mathematics JCSCM 2. Sallis, P. , Claster, W. , and Hernandez, S. (2011). A machine-learning algorithm for wind gust prediction. Journal of Computers and the Geosciences, Elsevier Press. 37 (2011) 1337 -1344. 3. Sallis, P. and Hernandez. (2011). An event-state depiction algorithm using CPA methods with continuous feed data. pp. 144148. 2011 Fifth Asia Modelling Symposium. ISBN 978 -0 -7695 -4414 -4/11 © 2011 IEEE DOI 10. 1109/AMS. 2011. 36. 4. Sallis, P. , Hernandez, S. and Shanmuganathan, S. (2011). Dynamic multivariate continuous data state-space estimation for agrometeorological event anticipation. In proceedings of [eds] Thatcher, S. , 2011 3 rd International Conference on Machine Learning and Computing (ICMLC 2011) Singapore, 26 -28 February 2011, ISBN 978 -1 -4244 -925 3 -4 /11 © 2011 IEEE Vol 1 pp 623 -627. 5. Sallis, P. , and Hernandez, S. (2011). Geospatial state space estimation using an Ensemble Kalman Filter, International Journal of Simulation, Systems, Science and Technology. Vol 11 (6) May 2011 pp: 1473 - 8031.

The methodology Chapman, Pete , et al. (2000) CRISP-DM 1. 0, Step-by-step data mining guide. SPSS Inc. CRISPMWP-1104, 2000. pp 73. Page 10 & page 12

The multi-sensor data 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. id Date. Time Node. Id Pressure_Rel Ind_Temp Ind_Hum Out_Temp Out_Hum Dewp Windc Winds Wind_Dir Gust Rain_Rate Act_rain Rain_Today Pressure_Abs Vinyard. Id Rain_Total Heat_Indx High_Gust Record high temperature 32. 8°C Record low temperature -8. 9°C Record high gust 172. 2 km/h Record high average 172. 2 km/h Record daily rain 82. 6 mm Record low wind chill -11. 5°C Record high barometer 1035. 6 h. Pa Record low barometer 977. 9 h. Pa From http: //www. binoscope. co. nz/Kumeu. htm 1. id 2. Date Time 3. Pressure_Rel 4. Out_Temp 5. Out_Hum 6. Dewp 7. Windc 8. Winds 9. Wind_Dir 10. Gust 11. class 12. Heat_Indx 13. E_code

20, "very" src="https://present5.com/presentation/7292af228af7c6571a7ffe9b58130fcf/image-8.jpg" alt="Data distribution gust classes <1, "no", <5, "low", <10, "med", <20, "high", >20, "very" /> Data distribution gust classes <1, "no", <5, "low", <10, "med", <20, "high", >20, "very high"

Data mining • C 5. 0 • C&RT (classification and regression trees-B) • CAHID (Chi-squared Automatic Interaction) Detector • ANN • Regression • PCA Sw: SPSS clementine

C 5. 0 for high gust

C 5. 0 for very high gust

C 5. 0 rule for high gust Rule 118 for high gust (8; 0. 75) if Pressure_Rel > 909 and Out_Temp > -9. 8 and Winds > 4. 9 and Dewp <= 18 and Winds <= 9. 9 and Heat_Indx <= 0 and Out_Temp > 2. 5 and Winds > 7. 3 and Dewp > 8. 3 and Out_Temp > 14. 9 and Out_Hum <= 90 and Pressure_Rel <= 1018. 9 and Winds <= 8. 8 and Wind_Dir <= 242 and Out_Hum > 45 and Winds > 8 and Pressure_Rel > 997. 7 and Out_Temp <= 24. 9 and Out_Hum > 69 and Dewp <= 16. 4 and Wind_Dir <= 135 and Out_Hum > 70 and Winds <= 8. 3 and Wind_Dir <= 67 and Wind_Dir <= 22 and Out_Hum <= 72 and Pressure_Rel > 1003. 1 and Pressure_Rel <= 1010. 5 then high gust

C&RT gust prediction

C&RT rules for Gust prediction

C&RT class: error & correct readings Wind speed Pressure relative & wind sp Wind direction & wind sp Outdoor temp wind dir & wind sp Wind sp

CHAID

CHAID wind speed <=0

Wind speed >13. 7

ANN predict gust class

ANN

PCA

Conclusions • Different primary predictors – – C 5. 0=>pressure relative C&RT => wind speed CHAID => wind speed Regression test model => wine speed, pressure relative, outdoor humidity, wind direction, wind chill, outdoor temperature, dew point – PCA=> pressure relative • outdoor temperature, outdoor humidity, dew point, wind chill, w speed, w direction • Future work – Deploy online – Test other location data

acknowledgements • Sara, Akbar & Philip – for access to data