Ecogrid update and future plan Collaboratory for Ecological

Ecogrid update and future plan: Collaboratory for Ecological from Local, Regional to Global Research in Taiwan Fang-Pang Lin 2005 NSC-NSF Annual Meeting National Center for High-performance Computing Taipei, 8, July, 2005 Fang-Pang Lin National Center for High-performance Computing National Applied Research Laboratory

New Frontier: Merging of Environmental Science and Information Technology Education & Capacity Building Sustained Collaboration • Build teams and trust Science Drivers • Develop human resources Previously Unobtainable Observations and Understanding Enabling Technology • Advance science • Focus development Persistent Infrastructure • Broaden impact Source: Tim Kratz, Peter Arzberger et al. at NSC-NSF 2005 annual meeting 2

“The World is Flat”* § Fundamental changes are underway in how: o Researchers interact with each other and with resources o Resources are distributed and accessed on the network o Science and education are conducted § This is leading to new o Research and educational paradigms o Discoveries and innovations o Networks of people • Multidisciplinary, multi-institutional, international § Ecology is a global science o Demanding global approaches o Providing future opportunities for creating previously unobtainable observations and understanding * Thomas Friedman 3

“Bandwidth” and “degree of connectivity” are the new measures of power … • Three distinguishing factors to harness power – Culture to exploit & share knowledge – Competitive setting that embraces change – Ability to partner Thomas Friedman, New York Times, P. 11 11/Apr/1998 4

The plan for the cyber. Infrastructure Grids Research Network Educational Network Innovation Education Commercial network Profits Source: Whey Fone Tsai, 2003 5

6

7

Sensor networks allow high frequency observations over broad spatial extents 100 km 10 km Spatial extent 1 km 100 m Existing Sensor Networks 10 m 1 m 10 cm random selection from Ecology 2003 Annual Monthly Weekly Daily Hourly Min. Sec. Frequency of measurement Source: John Porter et al. , Bioscience, July, 2005 8

KING sites The Vision: • Nerve ending of Taiwan in terms of Network (Last-Mile for TWAREN) • Focus on National Ecological Observation for supporting sustainable development of the island. The Plan： • 2003: Fushan, Nan-Jen-Shan, Kenting, Tatachia, Guan-Dai-Shi • 2004: Yuan-yan Lake, Remote islands • 2005: Coastal areas and Agricultural areas TERN sites The Technology： • Grid-based system design • Hierarchical Network structure. • Hybrid comm. Protocols: GSM, GPRS, WLAN, P 2 P, RF • Remote data communication (sensor-based) • Remote control & Grid-based automation (sensors & robots) 9

10

Date Time COND MSG FROM 05/07/08 01: 05: 30 S GVC "05/07/08 01: 05: 30", S GVC connected (CONNECT 9600/V 42). 05/07/08 01: 06: 21 S YYL_Met "05/07/08 01: 06: 21", S YYL_Met connected. 05/07/08 01: 06: 22 W YYL_Met "05/07/08 01: 06: 22", W YYL_Met E 08 error count: 2. 05/07/08 01: 06: 26 S YYL_Met "05/07/08 01: 06: 26", S YYL_Met Area 1 at location 16687, collected 504 FS words. 05/07/08 01: 06: 28 S YYL_Met "05/07/08 01: 06: 28", S YYL_Met disconnected. 05/07/08 01: 35: 29 S GVC "05/07/08 01: 35: 29", S GVC connected (CONNECT 9600/V 42). 05/07/08 01: 36: 26 S YYL_Buoy "05/07/08 01: 36: 26", S YYL_Buoy connected. 05/07/08 01: 36: 49 S YYL_Buoy "05/07/08 01: 36: 49", S YYL_Buoy Area 1 at location 58573, collected 1528 FS words. 05/07/08 01: 36: 50 F YYL_Buoy "05/07/08 01: 36: 50", F YYL_Buoy area 2 does not exist. All Network transmission status 11

12

An example of episodic events and Access can be difficult threshold dynamics during the most interesting times Yuan Yang Lake, Taiwan – August 2004 Used by NSF Director Feb 2005 Typhoon Photo by Peter Arzberger, October 2004 Part of a growing global lake observatory network - http: //lakemetabolism. org 13

Plan for Global Lake and Coral Reef Observation AS/NCHC 10 Lakes: New Zealand, Australia, Canada, US, Korea, UK, Finland, Taiwan 4 Coral Reefs sites: Breat Barrier Reef, World Bank Coral Reef efforts, US LTER, NOAA, TW LTER Meeting 7 -0, Mar 2005 at San Diego supported by NSF, Moore foundation and PRAGMA 14

15

16

Kenting Ecosite ~2. 4 G Wi. Fi Taipower Exibition Center Taipower Admin office Optical connector Field server 1 st camera 2 nd camera 3 rd camera 17

Architecture (I) 18

Architecture (II) 19

20

21

Basic Planning for IMS § Stakeholders: o Managers (Funding/Manpower/Strategy ) o Executive Managers (plan/coordination/auditing) o Scientists (e. g. Ecologists) o Information Managers 22

§ § § § § § § § § § 1. Information Manager (IM) 1. 1 IM Definition and Responsibility 1. 2 QA/QC 1. 3 Training courses 2. Portal 2. 1 Basics 2. 1. 1 Scenarios: Ecologists => login => upload => registration 2. 1. 3 Data description guidelines 1) Data/Information n composite attributes: numerics->place/time/objectives/people/equipment/owners…etc n cross references (Database layer) n Data Format First row Row should be Definition according to EML, Usage tables for cross references between the version of Chinese and that of English. 3. Sensors/equipment 4. Access Points 5. Network u Backbone issues: GSN and TANET 2/TWAREN u Last miles: ADSL/GPRS/RF u Local sensor network: Local Sensor Network n RF (~900 MHz) n RF (~150 MHz) <data modem: Chin> n IEEE 802. 11 (2. 4 GHz 54/11 Mbps, 100 meters depending on power) n IEEE 802. 16 (Wi. MAX, 75 Mbps, 30 miles) n Others u Hook-up Mechanism u Equipments n Pairing Structure: One to One, One to many n Details listed 6. Data format conversion Excel -> saved as ascii (csv=> mysqls, oracles) 23

Ecologist • Project registration • Define requirement • Device procurement Need to build a project registration interface yes Site IM same data collected already? NO No Site IM Yes Stream Data? Site IM • • IT executive manager • coordinate IT resources Create project db schema Register new schema to common data interface IT Specialist • Design infrastructure • Device procurement • Link new schema to common interface Site IM, IT Specialist, Ecologist • Test and deploy Site IM Operation Need to write operation manual Made by Hsiu-Mei, May 2005 24

GLEON Site Architecture (by Tony Fountain) Common Interface Layer Comm. & Ctl. Web Services Analytical Web Services Data-related Web Services Sensor Net Middleware Database Data Logger Models QA/QC Agent Sensor Data Acquisition . . . Data Management . . . Data Analysis and Modeling The components and the relationships are only tentative to start the discussion. 25

GLEON Network Architecture (by Tony Fountain) Common Interface Layer Comm. & Ctl. Web Services Sensor Net Middleware Portal Desktop & Mobile Users Analytical Web Services Data-related Web Services Database Data Logger QA/QC Agent Data Acquisition . . . Sensor Models Data Analysis and Modeling Data Management Lake Site 2 Common Interface Layer Comm. & Ctl. Web Services Common Interface Layer Analytical Web Services Data-related Web Services Comm. & Ctl. Web Services Sensor Net Middleware Database Data Logger QA/QC Agent Sensor Data Acquisition . . . Data Management Lake Site 1 Analytical Web Services Data-related Web Services Sensor Net Middleware Models Database Data Logger . . . Data Analysis and Modeling QA/QC Agent Sensor Data Acquisition Models . . . Data Management Data Analysis and Modeling Lake Site 3 Note that each site implements the Interface Layer, but the actual site architectures are independent and site-specific. 26

27

Fishery Observatory in Open Ocean (Fishery sites in plan: Pintong, Ilan and/or Penhu) Wireless network antenna Observational servers Fishery Platform/Observatory in Ocean (After Tim Acker et. Al. , Bio. Sonics Inc. ) Production management system Source: Chi-Yuan Lin, TFRIN 28

(2005. 03. 20, photo by 周秀美, NCHC) 29

Sensor Grid Portal (T. H. Chen and C. K. Sun, NCHC) § Provide Basics of a General Grid Platform and allow further development into a domain specific platform, such as IM for ecological research § Need a component which is aware of the status of resources in the grid environment, and be able to dispatch/redirect jobs to the most appropriate resources, § Integrate computing resource and storage resource using Grid middleware: o o § Computing middleware Storage middleware The ultimate goal is to have a complete grid platform and a user portal, so that NCHC could maximize the value of its hardware and knowledge center, enable end users to access various NCHC grid resources through the single entry, with user friendly Web interfaces. o Portal Server 30

Solutions § Part of Computing o o § Part of Storage o o o § Use Globus as computing middleware Provides both Web Services (utilizing Axis engine) and Java APIs as the interfaces for job submission and other functions. Utilize a RDB (My. SQL) to store Meta Scheduler data. Provides design information for the “Meta. Scheduler”. Use SRB as storage middleware SRB client and the Data mover component Authentication and authorization Part of Portal Server o o Life. Ray Portal server Portal user management and user profile integration with Globus and SRB. Integrate with GT 2 Co. G Integrate with SRB JARGON 31

NCHC Portal Architecture 32

Configuration § Hardware environment and Software function Service machine 2 Other Grid service machines SRB Server Service machine 1 Grid Portal Meta-scheduler Globus server Simple CA SRB Utilities GRIS Globus server SRB Utilities Certificate GIIS Other Grid service machine Globus server SRB Utilities Certificate GRIS TCP/IP Network 33

34