The future of Delft-FEWS Community Strategy Board Marc

The future of Delft-FEWS Community Strategy Board Marc van Dijk Medium term developments of Delft FEWS

>20 years of Delft-FEWS Applications Flood Forecasting Systems • Running models • Integration of (spatial) data • Ensembles etc Water Information Systems • Data validation • Data analysis • Long records of data Model Management platform • Off-line scenario analysis • Complex modelling systems

What is Delft- FEWS… • • • Tool for the hydrologist (forecaster) Community development Open model interface (FEWS-PI) Integration of large number of data sources (point, polygon, grid, . . ) Modular system …. Delft-FEWS: 2015 -2020

Some considerations… Why did Deltares develop Delft-FEWS? • Many nice projects, but no ‘world domination’ • Fast dissemination of new knowledge and developments • Access to organisations with ideas (and budget) • For further development of Delft-FEWS • Introduce new ideas and model systems from Deltares Further development is essential! In principle we never say no, but… • FEWS steering group, FEWS developers and Deltares project managers are leading. Watch out! • Grow → more complexity → holds up grow and innovation Delft-FEWS: 2015 -2020

Delft-FEWS worldwide

The Future of Delft-FEWS • • Make a plan for the medium range time period (3 -5 years) Focus on most important developments (not necessary to be complete) Who is involved in the discussion • Deltares FEWS core group • Deltares project managers, FEWS-developers, ICT-OWB • ‘External’ specialists • Cloud computing, Big Data, Apps, Remote Computing, . . • “Delft-FEWS Community Strategy Board” with important FEWS users. Delft-FEWS: 2015 -2020

Delft-FEWS architecture is mainly developed in 2003; technology has evolved: • Do we need to review the Delft-FEWS architecture? Where do we need to focus on: • Keep it simple, less software components (Jboss, Tomcat, . . ) • Use of standard components introduces many times more complexity! • Where possible share ideas and knowledge with other Deltares software systems Do not forget: • Why have we made certain decisions in the past, there is an idea behind! • Watch out for buzz-words; What is the role of FEWS? Delft-FEWS: 2015 -2020

Developments: interesting or hype? What is our view? Not everything is a hype! See the changes! Mostly project driven! Delft-FEWS: 2015 -2020

Some ideas 1. 2. 3. 4. 5. Big data and Large Datasets Open FEWS Archive Thick and Thin clients Cloud computing and Simplification of FEWS roll-out Webservices Delft-FEWS: 2015 -2020

Big Data • • • Large amounts of unstructured data from many sources (i. e. Twitter) FEWS will not use this data directly • We expect external services/software will use BIG data, structure it (find correlation) and provide information that can be used in FEWS must be able to deal with large amounts of structured data • Trends in meteorological forecasts, satellite data, . . Delft-FEWS: 2015 -2020

Delft-FEWS and Large Datasets We need to find a solution how FEWS handles large data sets • Large number of scalar data in Forecasting System seems no problem. • Operational meteorological forecasts (ensembles) also no problem. • Store data for longer period in operational database can be problematic • There must be a solution for storage of large model data sets that do not fit in FEWS operational database • Model states of Swan or Delft-3 D, 3 Di (store on file system, No. SQL) • Model results (first process, filter and only store important information) • In 2014 Development of new Open Archive has started • Will play a central role in storage of large datasets Delft-FEWS: 2015 -2020

Delft-FEWS and Open Archive • Development of new FEWS Archive is important and essential • Store validated historical data in the archive • Store historical grids in archive (i. e. Radar grids) • Users must have direct access to archive (Seamless integration in OC) • Fits in Open Data Strategy of many FEWS users Delft-FEWS: 2015 -2020

Delft-FEWS: Thick Client and Thin Client • Discussion on Delft-FEWS Client • Keep the FEWS Thick client; maybe redesign but keep it for the experts • For the users the look and feel of the application must not change too much (large investments in training) • CITRIX and Remote Desktop solutions are more and more popular • Will we initiate the development of a Thin client (Web-application) from Delft -FEWS • Sounds nice, but is complex to develop and maintain • Large dependency on popular browsers • Deltares has developed a Delta Data Viewer; use as complete solution for FEWS? • Current focus on webservices and Open-API for the development of Web viewers by others (Lizard, Hydro. Net, Delta Data Viewer, . . ) Delft-FEWS: 2015 -2020

Cloud Computing • • • Remote computing through the internet (Cluster is internal!) Make use of free capacity of external servers FEWS needs to have an answer on this development! Examples: • 3 Di as cloud solution (FEWS-3 Di, how to run models remotely) • Bring your model to the data; reduce data traffic over the internet Solutions: • Simple: Start a model that runs in the cloud • Complex: Send a complete FEWS system (FSS) with config to the data and run on a cloud computer. Minimise the data traffic and maintain a good performance • Important in the discussion is performance, security, robustness Delft-FEWS: 2015 -2020

Cloud Computing: Pilot 2015 The Deltares ID-lab is used to promote new ideas and applications developed by Deltares (also: initiate new developments) One of the new developments is a Proof of Concept (Po. C) with Microsoft to use the Azure cloud for FEWS. The Po. C for 2015 includes several steps: 1. Roll-out FEWS “as is” in the cloud (FSS and MC+Database) 2. Investigate with the ‘Experts’ what short term developments are needed to simplify the roll-out of FEWS in the cloud 3. Investigate if larger developments are needed to make better use of the cloud One of the Goals of this initiative is to simplify the current FEWS components for easier rollout of Server and Client components and automated upgrades Delft-FEWS: 2015 -2020

Web Services • A webservice makes it possible to request a service from a server. This can be a data request, execution of a task or process or send data. • • Difference between provider and consumer webservices Current FEWS webservice functionality will be made consistent and extended • FEWS-PI, UM-Aquo, Water. ML, Json, … Will support running FEWS form external servers (start workflow, modifier, etc. . ). • Delft-FEWS: 2015 -2020

Some Recent and planned Developments Development of FEWS Open-API • Can be used to access the Operational database and FEWS functions • Used by FEWS developers of Deltares • Used by external developers to add new modules • Currently only limited functionality • Will be extended in 2016 with more functionality FEWS Open-API FEWS Operational Database Delft-FEWS: 2015 -2020

Some Recent and planned Developments FEWS Archive can be accessed by FEWS Open-API • Observed scalar time series can be accessed directly by an OC • In future this will also be possible for FSS • Other time series like Grids, forecasts will be added in future • Other Open. Dap databases can be added later FEWS OC or SA FEWS Open-API FEWS Operational Database Delft-FEWS: 2015 -2020 FEWS Archive Open. Dap External

Some Recent and planned Developments Service layer can be added on top of FEWS Open-API • Web services developed for FEWS-PI, Aquo , Water. ML, Json • Source code of current web services is open and can be downloaded from WIKI TOMCAT Aquo Water. ML FEWS-PI JSON WMTS FEWS Open-API FEWS Operational Database Delft-FEWS: 2015 -2020 FEWS Archive

Some Recent and planned Developments Other application can be build on top of the Web-services Extern Hydronet FEWS Lizard TOMCAT Aquo Water. ML FEWS-PI JSON WMTS FEWS Open-API FEWS Operational Database Delft-FEWS: 2015 -2020 FEWS Archive

Feedback from Community Strategy Board Ask feedback from CSB. • What is important for Delft-FEWS? • What is good and less good in Delft-FEWS • What do they expect from Deltares in maintaining Delft-FEWS (Open Source? ) Delft-FEWS: 2015 -2020