National Backbone Extension Rolling out Multisite Gigabit connections

National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics Brian Boyle & Mike Norris

Topics 1. NBE – in context 2. Aims of the project 3. What has been achieved 4. Under the bonnet 5. The bigger picture 6. Summary 7. Next steps 8. Discussion 2

NBE – in context • • • Building on the NBN New requirements – hybrid networking New paradigm of service and support Proposal and resources Support from stakeholders and clients 3

Bandwidth Use

HPC facility ICHEC traffic ICHEC latency

General Internet Daily traffic Yearly traffic

NBE – the project • • • Project plan and management Inventory of sites Criteria, priorities Surveys and schedules Procurement, multivendor scenario 7

Project organisation

Project goals • A network for connecting clients, with the following features • • • High bandwidth Multi-service Resilient Scaleable Flexible Secure 9

Goals – Multi-service – Service orientated connectivity, e. g: • Private secure connection per Institute to An Chéim • Interconnectivity with the Government VPN – Not just focused on internet reachability – Capable of delivering access to multiple services using the same infrastructure – Advanced networking protocols supported – IPv 6, multicast 10

Goals - Resilience • Seek to build in circuit, equipment, path, and routing diversity • Equivalent service levels on primary and secondary • Using multiple ducts, termination points, equipment • Using a robust routing technology (BGP) • Can make use of existing connectivity via other service providers 11

Goals - Scalability • Can deliver multiple 1 Gigabit/s circuits • Can operate at line speed end to end • Capable of offering even higher speeds subject to requirements (10 Gbps, higher) • 10 Gbit/s access for clients being explored • Technology refresh available for 40 Gbit/s… • …and similar for 100 Gbps in development 12

Goals - Flexibility • Not subject to telco business case scenarios • Not tied by contract to any particular network implementation technology or model • Can use a mix of fibre and traditional managed Ethernet circuits • Fully compatible with existing networks (no big bang changes) • Capable of delivering wavelengths to clients if the need is proven 13

Client perspective • Client priorities are key throughout • Aim is to keep change to client sites minimal • HEAnet installs managed routers and switches in addition to current network equipment • When ready, services can be tested and migrated individually • No Big Bang change • No change without explicit client agreement 14

Typical installation • Research of available providers (fibre, managed circuits), usually via Tender process • Client site survey by provider with client § Paths identified, leading to “way leave” contracts • • Civil works commence Fibres are pulled into building, and subsequently spliced Routers and switches are delivered, configured by HEAnet Services can be tested and used by client 15

New link services • Point-to-point Ethernet links (p 2 p) – 1 Gbit/s, Ethernet framed – Ethernet Port Service – Ethernet VLAN Service – No aggregation – Automatic monitoring/graphing 16

Ethernet Port and VLAN services

Some examples • Services - connectivity - institutional access and security - p 2 p • Applications - Oireachtas - ICHEC - An Chéim - UCLP • Client examples - DCU 18

LAN Extensions – Ryan Academy • • • Centralise services, both voice and data Servers centrally located at DCU IP Telephony Extension to Nortel Meridian Seamless telephony integration Security cameras and DVR remote view and monitoring Buildings Management System High-End Video Conferencing > 2 Meg/sec Reliability No onsite technical support ( low maintenance ) 19

LAN Extensions – Mater Dei Institute • Sister College of DCU • All Servers consolidated in DCU • Previous difficulties with server location in MD ( Dust and no Aircon ) • Minimal on-site technical support ( mainly helpdesk 2/3 days per week ) • No requirement for Sysadmin • Reliability of service 20

Under the bonnet • • • DWDM, CWDM Ethernet switches MPLS New routing strategies Out of band access for network management CPE, routers Outsourcing of L 2 ops management Provisioning tool Contributions, SLA 21

Multiple sources • We make use of many fibre providers to connect at the lowest cost for our requirements – – – E-net Dublin City Council ESBT BT Ireland NTL Aurora 22

Fibre network end 2006 – approx 2300 km of fibre

MAN design

Transmission technologies • We use a heterogeneous mix of technologies to achieve our goals: – CWDM in metropolitan areas gives us multiplexed fibre services for short hops – DWDM between regions to give the same for longer hops – MPLS network to give multi-service offering – Multiple physical connections to add resiliency 25

HEAnet backbone network topology

How does it do it? • Circuits are managed using an automated provisioning system • Based on client requirements, the system is used to configure routers, service monitoring portal • Aim to have client use the tool directly, with administrator later checking/approving circuit change requests 27

Network provisioning tool

The bigger picture • • • NIBEST – NW link with NIRAN UKERNA DF – Super. JANET 5 links GÉANT – hybrid networking Transit – preparing for the next step INEX – peerings and more 29

INEX traffic growth • HEAnet-INEX (IPv 4) • Total INEX

Summary of changes Before After • Network consists of managed links connecting regional POPs • Clients used managed links to connect to Po. Ps • Service monitored by HEAnet (IP) and telco providers (physical) • More Po. Ps, connected by Dark Fibre • Clients used Dark Fibre and/or managed links to connect to Po. Ps • Service monitored by HEAnet (IP, Ethernet) and Fibre providers (physical)

Benefits • Improved bandwidth • Guaranteed uncontended bandwidth service levels • Reduced cost • Flexibility • Improved access to HEAnet services 35

Outline of services

Phase 2 progress, 2 nd Nov Confirmed Cancelled In progress Completed 7 4 17 13 Total number of links 43 37

Dark fibre statistics • No. of DF providers • Aggregate length of DF pair • No. of primary client sites with DF links • Total sites with DF connections 38 5 2, 300 km 18 (of 40) 31

Next steps • • • New model, service levels Phase 3 Client service reviews Review of research infrastructures Integrated e-infrastructures 39

Questions?

Project history • In 2004, we commenced implementation of dark fibre point to point links • Followed by additional of CWDM and DWDM equipment to increase service possibilities • Commenced work on an automated provisioning tool to create and manage configurations • Quickly grew into a heterogeneous network combining a mix of technologies and providers 41

Scalable capacity • • • Backbone – 10 Gbit/s Can add more 10 Gbit/s using existing infrastructure Client connections at 1 Gbit/s 42

What has been achieved • High levels of service to clients - high bandwidth - resilience – path, equipment, service • • • Scalability now and in the future Underpins e-infrastructure Complex and flexible Rich mix of services Secure 43