Large-scale testbeds for network R D Planet Lab Everywhere

Summary • We discuss R&D testbeds for networking and distributed systems, seen from the point of view of RNP • In addition to activities carried out in Brazil, we also examine some testbeds in use or planned in other countries, especially in relation to future Internet development • In conclusion suggestions are made about future steps to be followed in Brazil. Large-scale testbeds for R&D in Brazil 2

Introduction to Brazil • • • In 1494 Spain and Portugal had divided between themselves undiscovered lands by the Treaty of Tordesillas – The Tordesillas Line was to be the frontier between the dominions of Spain (W) and Portugal (E) Brazil is the successor country to the Portuguese dominions in South America – Rather over one half of present Brazil lies to the WEST of the Tordesillas Line Brazil is a BIG place! – diameter of about 4, 200 km Tordesillas Line to Spain to Portugal – 42 x 42 = 1764 ms 2 • Current population of about 180 millions, unevenly distributed – most of the population and communications infrastructure concentrated to the EAST of the Tordesillas Line Large-scale testbeds for R&D in Brazil 3

RNP – Rede Nacional de Ensino e Pesquisa • RNP is the Brazilian NREN – maintained by the Brazilian government (since 1989) to enable network access to the national research and education community – provides national (inter-state) and international R&E connectivity for more than 300 public and private universities and research centers through the provision of advanced networking infrastructure • also provides commodity access – one-stop shopping – promotes the development of advanced networking and applications • Since 2000, RNP is managed for the federal government by a non-profit private company, RNP-OS, legally recognised as an “Organização Social”, which allows the government to contract its services without competitive tender. Large-scale testbeds for R&D in Brazil 4

RNP’s service networks RNP includes the following funded connectivity: • National backbone network – Rede IPÊ – 1 Po. P (Point of Presence) in each state – usually a federal university – Link capacity depends on the available telco infrastructure – Currently from 2 Mbps to 10 Gbps • Direct intercity connections between state Po. P and non-local federal instituions (education, science and technology) – Currently from 2 to 155 Mbps (depends on the institution) • Community-based optical metro networks connected to Po. Ps – Currently being built out – 9 out of 27 already in operation No service charges are made to end user institutions • Non-federal institutions are normally required to fund their own access links Large-scale testbeds for R&D in Brazil 5

National integration

Red. Clara, beyond 2008 • Extend Red. Clara to all LA&C countries • Promote applications in education and health • Start of a new project (ALICE 2), with the support of the EC, in 2009 Source: www. redclara. net Large-scale testbeds for R&D in Brazil 7

A world-class network – RNP in GLIF Source: www. glif. is Large-scale testbeds for R&D in Brazil 8

A world-class network – worldwide GLIF Source: www. glif. is Large-scale testbeds for R&D in Brazil 9

Rede IPÊ – national backbone network Last big reform in 2005 (5 th phase) Capacity reflects available telco infrastructure Currently composed of: • Multigigabit core network – 4 Po. Ps at 10 Gbps, and 6 Po. Ps at 2. 5 Gbps – IP over lambdas (12. 000 km) • Terrestrial SDH connections to 15 Po. Ps – Most links are 34 Mbps – Some at 2 Mbps – Some upgrades in 2007 to 102, 155 and 622 Mbps • 2 Po. Ps connected by satellite at 2 Mbps Large-scale testbeds for R&D in Brazil 10

Evolution of academic networks in Brazil RNP Phase Year Technology Link capacities Comment 1988 BITNET up to 9. 6 kbps first national network 1 1992 Internet 9. 6 and 64 kbps first national IP network (RNP) 2 1995 up to 2 Mbps also: commercial IP deployed 3 1999 IP/ATM, IP/FR VC up to 45 Mbps, RNP 2 national backbone; access up to 155 testbed metro networks in 14 Mbps cities (using ATM/dark fiber) 4 2003 IP/SDH 34, 155, 622 Mbps also: IP/WDM interstate testbed network (Project GIGA) 5 2005 IP/WDM 2. 5 and 10 Gbps IPÊ national backbone; metro networks in 27 capitals Large-scale testbeds for R&D in Brazil 11

Evolution of academic networks in Brazil (Link capacity) Phase 5 Ipê Phase 4 RNP 2+ Phase 3 RNP 2 Phase 2 comercial Internet Phase 0 BITNET Phase 1 Internet Large-scale testbeds for R&D in Brazil 12

RNP and its innovating networks • The first version of RNP’s network was deployed in 1992, and pioneered in using TCP/IP technology nationwide • Since then, the different generations of network deployed and operated by RNP have innovated technically, at least within Brazil • With the development of the network industry here, since the beginning of commercial IP networks starting in 1995, RNP is no longer the only IP network operator in Brazil, but continues to lead technologically, in the pursuit of new models of infrastructure and applications Large-scale testbeds for R&D in Brazil 13

Technology changes in the network • The beginning of each new technology phase was a step in the dark – new circuits (or service, in the case of ATM/FR) were ordered and delivered, and equipment configured – after configuration by the network engineers, the new network would begin to work, and would be put into operation as soon as possible. – users could then begin to use the resources of the new network • Problems: – lack of familiarity with the new technologies before operational deployment – the technology transition became a singularity Large-scale testbeds for R&D in Brazil 14

New application services • Internet technology is very accommodating of new application services – any user can develop a new service, impelemnted as a distributed application using the sockets API – this permits and encourages experimentation with applications that can be built initially in the laboratory (in a LAN environment) and then migrated to the wide area network • Some problems which arise: – some distributed services require componentes “within the network” – security problems – performance monitoring may be a problem Large-scale testbeds for R&D in Brazil 15

Alternative solutions • Both for – preparing an upcoming network technology change – developing a a new large-scale distributed application we really need a testbed facility, isolated from the production network, and which reproduces its characteristics of scale and performance • This isolation can be real or virtual – Real – the testbed is based on separate physical infrastructure, independent of the production network • example: Project GIGA – Virtual – the testbed shares the same infrastructure used by the production network • example: Planet. Lab Large-scale testbeds for R&D in Brazil 16

Project GIGA – optical networking testbed • • • Partnership between – RNP , CPq. D (telco industry R&D centre in Campinas, SP), R&D community in networks and distributed systems – Financed by FUNTTEL between 2002 and 2007 – US$20 M – telcos – provide optical fiber at no cost Objectives: – build an advanced networking testbed for development and demonstration purposes – support R&D subprojects in optical and IP networking technology and advanced applications and services Network support (since May 2004) – R&D subproject consortia provided with internal connectivity using VLANs – 20 institutions connected – however: the testbed (with some exceptions) did not provide exteral connectivity, limiting its usefulness FUNTTEL Large-scale testbeds for R&D in Brazil 17

GIGA testbed network - location • dark fibre-based 700 -km inter-city backbone in states of São Paulo and Rio de Janeiro (south-east Brazil) • Initially 20 universities and R&D centers in 7 cities • 2. 5 G DWDM in the intercity backbone • 2. 5 G CWDM used in the metropolitan area Large-scale testbeds for R&D in Brazil testbed network 18

GIGA testbed network - location Universities IME PUC-Rio PUC-Campinas UERJ UFF UFRJ Mackenzie UNICAMP USP R&D Centers CBPF CPq. D CPTEC INCOR CTA FIOCRUZ IMPA INPE LNCC LNLS Large-scale testbeds for R&D in Brazil 19

Testbed network design • 2. 5 G DWDM inter-city network between Campinas and Rio de Janeiro (some upgrades to 10 G) Campinas – up to 6 waves per link (can use 8) • • 3λ 2. 5 G CWDM metro networks in São Paulo, Campinas São and Rio de Janeiro Paulo all links currently 1 Gigabit Ethernet – optical equipment from the Brazilian firm, Padtec 1λ 2λ 2λ 1λ Rio de Janeiro 2λ (www. padtec. com. br) – IP equipment from Extreme Networks São Paulo S. José dos Campos MAN SP Campinas 3λ S. J. dos Campos Cachoeira Paulista 1λ 2λ Rio de Janeiro Petrópolis Niterói MAN CP MAN RJ Large-scale testbeds for R&D in Brazil 20

What is Planet. Lab? (base d on slides by Marc Fiuczynski, Sept. 2007) • Facility: Planetary-scale “overlay” & “underlay” network – 900+ Linux-based servers at 400+ sites in 40+ countries – Currently there exist a handful of PL sites in Brazil operated by RNP and a few universities and research institutions

Planet. Lab Facility Today 1000+ servers at 450+ sites in 40+ countries Co-located throughout the world @ Uni. & Companies Co-located at network crossroads (Internet 2, RNP, CERNET, …) Large-scale testbeds for R&D in Brazil 22

Virtualization in Planet. Lab? • Research Community: Distributed Sys. & Networking – Researchers can get a set of “virtual machines” across these servers (SLICE) – In a SLICE researchers can deploy & evaluate … – … distributed systems services and applications “The next Internet will be created as an overlay in the current one” – … network architectures and protocols “The new Internet will be created in parallel next to the current one” (see later)

Example Network Services • Scalable Large-File Transfer: Co. Blitz—Princeton, Lo. CI—Tennessee • Content Distribution: Coral—NYU, Co. Dee. N—Princeton, Cob. Web— Cornell • • • Distributed Hash Tables: Open. DHT—UC Berkeley; Chord-MIT Routing Overlays: I 3 Internet Indirection Infrastructure—UC Berkeley Multicast Delivery Nets: End System Multicast—CMU, Tmesh-U. Michigan Serverless Email: e. POST—Rice University Publish-Subscribe News Access: Cor. ONA—Cornell Robust DNS Resolution: Co. DNS—Princeton, Co. Do. Ns—Cornell Mobile Access: DHARMA—U. of Pennsylvania Location/Anycast Services: OASIS—NYU, Meridian—Cornell Internet Measurement: Script. Route—U. of Maryland • Above services communicate with >1 M real users and transmit ~4 TB of data per day Large-scale testbeds for R&D in Brazil 24

Slice Manager (SM) Slice Planet. Lab Node Software Architecture Virtualization Software x 86 Server Hardware Large-scale testbeds for R&D in Brazil 25

Slices Large-scale testbeds for R&D in Brazil 26

Slices Large-scale testbeds for R&D in Brazil 27

Slices Large-scale testbeds for R&D in Brazil 28

Thoughts about the future Internet • The success of the Internet has been so enormous that it is tempting to imagine its future by extrapolating from the present • However, there are consequences of its design, based on decisions taken in the 1970 s, which severely limit its security, availability, flexibility and manageability • These limitations can not be removed through small incremental adjustments of the existing network, and, if they are not removed, they will create huge impediments to the ability to use and exploit the Internet in the future Large-scale testbeds for R&D in Brazil 29

Removing these limitations • For many years, combatting Internet limitations has been carried out through a series of “patches”, introduced to solve specific problems. • Unfortunately, these patches result in increased complexity, resulting in a less robust system, whose operation has become more difficult and costly • A growing consensus exists in the network research community that we have already reached the point where patches are inadequate, and a fundamental rethink of the Internet is required (from the GENI Research Plan, 2007) Large-scale testbeds for R&D in Brazil 30

GENI - http: //www. geni. net (Global Environment for Network Innovations) • An initiative of NSF (USA) to create a shared testbed environment to allow the validation of new network architectures – Initial phase: 2005 to 2007 – design of the GENI environment – Present phase: since 2008, deployment and use • GENI will support research which can lead to a future Internet with improved chacteristics – more comprehensive security – greater generality – better integration of optical and wireless technologies – integration of the world of sensors and embedded processors – improved options for the economic health of ISPs Large-scale testbeds for R&D in Brazil 31

The GENI testbed environment GENI will: • allow experiments with alternative large-scale network architectures, services and applications under realistic conditions • use vitualizations to permit carrying out multiple independent and simultaneous experiments • permit long-duration experiments, allowing mature prototypes to serve “living” user communities • facilitate experimental research through the use of extensive tools for measurement and data collection In summary, GENI will provide support for the taking ideas on large -scale ideas from their conception to their deployment, by means of experimental validation Large-scale testbeds for R&D in Brazil 32

How the GENI environment will be built • The GENI environment is inspired on Planet. Lab, and especially the “Meta-Testbed” VINI - http: //www. vini-veritas. net • VINI extends the scope of Planet. Lab to allow – “slicing” of links between the nodes (link virtualization) – substitution of level 3 protocols (IP) Large-scale testbeds for R&D in Brazil 33

Next Step: Meta Testbed (base d on slides by Marc Fiuczynski, Sept. 2007) • Goals – support experimental validation of new architectures • simultaneously support real users and clean slate designs • allow a thousand flowers to bloom – provide plausible deployment path • Key ideas – virtualization • multiple architectures on a shared infrastructure • shared management costs – opt-in on a per-user / per-application basis • attract real users • demand drives deployment / adoption Large-scale testbeds for R&D in Brazil 34

VINI: Our Meta Testbed approach • Infrastructure – Planet. Lab provides “access network” with global reach • user desktops run proxy that allows them to opt-in • treat nearby Planet. Lab node as ingress router – NLR/I 2 provides high-speed backbone in the United States • populate with programmable routers • extend slice abstraction to these routers • Usage model – each architecture (service) runs in its own slice – two modes of use • short-term experiments • long-running stable architectures and services Large-scale testbeds for R&D in Brazil 35

Extending Slices to a VINI testbed Large-scale testbeds for R&D in Brazil 36

Extending Slices to a VINI testbed Large-scale testbeds for R&D in Brazil 37

Extending Slices to a VINI testbed Large-scale testbeds for R&D in Brazil 38

User Opt-in Client Server NAT wireless Large-scale testbeds for R&D in Brazil 39

Internet in a Slice (IIAS) XORP in Network Container XORP (routing protocols) IPv 4 Fwd table User – Adds routes to copy of kernel IPv 4 forwarding table – Kernel forwards packets between virtual interfaces Kernel Filters and shapers – Add delay and loss, constrain bandwidth Virtual interfaces Filters, shapers vif 0 vif 1 vif 2 E-GRE tunnels – Appear as Ethernet devices in a slice – Reduce MTU for tunneling E-GRE tunnels – Hack standard GRE tunnels to preserve MAC headers Planet. Lab VM Large-scale testbeds for R&D in Brazil 40

GENI • Extenssion of VINI – key ideas: – virtualization • multiple network architectures sharing a common infrastructure – user opt-in: per user / per application • intended to attract real users • Infrastructure – NLR/Internet 2 provide high-capacity backbone in the US • populate with programmable resources (processors, storage) • populate with programmable routers – more sophisticated than the PCs used in VINI • extend the “slice” abstraction to these routers • include “extensions” to wireless and sensor networks Large-scale testbeds for R&D in Brazil 41

GENI: The Physical Network (slides by C. Qiao, 2008) • Large-Scale Facility of Networked Systems – – Reasonable Representation of the Internet’s Complexity A Nationwide Optical Network ~ 200 Universities Clusters for Processing/Storage Wireless Access Networks • Mobility, Location Awareness – Sensor Networks – Connected to a large number of User Communities • Partnerships to Extend GENI within the US – Add Technologies and Users • Federation to Extend GENI on a Global Scale Large-scale testbeds for R&D in Brazil 42

Schematic GENI Network Sensor Network Core Nodes Edge Site Federated International Facility Mobile Wireless Network Edge Nodes Large-scale testbeds for R&D in Brazil 43

Programmability All network elements programmable via open interfaces and/or downloadable user code GENI Control & Management Plane Programmable Open API Sensor Node Radio platform Programmable Core Node Large-scale testbeds for R&D in Brazil Programmable Edge Node 44

Slicing and Virtualization Sensor Network Edge Site Mobile Wireless Network - share resources to support many simultaneous experiments Large-scale testbeds for R&D in Brazil 45

GENI Design Principles • Physical network ‘substrate’ – building block components – elements / nodes / links / subnets • Software control & management framework – knits building blocks together – allows many parallel experiments (slices) – creates arbitrary logical topologies (virtualization) • Programmable for ‘Clean Slate’ research • Instrumented for accurate analysis • Flexible and Phased Design – Support Technology Introduction during GENI Lifetime Large-scale testbeds for R&D in Brazil 46

GENI Phase 1(2008 -9) (based on slides by Chip Elliott – GENI program director) • Provides the very first, national-scale prototype of an interoperable infrastructure suite for Network Science and Engineering experiments • Creates an end-to-end GENI prototype in 6 -12 months with broad academic and industrial participation, while encouraging strong competition in the design and implementation of GENI’s control framework and clearinghouse • Includes multiple national backbones and regional optical networks, campuses, compute and storage clusters, metropolitan wireless and sensor networks, instrumentation and measurement, and user opt-in • Because the GENI control framework software presents very high technical and programmatic risk, the GPO has funded multiple, competing teams to integrate and demonstrate competing versions of the control software in Phase 1 Large-scale testbeds for R&D in Brazil 47

GENI phase 1: integration: 5 competing control schemes Large-scale testbeds for R&D in Brazil 48

Research using GENI - FIND • Prior to the building of the GENI testbed, NSF launched the initiative FIND – Future Internet Design, with the aim of identifying and financing research activities • http: //www. nets-find. net/ • One product of this initiative is the GENI Research Plan, which details the research motivation for GENI, and some of the research goals which will become possible: • http: //www. geni. net/GDD-06 -28. pdf Large-scale testbeds for R&D in Brazil 49

Experimental activities in Europe FIRE - Future Internet Research & Experimentation http: //cordis. europa. eu/fp 7/ict/fire/home_en. html • European initiative directed towards the design of the future Internet, similar to the FIND and GENI initiatives of the NSF • promotes the concept of experimental research, combining visionary academic research with validation and experimentation typical of industry • aims to create a multidisciplinary research environment to investigate and validate experimentally innovative ideas for new paradigms of networks and services • plans to create a “European Experimental Facility” (EEF), formed by the interconnection and federation of both existing and future testbeds, for emerging and future Internet technologies • First projects selected in 2008 Large-scale testbeds for R&D in Brazil 50

FIRE/Panlab (FOKUS/DE)– www. panlab. net Large-scale testbeds for R&D in Brazil 51

FIRE/One. Lab (UPMC, FR) – www. onelab. eu • History: – Mar/04: based on ENEXT (Identification of critical testbeds for networking research) – Sep/06: One. Lab 1 project funded as an IST project under the FP 6 funding program: 10 partners, 2 years – Sep/08: proposal funded as an IST project under the FP 7 funding program: 26 partners, 2 years • One. Lab 1 – Goals – extend Planet. Lab into new environments, beyond the traditional wired internet – improve Planet. Lab’s monitoring capabilities – provide a European administration for Planet. Lab nodes in Europe Large-scale testbeds for R&D in Brazil 52

FIRE/Federica – www. fp 7 -federica. eu Large-scale testbeds for R&D in Brazil 53

Comparison of FIRE prototypes Large-scale testbeds for R&D in Brazil 54

Future Internet activities in Japan • AKARI - http: //akari-project. nict. go. jp/eng/overview. htm – plans to deploy a new generation network by 2015, based on a new architecture – supposes the use of an experimental testbed incorporating virtualization techniques, as in GENI Large-scale testbeds for R&D in Brazil 55

Perspectives in Brazil • Phase 2 of Project GIGA (RNP- CPq. D) – original Project GIGA funding ended in Dec/2007, although the experimental network continues to operate – since 2007 RNP and CPq. D have been seeking to maintain an experimental facility for their research communities – RNP’s proposal is to place greater emphasis on research into architectures for a future Internet Large-scale testbeds for R&D in Brazil 56

Perspectives in Brazil • INCT/Web Science (consortium led by UFRJ) – more than 100 researchers – approved in 2008 (3 to 5 years) • A group of 8 researchers from (RNP, UFF, UFPA, UNIFACS, USP) included a research proposal in “Future Internet Architectures” – main emphasis on experimental research, using an environment based on Planet. Lab / VINI, with extensions for wireless access networks – RNP network will be used for long-distance communications (within Brazil and externally) • Note: – VINI requires access to a “lower than level 3” network – this depends on the next phase of the RNP network (Futura. RNP), expected in 2010 Large-scale testbeds for R&D in Brazil 57

Futura. RNP and 2010 • RNP is engaged in several initiatives, which are changing the face of its infrastructure: – adoption of a layer 2 national backbone, and introduction of static and dynamic end-to-end circuits as well as routed IP (Hybrid Network) – extension to all 27 capitals and 10 other cities of optical metro networks based on Ethernet technology (layer 2) – huge increase in capacity, when possible through multiple lambdas in DWDM systems • A possible desirable consequence would be the permanent reservation of capacity for experimental activities, segregated from production traffic (à la GENI) Large-scale testbeds for R&D in Brazil 58

Federation with other initiatives • The deployment of an experimental facility in Brazil to support research into new architectures and applications will simplify international collaboration with similar initiatives abroad • This would be brought about by the interconnection (federation) of the national facility with similar facilities in other countries • It should be noted that this style of federation is one of the characteristics of the projects we have described from the USA and Europe. Large-scale testbeds for R&D in Brazil 59

The Future Internet will be polymorphic Large-scale testbeds for R&D in Brazil 60

The Future Internet will be polymorphic Large-scale testbeds for R&D in Brazil 61

Thank you! Michael Stanton (michael@rnp. br) www. rnp. br Yellow ipê in blossom