UC Berkeley CS 98 198 Intro to Web 2

UC Berkeley CS 98/198: Intro. to Web 2. 0 Development Using Ruby on Rails Armando Fox, Will Sobel, Dave Patterson

RESERVED: MW 4– 5: 30 271 Soda is RESERVED for CS 98 -10/198 -10, Intro. to Web 2. 0 Development Using Ruby on Rails, MONDAYS & WEDNESDAYS 4– 5: 30 PM. Space permitting, students not enrolled in the class may use the space during those times if their activities don’t interfere with running the class. Thanks. Questions: fox@cs. berkeley. edu CS 98 -10/198 -10, Spr 08

Today • About the course – administrivia, prereqs, course format, projects – about the RAD Lab – course projects & Entrepreneurship tie-in • Movies – zero to blogging in 58 lines of code – Rails “TV ads” • Background concepts review – Web app anatomy & brief history of Web programming – TCP/IP, HTTP, HTML, Web servers CS 98 -10/198 -10, Spr 08

Why this course? • Important trends in “Web 2. 0” style applications – – Reinventing client-server: the datacenter is the computer New models of software development & deployment Rich, sophisticated UI’s Modular apps (mashups, etc. ) • Attract “end users” to RAD Lab platform – Student-conceived apps generate traffic – We provide “hosting service” • Exercise RAD Lab infrastructure Opportunity: combine sound pedagogy with “real world” skills & gratification of crafting a useful artifact early in your CS career. CS 98 -10/198 -10, Spr 08

Web 1. 0 • Static and some dynamic content – lots of ad-hoc scripting languages, cgi-bin – “homegrown” sites not yet standardized on a stack (like LAMP); eg, variants of Berkeley DB or filesystem for storage – Big companies: $$$ relational DB (eg Oracle) • Primitive web page GUIs – 1 interaction == 1 roundtrip to server for page view • “Services” available but have to be a company to get them (small sites use libraries instead) – Doubleclick (ads) – Inktomi et al. (web search on your site) – Visa/MC (accept credit cards) CS 98 -10/198 -10, Spr 08

Software as a Service (Saa. S) • Run your own database => Salesforce • Email Word files around => Google Docs • Shrinkwrap mapping software => Google Maps, Google Earth • Buy a hard disk => Amazon S 3 • Buy servers => Amazon EC 2 • Desktop GUI app => AJAX (Javascriptbased) GUI in browser CS 98 -10/198 -10, Spr 08

Web 2. 0 Trends • Company => Individual can get services – Double. Click => Google Ad. Sense; Visa => Paypal • Sophisticated UI: AJAX decouples interactions/page updates from server roundtrips – Effect 1: “desktop-like” interactive Web apps – Effect 2: Desktop => cell phone (Browser w/Javascript == universal platform) • “Mass customize” to 1 => Wisdom of crowds – global search, etc are more compelling than local because learns from everybody • Libraries => Services, thanks to arch. standards – REST/CRUD for state management – Javascript/DOM for advanced browser interactions CS 98 -10/198 -10, Spr 08

Effect: Complexity Curve • History: startups can’t compete effectively because of cost of developing large scale or complex infrastructure – Full-text index for 200 MM documents – Geocoded real-time messages – Quickly index and navigate mesh of millions of friends, compute shortest-path distance – Validate identity for secure transactions • Services provide components and provide pay as you go model • Scalable costs allow small companies to focus on core competency CS 98 -10/198 -10, Spr 08

New models of software development • Waterfall: Static Handoff • Process: Support DADO Evolution, 1 group Model, N groups Develop Assess Deploy Operate CS 98 -10/198 -10, Spr 08

Agile (in 5 steps) Client Focused Development Light Weight Process for Small Teams 1. 2. 3. 4. 5. Write Stories not Requirements Estimate Stories using Points Compute Your Velocity in Points per Day Scope Your Project Based on Your Velocity Spike Where You Can’t Estimate CS 98 -10/198 -10, Spr 08

Real World Practices • Development vs. deployment environments & tools – Real version control (Subversion) – Automated deployment tools (Capistrano) – Regression testing techniques (Rspec, Selenium) – Database schema migration • User Stories. Customer focused requirements. • CRC cards (Class – Responsibility – Collaborator). Minimalist Object Modeling. • Iterations and velocity to scope releases. • Discussion of Ro. R and Web app entrepreneurship toward end of sem. CS 98 -10/198 -10, Spr 08

Prereqs & course format • • “Guided individual study”, 2 -4 units C/NC Prereqs: CS 61 A or equiv. ; OOP concepts Hybrid lecture/lab format Realistic development environment – develop primarily on your own machines – Deploy to “production ISP” (RAD Lab cluster), also available for development use – Each project gets an external sub-domain – Option to purchase your own domain CS 98 -10/198 -10, Spr 08

Required Team Project • You pick the app – Tie-in to Enterpreneurship (Eng 190 C) to help brainstorm • We help you scope & design the project • We help you DADO the project • You give live demo & 5 minute “pitch” at end of semester • “Best” project team members offered RAD Lab internships if they wish • Some previous projects already. com’s CS 98 -10/198 -10, Spr 08

Some projects from previous iterations • Spring 07 Projects: heavy comments; test rigs; Good GUI; Fine in browser; 1 k to 3 k LOC – We. Join. In (real users!): staff your volunteer events – Calentopia: organize your academic life – 5 um (next generation forum): organize & stay on top of all forums you read • Fall 07 Projects with real users (used 5 um in class); – Yumify: where go to dinner? http: //www. yumify. com/ • 2. 5 K LOC, 0. 3 K Test rig, 1 K Lines of comments – Hesperian: multilingual community health access http: //hesperian. rorclass. org/ • 0. 5 K LOC, 0. 3 K Test rig, 0. 2 K Lines of comments Students from “award winning” projects already working in RAD Lab as ugrad research assistants CS 98 -10/198 -10, Spr 08

Datacenter is the Computer • Program == Web search, Mail, … • Computer == Building full of 1000 s of computers, networking & storage devices • Warehouse-sized facilities and workloads may be unusual today but are likely to be more common in the next few years • Re-inventing Client/Server Computing • The Datacenter is the (Server) Computer • The Laptop/Handheld is the (Client) Computer – HP shipped more laptops than desktops in 2007 – Million cells phones per day, increasing functionality • Apple i. Phone as one example CS 98 -10/198 -10, Spr 08

Datacenter is the Computer • Interesting design point: datacenter composed of 20 ft. containers – Power/cooling for 200 KW of racked HW – External taps for electricity, network, water – ~250 Servers, 7 TB DRAM, or 1. 5 PB disk • Black. Box to arrive next to Soda Spring 2008? Sun Project Blackbox 10/17/06 CS 98 -10/198 -10, Spr 08

RAD Lab research goal: Address Datacenter management challenges • Management == efficient use of resources under dynamic demand, cost model, reliability – – Resources: Order(s) of magnitude more devices “Dynamic demand”: peaks 5 -10 X averages, provisioning hard “Reliability”: SW/HW failures commonplace “Software Churn”: Google search rewritten twice in last 5 years; Ebay rewritten 4 times in 9 years – “Dynamic cost model”: nonlinear cost models for power, cooling, network traffic Too large scale, too complex for manual administration by people use Statistical Machine Learning to tackle CS 98 -10/198 -10, Spr 08

Ro. R as Datacenter Programming Language • We believe Ro. R will have lasting impact – Tastefully chosen features in language & framework combine best ideas from programming language literature – Intellectual/academic value in reinforcing key ideas of programming structures – Bootstrap Ro. R expertise within Berkeley EECS • Exceptionally rapid prototyping of Web 2. 0 apps – Incremental/agile development support – Sophisticated UI support (Ajax, etc. ) – Rapid bootstrapping doesn’t require learning bad habits • Maximizes programmer productivity, which is increasingly a scarce resource CS 98 -10/198 -10, Spr 08

So, why this course? • We (researchers) are great at creating technologies, terrible at creating apps • You create popular apps, your friends & colleagues flock to them • We get apps/data to test our ideas • You get fame, etc. • Example: two recent RAD Lab MS grads. . . CS 98 -10/198 -10, Spr 08

CS 98/198 Development & Deployment Environment My. SQL Dev & Testing Mongrel, Webrick, or IIS Shared My. SQL instance Mongrel Capistrano Subversion repository Your laptop CS 98 -10/198 -10, Spr 08 RAD Lab/ Millennium cluster www. My. Vanity. URL. com Apache reverse proxy . . . Static content Internet Mongrel Apache reverse proxy

Toolz U Will Uze • • • Lab 0: install these items from class page Ro. R development environment Capistrano deployment tools An editor (emacs, vim. . . ) or IDE (Eclipse) Subversion client for source code management (with SSL & Kerberos support compiled in) • My. SQLAdmin or Cocoa. My. SQL database GUI • and signup for class forum on 5 um. rorclass. org CS 98 -10/198 -10, Spr 08

Movie time. . . • radlab. cs. berkeley. edu/wiki/Ro. R CS 98 -10/198 -10, Spr 08

Today: technical stuff review • Web Apps 101, from Web 1. 0 to Web 2. 0 – TCP/IP, HTTP, HTML, dynamic content (CGI) – Web 1. 0 vs Web 2. 0 – Model/view/controller • Lifecycle of a Web app • REST • Get Ro. R install, etc. running on your computer CS 98 -10/198 -10, Spr 08

The Web as remote procedure call • RPC protocol == HTTP – ASCII based request/reply protocol run over TCP/IP – protocol headers specify metadata about the request – Stateless: notion of “session” must be synthesized separately • RPC arguments == URL’s, HTML-form contents – URL names a server & resource – URL may embed “argument values”, or these can be “uploaded” as encoded HTML form submission – REST goes a long way toward cleaning up this sorry state browser CS 98 -10/198 -10, Spr 08 The Internet server

A conversation with a Web server • Browser opens TCP connection to server on port 80 (default) and sends: GET /index. html HTTP/1. 0 User-Agent: Mozilla/4. 73 [en] (X 11; U; Linux 2. 0. 35 i 686). . . other boring headers. . . Cookie: B=2 vsconq 5 p 0 h 2 n • Server replies: HTTP/1. 0 200 OK Content-Length: 16018 Content-Type: text/html <html><head><title>Yahoo!</title><base href=http: //www. yahoo. com/> …etc. • Repeat for any embedded images, scripts, . . . CS 98 -10/198 -10, Spr 08

HTML in one slide • Roughly hierarchical collection of elements that make up a viewable page – inline (headings, tables, lists. . . ) – embedded (images, video, Java applets, Java. Script code. . . ) – forms—allow user to submit simple input (text, radio/check buttons, dropdown menus. . . ) • Each element can have attributes (many optional) – of particular interest are id and class attributes – CSS (Cascading Style Sheets) allow specification of visual appearance of HTML pages based on the id’s and/or classes of elements • Old “styling” elements (like <b>, <i>, etc. ) are deprecated; use CSS instead, or where possible, logical styling like <emph>, <strong> • Current incarnation, XHTML, more device-portable by being strict about syntax that went to pot in HTML – Ro. R and many other frameworks generate XHTML-compliant code CS 98 -10/198 -10, Spr 08

XHTML/CSS in 1 slide • Declarative language to specify visual appearance of your HTML • Style attributes include font color/size, background color/picture, border colors, padding, spacing, justification/position. . . • Attributes can apply to specific elements, elements of a given class. . banner. Head { background-color: #883333; color: white; padding: 8 px; margin: 4 px; border: 2 px solid #d 3 bf 8 e; text-align: center; }. banner. Head a { color: white; text-decoration: none; } CS 98 -10/198 -10, Spr 08 pre { background-color: #eee; padding: 10 px; font-size: 11 px; }

Web 0. 9 and Web 1. 0 • Web <1. 0: web page == file (“static content”) – Web server (eg Apache) maps URL to file name • Web 1. 0: web page == output of running a program – URL or HTML form specifies name of program and parameters to pass to it, via CGI (Common Gateway Interface) API – example: http: //www. foo. com/search? term=white%20 rabbit&show=10&page=1 – App runs and generates HTML – Server passes HTML back to client via HTTP • How to identify same user returning? – – • Server gives Web browser a cookie on 1 st visit Browser passes cookie back to server on each request Server can modify cookie before returning it to browser Cookie typically used to look up session info in database or other store Various frameworks now capture this common structure CS 98 -10/198 -10, Spr 08 Filesystem or database mysql Ruby your app interp. CGI apache firefox

The MVC Design Pattern • Goal: separate organization of data (model) from UI & presentation (view) by introducing controller – mediates user actions requesting access to data – presents data for rendering by the view • Web apps are “sort of” MVC by design • User actions • Directives for rendering data . rb Controller (Ruby) code . rhtml template View (or. rjs, . rxml. . . ) • Data provided to views CS 98 -10/198 -10, Spr 08 • Read data • Update data SQL table + Model Ruby class

What is Ruby on Rails? • Ruby is a language that is. . . – dynamically typed, interpreted, object-oriented, functionally-inspired • Rails is a web application framework that. . . – embodies the MVC design pattern – emphasizes convention over configuration – leverages Ruby language features incl. dynamic typing, metaprogramming, & object-orientation to provide elegant support for both goals • Rails handles everything up to the point where your code is called • And everything past the point where your code delivers stuff to the user. CS 98 -10/198 -10, Spr 08

Web 1. 0 • Web 1. 0: user interaction == server roundtrip – Other than filling out form fields – Every user interaction causes server roundtrip – Every roundtrip causes full page redraw • Web 1. 5: user interactions without contacting server – e. g. form validation before submit – e. g. selecting something from menu A causes contents of menu B to change – But every roundtrip still causes full page redraw CS 98 -10/198 -10, Spr 08

Web 2. 0 • Separation of server roundtrip from page rendering – Initial load of page & draw page – User interaction causes backgroundtrip to server – Response from server “captured” and passed to a programmer-defined Java. Script function – That function can redraw part of the page in place (using same mechanisms as Web 1. 5) • Result: “desktop-like” responsive UI’s that can contact server – Auto completion – “Lazy” fetch of complicated parts of page CS 98 -10/198 -10, Spr 08 – etc

Managing Your Source • Source/configuration management (SCM) – a/k/a revision control, version control – history in Unix world: sccs => rcs => cvs => svn • Idea: take “snapshots” of code at different points – roll back to snapshot if screw up – everyone can bring their working copies up-to-date relative to some recent snapshot – everyone can contribute changes to new snapshot (supports multiple developers working on same tree) – “conflicts” detected at check-in time CS 98 -10/198 -10, Spr 08

Basic operations of source code management • First time: Check out a copy of current revision (or some previous revision) of code from repository • Subsequent times: Update your working copy from repository (to get latest changes by others) • Edit code, etc. • Commit your changes • Conflicts during commit? (if others modified same files as you) – identify bug in communication among developers – 1 person manually edits file to resolve conflict; marks file "resolved" and commits definitive version – others update their version from newly-committed file CS 98 -10/198 -10, Spr 08

Lifecycle of Web app My. SQL Dev & Testing Mongrel, Webrick, or IIS 4. deploy 2. edit & test CS 98 -10/198 -10, Spr 08 Shared My. SQL instance Mongrel Capistrano IDE or editor Your laptop migrate 1. Checkout or update 3. commit Apache reverse proxy Subversion repo- update sitory . . . Static content Internet Mongrel Apache reverse proxy

REST is a Software Architectural Style • What's software architecture? – subdivides complex systems into components – assigns each component specific responsibilities – dictates how components communicate and which ones communicate with which others – Models the information (domain) into entities and specifies how it will be transformed and flow through the application – State management CS 98 -10/198 -10, Spr 08

REST • REST: Representational State Transfer – Documents are transferred around – Application state is contained entirely in the documents – URI's name documents and actions to be performed on them – The definitive document on REST is Roy Fielding's dissertation: http: //www. ics. uci. edu/~fielding/pubs/dissertation/fieldin g_dissertation. pdf CS 98 -10/198 -10, Spr 08

Web Architecture • Network vs. Distributed – Distributed: A single system that distributes its work across multiple machines – Networked: A system that is utilized across a network • We are discussing networked architecture • A networked architecture can have a distributed architecture at the server CS 98 -10/198 -10, Spr 08

REST and CRUD • CRUD = basic operations on data objects (documents) • REST maps CRUD actions to HTTP methods (commands) CRUD HTTP Create POST Read GET Update PUT Delete DELETE CS 98 -10/198 -10, Spr 08

URI Refers to Resources • A resource is an object that can be operated upon • A URI will always refer to the same semantic resource – "customer with ID # 27" – "first order belonging to customer with ID # 27" • Resources are delivered in the manner best suited for the user agent – – HTML XML WML Etc… CS 98 -10/198 -10, Spr 08