Platform Quality APIs Jim des Rivières IBM OTI

Outline of Talk § Platform quality APIs § Long-haul APIs to support open-ended set of clients § Best Practices for API Design § Design advice for good APIs § API Evolution § Changing APIs while keeping existing clients happy § Best Practices for API Development § How to work effectively on components with APIs 2 © 2004 IBM Corporation

Topic – Platform quality APIs § Platforms are… § Useful § Platform provides useful services to clients § Open § Platform intended for open-ended set of diverse clients § Stable § Platform provides stability across multiple releases § Growing § Platform evolves from release to better meet client needs 3 © 2004 IBM Corporation

Successful platforms § Examples of successful platforms § Win 32 § Intel x 86 § IBM 360 § J 2 SE § Mac OS § Successful platforms last a long time § They don’t break existing clients – compatible evolution § They keep existing binaries working – binary compatibility 4 © 2004 IBM Corporation

Platform quality APIs § APIs just among friends (adjacent components in same release) § Unsupported for arbitrary clients § Don’t need to be stable from release-to-release § You get chance to correct your mistakes § Stark contrast to platform APIs § Platform APIs must support open-ended set of clients and be stable § Once API is released there’s no going back § Breaking changes are bad news § Either extend API in next release § Or provide improved API alongside old API 5 © 2004 IBM Corporation

Platform quality APIs are unforgiving § One chance § To tell clients how to use API – the “API specs” § To choose helpful class and method names § To decide between class or interface § Once API is in service § Constrained to work within framework laid down § Repair and improve without breaking clients § Unclear, inadequate, or incorrect API specs are bad news § Confusing to clients § Fixing API specs may make things worse 6 © 2004 IBM Corporation

Good specs are key to platform quality APIs § Good APIs needs good specs § Code is not enough (even source code) § API specs § Describe intended behavior of API element § How it’s supposed to work § How clients should behave to ensure they will not be broken by future changes § Client code written to spec § Every reason to believe it will work with any conforming implementation § Implementation code written to spec § Every reason to believe it will support all conforming clients § Best (&only) known recipe for long-term survival § Allows API to endure, evolve, and satisfy clients that build upon it 7 © 2004 IBM Corporation

Topic – Best Practices for API Design § Design advice for good APIs 8

Clearly separate API from internals § APIs should be highly visible so that clients can find § Publish API specs § Draw clear distinction between API and non-API § Eclipse non-API packages “internal”; e. g. , org. eclipse. ui. internal § API consists of public classes and interfaces in API packages § All interface members § Public and protected members of classes § No “visible” connection from API to non-API § API should be closed story § Implementation details should not leak into API 9 © 2004 IBM Corporation

Separate Core from UI § “Core” component § Provides basic infrastructure that can be used in headless manner § Usually provides API § May depend on other core components § Must not depend directly or indirectly on UI components § “UI” component § Provides elements or frameworks for use in GUI § May have API (but often not) § May depend on other UI components § May depend on core components § All inter-component dependencies are via APIs 10 © 2004 IBM Corporation

Segregate client API from provider API § Component often provide APIs for different parties § Client APIs § Used by bread-and-butter clients, like a UI § Provider APIs (aka SPIs) § Used only for plugging in new pieces § Keep different kinds of APIs segregated § Client API should be easy for clients § SPI should be flexible, powerful for service provider § Clients shouldn’t have to understand SPI too § Clients shouldn’t be able to meddle in SP affairs 11 © 2004 IBM Corporation

Use Java classes and interfaces wisely § API interfaces that clients do not implement are useful § Excellent way to insulate client from implementation internals § Allows there to be multiple implementations § Adding methods in future is non-breaking § API interfaces that clients may implement are inflexible § Adding method breaks binary compatibility § Abstract classes work better for this § API classes that clients do not subclass are useful § Use factory methods and hide constructors § API classes that clients may subclass are useful, but difficult § API frameworks are notoriously hard to specify § Needs additional contracts for subclassers 12 © 2004 IBM Corporation

Use Java modifiers to block unintended usage § If API class is not intended to be subclassed by clients § Declare class final, or make constructors package-private § If API class is not intended to be instantiated by clients § Declare constructors private § If API method is not intended to be overridden by client subclasses § Declare method final § If API class is intended to be subclassed but not instantiated by clients § Declare class abstract § Declare constructors protected § Use “soft” injunctions in specs if there’s no way to block § “This interface is not intended to be implemented by clients. ” 13 © 2004 IBM Corporation

Factory methods are better than constructors § Constructors are limiting as API § All constructors effectively have same name § Cannot have two with same type parameters § public Path(String platform. Neutral. String) § public Path(String os. Specific. String) § Constructors cannot return instance of subclass § Factory methods are more flexible than constructors § Factory method have names § public static Path new. Path(String platform. Specific. String) § public static Path new. OSPath(String os. Specific. String) § Factory method can return instance of subclass § Keep constructors hidden (private, package-private, or protected) 14 © 2004 IBM Corporation

Methods are better than fields § API fields are very weak § public String host. Name; § No way to intercept field accesses either read or write § No lazy init on first read § No possibility to report changes § Non-final fields vulnerable to overwrite § More flexible to expose get/set API methods § public final String get. Host. Name(); § public final void set. Host. Name(String host. Name); § Use fields in API only for constants (public static final) 15 © 2004 IBM Corporation

Program extra defensively at API boundary § Clients do the strangest things § Usually accidentally, but sometimes intentionally § Check incoming arguments are to spec § Don’t ignore - throw some form of Runtime. Exception § In Eclipse null argument is illegal unless explicitly allowed by spec § Worry about argument capture § What are consequences if client later modifies object being captured? § Consider copying arrays and collections § Worry about result disclosure § What are consequences if client later modifies object being return? § Consider copying arrays, returning unmodifiable collections § Worry about threading, or reentrancy § Are you holding locks when you issue that callback? 16 © 2004 IBM Corporation

Conclusion § No one can really tell you how to design good APIs § Any more than they can really tell you how to write good programs § API design is just a form of high-stakes program design § Talk to other programmers § Ask how they’d approach it § Study successful APIs § APIs are generally available and well-documented § Recommended reading § Effective Java Programming Language Guide, by Josh Bloch http: //java. sun. com/docs/books/effective/ § How to Design a (module) API http: //openide. netbeans. org/nonav/tutorial/api-design. html 17 © 2004 IBM Corporation

Topic – API Evolution § APIs need to evolve from release to release § Changes to API could invalidate existing clients § Evolve API in compatible ways § Preserve as much value as possible across API changes § Keep existing clients working § Two general considerations § Contract compatibility – Honor existing API contracts § Binary compatibility – Keeping the JVM happy 18 © 2004 IBM Corporation

Contract Compatibility Before /** Returns the non-empty list of indices. */ public int[] get. Indices(); After /** Returns the list of indices. The list may be empty. */ public int[] get. Indices(); Breaks some existing callers int[] d = get. Indices(); System. print(d[0]); // possible array index out of bounds However, existing implementers are fine public int[] get. Indices() { …; return result; // result is non-empty } 19 © 2004 IBM Corporation

Evolving API Contracts § API contracts are expressed in API specs § API contracts promise the client certain things § Clients can play multiple roles – e. g. , caller, implementer § Different roles have different contracts § Changes to contracts should not invalidate existing clients 20 © 2004 IBM Corporation

Binary Compatibility Before public void register(String key); After public void register(Object key); Existing calls re-compile as expected register(“foo”); // no compile error But existing binaries no longer link register(“foo”); // link error 21 © 2004 IBM Corporation

Binary Compatibility DON’Ts for API Elements 1. 2. 3. 4. 5. 6. 7. 8. 9. Rename a package, class, method, or field Delete a package, class, method, or field Decrease visibility (change public to non-public) Add or delete method parameters Change type of a method parameter Add or delete checked exceptions to a method Change return type of a method Change type of a field Change value of a compile-time constant field 10. Change an instance method to/from a static method 11. Change an instance field to/from a static field 12. Change a class to/from an interface 13. Make a class final (if clients may subclass) 14. Make a class abstract (if clients may subclass) … 22 © 2004 IBM Corporation

Binary Compatibility DOs for API Elements 1. 2. 3. 4. 5. 6. 7. 8. 9. Add packages, classes, and interfaces Change body of a method Do anything you want with non-API elements Add fields and type members to classes and interfaces Add methods to classes (if clients cannot subclass) Add methods to interfaces (if clients cannot implement) Add non-abstract methods to classes (if clients may implement) Reorder class and interface member declarations Change value of a field (if not compile-time constant) 10. Move a method up to a superclass 11. Make a final class non-final 12. Make an abstract class non-abstract 13. Change name of method formal parameter … 23 © 2004 IBM Corporation

Binary Compatibility § Java VM has special rules for binary compatibility § API changes should be binary compatible § Existing clients should continue to work without recompiling § N. B. Java compiler does not detect this kind of breakage § Ref: Evolving Java-based APIs http: //eclipse. org/eclipse/development/java-api-evolution. html 24 © 2004 IBM Corporation

Adding Methods to API Interfaces § API interfaces used to hide implementation work well § “/** … This interface is not intended to be implemented by clients */” § Add new methods to API interface § Add corresponding methods to implementing class package org. eclipse. core. resource; /** … This interface is not intended to be implemented by clients */ public interface IWorkspace { boolean is. Tree. Locked(); // new … } package org. eclipse. core. internal. resource; publicclass Workspace implements IWorkspace { boolean is. Tree. Locked() {…} … } 25 © 2004 IBM Corporation

Avoid API Interfaces that Clients May Implement § API interfaces that clients may implement are problematic § Adding method breaks binary compatibility § Use API class instead of API interface… § When client may implement § When there is a chance new methods needed in future § N. B. converting interface to class breaks binary compatibility 26 © 2004 IBM Corporation

Adding Methods via I*2 Extension Interfaces § If no choice, add new methods in extending API interface § Avoids breaking existing clients that implement package org. eclipse. ui; public interface IAction. Delegate { … } // original interface public interface IAction. Delegate 2 extends IAction. Delegate { void dispose(); // new } Usage IAction. Delegate d = new IAction. Delegate 2() {…}; if (d instanceof IAction. Delegate 2) { IAction. Delegate 2 d 2 = (IAction. Delegate 2) d; d 2. dispose(); // call new method } 27 © 2004 IBM Corporation

How to Delete API § API deletion always breaks any existing clients § But

Replacing API Methods § Add replacement API method § Deprecate original method § Ensure original method continues to work package org. eclipse. jdt. core. dom; public class Message { … /** … * @deprecated Use get. Start. Position() instead */ public int get. Source. Position() { // rename get. Start. Position() return get. Start. Position(); // forward to new method … } } public int get. Start. Position() { … } } 29 © 2004 IBM Corporation

API Evolution – Summary § Evolve API in compatible ways § Honor existing API contracts § Observe technical rules for Java binary compatibility § Usually feasible to find way to improve API and keep existing clients working without recompiling § Design APIs with future evolution in mind 30 © 2004 IBM Corporation

Topic – Best Practices for API Development § Good APIs don’t just appear overnight § Significant design effort § Good APIs require design iteration § Feedback loop involving clients and implementers § Improve API over time § Components build upon the APIs of other components § Need collaborative working relationship between teams § Some ways to work effectively on components with APIs, based on Eclipse Project experience 31 © 2004 IBM Corporation

Before you begin § Have and agree on common API guidelines § Eclipse Naming conventions http: //dev. eclipse. org/naming. html § How to Use the Eclipse API http: //www. eclipse. org/articles/Article-API%20 use/eclipse-api-usagerules. html § Have someone in charge of the API early on § Have well-defined component boundaries and dependencies § core vs. UI 32 © 2004 IBM Corporation

Work with API clients § APIs exist to serve the needs of clients § Where are those clients? What do they need? § Important to work with actual clients when designing API § Designing APIs requires feedback from real clients who will use it § Risks crummy API that real clients cannot use § Find a primary client § Ideally: adjacent component, different team, same release § E. g. , JDT UI is primary client of JDT Core § Work closely with primary client § Listen to their problems with using API § Watch out for lots of utility classes in client code symptomatic of mismatch between API and what client really needs § Work together to find solutions 33 © 2004 IBM Corporation

APIs First § Basic “APIs First” workflow 1. Work with primary client to decide what API you want/need (design API) 2. Write API specs 3. Write API test suites 4. Implement API (Expect numerous iterations within basic workflow) § Helps ensure APIs are § Ready for clients to use (specs and impl. in place) § Of high quality and stable § Ready to evolve to meet next requirement 34 © 2004 IBM Corporation

Don’t reveal API too early § “We shall ship no APIs before its time” (to paraphrase Orson Welles' old wine commercial) § Keep work in internal packages until new API is ready § API specs in place § API test suite § Credible implementation § When ready, release by moving to API package § Keeps everyone’s expectations realistic § Clients § Component owner § Avoids embarrassing recall of unfinished API just before shipping 35 © 2004 IBM Corporation

Write comprehensive API unit tests § Unit tests for each API element § Tests written “to spec” § Assumes and tests only what is covered in spec § Plays role of idealized client § Helps you to keep client view of the API § Implementer’s safety net § Catches stupid mistakes before they can screw over clients § Helps working relationship with primary client § Core components § Complete API coverage by automated tests § UI components § Partial coverage by automated tests § May required additional manual tests for visual elements 36 © 2004 IBM Corporation

Keep API specs consistent at all times § You write a story. Someone else changes the story. How do you know whether the story still reads well? § Errors and inconsistencies within specs are hard to detect/remove § API spec is more like a book than like code § Maintaining consistency require re-reading § Start with initial set of consistent API specs § Scrupulously maintain consistency as API spec grows/changes § Have a “Keeper of the Specs” § Responsible long-term for maintaining specs and reviewing spec changes § Requires thorough knowledge of API and how it got its spots 37 © 2004 IBM Corporation

Keep API tests up to date at all times § Errors and inconsistencies within tests are found immediately § But insufficient test coverage is hard to detect/remove § Start with a set of tests that completely cover API § Scrupulously maintain tests as API spec grows/changes § Can be done as part and parcel of changing API § Or by “Keeper of the Specs” if they keep the tests as well 38 © 2004 IBM Corporation

Funnel all external dependencies thru API § Component provides API for arbitrary clients § API exists to tame inter-component coupling § Client components are expected to use API “to spec” § Not depend on behavior not covered in API spec § Not depend on internals § Foolish to make exceptions for close friends § Close friends don’t point out flaws § Sets bad example for others § Most common form of Eclipse component is single plug-in § Internal packages have “. internal. ” in name § Plug-ins should not reference internal packages of other plug-ins 39 © 2004 IBM Corporation

Focus clients on API § Encourage clients to discuss/propose API changes in terms of existing API § Grounds discussion in reality § API owner ultimately decides how best to address issue § API needs to make sense for all clients § Final decision may differ for what was proposed 40 © 2004 IBM Corporation

Tread carefully in vicinity of APIs § Be especially careful when working on code in component with an API § Adding/deleting public method § internal class - no problem § API class - changes API § Be careful with renaming/refactoring tools § Make sure APIs not affected 41 © 2004 IBM Corporation

Tag API elements with @since § Scrupulously record which release API element first appeared in § In Javadoc with @since tag § @since 3. 1 § Elsewhere in words § During release cycle § Distinguishes old API from new API added during current release cycle § Still flexibility to change (or delete) newly added API § After release has shipped § Help clients targeting release >= N to avoid API added after N 42 © 2004 IBM Corporation

Minimize disruptive API changes § Breaking API changes are very disruptive to clients § Non-breaking API changes also cause work for clients § Work required to port to use new/improved APIs § During release cycle § Schedule API work for early in cycle § Whenever possible, preserve API contracts § When not possible, coordinate with clients to arrange cut-over plan § After release has shipped § Evolve APIs preserving API contract and binary compatibility 43 © 2004 IBM Corporation

Replace, deprecate, and forward § Add replacement API in non-breaking way § Deprecate old API but keep it working § @deprecated This method has been replaced by {@link #bar()}. § For new API added earlier in release cycle § Negotiate a short fuse on deleting deprecated stuff § Make sure clean up happens well before end of release cycle § Schedule API review & polish before too late to fix § Reviews often uncover inconsistencies § For API added in earlier release § Deprecation warnings inform of better way § API contract and binary compatibility preserved 44 © 2004 IBM Corporation

Outgoing API changes for a component § Component team works in HEAD stream § Other component teams compile and run against version as of latest (weekly) integration build § Keep HEAD compiling and working cleanly § Do not release changes that would hose other team members § Re-run all unit tests before releasing § Outgoing API changes require coordination with downstream teams § Post “preview version” of component a few days before I-build § Allows downstream components to coordinate their changes § Use nightly builds to reduce risk for a broken integration build 45 © 2004 IBM Corporation

Incoming API changes affecting a component § For dependent components, use version as of latest (weekly) integration build § Team members upgrade to latest I-build each week § Laggards slow the team down § Puts premium on everyone making each I-build better than last § Incoming API changes from upstream teams § Receive “preview version” of component a few days before I-build § Allows component to prepare coordinated changes for next I-build 46 © 2004 IBM Corporation

Staging large changes to existing APIs § E. g. , JDT Core support for new J 2 SE 5 language features § Work done in stages, over several months § Coordinated with primary client § Parcels of API changes with stub implementations § throw new Runtime. Exception(“Not implemented yet”); § Allows clients to write and compile code (but not run/test) § Implementation of API parcels done later § Clients can run/test code as soon as API is implemented 47 © 2004 IBM Corporation

API-related Resources § How to Use the Eclipse API, by Jim des Rivieres http: //www. eclipse. org/articles/Article-API%20 use/eclipse-api-usagerules. html § Effective Java Programming Language Guide, by Josh Bloch http: //java. sun. com/docs/books/effective/ § Requirements for Writing Java API Specifications http: //java. sun. com/products/jdk/javadoc/writingapispecs/index. html § How to Write Doc Comments for the Javadoc Tool http: //java. sun. com/products/jdk/javadoc/writingdoccomments/index. html § Evolving Java-based APIs http: //eclipse. org/eclipse/development/java-api-evolution. html § Contributing to Eclipse, by Erich Gamma and Kent Beck http: //www. aw-bc. com/catalog/academic/product/0, 4096, 0321205758, 00. html § Internal Tool (reports cross-plug-in references to internals) http: //dev. eclipse. org/viewcvs/index. cgi/%7 Echeckout%7 E/jdt-corehome/tools/internal/index. html § How to Design a (module) API http: //openide. netbeans. org/nonav/tutorial/api-design. html 48 © 2004 IBM Corporation