Achieving Language Interoperability with Gary Kumfert with Bill

Achieving Language Interoperability with Gary Kumfert with Bill Bosl, Tammy Dahlgren, Tom Epperly, Scott Kohn, & Steve Smith

Babel’s Scope & CCA Compliant Frameworks Component Semantics Compilers & Linkers Operating System CASC l Babel provides language interoperability, not components. l We collaborate with CCA to add parallel distributed support l We also provide tools (Quorum & Alexandria) to facilitate component development and deployment GKK 2

Release Announcement l C, CASC C++, F 77, Python(client) GKK 3

babel-0. 5. 0. tar. gz l Babel code generator u written l Babel runtime library u written l Docs in Java in ANSI C (minimal) u papers, talks, javadoc html u babel 101 tutorial CASC GKK 4

Hand Coded Language Interoperability f 77 JNI Native SWIG Platform Dependent C f 90 C++ Python Java CASC GKK 5

Babel Enabled Language Interoperability l Truly Object Oriented l Reference Counting f 77 l Exception l RMI Handling (future) C f 90 C++ Python Java CASC GKK 6

Babel Enabled Language Interoperability What’s In This Release: f 77 C f 90 C++ Python Java CASC GKK 7

Babel Has Two Types of Customers Developers Users l Have a code l Want to increase their user base l Want l Will l May a problem learn SIDL l Want Babel general and powerful CASC to solve their problem never see SIDL l Want software that’s easy & trustworthy GKK 8

Babel’s Design Priorities l Performance l Developer/User dichotomy l What’s natural for each language? u Could expose C array structs in C++ u C++ style would be SIDL: : array<T> template <> array<item_t> : public array_mixin < array_t, item_cxx_wrapper_t> { } CASC GKK 9

SIDL (Scientific Interface Definition Language) l Builds on Industry version Hello 1. 0; IDL technology CORBA u COM u l Designed for Scientific Apps complex types u dynamic multidimensional arrays u CASC package Hello { class World { string get. Msg(); } } GKK 10

version My. Solver. Lib 0. 1. 0; import ESI; package My. Solver. Lib { interface Matrix. Generator {. . . } class Option. Database { void get. Option( in string name, out string val); } class Vector implements-all ESI. Vector { void set. Options( in Option. Database db ); } class Bizarre implements Matrix. Generator {. . . void set. Data( in array<dcomplex, 2> a ); } } CASC GKK 11

Many forms of language interoperable interfaces ma Hu e tibl pa m Co n SIDL C Scientific Interface Definition Language pil om e tibl pa m Co er IOR Internal Object Representation XML e. Xtensible Markup ible Language pat m Co eb W CASC GKK 12

XML enables. . . Type Descriptions on Shared Repositories . . . Browsing for Types CASC . . . automated creation via higher-level tools . . . automated search & discovery by advanced builders GKK 13

Language Interoperability: How Babel Makes it Work Application l Stubs Application: user’s LOP (Language Of Preference) If developer used Babel, they also have a SIDL file. IORs SIDL Skels Impls CASC l Implementation: developers LOP (Can be wrappers to legacy code) GKK 14

Language Interoperability: How Babel Makes it Work Application l Application: user’s LOP (Language Of Preference) Stubs l Client Side Stubs: user’s LOP to C IORs l Internal Object Representation (IOR): Always in C Skels l Server Side Skeletons: translates IOR (in C) to developer’s LOP Impls l Implementation: developers LOP (Can be wrappers to legacy code) CASC GKK 15

How Much Language Interoperability Have We Achieved? l Basic Types l Modes u bool u in l Extended Types u char u out l OO Method Dispatch u Objects u int u inout u regular u enumerations u long l Exception Handling u return u float final u arrays of basic types value l For All Combinations of Languages u double static u arrays of objects u. C C u fcomplex interfaces u (arrays are u C++ u dcomplex l multidimensional, no u classes u F 77 u string arrays of arrays) u opaque u Python CASC 1431 test cases (and counting) GKK 16

What we foresee, based on experience with our tests. . . Language Interoperability Developer Concerns: Configuration, Packaging, & Deployment User Concerns: Installation Trust CASC GKK 18

These aren’t new problems. . . But they are on a larger scale Developer Concerns: Configuration, Packaging, & Deployment User Concerns: Installation Trust CASC GKK 19

For Example. . . l To u support Python and Java All libraries must be shared (*. so) not statically linked (*. a) l C++ shared libraries are problematic Exception support is platform/compiler dependent u Linking issues when interoperating with other languages u l Can create valid shared library with uncatchable exceptions CASC GKK 21

Babel’s Configuration/Build l GNU Make l Autoconfiguration l Automake build Makefiles l Libtool shared libraries l CUTE l python’s own build system l java’s built-in (broken) make l helper scripts l fixes to autoconf, automake & libtool l lots of hacks custom testing CASC GKK 22

Test History sparc-sun-solaris 2. 7 -gcc babel 0. 5. x babel 0. 4. x IOR Rewrite # Test Cases Configure/Build/Repository faults, not software failures!!! Date CASC GKK 23

Problems affect the User too. . . l How does a user get and install “language interoperable” software? u Binary: if supplied by developer u Source: 4 Assume “configure; make install”? l How to link into application? u If any C++ code, must use C++ linker u Which C++ to use? C++ has no std binary interface CASC GKK 24

Crux of the problem l We’re building 21 st century C++ F 77 Python Java technology. . . C l. . . using 30 year old tools. Autoheader libtool (perl/sh) aclocal Automake (perl) Autoconf (M 4) Make Bourne shell CASC GKK 25

Solution l Integrated config, build, package, test and management system. u no make inside! 4 can have action create many files 4 understands directories u uses real database u program all aspects in one language u MUST BE OPEN SOURCE CASC GKK 26

In the mean time. . . l Babel works on other platforms, just not automated config/testing u Java code generator (precompiled) u ANSI C runtime library (no problem) l Babel’s tests are (necessarily) pathological worst-case examples l We didn’t create these problems, we just exercise them aggressively CASC GKK 27

Future Babel: Will Provide More Build Help l “babel. make” u currently lists code generated u may additional flags, macros, etc. l configure u currently used for regression tests u may generate artifacts useful for developers 4 helper scripts 4 warnings CASC GKK 28

Farther Future Babel: Will Do Distributed Computing Application Line Protocol Stubs Unmarshaler IORs Internet Marshaler Skels Line Protocol Impls CASC GKK 29

Closing Remarks l Babel Beta 0. 5 is released l Babel enables language interoperability u connect C, C++, F 77, and Python u provide a uniform object-model, even in non-OO languages. l Deploying & Installing Language Interoperable Code in General u is still very hard u has broken every CASC tool we use GKK 30

The End babel-announce@llnl. gov babel-users@llnl. gov http: //www. llnl. gov/CASC/components@llnl. gov Bill Bosl, Tammy Dahlgren, Tom Epperly, Scott Kohn, Gary Kumfert, & Steve Smith CASC GKK 31

A. 3. Can HPC and Component Technology REALISTICALLY be integrated? l Yes. l But HPC Components have huge (and unique) hurdles: u Diverse Architectures u Diverse OS’s u Integration of SPMD and Dist. Comp. u Archaic Pkg/Devel/Config/Build tools u Non-CS trained (or interested) users CASC GKK 32

B. 3. Can the HPC community really afford yet another compiler such as Babel? l Is language interoperability important? l How important? CASC GKK 33

B. 3. How is the Java subset of C++ inadequate as an HPC IDL? l What is a “Java subset of C++” ? l How does one u use it to bind to other languages? u get a common inheritance model? u get a common exception model? CASC GKK 34

B. 5. What is the role of traditional (parallel) tools in component technology? l Hopefully, they’re replaced by modern parallel tools. CASC GKK 35

B. 9. What will be the configuration issues for components. . . to be portable and high-performance? l Lots. l Lack of Configuration, Packaging, & Deployment tools u is the #1 Achilles heel for components u is the #1 day-to-day pain in Babel development u #1 cause for failure in regression tests CASC GKK 36

C. 1. Will anyone actually make the effort to componentize their applications software? l Yes. u But it will be messy. l Efforts to Babelize at LLNL: u hypre - want OOP in ANSI C & automatic F 77 bindings u ALPS - want scripting interface for laser plasma physics u SAMRAI - framework used in ALPS CASC GKK 37

D. 4. Should components be viewed as mostly a library/runtime developer technology? l No. l. I used components in this Power. Point Presentation l Users can use components without knowing they’re using them. u This is harder to achieve in UNIX than other platforms CASC GKK 38

UCRL-PRES-144649 23 Jul 2001 Work performed under the auspices of the U. S. Department of Energy by the University of California, Lawrence Livermore National Laboratory under Contract W-7405 -Eng-48 CASC GKK 39