Скачать презентацию Scalla Developer Summary xrootd cmsd Andrew Hanushevsky SLAC Скачать презентацию Scalla Developer Summary xrootd cmsd Andrew Hanushevsky SLAC

8b50d29f10d3ebcb239ed619de18bfa5.ppt

  • Количество слайдов: 34

Scalla Developer Summary xrootd /cmsd Andrew Hanushevsky SLAC National Accelerator Laboratory CERN Workshop 10 Scalla Developer Summary xrootd /cmsd Andrew Hanushevsky SLAC National Accelerator Laboratory CERN Workshop 10 -November-08 http: //xrootd. slac. stanford. edu

Supported Platforms Free. BSD n New addition (best-effort support only) Linux n i 386_linux Supported Platforms Free. BSD n New addition (best-effort support only) Linux n i 386_linux 24, i 386_linux 26, x 86_64_linux_26 Mac. OS n ppc_darwin_70, ppc_darwin_80, x 86_darwin_90 Solaris n sun 4 x_58, sun 4 x_59, sun 4 x_510, sunx 86_510 Windows n XP (client only) 10 -November-08 2: http: //xrootd. slac. stanford. edu

Supported Compilers g++ Up to version 4. 3. 2 n Used for Free. BSD, Supported Compilers g++ Up to version 4. 3. 2 n Used for Free. BSD, Linux, and Mac. OS n icc n Used for Itanium architectures n SGI Linux Sun CC n Always used for Solaris compilations 10 -November-08 3: http: //xrootd. slac. stanford. edu

Supported Build Environments In-House configure. classic Works on all platforms without problems n Missing Supported Build Environments In-House configure. classic Works on all platforms without problems n Missing some features n n Make install Autotools make Supported by Derek Feichtinger n Has some issues on certain platforms n n E. G. , n 10 -November-08 Unable to build Xrd. Posix. Preload. so 32/64 bit compilation/linking on 64/32 bit platforms 4: http: //xrootd. slac. stanford. edu

General Source Layout I XProtocol/ n cmsd and xrootd protocol data structures Xrd/ n General Source Layout I XProtocol/ n cmsd and xrootd protocol data structures Xrd/ n Protocol driver for cmsd and xrootd (networking and scheduling) Xrd. Acc/ n Default access control (i. e. , authorization) Xrd. Bwm/ n Bandwidth manager plug-in Xrd. CS 2/ n Castor plug-in (obsolete) Xrd. Client/ n Client related code (e. g. , TXNet. File, xrdcp, etc) Xrd. Cms/ n Cluster Management Services protocol plug-in Xrd. Cns/ n Cluster Name Space daemon and other components 10 -November-08 5: http: //xrootd. slac. stanford. edu

General Source Layout II Xrd. Crypto/ n Cryptography support classes for security plug-ins Xrd. General Source Layout II Xrd. Crypto/ n Cryptography support classes for security plug-ins Xrd. Mon/ n Monitor data collection agent Xrd. Net/ n All networking related classes (used by everyone) Xrd. Odc/ n Open Distributed Cluster support (deprecated) Xrd. Ofs/ n Open File System plug-in Xrd. Olb/ n Open Load Balancing protocol implementation (deprecated) 10 -November-08 6: http: //xrootd. slac. stanford. edu

General Source Layout III Xrd. Oss/ n Open Storage System plug-in Xrd. Ouc/ n General Source Layout III Xrd. Oss/ n Open Storage System plug-in Xrd. Ouc/ n Object Utility Classes (used by everyone) Xrd. Posix/ n Posix compatibility libraries n Includes preload library Xrd. Pss/ n Proxy plug-in n Used as a storage system plug-in 10 -November-08 7: http: //xrootd. slac. stanford. edu

General Source Layout IV Xrd. Sec/ n Authentication protocol driver plug-in Xrd. Secgsi/ n General Source Layout IV Xrd. Sec/ n Authentication protocol driver plug-in Xrd. Secgsi/ n Gsi-based authentication plug-ins Xrd. Seckrb 4/ n Kerberos IV authentication plug-in Xrd. Seckrb 5/ n Kerberos V authentication plug-in Xrd. Secpwd/ n Password authentication plug-in Xrd. Secsss/ n Simple Shared Secret authentication plug-in Xrd. Secunix/ n Unix (i. e. , NFS-like) authentication plug-in 10 -November-08 8: http: //xrootd. slac. stanford. edu

General Source Layout V Xrd. Sfs/ n File System Interface definition n Includes the General Source Layout V Xrd. Sfs/ n File System Interface definition n Includes the default implementation Xrd. Sut/ n Security utility classes Xrd. Sys/ n OS-dependent classes (used by everyone) Xrd. Token. Authz. Ofs/ n ALICE ofs plug-in implementing ALICE security Xrd. Version. hh n Holder to identify the version everywhere Xrd. Xr/ n Proxy plug-in (obsolete, to be deleted) Xrd. Xrootd/ n Xrootd protocol plug-in 10 -November-08 9: http: //xrootd. slac. stanford. edu

So, What’s A Plug-In? Run-Time Loadable Code n A single class based on an So, What’s A Plug-In? Run-Time Loadable Code n A single class based on an abstract interface n n Plus an extern ‘C’ object instantiator Always packaged as a shared library n Configuration file specifies location of library n n Automatically loaded Objects created as needed Extends xrootd or cmsd functionality n Avoids massive rebuilds and code branching n The base system uses pre-defined statically linked plug-ins 10 -November-08 10: http: //xrootd. slac. stanford. edu

Scalla Plugin Architecture authentication (gsi, krb 5, etc) Protocol Driver (Xrd) Protocol (1 of Scalla Plugin Architecture authentication (gsi, krb 5, etc) Protocol Driver (Xrd) Protocol (1 of n) (xrootd, cmsd) authorization lfn 2 pfn prefix encoding (name based) File System Storage System (ofs, sfs, alice, etc) (Flash drm/srm, etc) (oss, Based System) Clustering (cmsd) 10 -November-08 11: http: //xrootd. slac. stanford. edu

The Plug-Ins Dynamic plug-in always packaged as a shared library n Authentication plug-in n The Plug-Ins Dynamic plug-in always packaged as a shared library n Authentication plug-in n Authorization plug-in n n Available: lib. Xrd. Proxy. so (provides proxy access) Protocol plug-in n n Available: lib. Xrd. Ofs. so Storage System plug-in n n Currently, no plug-ins exist (default statically linked-in) File System plug-in n n Always as lib. Xrd. Sec. so Others as lib. Xrd. Secxxxx. so xxxx PROOF team uses a plug-in to implement PROOF protocol External Management Interface plug-in (XMI) n Used by Castor to interface cmsd to Castor name space 10 -November-08 12: http: //xrootd. slac. stanford. edu

Closer Look At Security Plug-Ins xrootd has no security n FALSE! By default, security Closer Look At Security Plug-Ins xrootd has no security n FALSE! By default, security is not enabled n This simplifies setup for most sites You must configure security to get it n Not difficult but yet another thoughtful step Once enabled, xrootd is as secure as you want n So, the myth is now busted! 10 -November-08 13: http: //xrootd. slac. stanford. edu

What Is Security? Two phases to security n Authentication n Done in the xrootd/cmsd What Is Security? Two phases to security n Authentication n Done in the xrootd/cmsd protocol layer n Multiple simultaneous authentication modes supported n Authorization n Done in the ofs filesystem layer n Basic user/group/netgroup capability list mode available n But, authorization is a plug-in n 10 -November-08 Can substitute any other mode you’d like 14: http: //xrootd. slac. stanford. edu

xrootd Authentication Multi-protocol design Server provides available protocols n Client chooses one of the xrootd Authentication Multi-protocol design Server provides available protocols n Client chooses one of the possibilities n Each protocol name comes with configuration data n n Allows client to self-configure for the protocol Each protocol implemented as a plug-in n Easy to add new protocols 10 -November-08 15: http: //xrootd. slac. stanford. edu

Currently Available Protocols gsi n Grid certificate based authentication n Extension gsi. GMAPLDAP maps Currently Available Protocols gsi n Grid certificate based authentication n Extension gsi. GMAPLDAP maps DN to username via LDAP krb 4 n Standard Kerberos IV krb 5 n Standard Kerberos V pwd n Hidden password authentication sss n Simple shared secret unix n Basic Unix NFS-like [non-]security 10 -November-08 16: http: //xrootd. slac. stanford. edu

Authentication Protocol Plug-Ins Each protocol is a shared library plug-in n lib. Xrd. Secxxxx. Authentication Protocol Plug-Ins Each protocol is a shared library plug-in n lib. Xrd. Secxxxx. so xxxx n The xxxx is the protocol name (e. g. , lib. Xrd. Seckrb 4. so) krb 4 The plug-ins are managed by lib. Xrd. Sec. so n Authentication protocol client/server driver n Finds n and loads appropriate shared libraries Uses abstract security interface for all interactions n Credential 10 -November-08 generation & authentication 17: http: //xrootd. slac. stanford. edu

Abstract Class Xrd. Sec • C++ framework to manage protocols as plug-ins • Generic Abstract Class Xrd. Sec • C++ framework to manage protocols as plug-ins • Generic protocol (Xrd. Sec. Protocol) class Xrd. Sec. Protocol { public: virtual int Authenticate( Xrd. Sec. Credentials *cred, Xrd. Sec. Parameters **parms, Xrd. Sec. Client. Name &client, Xrd. Ouc. Err. Info *einfo=0 ) virtual Xrd. Sec. Credentials *get. Credentials( Xrd. Sec. Parameters Xrd. Ouc. Err. Info // // In Out Out *parms=0, // In *einfo=0 ) // Out virtual const char *get. Parms( int &psize, const char *host ) }; server client // Out // In • Protocol implementations inherit from Xrd. Sec. Protocol 10 -November-08 18: http: //xrootd. slac. stanford. edu Courtesy of Gerri Ganis

lib. Xrd. Sec. so • lib. Xrd. Sec. so provides the Protocol Manager - lib. Xrd. Sec. so • lib. Xrd. Sec. so provides the Protocol Manager - Server: instantiated at start-up from configuration file: - load protocol plug-ins that server can / wants to run - binds (subsets of) the list to hosts or host patterns - controls authentication mode by host - Client: build-up list loading protocols the first time needed - library loaded only if authentication is required • Plug-in implementations provide a public instantiator to create an instance of the protocol • Simple negotiation: list of allowed protocols sent to the client, who chooses the one to try first Courtesy of Gerri Ganis 10 -November-08 19: http: //xrootd. slac. stanford. edu

Xrd. Sec Implementation Is Generic • Depends only on network and utility modules • Xrd. Sec Implementation Is Generic • Depends only on network and utility modules • Xrd. Net, Xrd. Ouc, and Xrd. Sys • Can be easily used in a non-XROOTD context • Working example of standalone client and server programs using Xrd. Sec available at http: //ganis. home. cern. ch/ganis/ROOT/SECURITY/test. Xrd. tgz Courtesy of Gerri Ganis 10 -November-08 20: http: //xrootd. slac. stanford. edu

Authentication Architecture 1 Get Credentials login Send client-specific security configuration (Select Protocol) 3 Send Authentication Architecture 1 Get Credentials login Send client-specific security configuration (Select Protocol) 3 Send credentials Ask for more Get Security Configuration 2 n Multiple exchanges allowed get credentials authenticate 0 Config File lib. Xrd. Secgsi. so lib. Xrd. Seckrb 4. so lib. Xrd. Seckrb 5. so lib. Xrd. Secpwd. so Dynamically selected by client Server specifies availability Libraries managed by lib. Xrd. Sec. so 10 -November-08 21: http: //xrootd. slac. stanford. edu lib. Xrd. Sec. so

Authenticated Identity Xrd. Sec. Entity. hh char char void int prot[8]; *name; *host; *vorg; Authenticated Identity Xrd. Sec. Entity. hh char char void int prot[8]; *name; *host; *vorg; *role; *grps; *endorsements; *tident; *cert clen; // // // Protocol used Entity's name Entity's host name Entity's virtual organization Entity's role Entity’s groups Protocol specific endorsements Trace identifier (do not modify) Pointer to certificate (future) Length of certificate (future) Passed to file system layer to be used for authorization 10 -November-08 22: http: //xrootd. slac. stanford. edu

Why Do It This Way? Can implement almost any model needed n Without changing Why Do It This Way? Can implement almost any model needed n Without changing any server/client code at all n Simplifies security audit procedures Can quickly evolve as requirements change n And support different modes for different moods n SLAC Atlas uses sss authentication n Fermi (a. k. a. GLAST) uses unix authentication n But generally many experiments don’t want anything n 10 -November-08 Only for reading but not if they are writing to xrootd 23: http: //xrootd. slac. stanford. edu

But Wait! Where is SSL? n SSL applies to the transport layer n The But Wait! Where is SSL? n SSL applies to the transport layer n The security framework applies to the protocol layer n Recall, any protocol can optionally use this framework n A transport protocol does not easily give you any options And SSH? n You really mean using ssh keys, don’t you? n No 10 -November-08 one wrote a plug-in for that yet 24: http: //xrootd. slac. stanford. edu

Let’s Recap xrootd/cmsd are all about plug-ins Expands applicability within the design focus n Let’s Recap xrootd/cmsd are all about plug-ins Expands applicability within the design focus n Allows you to piggy-back new functionality n n For instance, PROOF Are all points pluggable? n Just the obvious ones n There n is always room for improvement here E. G. , MSS plug-in now part of the oss plug-in So, let’s see talk about writing plug-ins 10 -November-08 25: http: //xrootd. slac. stanford. edu

Writing Plug-Ins I Read the documentation in the plug-in “hh” n Xrd. Acc. Authorize. Writing Plug-Ins I Read the documentation in the plug-in “hh” n Xrd. Acc. Authorize. hh n n Xrd. Cms. Xmi. hh n n Protocol abstract interface Xrd. Sec. Interface. hh n n Storage System Abstract Interface Xrd. Protocol. hh n n XMI abstract interface Xrd. Oss. hh n n Authorization abstract interface Authentication abstract interface Xrd. Sfs. Interface. hh n File system abstract interface Look at an existing plug-in of the same type 10 -November-08 26: http: //xrootd. slac. stanford. edu

Writing Plug-Ins II General things All plug-ins must be thread-safe n Avoid high-latency actions Writing Plug-Ins II General things All plug-ins must be thread-safe n Avoid high-latency actions n n This n generally causes pile-up/melt-down Usually because of thread starvation and timeouts n Some n n interfaces allow background processing High latency is not an issue then Use existing classes in the repository n lib. Crypto. a, lib. Crypto. Lite. a, n lib. Xrd. Net. a, lib. Xrd. Ouc. a, and lib. Xrd. Sys. a 10 -November-08 27: http: //xrootd. slac. stanford. edu

Writing Plug-Ins III For a plug-in to be included in the CVS repository n Writing Plug-Ins III For a plug-in to be included in the CVS repository n n Must follow naming conventions Adhere to the abstract interface n n Interface changes are rare and always backward compatible Be stand-alone n Cannot rely on frameworks and external add-ons n n This includes STL • A sore point but it’s saved us countless hours of debugging Have a good Makefile and Makefile. am Compile and run on all supported platforms Usually have detailed documentation n See existing references (e. g. , Security) n http: //xrootd. slac. stanford. edu/doc/sec_config. htm Otherwise, we can simply reference your plug-in web page 10 -November-08 28: http: //xrootd. slac. stanford. edu

Packaging Official releases are available via web site n http: //xrootd. slac. stanford. edu/ Packaging Official releases are available via web site n http: //xrootd. slac. stanford. edu/ n n We are working on getting a better more generic URL Currently, Wilko Kroeger cuts official releases n This does not preclude special integrated releases n n n ALICE Castor PROOF Root These usually suffer a little drift from the official release We are working on making the CVS available r/o n The fastest way is an AFS accessible directory n /afs/slac. stanford. edu/public/software/scalla n n CVS head checked out for viewing and a gtar file of the same Planning for a web interface to the repository 10 -November-08 29: http: //xrootd. slac. stanford. edu

Licensing & Contributions Currently, Scalla is under a BSD License n http: //www. opensource. Licensing & Contributions Currently, Scalla is under a BSD License n http: //www. opensource. org/licenses/bsd-license. php May change to Apache 2. 0 license n http: //www. opensource. org/licenses/apache 2. 0. php We may need to restrict how names are used n cmsd, Scalla, and xrootd n This is to prevent confusion relating to derivative works Contributions happily accepted n n Must conform to licensing requirements Handled in the Linux tradition 10 -November-08 30: http: //xrootd. slac. stanford. edu

SLAC Support Memo of understanding… The SLAC science program is heavily dependent on xrootd. SLAC Support Memo of understanding… The SLAC science program is heavily dependent on xrootd. I can therefore xrootd. assure you that xrootd will be maintained by SLAC for at least five years. As you are aware, xrootd is an open source product and will remain freely available. I believe that xrootd is brings valuable and currently unique capabilities in scalable high-volume data analysis. It is part of SLAC's mission to encourage wide use of developments like xrootd where they can benefit national and international science programs. SLAC staff supporting xrootd will be encouraged to examine, as time permits, problems and suggestions submitted by users who are not connected with the SLAC program. Non-trivial work in response to such submissions would require that work be also beneficial to SLAC's use of xrootd. I particularly encourage a collaborative approach to maintaining and developing products like xrootd. This approach promotes wide use, and xrootd. creates a situation where effort spent on issues raised by a collaborator would be considered valuable to SLAC by default. Richard Mount 10 -November-08 31: http: //xrootd. slac. stanford. edu

Active Developers Current Active Software Developers n Andreas Peters (Andreas. Joachim. Peters cern. ch) Active Developers Current Active Software Developers n Andreas Peters (Andreas. Joachim. Peters cern. ch) n n Andrew Hanushevsky (abh stanford. edu or Andrew. Bohdan. Hanushevsky cern. ch) n n Classic make, Cross-Platform issues, Security and PROOF Tofigh Azemoon (azemoon slac. stanford. edu) n n Client Gerardo Ganis (Gerardo. Ganis cern. ch) n n Autotools Fabrizio Furano (Fabrizio. Furano cern. ch) n n Free. BSD issues, Windows Derek Feichtinger (Derek. Feichtinger cern. ch) n n Server Bertrand Bellenot (Bertrand. Bellenot cern. ch) n n Castor/xrootd Monitoring Wilko Kroeger (wilko slac. stanford. edu) n MPS scripts, packaging, and release issues 10 -November-08 32: http: //xrootd. slac. stanford. edu

Getting Support Currently available venues n Official web site (always check there first) n Getting Support Currently available venues n Official web site (always check there first) n n General problem mailing list ([email protected] stanford. edu) n n http: //project-arda-dev. web. cern. ch/project-arda-dev/xrootd/site/index. html Request Tracker (RT) problem system at SLAC n n Must be subscribed (see http: //xrootd. slac. stanford. edu/xrootdlist. html) CERN Web Site n n http: //xrootd. slac. stanford. edu/ This is still experimental E-Mail the right developer n Actual bugs, contributions, and enhancement requests User support n An evolving issue for experiments and groups n n E. G. , OSG provides 1 st level support for VDT Developers generally cannot provide direct user support 10 -November-08 33: http: //xrootd. slac. stanford. edu

Acknowledgements Software Contributors n n CERN: Derek Feichtinger, Fabrizio Furano, Andreas Peters Fermi: Tony Acknowledgements Software Contributors n n CERN: Derek Feichtinger, Fabrizio Furano, Andreas Peters Fermi: Tony Johnson (Java) Root: Gerri Ganis, Bertrand Bellenot SLAC: Jacek Becla, Tofigh Azemoon, Wilko Kroeger Operational Collaborators n BNL, INFN, IN 2 P 3 Partial Funding n US Department of Energy n Contract DE-AC 02 -76 SF 00515 with 10 -November-08 Stanford University 34: http: //xrootd. slac. stanford. edu