Windows and UNIX Interoperability

Windows and UNIX Interoperability - tips, tricks, and secrets Peter Skjøtt Larsen Lead PM Microsoft Corporation

Vision In 5 years, most enterprises will have Windows and. NET central to their platform strategy for business computing

More than an operating system

Computing Reality There is a natural separation between: Server side rigorous expensive high performance computing Client side human-time intensive flexible modeling, planning, programming, etc. Most UNIX based scientific applications require a UNIX environment for client side activities. Code is written for X or use special libraries, scripting languages, UNIX text processing.

Windows Attraction Client Side Many sophisticated and volume priced data management apps available on Windows Person-time consuming aspects of scientific computing is conveniently done on laptops. Server Side Windows is being used increasingly for diverse server side workloads. Windows is now a highly scalable serious workload contender (great TPC results)

Client Options for UNIX code A number of alternatives exist today: Improved UNIX clients with better applications Better desktops apps for Linux, etc. UNIX like environments on Win 32 API Cygwin, uwin, mks UNIX emulation on Windows Kernel Microsoft Services for Unix Virtual Machines Microsoft Virtual Server Windows like environment on UNIX Wine

What Is Services For Unix? Interoperability NFS Client, Server, Gateway Password Synchronization NIS/Active Directory migration Portability for Unix… Scripts Applications Users Supportability Commercial MS product

All the comforts of home … Replaces Posix subsystem (in Windows) C Shell and Korn shell Single-rooted file system Symbolic links Win 32® programs Terminals and other devices Services and daemons Man pages X windows

Windows And SFU UNIX Applications Open Source tools: Apache, Windows Applications Motif Tcl/Tk, bash, etc. UNIX, XPG, POSIX. 2 commands & utilities U N I X X 11 UNIX shells X 11 R 6 Windows Applications server Windows GUI S D K Windows system admin, commands & networking (gcc) Windows command Shell SFU/Interix telnetd 3 rd Party UNIX /POSIX APIs BSD Sockets Windows APIs winsock Color Legend Win 32 Subsystem Interix Subsystem win 32 k. sys Windows Kernel Other device drivers CDFS FAT NTFS Hardware Abstraction Layer Client NFS Server Gateway

Infrastructure: NFS UNIX NFS Servers SFU NFS Clients SFU NFS Servers SFU NFS Gateway UNIX NFS Clients

Mapping Windows to UNIX Access request for NFS resource John Request Windows to UNIX mapping SFU NFS Client NFS access granted Return UNIX credentials Windows domain NFS request granted using UNIX credentials /home/john_smith NIS Server Users UNIX Machine NFS Server SFU User Name Mapping Server NFS request sent using UNIX credentials User ‘john_smith ’ in UNIX domain Windows user UNIX user John john_smith

NFS request NFS access Unix Machine II Mapping UNIX to Windows User ‘john_smith’ in Unix domain NFS Client NIS Client NFS request sent using Windows credentials SFU User Name Mapping Server User Name Mappings SFU NFS Server Windows domain Mapped Windows user’s access provided Authenticate Domain controller mapped with SFU Windows users [Active Directory®/ NIS Database]

Infrastructure: Password Synchronization Bidirectional - Change passwords from Windows NT® or UNIX Fine grained control - updates filtered by user name Password updates propagate to selected systems Passwords Triple-DES encrypted in transit Limited to users with identical names

Infrastructure: Printing Basic steps: On Unix system, define and configure a remote printer On Windows, install Print Services for Unix lp and lpr commands can be used Standard Unix printing protocols do no user authentication Secure by client IP address

BSD: /etc/printcap lp 1: : of=/usr/lbin/pcfof +Chp 5 mp. pcf: : sd=/usr/spool/lpd-hp 5 mp: : lp=: : rp=hp-5 mp: : rm=winsvr 1: of: filter driver (omit if unnecessary) sd: spool directory lp: printer device name (empty for network printers) rp: remote printer name (windows share name) rm: remote system name or IP address

Set up Windows printer Install and create a share in the usual way (e. g. \winsrv 1hp-5 mp) Install Windows Print Services for UNIX Add/remove | Windows Components | Other Network File and Print Services | Print Services For UNIX Enables server support for BSD/LPR printing Also enables printing from Windows to remote UNIX printers via lpr Check Services Manager to ensure “TCP/IP Print Server” (tcpsvcs) is running

SFU for the developer Command line environment Korn/Bourne shells, vi, more, etc. awk, grep, perl, sed, etc. at/cron/batch, rlogin, telnet, etc. Application SDK and runtime support POSIX. 1, POSIX. 2 (IEEE 1003, ISO 9945) Most of UNIX 98 gcc, rcs, make, etc. cc/c 89 tied to MSVC

What’s New in SFU 3. 5 Pthreads support Threading New synchronization primitives Internationalization support I 18 n APIs Limited localizability for some utilities X 11 R 6. 6 gcc 3. 3 Performance improvements

Cool Tricks with Threads Although Interix apps can’t call Win 32 APIs, they can call ntdll. dll WARNING! Microsoft doesn’t support direct use of the NT kernel interface However, there are several books on “Undocumented Windows NT Calls” If you create a separate thread and mask all Unix signals in that thread, making ntdll calls is relatively safe Test the heck out of this!

Now, Why Would You Do That? I/O to devices not exposed by Interix Use Nt. Open. File, Nt. Read. File, Nt. Write. File, Nt. Device. IOControl, etc. to talk to the device Using NT synchronization primitives Mutexes, critical sections, events

Flow of Control ux_thr nt_thr … Block all signals except SIGUSR 1 Set up request data in known mem sigpause() pthread_kill(nt_thr, SIGUSR 1) … Block signals; pick up request Block or do other work until SIGUSR 2 Make desired calls (do I/O, wait on is delivered synchronization object, etc. ) … pthread_kill(ux_thr, SIGUSR 2) … Signal handler for SIGUSR 2 invoked

Building Against ntdll. dll Can’t use Interix build tools Compilation trips over headers Can’t link Interix objects directly against ntdll Solution: Build your own DLL Contains just functions which call ntdll. dll Be sure you don’t touch kernel 32. dll Use VS to build and link; generate FOO. LIB file Interix code links against FOO. LIB file Rename to foo. a to make ld happy

But, I want to call Win 32 ! In a coexistence scenario it is possible that an application on the SFU side can benefit from calling functionality on the Windows side (and visa versa). Such cases must be carefully studied for the performance impact of marshalling and the extra development cost involved. Low volume communication between loosely coupled functionalities are good candidates.

Creating a Proxy DLL All DLL Functions are duplicated with stub code on the SFU side. All calls go through a marshalling mechanism to a DLL host on the Windows which loads the DLL and calls the function. Could be RPC SFU APP SFU Std. Libs Win DLL Host Proxy Lib Marshalling Lib Services for UNIX Marshalling Lib Win DLL Windows

Almost shared memory … A tighter coupling can be achieved with memory mapped files. SFU APP SFU Std. Libs Services for UNIX Win App Win Std. Libs Windows NTFS

Call Win 32 App from SFU, Easy! exec() will naturally invoke Win 32 applications and map stdin, stdout and stderr as expected. The shell will also seamlessly start Win 32 applications.

Call SFU App from Windows has access to SFU commands by invoking the SFU command directly. The Posix. exe command is designed to translate between path formats and invoke the SFU subsystem.

Porting: Satisfying Dependencies “configure” around missing libraries Search for latest versions of required libraries http: //google. com http: //freshmeat. net http: //interopsystems. com/tools Expect dependencies to have their own dependencies in turn

Correcting Compile Errors Use #ifdef _INTERIX / #endif Look for existing portability #ifdefs Most are really portability bugs Look for similar functions in POSIX, SUS Consider platform “assumptions” that aren’t guaranteed to be true everywhere Disable sections of code to get something running

Common Problems Missing functions or constants Check for _ALL_SOURCE Search for “standard” equivalent Missing ioctl constants Look for equivalents Missing functions at link time Check order of libraries in link command Functions may appear in different libraries Some core functions appear in libpsxdll. a instead of libc. a

Running Unix/Linux Applications in a Virtual Machine When is Virtualization Appropriate? No logical way to run app on Windows or SFU App and supporting libs exist on Linux

Managed Co-Existence with Virtual Server Shell Cmd & Util Shell UNIX API NT 4. 0 API Windows 2003 API UNIX Kernel NT 4. 0 Kernel Windows 2003 Kernel Virtual Server Hardware Abstraction Layer Virtual Server Gui X 11 Cmd & Util Windows APP NT 4. 0 APP UNIX APP

Virtualization Results Linux app runs in the Windows environment with integrated … User file store Security context Command execution environment Access Linux transparently from Windows Linux / UNIX apps run out of the box Performance acceptable for many classes of apps

UNIX Application Migration Guide ANSI C/C++ and Fortran Applications Architecture Differences of Unix and Windows Code Assessment Guidelines and Best Practices on: Prototyping Implementing Testing Code samples illustrate key concepts Freely available on MSDN http: //msdn. microsoft. com/library/en-us/dnucmg/html/ucmglp. asp Exposing The Best Known Practices From Field Experts

Unix Migration Guides

All Technology Options Making the move from UNIX to Windows easy Visual Studio &. NET Services for UNIX “Simplify evolving UNIX code to Windows &. NET” “Complete UNIX compatibility” q Code conversion tools q Full “real UNIX” scripting q UNIX tools & libraries q APIs/tools for Solaris / Linux compat q Make file conversion Guidance q Tight integration with Windows Migration guides Automation & tools Scenario focused Community Virtual PC & Virtual Server Community forum & support Customer contributions Shared tools “Run UNIX binary apps” q Minimal support for selected scenarios q Selected guidance q Respond to customer usage needs Interop with UNIX “Seamless integration with UNIX legacy” q Better interop during legacy transition q Support UNIX clients for Windows, SQL q Leverage Biz. Talk, SFU, . NET interop

More info … è http: //www. microsoft. com/windows 2000/migrate/unix Email … è migrate@microsoft. com è petela@microsoft. com

Windows and UNIX Interoperability — tips tricks and