Java New I O Ron Hitchens ron ronsoft com http

Java New I/O Ron Hitchens ron@ronsoft. com http: //www. ronsoft. com Java NIO Book Website http: //javanio. info Sand Diego Java User's Group May 20, 2003 © 2003, Ronsoft Technologies

Pig-Footed Bandicoot See http: //javanio. info for details

Check Your Local Bookstore

Speaker Bio 25+ Years Computer Experience Mainframe to Micro Unix/Linux kernel, academic, internet ~20 years 5+ Years Java Experience Heavy server-side, web apps, startups Independent Consultant “Bit twiddling at it's finest” Will hack NIO for fun and profit Dot Com Survivor Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Why NIO? Where did it come from? What does it do for me? When should I use it? Should I stop using java. io? Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Efficiency – Scalability – Reliability Efficiency – The Need For Speed Why should the JVM do what the OS can do better? Scalability – Livin' Large Big applications have big appetites Reliability – Enough Wheels Invented The infrastructure exists – concentrate on the app No Longer CPU Bound JSR 51 (http: //www. jcp. org/en/jsr/detail? id=51) Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

What Does NIO Do For Me? New Abstractions Buffers, Channels and Selectors New Capabilities Non-Blocking Sockets File Locking Memory Mapping Readiness Selection Regular Expressions Pluggable Charset Transcoders Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Use NIO When You Need To: Move large amounts of data efficiently NIO is primarily block oriented – java. io uses streams Uses direct buffers to do raw I/O – bypassing the JVM Multiplex large numbers of open sockets Operates in non-blocking mode One thread can manage huge numbers of socket channels Use OS-level file locking or memory mapping Do character set Transcoding Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Should I Stop Using java. io? Nope java. nio is not a replacement for java. io NIO addresses different needs java. io is not going away Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

What Makes Up NIO? Buffers Channels Selectors Regular Expressions Character Set Coding Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

NIO Buffers Fixed size containers of primitive data types Byte. Buffer, Char. Buffer, Float. Buffer, etc. Byte buffers are special, used for I/O with channels Direct and Non-direct Byte. Buffers Direct Byte. Buffers address raw memory – direct I/O Buffers can be views of other buffers or wrap arrays Byte order (endian-ness) Affects byte swabbing in views of Byte. Buffers Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Buffer Classes Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Buffer Objects (Empty/Fill) Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Buffer Objects (Flip) Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Buffer Views (Dupe/Slice) Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Buffer Views (Char View) Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

I'm Confused. . . Show Me Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Hello? public class Hello. World { public static void main (String [] argv) { System. out. println ("Hello World"); } } Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Hello NIO? import java. nio. Byte. Buffer; import java. nio. channels. Writable. Byte. Channel; import java. nio. channels. Channels; public class Hello. World. Nio { public static void main (String [] argv) throws Exception { String hello = "Hello World" + System. get. Property ("line. separator"); Byte. Buffer bb = Byte. Buffer. wrap (hello. get. Bytes ("UTF-8")); Writable. Byte. Channel wbc = Channels. new. Channel (System. out); } } wbc. write (bb); wbc. close(); Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

NIO Channels New I/O metaphor: Conduit to an I/O service (“nexus”) Channels do bulk data transfers to and from buffers channel. write (buffer) channel. read (buffer) ~= ~= buffer. get (byte. Array) buffer. put (byte. Array) Scatter/gather, channel-to-channel transfers Three primary channel implementations File. Channel: File locks, memory mapping, cross-connect transfers Sockets: Non-blocking, selectable, async connections, peers Pipe: loopback channel pair, selectable, generic channels Selectable Channel Implementations are pluggable (SPI) Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Channel Copy – Simple #1* public void channel. Copy (Readable. Byte. Channel src, Writable. Byte. Channel dest) throws IOException { Byte. Buffer buffer = Byte. Buffer. allocate (16 * 1024); while (src. read (buffer) != -1) { // prepare the buffer to be drained buffer. flip(); // make sure the buffer was fully drained. while (buffer. has. Remaining()) { dest. write (buffer); } } } // make the buffer empty, ready for filling buffer. clear(); * No buffer copies, but potentially more system calls. Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Channel Copy – Simple #2* public void channel. Copy (Readable. Byte. Channel src, Writable. Byte. Channel dest) throws IOException { Byte. Buffer buffer = Byte. Buffer. allocate (16 * 1024); while (src. read (buffer) != -1) { // prepare the buffer to be drained buffer. flip(); // write to the channel, may block dest. write (buffer); } // if partial transfer, shift remaining elements down // if buffer was empty, same as doing clear buffer. compact(); buffer. flip(); } // EOF leaves buffer in fill state while (buffer. has. Remaining()) { dest. write (buffer); } * Minimal system calls, but may do buffer copies. Post loop cleanup needed. Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Channel Copy – Transfer* public void channel. Copy (File. Channel src, Writable. Byte. Channel dest) throws IOException { src. transfer. To (0, src. size(), dest); } public void channel. Copy (Readable. Byte. Channel src, File. Channel dest) throws IOException { dest. transfer. From (src, 0, Long. MAX_VALUE); } * Very easy, but one end must always be a File. Channel. Transfer could occur entirely in kernel space. Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Memory Mapped Buffers Random. Access. File raf = new Random. Access. File (file. Name, "rw"); File. Channel fc = raf. get. Channel(); Mapped. Byte. Buffer buffer = fc. map (File. Channel. Map. Mode. READ_WRITE, 0, fc. size()); byte b = buffer. get(); . . . buffer. put (some. Other. Byte); // reads from file // writes to file The content of buffer is the content of file. Name Any change to one affects the other Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Non-Blocking Sockets – Simple Really Byte. Buffer buffer = Byte. Buffer. allocate (1024); Socket. Channel socket. Channel = Socket. Channel. open(); socket. Channel. configure. Blocking (false); . . . while (true) {. . . if (socket. Channel. read (buffer) != 0) { process. Input (buffer); }. . . } Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Non-Blocking Server Socket Server. Socket. Channel ssc = Server. Socket. Channel. open(); ssc. socket(). bind (new Inet. Socket. Address (port)); ssc. configure. Blocking (false); while (true) { Socket. Channel new. Connection = ssc. accept(); } if (new. Connection == null) { do. Something. To. Keep. Busy(); } else { do. Something. With. Socket (new. Connection); } Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

NIO Selectors Multiplexing Channels – Readiness Selection Selectable Channels are registered with Selectors Selection. Key encapsulates selector/channel relationship A subset of ready channels is selected from the Selector's set of registered channels (Selector. select()) Selected Set contains those keys with non-empty Ready Sets Each Selection. Key holds an Interest Set and a Ready Set Possible members of Interest Set: accept, read, write, connect Ready set is a subset of interest set –as-of the last select() call Readiness Selection means less work – ignore idle channels Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Selectors, Keys and Channels Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Registering With a Selector Server. Socket. Channel server. Channel = Server. Socket. Channel. open(); Selector selector = Selector. open(); server. Channel. socket(). bind (new Inet. Socket. Address (port)); server. Channel. configure. Blocking (false); server. Channel. register (selector, Selection. Key. OP_ACCEPT); Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

The Selection Process Create a Selector and register channels with it The register() method is on Selectable. Channel, not Selector Invoke select() on the Selector object Retrieve the Selected Set of keys from the Selector Selected set: Registered keys with non-empty Ready Sets keys = selector. selected. Keys() Iterate over the Selected Set Check each key's Ready Set (set of operations ready to go as-of last select()) Remove the key from the Selected Set (iterator. remove()) Bits in the Ready Sets are never reset while the key is in the Selected Set The Selector never removes keys from the Selected Set – you must do so Service the channel (key. channel()) as appropriate (read, write, etc) Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Running a Selection Loop while (true) { selector. select(); Iterator it = selector. selected. Keys(). iterator(); while (it. has. Next()) { Selection. Key key = (Selection. Key) it. next(); it. remove(); if (key. is. Acceptable()) { Server. Socket. Channel server = (Server. Socket. Channel) key. channel(); Socket. Channel channel = server. accept(); channel. configure. Blocking (false); channel. register (selector, Selection. Key. OP_READ); } } } if (key. is. Readable()) read. Data. From. Socket (key); Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Scalability With Selectors One Thread to Rule Them All More threads != More Efficient – Context Switching, CPU Availability OS and/or JVM do the hard work for you Only the kernel can efficiently do Readiness Selection No more thread-per-socket nonsense Simpler, easier to maintain code Less concurrency hassles – locking overhead, thread races Single point of dispatch Not necessarily single-threaded Single selection thread can dispatch to multiple worker threads Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

How Does That Work. . . Exactly? Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

NIO Regular Expressions Perl 5 -ish syntax New Char. Sequence interface in java. lang Pattern and Matcher objects String class has regex convenience methods added Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

java. lang. Char. Sequence Package java. lang; public interface Char. Sequence { int length(); char. At(int index); Char. Sequence sub. Sequence(int start, int end); public String to. String(); } Implemented by String, String. Buffer and Char. Buffer Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Regex CSV Tokenizer String [] tokens = line. Buffer. split ("\s*, \s*"); for (int i = 0; i < tokens. length; i++) { System. out. println ("" + i + ": " + tokens [i]); } Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Regex Email Address Parsing public static final String VALID_EMAIL_PATTERN = "([a-z. A-Z 0 -9_\-\. ]+)@((\[[0 -9]{1, 3}\. [0 -9]" + "{1, 3}\. [0 -9]{1, 3}\. )|(([a-z. A-Z 0 -9\-]+\. )+))" + "([a-z. A-Z]{2, 4}|[0 -9]{1, 3})(\]? )"; . . . public void setup. Person (Person person, . . . , String email. Address) {. . . if (email. Address. matches (VALID_EMAIL_PATTERN)) { person. set. Email. Address (email. Address); } else { throw new Illegal. Argument. Exception (email. Address); }. . . } Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

NIO Charsets Character Set Coding Character Set, Coded Character Set, Coding Scheme Encoding and decoding objects Character sets are pluggable (SPI) Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

The JNI Connection Java Code Can Allocate Native Memory (Direct) Native Code Can Create and/or Use Buffers Can Wrap Arbitrary Memory Spaces Video memory, device controllers, etc. All Buffers Are Java Objects Scoping, Garbage Collection, Typing, Etc. Zoom Open. GL For Java (http: //www. jausoft. com) JCanyon F 16 Flight Simulator Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

What Did They Leave Out? Formatted I/O (ala printf/scanf) Will leverage Regular Expressions Enhanced Filesystem Interface More consistent across OS platforms Better access to file/directory attributes Pluggable access to new filesystem types True Asynchronous I/O Under consideration, may never happen Ronsoft Technologies http: //javanio. info http: //www. ronsoft. com

Questions ? ? ? ? ? ?

Bye Buy my Daddy's book. I think I see one right over there. Ron (and Miranda) Hitchens ron@ronsoft. com http: //www. ronsoft. com http: //javanio. info