Stereo 3 D Technologies in Computing Entertainment and

Stereo 3 D Technologies in Computing, Entertainment and Art A Reminder of Today and a Look at Tomorrow by Peter Calvache petercalvache@gmail. com University of Applied Arts Vienna

The 3 D Experience You attempt to test the 3 D monitor capabilities and search the net for appropriate images with little success. Everything seems to be geared for conventional displays. Even general-purpose software products don't use these facilities. You finally locate suitable image files but have misplaced the 3 D glasses. At last you find them. Stunning 3 D scenes appear to reach out towards you. You want to interact with them but interaction using keyboard and mouse is far from easy. They are unsuitable for navigation in 3 D space. And now you notice the image quality of the new display in 2 D mode is not as good as your old monitor either. You consign it to the attic and your conventional display is again your window into the digital world. Dr. Barry Blundell British Computer Society

3 D for Computing • Initial target audience is used to interpreting 3 D spatial relationships from 2 D images – PC Enthusiasts, Technicians, Engineers, . . . • Common preconceptions about 3 D displays – – – At best, perceptive augumentation At worst, distraction Scarce compatibility More trouble than it‘s worth Overpriced Not necessary • Greatest competition from high quality 3 D images on 2 D displays – „It looks real enough to me!“

3 D for Computing • Computer interfaces designed around 2 D paradigms – Approximation of paper – Easiest to comprehend • Experiments with 3 D user interfaces – Gimmicky? – No increase in productivity? – 3 D not useful for user interface technology?

3 D for Computing i. Z 3 D 17“ 3 D LCD „This 3 D monitor is only good if you can stand wearing the glasses every time you work in 3 D. Thanks to the doubled-up internal LCDs, the regular Windows desktop looks fuzzy when compared to my Dell 19 inch LCD. Good for 3 D, so-so for 2 D. “ gizmodo. com

3 D for Computing • Banal but important point – 3 D is only useful for displaying natively 3 D content „mapping“ natively 2 D content into 3 D space for visual attraction is unnecessary, distracting and very consciously seen as such – Most content produced for the screen will always be rooted in 2 D • • • Web Browsing Word Processing Text Data Editing Tabular Data Editing Graphical Data Editing (excl. 3 D Graphics) Multimedia Production (excl. 3 D Media) – The customer expects to be able to display 2 D data in perfect quality A good product can have no drawbacks while in 2 D mode

The ideal 3 D Monitor for the PC • Customers are less and less interested as these requirements are not met, with good reason! – – – – High-Resolution capability on par with native 2 D displays 2 D display quality on par with native 2 D displays Hassle-free installation (not more than a driver) Compatibility with a huge range of existing 3 D media and applications Intuitive compatibility (this program is 3 D, so it will display as 3 D!) No glasses required (Vanity!) Affordable (? ? ? ) • All of these points, save for the last, are nearly achievable ☺

The Compatibility Hurdle • Very few programs, if any, are compatible with 3 D monitor libraries such as Philips WOWvx No market share? No incentive to develop for WOWvx! Viscious Cycle No programs for WOWvx? No customer interest! • Very hard to overcome, virtually impossible in the short run

The Compatibility Hurdle „Although 3 D technology poses a myriad of benefits and potential, the infrastructure to support this industry is extremely fragmented and embryonic. While there are numerous companies working on bringing this technology to the mainstream, there is no central platform coordinating this sector. “ U. S. Display Consortium “What is remarkable is that each application has developed its own solutions, culture and technology with little awareness of other applications with similar needs and technology. ” Insight Media

The Compatibility Hurdle • Solution – Make programs compatible automatically (no developer intervention!) – Open. GL or Direct. X libraries used for most 3 D applications. Take advantage of that! • n. Vidia Stereo Driver – Works on nearly all Open. GL or Direct. X 3 D applications and games including legacy applications and games! – Works on nearly all n. Vidia Graphics Cards including legacy cards! – Compatible with select 3 D equipment from Sharp, Asus, See. Real, Dimension Techn. (requires 3 D Glasses) – Inconsistent support from n. Vidia (Requires year-old video drivers)

The 3 D Monitor Killer App • Example: Next-Gen Philips WOWvx Monitor

The 3 D Monitor Killer App • Next-Gen Philips WOWvx Monitor • Customer expectations satisfied: ü ü ü ? High-Resolution capability on par with native 2 D displays 2 D display quality on par with native 2 D displays Hassle-free installation (not more than a driver) Compatibility with a huge range of existing 3 D media and applications Intuitive compatibility (this program is 3 D, so it will display as 3 D!) No glasses required (Vanity!) Affordable (Eventually!)

To Capture the Customer Imagination • 3 D media must not be hard to come by • 3 D media should be producible by the customer • Consider a stereoscopic camera phone (!)

Groups and Consortia • „Meant to be Seen“ (www. mtbs 3 d. com) – Stereoscopic 3 D certification and advocacy group – Stereoscopic 3 D programming guide for video game developers – Program to certify 3 D quality and compatibility • “Interactive Digital Center” (www. eonreality. com/idc) – Consortium made up of n. Vidia, Christie Digital, HP and EON – Raising awareness of interactive 3 D visualization – Establishing a de facto standard for real time visualization solutions • „Imaging and Display Research Group“ – Developing laser-based 3 D rear projection display systems

Stereoscopic 3 D in Entertainment and Art

History of 3 D Cinema • First known 3 D movie in 1903 • 3 D „fad“ in 1953 (61 3 D films shown!) • Episodic resurgence – In the 70 s due to 3 D adult films – In the 90 s due to 3 D theme park attractions [Insight Media]

3 D Cinema • 3 D screens attracted ~2. 5 x more customers on opening weekends while 2 D screens were also available [Screen Digest] • Cinemas who do not invest may be losing out to rival theaters • Long term survival of 3 D Cinema depends on – Quality of the content attracts audiences – Wide enough screenbase to justify production of 3 D features attracts producers

Stereoscopic Filming • „Native“ process • Depth of Field Problem – Multiple focal planes create „blurry“ regions – Age-old artistic device of photographers – Distraction during stereoscopic viewing • Looks unnatural during 3 D viewing • May cause eye strain and headaches – Ideal stereoscopic camera: Infinite depth of field • Small frame or sensor (diminished quality) • Narrow aperture – Current film equipment • Large 35 mm frame (for narrow D-O-F), even larger on IMAX! (70 mm) • Wide aperture (more flexible during lighting) – Ideal can be achieved for CG animated movies!

Stereoscopic Filming • Artistic limitations impossible to enforce? – No shallow depth of field? – No telescopic lenses? – No close-up shots? “We learned that you can't use the same filmmaking process. The shots can't be too close up, or the shots have to be longer sometimes. ” Tim Cheung Head of character animation Dream. Works Animation

3 D Conversion of 2 D Content • Recent examples (partial conversions) – Superman Returns – Harry Potter and the Order of the Phoenix • For shots with heavy movement very time consuming – Can only achieve basic effect • For feature films it‘s heavily criticized! “It looked blurrier than Polar Express 3 D and about as good as the old colored-glasses films. The scenes weren't designed for 3 D so it becomes a distraction. It gets to the big action scene and you have put your glasses on. Suddenly you can't see Superman's face clearly and the screen looks smaller and faded. I was actually wishing it wasn't in 3 D. ” www. geocities. com/outlawvern (on Superman Returns)

Digital 3 D Cinema • • More advanced than earlier renditions of 3 D cinema No reported side effects (eye strain, headaches, . . . ) Can be installed at any theater on top of basic d-cinema equipment Various methods exist for displaying digital 3 D content – Once converted into digital 3 D, a film can be shown in any 3 D formats guaranteed one-time premium during production • Real D (leading provider) – Cooperation with Doremi, Kodak, Qu. Vis, GDC, Sony, XDC, Dolby, . . .

3 D Cinema Adoption (US) • 34 cinema chains have 3 D equipment (H 1 2007) • 639 digital 3 D screens (H 1 2007) [Screen Digest] • 10 biggest chains own ~1900 cinema sites ~20000 screens most large cinema sites have a digital 3 D screen • Wide cinema releases total ~4000 screens • 3 D screens in H 1 2007 capable of handling ~10% of a wide release

3 D Cinema Adoption (International) • About ~150 Digital 3 D Screens • International territories severely stalling behind the US • Leading countries: – Germany, South Korea, Australia

3 D Cinema Adoption (Future) „ 5000 digital 3 D screens will be installed in the U. S. by May 2009 “ Joshua Greer President of Real D • ~1200 digital 3 D screens by November 2007 [Hollywood Reporter] – for release of „Beowulf“ (dir. Robert Zemeckis) • enough 3 D screens by 2009 to lauch Avatar in 3 D only [Insight Media] – (dir. James Cameron)

Imax 3 D • First large-scale commercial 3 D cinema technology • Analogoue 70 mm Projector • Difficult for feature-length presentations (~60 minute film roll buffer) • Slow switch to digital 3 D based technology – Dual Sony 4 k projectors

Imax 3 D (Box Office Analysis)

Imax 3 D (Box Office Analysis)

Imax 3 D (Box Office Analysis)

Times Change • Roger Ebert Chicago Sun Times „As a movie device, 3 D sucks, always has, maybe always will. The problems are: (1) It‘s pointless except when sticking things in the audience's eyes; (2) It is distracting when not pointless; and (3) It dims the colors and makes the image indistinct. “ Spy-Kids 3 D: Game Over (Reality Camera System 3 D, 2003) „In general, I haven't been a fan of 3 D, with its murky images and pathetic little cardboard glasses. But IMAX does it right, and I was astonished by how effective it is on the big screen. That made it the best 3 D viewing experience I had ever had. “ Polar Express (Imax Digital 3 D, 2005)

Influence of 3 D Cinema • Non-cinematic entertainment venues have relied on 3 D imaging – Theme Parks, Pop Concerts, . . . ( More innovation in these areas) • „ 4 D“ experiences become necessary – Physical Effects (water splashes, wind, smell, . . . ) – Not applicable in a cinema or home cinema context competitive advantage • Expansion of 3 D movie paradigm? – 3 D trailers in front of 2 D films? impractical with 3 D glasses – 2 D trailers in front of 3 D films? bad marketing

3 D releases to watch out for • Harry Potter and the Order of the Phoenix (dir. David Yates) – Partial 2 D to 3 D conversion (now showing) • Beowulf (dir. Robert Zemeckis) – Native 3 D, CG Animated (2007) • Star Wars Series 3 D Re-Release (dir. George Lucas, et al. ) – Full 2 D to 3 D conversion (2007 to 2008) • Battle Angel (dir. James Cameron) – Native 3 D Production with the Fusion Camera System (2009) • Avatar (dir. James Cameron) – Native 3 D Production with the Fusion Camera System (2009)

Volumetric Displays Not stereoscopic, but related!

Volumetric Displays • Plasma Flashpoints – National Institute of Advanced Industrial Science and Technology – Keio University – Burton Inc.

Volumetric Displays • Plasma Flashpoints – Emission time on the order of 1 nano-second (1 pulse for each dot) – Human recognition due to after-image effect enables 100 dots/sec

Volumetric Displays • Perspecta – Actuality Systems

Volumetric Displays • Perspecta – Open. GL Compatible!

Volumetric Displays • Perspecta – Bandwidth bottleneck – 1600 x 1200 x 32 bit colors at 85 Hz requires 652 MB/s – 3 rd Dimension: 851 GB/s

Here‘s to a good future for 3 D