
60a2757d72bad6a17e6691c3ca1c5c71.ppt
- Количество слайдов: 41
Audio for Games Scott Selfon Xbox Advanced Technology Group Microsoft Confidential
Overview The roles of audio in games Audio challenges on a game console Score, Sound Effects, and Dialog Challenges Typical Solutions Content creation / implementation process Audio integration and end game Case studies on Xbox Leading edge / future of game audio Q&A Microsoft Confidential
Game Audio Has Come a Long Way… Fidelity Tinny mono speaker 5. 1 home theater systems 1 channel/4 bits 256 channels, 24 bits FM oscillators 48 k. Hz wavetable synthesis Audio processing power Off-line effects processing real-time DSP effects 4 -channel mixing Realtime Dolby Digital encode Realtime 3 D positioning Production pipeline Low level chip programming Separate sound designer/composer and audio programmer Microsoft Confidential
The Roles of Audio in Games Audio is 50% of the experience Support (improve? ) the visuals Provide information about what we don’t see Provide emotional support for gameplay Rewards/encouragement The best audio implementations are rarely noticed by the gamer 1995 MIT study Game reviews and audio Microsoft Confidential
Audio Challenges on a Game Console: Aesthetic Competing with other ‘living room’ experiences But much less linear Events may be skipped, repeat, play in unpredictable order Players expect same kind of realism as they get from non-interactive media Microsoft Confidential
Audio Challenges on a Game Console: Technical Finite Resources Storage space (not as much of an issue currently) Memory (actual playing sound data, audio library size, etc. ) Processing power (realtime effects, mixing, etc. ) Interactivity Avoiding repetition is perhaps most important Secondary goal: compelling non-linear audio Need solutions that don’t require a lot of development time Microsoft Confidential
Audio Challenges on a Game Console: Organizational “Post-Production” mentality towards audio – scheduled late Often appropriate in film, television Often viewed in budgeting as expense rather than investment Tradition & expectations for game audio - hasn’t been always been of highest quality (see previous slide) No dedicated audio programming resource Often up to composer/sound designer to understand determine what’s possible and feasible Cross-platform audio solutions Simplest-to-implement programmatic solutions used Microsoft Confidential
Audio Role Comparison Film/TV/Radio Games Sound Designer Composer/arranger(s) Orchestrator(s) Recording Engineer(s) Mixing/Mastering Engineer(s) Music Supervisor Music Editor SFX Editor ADR/Dialog Editor Foley Artist(s) Casting (for voices) Sound Designer/Composer (maybe 2 separate people) Audio programmer (part time) Microsoft Confidential
Hiring a Content Creator Generally ‘buy-out’ (work for hire) Composer paid straight fee (per platform? ) Developer/publisher generally owns all rights to music May or may not include cost of recording, musicians, mixing, etc. May negotiate for publishing Per copy sold royalty (rare) Paid for TV/radio spots Royalty for thematic use in sequels, spinoffs Opportunity for game soundtrack CD Microsoft Confidential
Budgeting for Audio Musical underscore Live, synth, or mix? And how much? How does it start and stop? (linear, looping, branching) Licensing existing music Sound Effects / Ambience Live recordings or sample libraries? Creation of new modeled sounds Sub-contracting Often essential for large projects Microsoft Confidential
Laying out an Attack Plan Look at all ways system can make noise Work with technical document to determine what’s preferable/do-able E. g. MIDI or streamed (CD/DVD/HD) music Real-time effects (reverb, filtering, etc. ) What other game resources are being used Maximize audio capabilities while minimizing impact on rest of game Often different for different parts of game Microsoft Confidential
Tools, Tools… Evaluating for technical and aesthetic criteria Make authoring and programming pipelines relatively independent Maintain creative flexibility Linear audio Authoring: Same as for other media Programming: Cross-platform and platform specific options Non-linear audio = generally proprietary PC: Direct. Music or ‘in-house’ engines Consoles: Platform-specific or ‘in-house’ solutions Microsoft Confidential
Underscore Challenges Film ~90 minutes, often <60 minutes of score Linear, unchanging storyline Game 10+ hours of gameplay Gameplay events may be quite variable (in order, duration, etc. ) Don’t need “music” everywhere (but at least ambience) Score: Often linear tracks several minutes long (may loop) Microsoft Confidential
Typical Underscore Solutions for Consoles Create several long linear tracks May loop May be played ‘soundtrack’ style (no interaction with gameplay) Save purely musical score for key one-time game points Use ambience instead to set mood Avoids repetition of recognizable thematic material Advanced: Non-linear audio Alternate versions with looping and branching Various intensity levels of music that crossfade based on game state Microsoft Confidential
Licensed Music Not appropriate for all games Often difficult to make interactive/dynamic Disconnects score from moment-to-moment gameplay Can be effective in some situations Name recognition/cross-marketing potential Can set titles up for user soundtrack support Typical uses Existing brands/licenses (esp. movies and TV) Heavily used by racing games Microsoft Confidential
Composing a Game Score Spotting session Where can music support action? Compose cues for linear sections of game (for instance, cutscenes) Need final cut for accurate timings (hah hah) Record and mix Pre-rendered: Synth, live players, mixture Real-time rendered: MIDI, individual pads, etc. Implement playback in game Triggers to start, stop, transition, etc. Edit to match ‘final’ animations Microsoft Confidential
Sound Effects Challenges No pre-existing soundtrack to work with Same sound event may be heard hundreds of times Variation Environmental interaction Tying FX to game physics and AI Sheer number of sound elements to create Combinations and permutations Everything that can make a noise should Microsoft Confidential
Typical Sound Effects Solutions Variation Implement basic pitch randomization Several versions of each wave Break a sound into components and record versions of each of those Tying effects to game physics / AI Implementation varies per-title Often vast majority of dev time spent here Advanced: Dynamic processed sounds Environmental effects (reverb) Filtering (force of impacts, radioizing, etc. ) Microsoft Confidential
Creating Ambience and Sound Effects Spotting session (if game playable), or look at artwork (environments, objects) List everything that can make noise, prioritize Object interaction matrix Often start with organic sounds Sound effects library ($$$, can be generic) Create own samples (free, unique) Sound processing (pre-rendered or dynamic) Reverb, filtering, chorus, effects Microsoft Confidential
Dialog Challenges Casting – producer may want involvement Directing / Coaching Stitching (first | and | 10 | from the | 25) Dialog repetition is easily recognized Localization Re-recording voice talent Engine integration Grammatical differences Recreating pre-rendered processing Microsoft Confidential
Typical Dialog Solutions Triggers for dialog (just like music) Variation for common lines (often too few) Stitching – coach the talent Localization Solutions vary per-title Very careful asset management Sometimes source assets can just be swapped Advanced: Sophisticated dialog engine Variation tracking to avoid repetition People don’t always have to say anything Microsoft Confidential
Prepping for Dialog Content creator works with producer, game designer, and/or writer to: List and describe characters Write each character’s scripts Cast person for each character Real voice talent makes a difference Many actors can do multiple roles Microsoft Confidential
Dialog Production Record all dialog Encourage improvisation Coaching for creating convincing stitchable dialog Cut up dialog into separate files Clean up for stitching – sound designer for sports title can spend majority of time here Post-recording processing Document everything for localization Microsoft Confidential
Squeezing it all into 5 bytes Best to determine resources up front Memory and CPU available Disk space and bandwidth available Inevitably, not enough room Asset swapping (programming time) Compression Use of streaming where possible Cut assets Microsoft Confidential
And the Handoff… Source assets given to game Typically high fidelity + compressed versions delivered Freelance composer/sound designer: often end of work Programmer integrates audio assets into title Often tasked with mixing (either with or without content creator feedback) Implement runtime sound processing (reverb, effects, filtering, etc. ) unless engine natively supports Testers confirm sounds are in place Asset/trigger lists, scripts, and/or manual triggers Variation testing? Microsoft Confidential
After the Fact: More Challenges Archive, archive Create a recording of typical gameplay for reference Disaster always strikes at the 11 th hour Keep an organized collection of any sounds created (spreadsheet, database) Localization Voice talent recasting, re-recording Recreate processing (unless real-time used) Releasing a CD Microsoft Confidential
Audio Case Study: Project Gotham Racing Stereo ambience, with some 3 D positioned foreground ambient elements Player’s car Stereo engine ambience Engine sound varies with RPM, velocity, physics, etc. Tire sound varies with surface (sidewalk, stairs, street, etc. ) 3 D positioning on other cars Engines (varies with RPM, velocity, physics) Car horns Cornering, tire squeals, collisions, etc. (all with some variation) Dynamic sounds Tire squeals (randomized between several versions and pitches) Collisions (depend on speed, material involved, etc. ) Microsoft Confidential
Project Gotham Racing: Music Game disc-supplied licensed songs (WMA) User Soundtrack support Allows for personalized driving music mixes Music uniquely integrated into game Use of ‘car radio’ or ‘CD’ playback ‘Radio’: Location-specific DJ interstitials Microsoft Confidential
Audio Case Study: Splinter Cell Detailed ambiences, foley, sound effects Audio as a gameplay element (stealth) Stealth: Noise tied to game AI Footsteps, interaction with objects, weapons Audio as an informational tool Interactive musical score Tells player when enemies alerted Use of musically appropriate boundaries for transitions Microsoft Confidential
Audio Case Study: Halo 4, 311 triggerable events (soundtags) 11, 629 source waves (soundfiles) 2. 5 GB uncompressed audio 22 k. Hz mono and 44 k. Hz stereo 80 minutes of source music 20+ hours of gameplay 12 voice talents, 7, 735 lines of dialog …all in 2 -MB memory footprint Microsoft Confidential
Halo: Audio Tools Used Pre-rendered linear music/FX = traditional standard audio post production tools Pro Tools, Studio Vision, Peak, Digital Performer Dynamic audio = custom tool/engine, randomization/permutation for each event Pick one or more waves to play Random variation in pitch Weighting for how often a sound will be used Microsoft Confidential
Halo Audio Engine: Dialog Simulation of actual speech patterns Close connection to game AI Large number of events a character can respond to, multiple phrases per action Characters don’t always speak Tracking speech Multiple voice talents for multiple characters Character only says one thing at a time Positioning (3 D versus locked-to-speaker) Repetition avoidance Microsoft Confidential
Halo Audio Engine: Sound Effects Variations in all sound effects Weighting (frequency for multiple versions of sound) Randomized pitch control Variations in ambience Various loops of different lengths Layered 3 D-positioned detail sounds (birds, crickets, machinery hum, etc. ) “Cascading” sound effects If many instances of an effect are playing, play a single more complex sound Response to game environment Specific conditions/events/game physics Environment interaction (reverb) 3 D spatial positioning versus static Microsoft Confidential
Halo Audio Engine: Music Typically linear or looping If looped, alternate versions with varying lengths Dynamic execution (“spotted” with programmer) Varying transition types based on situation Branching Crossfades Game event tags Appropriate absence of music Silence is golden - not scored wall to wall Timeouts Microsoft Confidential
Halo 5. 1 Usage Sound effects typically 3 D-positioned (4 channels) Some ambience 3 D-positioned as well “Quad” music and ambience Doubled stereo (fronts 100% / rears 50%) Crossfades based on location Center channel Cortana (“first person” dialog) Radio communication LFE channel Used to emphasize more massive sounds Heartbeat (low health) Explosions Microsoft Confidential
Halo Environmental Reverb Usage Reverb settings changed based on environment Indoors versus outside Confined spaces versus large spaces Caves versus hallways Sound interactions with reverb Especially 3 D-positioned sound effects Some ambient elements also Microsoft Confidential
Halo Implementation (1) 2 -MB in-memory footprint 22 -k. Hz mono ADPCM for 3 D sounds 44 -k. Hz stereo ADPCM for music + ambience ADPCM = ~3. 5: 1 compression Use of Xbox hard disk Copy all sound data to drive at start of level (faster to read than DVD) Pull small chunks into memory rather than entire banks (pseudo-”streaming”) Drive persistence = faster load next time Microsoft Confidential
Halo Implementation (2) Sound during load screens Small looped sample Multiple instances played with pitch shifting and filtering as load progresses “Dynamic” – speed and actual sound vary with read time Microsoft Confidential
Leading Edge Audio Challenges Voice integration (esp. for online titles) Chat and gameplay implications Manipulation (masking, radioizing, etc. ) Voice recognition Command control Further multichannel audio usage Fuller 5. 1 implementations Dynamic audio mixing Physical modeling Still too expensive (CPU, programming effort) for most games Microsoft Confidential
Wrap Up: The State of Game Audio Lots of independent composers/sound designers Many coming from outside game industry G. A. N. G. (www. audiogang. com) Still typical for developers to reinvent audio engine with each title (newer tools try to solve) Xbox efforts Xbox Registered Developer and Registered Content Creator Programs Tools and dedicated support for audio content creators and programmers Game Designer/Producer education Microsoft Confidential
Q&A scottsel@xbox. com Resources G. A. N. G. (www. audiogang. com) Game Audio Pro e-mail list (Yahoo) Direct. Music (www. microsoft. com/directx) 2003 Game Developer Conference lectures Microsoft Confidential
60a2757d72bad6a17e6691c3ca1c5c71.ppt