Presented as a keynote speech on the International

Скачать презентацию Presented as a keynote speech on the International

2bd8bb10355ed3668bbab33d21423e3e.ppt

Количество слайдов: 60

Presented as a keynote speech on the International Workshop on Acoustic Echo and Noise Control (IWAENC) in Tel Aviv, Israel on September 2, 2010 Noise and Echo Control for Immersive Voice Communication in Spacesuits Yiteng (Arden) Huang We. Voice, Inc. , Bridgewater, New Jersey, USA arden_huang@ieee. org 9/2/2010

About the Project § Financially sponsored by the NASA SBIR (Small Business Innovation Research) program § Phase I feasibility research: Jan. 2008 – July 2008 § Phase II prototype development: Jan. 2009 – Jan. 2011 § Other team members: • Jingdong Chen, We. Voice, Inc. , Bridgewater, New Jersey, USA • Scott Sands, NASA Glenn Research Center (GRC), Cleveland, Ohio, USA • Jacob Benesty, University of Quebec, Montreal, Quebec, Canada 2 All Rights Reserved © We. Voice, Inc. 2010

Outline 1. Problem Identification and Research Motivation 2. Problem Analysis and Technical Challenges 3. Noise Control with Microphone Arrays 4. Hardware Development 5. Software Development 6. A Portable, Real-Time Demonstration System 7. Towards Immersive Voice Communication in Spacesuits 3 All Rights Reserved © We. Voice, Inc. 2010

Section 1 1. Problem Identification and Research Motivation 2. Problem Analysis and Technical Challenges 3. Noise Control with Microphone Arrays 4. Hardware Development 5. Software Development 6. A Portable, Real-Time Demonstration System 7. Towards Immersive Voice Communication in Spacesuits 4 All Rights Reserved © We. Voice, Inc. 2010

Requirements of In-Suit Audio § Speech Quality and Intelligibility: Ø 90% word identification rate § Hearing Protection: Ø Limits total noise dose, hazard noise, and on-orbit continuous and impulse noise for waking and sleeping periods Ø Noise loads are very high during launch and orbital maneuvers. § Audio Control and Interfaces: Ø Provides manual silencing features and volume controls § Operation at Non-Standard Barometric Pressure Levels (BPLs): Ø Operates effectively between 30 k. Pa and 105 k. Pa 5 All Rights Reserved © We. Voice, Inc. 2010

Current In-Suit Audio System Current Solution: Communication Carrier Assembly (CCA) Audio System Skullcap Perspiration Absorption Area Helmet Earpiece Helmet Ring Chin Cup Microphone Module Microphone Boom 6 All Rights Reserved © We. Voice, Inc. 2010

Extravehicular Mobility Unit (EMU) CCA • For shuttle and International Space Station (ISS) operations Interconnect wiring Nylon/spondex top Teflon sidepiece and pocket Ear cup Electret Microphone Ear seal Electret Microphone 7 Interface cable and connector • A large gain applied to the outbound speech for sufficient sound volume at low static pressure levels (30 k. Pa) leads to clipping and strong distortion during operations near sea-level BPL. Source: O. Sands, NASA GRC All Rights Reserved © We. Voice, Inc. 2010

Advanced Crew Escape Suit (ACES) CCA • For shuttle launch and entry operations Dynamic Microphones Source: O. Sands, NASA GRC • Hearing protection provided by the ACES CCA may not be sufficient. 8 All Rights Reserved © We. Voice, Inc. 2010

Developmental CCA Source: O. Sands, NASA GRC Noise Canceling Microphones Source: O. Sands, NASA GRC Ear Cups Active In-Canal Earpieces • The active earpieces will be used in conjunction with the CCA ear cups during launch and other high noise events and can be removed for other suited operations. • The active earpieces alone nearly provide the required level of hearing protection. 9 All Rights Reserved © We. Voice, Inc. 2010

CCA Systems: Pros • High outbound speech intelligibility and quality, SNR near optimum Ø Use close-talking microphones Ø A high degree of acoustic isolation between the in-suit noise and the suit subject’s vocalizations Ø A high degree of acoustic isolation between the inbound and outbound signals Ø The human body does NOT transmit vibration-borne noise • Provide very good hearing protection. 10 All Rights Reserved © We. Voice, Inc. 2010

CCA Systems: Cons • The microphones need to be close to the mouth of a suited subject. • A number of recognized logistical issues and inconveniences: Ø Cannot adjust the cap and the microphone booms during EVA operations, which can last from 4 to 8 hours Ø The close-talking microphones interfere with the suited subject’s eating and drinking, and are susceptible to contamination. Ø The communication cap needs to fit well. Caps in a variety of different sizes need to be built and maintained, e. g. , 5 sizes for EMU caps. Ø Wire fatigue for the microphone booms • These problems cannot be resolved with incremental improvements to the basic design of the CCA systems. 11 All Rights Reserved © We. Voice, Inc. 2010

Stakeholder Interviews • The CCA ear cups produce pressure points that cause discomfort. • Microphone arrays and helmet speakers are suggested to be used. • Suit subject comfort should be maximized as much as possible, given that other constraints can be met (relaxed and traded off): Ø Clear two-way voice communications Ø Hearing protection from the fan noise in the life support system ventilation loop Ø Properly containing and managing hair and sweat inside the helmet Ø Adequate SNR for the potential use of automatic speech recognition for the suit’s information system 12 All Rights Reserved © We. Voice, Inc. 2010

Two Alternative Architectural Options for In-Suit Audio Helmet Speaker 1. Integrated Audio (IA): Instead of being housed in a separate subassembly, both the microphones and the speakers are integrated into the suit/helmet. 2. Hybrid Approach: Employs the inbound portion of a CCA system with the outbound portion of an IA system. 13 All Rights Reserved © We. Voice, Inc. 2010

Section 2 1. Problem Identification and Research Motivation 2. Problem Analysis and Technical Challenges 3. Noise Control with Microphone Arrays 4. Hardware Development 5. Software Development 6. A Portable, Real-Time Demonstration System 7. Towards Immersive Voice Communication in Spacesuits 14 All Rights Reserved © We. Voice, Inc. 2010

Noise from Outside the Spacesuit • During launch, entry descent, and landing: Ø Impulse noise < 140 d. BSPL, Hazard noise < 105 d. BA • On orbit: Ø Impulse noise: < 140 d. BSPL waking hours and < 83 d. BSPL sleeping Ø Limits on continuous on-orbit noise levels by frequency: Band Center Frequency (Hz) 63 125 250 500 1 k 2 k 4 k 8 k 16 k Sound Pressure Level (d. B) 72 65 60 56 53 51 50 48 48 SPL (d. B) 85 – 95 75 – 85 65 – 75 55 – 65 Perception Very High Noise: speech almost impossible to hear High Noise: speech is difficult to hear Medium Noise: Must Raise Voice to be Heard Low Noise: speech is easy to hear Assembly Line Crowded Bus/Transit Waiting Area Very Noisy Restaurant/Bar Department Store Band/Public Area Supermarket Doctor’s Office Hospital Hotel Lobby Construction Site Typical Loud Machine Shop Environments Noisy Manufacturing • Remark: During EVA operations, ambient noise is at most a minor problem. 15 All Rights Reserved © We. Voice, Inc. 2010

Structure-Borne Noise Inside the Spacesuit • Four noise sources (Begault & Hieronymus 2007): 1. Airflow and air inlet hissing noise, as well as fan/pump noise due to required air supply and circulation 2. Arm, leg, and hip bearing noise 3. Suit-impact noise, e. g. , footfall 4. Swishing-like noise due to air movement caused by walking (since the suits are closed pressure environments) • For CCA systems, since the suit subject’s body does not transmit bearing and impact noise, only airflow-related noise needs to be controlled. • For Integrated Audio (IA) systems, microphones are mounted directly on the suit structure and vibration noise is loud. 16 All Rights Reserved © We. Voice, Inc. 2010

Acoustic Challenges • Complicated noise field: Ø Temporal domain: Has both stationary and non-stationary noise Ø Spectral domain: Inherently wideband Ø Spatial domain: Near field; Possibly either directional or dispersive • Highly reverberant enclosure: Ø The helmet is made of highly reflective materials. Ø Strong reverberation dramatically reduces the intelligibility of speech uttered by the suit subject and degrades the performance of an automatic speech recognizer. Ø Strong reverberation leads to a more dispersive noise field, which makes beamforming less effective. 17 All Rights Reserved © We. Voice, Inc. 2010

Section 3 1. Problem Identification and Research Motivation 2. Problem Analysis and Technical Challenges 3. Noise Control with Microphone Arrays 4. Hardware Development 5. Software Development 6. A Portable, Real-Time Demonstration System 7. Towards Immersive Voice Communication in Spacesuits 18 All Rights Reserved © We. Voice, Inc. 2010

Proposed Noise Control Scheme for IA/Hybrid Systems Head Position Calibration Head Motion Tracker 4 3 2 Acoustic Source Localization Mouth range and incident angle with respect to the microphone array Microphone Array Beamforming 1 Multichannel Noise Reduction 5 Adaptive Noise Cancellation Single Channel Noise Reduction Outbound Speech Noise Reference 19 All Rights Reserved © We. Voice, Inc. 2010

Beamforming: Far-Field vs. Near-Field Far-Field S(f, θ) Sound Source of Interest Far-Field Noise Plane Waves V(f, ψ) Near-Field Sound Source ·co s( ψ ) …V(f, ψ) … ψ S(f, rs) rs Plane Waves (N -1 ) ·d … N XN(f) h. N θ . . . X (f). . . h 2 2 2 Σ N ψ d 1 XN(f) X 1(f) h. N h 1 . . θ 2 X 2(f) h 2 d 1 X 1(f) h 1 Σ Y(f, ψ, rs) Y(f, ψ, θ) 21 All Rights Reserved © We. Voice, Inc. 2010

Fixed Beamformer vs. Adaptive Beamformer Microphone Array Beamformers Stationary, Known before the design Isotropic noise generally assumed Noise Field? Fixed Beamformers Time Varying, Unknown Adaptive Beamformers Not Concerned Delay-and-Sum Filter-and-Sum Reverberation? Significant LCMV (Frost)/GSC MVDR (Capon) Delay-and-Sum Filter-and-Sum MVDR (Capon) LCMV (Frost)/GSC • Simple • Non-uniform directional responses over a wide spectrum of frequencies • Complicated • Uniform directional responses over a wide spectrum of frequencies: good for wideband signals, like speech • Only the TDOAs of the interested speech source need to be known – simple requirements. • Reverberation causes the signal cancellation problem. • Time-domain or frequency-domain • The impulse responses (IRs) from the source to the microphones have to be known or estimated. • Errors in the IRs lead to the signal cancellation problem. 22 All Rights Reserved © We. Voice, Inc. 2010

Comments on Traditional Microphone Array Beamforming • For incoherent noise sources, the gain in SNR is low if the number of microphones is small. • For coherent noise sources whose directions are different from that of the speech source, a theoretically optimal gain in SNR can be high but is difficult to obtain due to a number of practical limitations: Ø Unavailability of precise a priori knowledge of the acoustic impulse responses from the speech sources to the microphones. Ø Inconsistent responses of the microphones across the array. • For coherent noise sources that are in the same direction as the speech source, beamforming (as a spatial filter) is ineffective. 23 All Rights Reserved © We. Voice, Inc. 2010

Multichannel Noise Reduction (MCNR) • A conceptual comparison of beamforming and MCNR: Speech Source of Interest • Signal Model: s(k) Noise g. N x. N(k) v(k) Impulse Responses N . . . g. . . x. (k) g 1 2 2 2 d x 1(k) 1 Knowledge related to the source position or gn Beamforming Dereverberation and s(k) Denoising • Beamformer: Spatial Filtering • Array Setup: Calibration is necessary – possibly time/effort consuming 24 g. N N x. N(k) . . x (k) g 2 g 1 1 2 2 x 1(k) MCNR Only Denoising x 1, s(k) • MCNR: Statistical Filtering • Array Setup: No need to strictly demand a specific array geometry/pattern All Rights Reserved © We. Voice, Inc. 2010

Frequency-Domain MVDR Filter for MCNR • The problem formulation: • The MVDR filter: • A more practical implementation: where • Similar to traditional single-channel noise reduction methods, the noise PSD matrix is estimated during silent periods and the signal PSD matrix is estimated during speech periods. 25 All Rights Reserved © We. Voice, Inc. 2010

Comparison of the MVDR Filters for Beamforming and MCNR • MVDR for Beamforming (BF): • MVDR for MCNR: • The acoustic impulse responses can at best be estimated up to a scale: where Leads to speech distortion. 26 denotes the true response vector. • Note: In the implementation of the MVDR-MCNR, the channel responses do not need to be known. All Rights Reserved © We. Voice, Inc. 2010

Distortionless Multichannel Wiener Filter for MCNR • Use what we called the spatial prediction: • Formulate the following optimization problem: where • The distortionless multichannel Wiener (DW) filter for MCNR: • The optimal Wiener solution for the non-causal spatial prediction filters: where So, • It was found that 27 All Rights Reserved © We. Voice, Inc. 2010

Single-Channel Noise Reduction (SCNR) for Post-Filtering • Beamforming: The Wiener filter (the optimal solution in the MMSE sense) can be factorized as MVDR Beamformer Wiener Filter for SCNR Note: For a complete and detailed development of this factorization, please refer to Eq. (3. 19) of the following book. q M. Brandstein and D. Ward, eds, Microphone Arrays: Signal Processing Techniques and Applications, Berlin, Germany: Sprinter, 2001. • MCNR: Again, the Wiener filter can be factorized as MVDR for MCNR Wiener Filter for SCNR Note: For a complete and detailed development of this factorization, please refer to Eq. (6. 117) of the following book. q J. Benesty, J. Chen, and Y. Huang, Microphone Array Signal Processing, Berlin, Germany: Springer, 2008. 28 All Rights Reserved © We. Voice, Inc. 2010

Single-Channel Noise Reduction (SCNR) • The signal model: • SCNR filter: • Error signal: • MSE cost function: • The Wiener filter: where and • Other SCNR methods: Parametric Wiener filter, Tradeoff filter. • A well-known feature: Noise reduction is achieved at the cost of adding speech distortion. 29 All Rights Reserved © We. Voice, Inc. 2010

New Idea for SCNR • A second-order complex circular random variable (CCRV) which implies that and its conjugate has: are uncorrelated. • In general, speech is not a second-order CCRV: • But noise is a second-order CCRV if stationary, and not otherwise. • Examine Correlated but not completely coherent Uncorrelated or coherent • This is similar to the signal model of a two-element microphone array. So there is a chance to reduce noise without adding any speech distortion. 30 All Rights Reserved © We. Voice, Inc. 2010

Widely Linear Wiener Filter • New filter for SCNR: • Error signal: • Widely linear MSE: • Then the widely linear Wiener filter or MVDR type of filters can be developed. 31 All Rights Reserved © We. Voice, Inc. 2010

Section 4 1. Problem Identification and Research Motivation 2. Problem Analysis and Technical Challenges 3. Noise Control with Microphone Arrays 4. Hardware Development 5. Software Development 6. A Portable, Real-Time Demonstration System 7. Towards Immersive Voice Communication in Spacesuits 32 All Rights Reserved © We. Voice, Inc. 2010

Computational Platform/Technology Selection § Three platforms under consideration: § Four competing factors: • ASIC • The count of transistors employed • DSP • The number of clock cycles required • FPGA • The time taken to develop an application § Trade-off among performance, power consumption, size, and costs • Nonrecurring engineering (NRE) costs ASIC DSP FPGA • Low numbers of transistors and clock cycles • Long development time and high NRE costs • Effective in performance, power, and size, but not in cost • Low development and NRE costs • Low power consumption • More efforts to convert the design to ASICs • Not suited to processing sequential conditional data flow, but efficient in concurrent applications • Support faster I/O than DSPs • One step closer to ASIC than DSP • High development cost due to performance optimization 33 All Rights Reserved © We. Voice, Inc. 2010

System Block Diagram XLR Male Female 6 7 8 34 2 1 1 3 Mic. Preamps 3 G 2 1 1 2 2 1 1 G 3 G 3 G HOT Mic. COLD Powering GND Circuit Mic. HOT COLD Powering GND Circuit Power Mgmt IC. . . 2 3 Digital Output Interface (USB 2. 0) JTAG (Male) 2 3 2 1 1 2 3 2 1 2 3 1 . . 48 k. Hz ADC. Altera FPGA 2 1 8 -ch 24 -bit Power Jack Flash 5 Mic. HOT COLD Powering GND Circuit 1 DB 25 Female 3 G G 3 3 . . . 4 Mic. HOT COLD Powering GND Circuit 3 1 2 3 DB 25 Male 2 . . . 3 Mic. HOT COLD Powering GND Circuit 2 . . . 2 Mic. HOT COLD Powering GND Circuit Analog Input 1 . . . MIC CAPSULE G Jumpers (for Gain Control) SDRAM FPGA Board 3 All Rights Reserved © We. Voice, Inc. 2010

FPGA Board Block Diagram USB 2. 0 (High Speed) User LED/IOs OPA 1632 (1) OPA 1632 (2) ADS 1278 Altera Cyclone III EP 3 C 55 F 484 C 8 OPA 1632 (8) FPGA EPCS 16 50 MHz XTAL 35 16 MB SDRAM (× 32) 3. 3 V 16 MB SDRAM (× 32) 16 MB Flash (× 16) 24. 576 MHz XTAL All Rights Reserved © We. Voice, Inc. 2010

Prototype FPGA Board: the Top View Phantom Power Feeding User I/Os FT 2232 H USB 2. 0 GND Jack REF 1004 ADS 1278 EPCS 16 S JTAG 12 MHz Crystal OPA 1632 User LEDs TPS 65053 Flash DB 25 DC Power Jack Power LED Mic. Pream Gain Jumpers 36 Analog Power DC 9 V Analog Power Cyclone III DC 5 V FPGA SDRAMs 174. 8 mm × 101 mm All Rights Reserved © We. Voice, Inc. 2010

FPGA System Development Flow Adopted in the Project System on Programmable Chip (So. PC) + C/C++ Programming: 1) Use So. PC Builder to construct a soft-core NIOS II processor embedded on the Altera FPGA 2) Develop software/DSP systems in C/C++ on the NIOS II processor • Advantages: ü Short development cycle/time ü Low cost ü High reliability ü Reusability of intellectual property 38 I/O ROM RAM CPU (NIOS II) UART DSP • Drawbacks: û Poor efficiency and low performance: Ø Efficiency can be improved by identifying those time-consuming functions (e. g. , FFT and IFFT) and accelerating them with the tool of C 2 H (C-to-Hardware) All Rights Reserved © We. Voice, Inc. 2010

MEMS Microphone Array 20 mm 5 mm b a 1 b c a d 2 b c d Pin 18 7 Subarrays 5 mm a 3 b c d a 4 b c d a 5 b c a d 6 b c d a 7 c d XG-MPC-MEMS Pin 1 Analog Device ADMP 402 MEMS Microphones: 2. 5 mm × 3. 35 mm Samsung 18 -pin Connector 39 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 All Rights Reserved © We. Voice, Inc. 2010

Section 5 1. Problem Identification and Research Motivation 2. Problem Analysis and Technical Challenges 3. Noise Control with Microphone Arrays 4. Hardware Development 5. Software Development 6. A Portable, Real-Time Demonstration System 7. Towards Immersive Voice Communication in Spacesuits 41 All Rights Reserved © We. Voice, Inc. 2010

FPGA Program Flowchart From ADC To USB FPGA overlap add data in & preprocessing Nios II Soft Core USB trans. overlap add data in & preprocessing MCNR+SCNR . . . MCNR+SCNR USB trans. FFT/IFFT Processor 4 -ch FFT 1 -ch IFFT t 1 time frame 1 -ch IFFT t+4 t+8 time (ms) Processing delay < 8 ms 42 All Rights Reserved © We. Voice, Inc. 2010

Section 6 1. Problem Identification and Research Motivation 2. Problem Analysis and Technical Challenges 3. Noise Control with Microphone Arrays 4. Hardware Development 5. Software Development 6. A Portable, Real-Time Demonstration System 7. Towards Immersive Voice Communication in Spacesuits 47 All Rights Reserved © We. Voice, Inc. 2010

The Portable, Real-Time Demo System MEMS Microphone Array PC DB 25 Connectors USB 2. 0 Cable Audio Cable Suited Subject 48 FPGA Board Power Supply: Linear DC 12 -20 V/1 A All Rights Reserved © We. Voice, Inc. 2010

Section 7 1. Problem Identification and Research Motivation 2. Problem Analysis and Technical Challenges 3. Noise Control with Microphone Arrays 4. Hardware Development 5. Software Development 6. A Portable, Real-Time Demonstration System 7. Towards Immersive Voice Communication in Spacesuits 49 All Rights Reserved © We. Voice, Inc. 2010

What is and Why do we want Immersive Communication? § Telecommunication helps people collaborate and share information by cutting across the following 3 separations/constraints: Ø Long distance Ø Real time Ø Physical boundaries § Modern telecommunication technologies are successful so far in transcending the first two constraints: i. e. , the long-distance and real-time constraints. § Immersive communication offers an feeling of being together and sharing a common environment during collaboration. § Immersive communication targets at breaking the physical boundaries, which is the “last mile” problem in communication. 50 All Rights Reserved © We. Voice, Inc. 2010

Why Immersive Voice Communication in Spacesuits? § Immersive voice communication exploits human’s binaural hearing. § Provides enhanced situational awareness for a suited crewmember: ü Can improve the productivity of collaboration among the crewmembers ü Can produce potential safety benefits § Crew comfort can be optimized. 58 All Rights Reserved © We. Voice, Inc. 2010

Conclusions • While it has been more than 40 years since Neil Armstrong landed on the Moon, the astronauts are still using the communication carrier assembly (CCA) based audio system for voice communication in spacesuits. • The new spacesuit design is going to take advantage of the most recent advances in multichannel acoustic and speech signal processing for echo and noise control and meanwhile with significantly improved crew comfort and ease of use. Ø Noise reduction with microphone arrays Ø Multichannel echo cancellation Ø Integrated echo and noise control Ø 3 D audio • We explained the difference between the traditional beamforming method and what we called the multichannel noise reduction approach. • We presented an intuitive interpretation of the widely linear Wiener filter for singlechannel noise reduction. • We described a new application of immersive communication in space exploration, ancillary to its mainstream use in commercial telecommunication systems. 60 All Rights Reserved © We. Voice, Inc. 2010