Скачать презентацию Chinese-Speaking 3 D Talking Head Project No H Скачать презентацию Chinese-Speaking 3 D Talking Head Project No H

f04ad88e98ce876f0785c3640f91be1e.ppt

  • Количество слайдов: 23

Chinese-Speaking 3 D Talking Head Project No: H 08040 Sang Siew Hoon Supervisor: Dr Chinese-Speaking 3 D Talking Head Project No: H 08040 Sang Siew Hoon Supervisor: Dr Ng Teck Khim 1

System Objectives n create a realistic Chinese-speaking 3 D talking model - automatic - System Objectives n create a realistic Chinese-speaking 3 D talking model - automatic - accurate and realistic - portable 2

Problems n Lack of references on Chinese n Involves various fields - Face modeling Problems n Lack of references on Chinese n Involves various fields - Face modeling Text-to-speech Face Animation (Focus) 3

System Overview 5 modules 1. Face Model Preparation 2. Chinese Visemes 3. Text-to-Speech (TTS) System Overview 5 modules 1. Face Model Preparation 2. Chinese Visemes 3. Text-to-Speech (TTS) 4. Animation 5. Rendering 4

5 5

Face Model Preparation 6 Face Model Preparation 6

Face Model Preparation n Face Modeling 3 DS Max 7 Face Model Preparation n Face Modeling 3 DS Max 7

Face Model Preparation n Defining MPEG-4 feature points 8 Face Model Preparation n Defining MPEG-4 feature points 8

Face Model Preparation n Assigning lip vertices 9 Face Model Preparation n Assigning lip vertices 9

Face Model Preparation n Cloning morph targets FAP#3 open_jaw FAP#4 lower_t_midlip 10 Face Model Preparation n Cloning morph targets FAP#3 open_jaw FAP#4 lower_t_midlip 10

Chinese Visemes - the visual equivalent of a phoneme 1. 2. 3. Classification of Chinese Visemes - the visual equivalent of a phoneme 1. 2. 3. Classification of Chinese Syllables Definition of Chinese Visemes Refining Dynamic Visemes 11

Chinese Visemes n Definition of Chinese Visemes 12 Chinese Visemes n Definition of Chinese Visemes 12

Chinese Visemes n Definition of Chinese Visemes 13 Chinese Visemes n Definition of Chinese Visemes 13

Chinese Visemes n Definition of Chinese Visemes 14 Chinese Visemes n Definition of Chinese Visemes 14

Chinese Visemes n Refining Dynamic Visemes 1. <i> + e only replace <f> er Chinese Visemes n Refining Dynamic Visemes 1. + e only replace er zh, z, y + I drop i d, l, g, j, zh, z, y, w + u | ü drop j, y + ending with an replace the an by en j + complex finals headed by i and followed by more than one drop i in complex finals 2. 3. 4. 5. 15

Text-to-Speech 16 Text-to-Speech 16

Text-to-Speech 17 Text-to-Speech 17

Animation 1. 2. 3. Coarticulation Automatic Generation of Face Animation Enhancing Realism 18 Animation 1. 2. 3. Coarticulation Automatic Generation of Face Animation Enhancing Realism 18

Animation n Enhancing realism - Eye blinks - Eyebrow raising - Gaze - Head Animation n Enhancing realism - Eye blinks - Eyebrow raising - Gaze - Head Rotation 19

Results n n n Accurate and realistic No discrepancies for new face models Automated Results n n n Accurate and realistic No discrepancies for new face models Automated results need slight manual intervention http: //www. slamstudio. com/HYP/videos 20

Conclusion n Contributions 1. own system to define Chinese visemes - adds to the Conclusion n Contributions 1. own system to define Chinese visemes - adds to the current research works of Chinese talking heads (currently limited). an automatic Chinese lip-synchronization system - saves an animator much time and effort 2. 21

Conclusion n 1. 2. 3. System Limitations TTS limitation FAPs conflicts Not integrated – Conclusion n 1. 2. 3. System Limitations TTS limitation FAPs conflicts Not integrated – inconvenient for untrained users 22

Conclusion n Future Works 1. Automatic audio signal processing Resolve FAPs values conflicts Automatic Conclusion n Future Works 1. Automatic audio signal processing Resolve FAPs values conflicts Automatic feature points assignment system Integration of system 2. 3. 4. 23