柔性計算的研究現況與發展 淡江大學 資訊管理系所 侯 永 昌 http: //mail. im. tku. edu. tw/~ychou ftp: //mail. im. tku. edu. tw/Prof_Hou
大綱 § § § 前言 模糊理論 類神經網路 遺傳演算法 柔性技術的結合 淡江大學資管所侯永昌 1
前言 淡江大學資管所侯永昌 2
柔性計算 § Dr. Lotfi Zadeh: in 1965 published his seminal work "Fuzzy sets" in the journal Information and Control. § Unfortunately, fuzzy logic did not receive serious notice in this world until the last decade. § The attention currently being paid to fuzzy logic is most likely the result of present popular consumer products employing fuzzy logic. 淡江大學資管所侯永昌 3
柔性計算 § In Japan, Fuzzy logic has become a common advertising slogan. § Whereas the Eastern world equates the word fuzzy with a form of computer intelligence, the Western world still largely associates the world derisively within the context of imprecise or approximate science. § Appliances with fuzzy logic controllers provides the consumer with optimum settings that more closely approximate human perceptions and reactions than those associated with standard control systems. 淡江大學資管所侯永昌 4
柔性計算 § Over the last several years, the Japanese alone have filed for well over 1000 patents in fuzzy logic technology, and they have already grossed billions of U. S. dollars in the sales of fuzzy logic-based products to consumers the world over. § The integration of fuzzy logic with neural networks and genetic algorithms is now making automated cognitive systems a reality in many disciplines. 淡江大學資管所侯永昌 5
柔性計算 § the reasoning power of fuzzy system, when integrated with the learning capabilities of artificial neural networks and genetic algorithms, is responsible for new commercial products and processes. § The marketing research firm of Frost & Sullivan projected that fuzzy logic, with an annual growth rate of 20 percent, would be one of the world's 10 hottest technologies going into the twenty-first century. 淡江大學資管所侯永昌 6
模糊理論 淡江大學資管所侯永昌 7
Uncertainty and Impression 愈複雜的系統， 就愈不容易精 確的表現 淡江大學資管所侯永昌 8
Uncertainty and Impression § Probability theory： 研究的是語意清楚，但是 行為 (outcome)不確定 (random)的事件，也就 是對於事件的出現與否，完全沒有把握。 The occurrences arise by chance § 例如：擲骰子、 … § Fuzzy theory： 研究的是 outcome明確，但是其 語意不確定 (vagueness, impression)的事件，也 就是一個現象是否符合這一個模糊概念， 不能只用一個單一的 yes or no來回答 § 例如：高矮、美醜、大小、 … 淡江大學資管所侯永昌 9
Uncertainty and Impression § Uncertainty in Information: The source of impression is the absence of sharply defined criteria of class membership rather than the presence of random variable. 淡江大學資管所侯永昌 10
Uncertainty and Impression By chance 在學校中學的多為理想 狀況；而在真實世界中， 確定的資料是不多的。 Inability to perform adequate measurement Lack of knowledge Fuzziness inherent in our natural language 淡江大學資管所侯永昌 11
Fuzzy logic is most successful in § very complex model where understanding is strictly limited or quite judgmental. § processes where human reasoning, human perception, or human decision making are inextricably involved. § Because: l Human use linguistic variables, rather than quantitative variables to represent imprecise concepts. l Human reasoning is based largely on imprecise intuition or judgment, more precision entails higher cost. 淡江大學資管所侯永昌 12
Uncertainty and Impression 淡江大學資管所侯永昌 13
Uncertainty and Impression 淡江大學資管所侯永昌 14
Uncertainty and Impression 淡江大學資管所侯永昌 15
Fuzzy logic is most successful in § § § focusing and image stabilization: Fisher, Sanyo air conditioner: Mitsubishi washing machine : Matsushita subway automatic system controller : Sendai, Hitachi automatic transmission/ anti-skid braking system: Nissan golf diagnostic system toaster rice cooker vacuum cleaner pattern recognition and classification: US DOD space docking control: NASA stock-trading portfolio: Japan 淡江大學資管所侯永昌 16
Fuzzy Set and Membership § 針對一個 crisp set A： 某一個元素 x， 要就 完全屬於集合 A， 要就完全不屬於集合 A § 但是對於 Fuzzy set A： 某一個元素 x， 是否 屬於集合 A， a matter of degree and is is relative § A(x) [0, 1] 淡江大學資管所侯永昌 17
Fuzzy Set and Membership § 例如： height around 6 feet § 多少有點自由心証，但是至少他要滿足： normal、 monotonicity、 symmetry 淡江大學資管所侯永昌 18
Fuzzy Set and Membership § probability provides knowledge about relative frequencies § fuzzy membership function represents similarities of objects to ambiguous properties 淡江大學資管所侯永昌 19
Chance versus Ambiguity § 例： persons: two 1. 95% chance of being over 7 feet tall 2. high membership in the set of very tall people § 例： glasses of water: two 1. 95% chance of being healthful and good 2. 0. 95 membership of being healthful and good § Which one will you choose? 淡江大學資管所侯永昌 20
Chance versus Ambiguity § The prior probability of 0. 95 in each case becomes a posterior probability of 1. 0 or 0. 0. However, the membership value of 0. 95 remains 0. 95 after measuring or testing! § Fuzziness describes the ambiguity of an event, whereas randomness describes the uncertainty in the occurrence of the event. 淡江大學資管所侯永昌 21
Chance versus Ambiguity 淡江大學資管所侯永昌 22
Fuzzy Control System Design 淡江大學資管所侯永昌 23
Simple Fuzzy Logic Controllers § The steps in designing a simple fuzzy logic control system are as follows： 1、Identify the variables (inputs, states, and outputs) of the plant. 2、Partition the universe of discourse or the interval spanned by each variable into a number of fuzzy subsets, assigning each a linguistic label (subsets include all the elements in the universe). 3、Assign or determine a membership function for each fuzzy subset. 淡江大學資管所侯永昌 24
Simple Fuzzy Logic Controllers 4、Assign the fuzzy relationships between the inputs’ or states’ fuzzy subsets and the outputs’ subsets, thus forming the rule-base. 5、Choose appropriate scaling factors for the input and output variables in order to normalize the variables to the [0, 1] or the [-1, 1] interval. 6、Fuzzify the inputs to the controller. 7、Use fuzzy approximate reasoning to infer the output contributed from each rule. 8、Aggregate the fuzzy outputs recommended by each rule. 9、Apply defuzzification to form a crisp output. 淡江大學資管所侯永昌 25
Example: A pressure process controlled by a fuzzy controller § Step 1. Value assignment for the fuzzy input and output variables. § 主要的控制變數有兩個。 the error (e) 其值域 為 [-em, +em]， 可以區分為 8 個語言變數 (Ai)。 the error rate ( )其值域為 [- m, + m]， 可以區 分為 7 個語言變數 (Bi) § 為了避免控制變數的值域相差太大，在處理 時需先將它們正規化到 [-a, +a]之間 淡江大學資管所侯永昌 26
Example: 淡江大學資管所侯永昌 27
Example: 淡江大學資管所侯永昌 28
Example: § Step 2. Summary of control rules. § 根據專家的經驗，可以得到控制的法則，其形 態如下：If e is A i and is B j , Then z is C ij § 例如： pressure error is PB and the rate of change in If the pressure error is NS, then heat input change is NM. 淡江大學資管所侯永昌 29
Rule的轉換過程 § 每一個 rule 都可以轉換為一個 其轉換的過程如下： fuzzy relation R， 1、先求 Ai * Bj ===> 然後將 RAB 展開成為一階的陣列 [ ] 182*1 淡江大學資管所侯永昌 30
Rule的轉換過程 2、再求 Ai Bj * Cij ===> 3、然後將所有的 (一共有 48 個 ) Ai Bj Cij union 起來 淡江大學資管所侯永昌 31
Rule的轉換過程 § Step 3. Conversion between fuzzy variables and precise quantities: 利用重心法或最大值法，求出每一列 ( 代表一種輸入的情況 ) 的對應輸出值 § ===> 淡江大學資管所侯永昌 32
Rule的轉換過程 § Step 4. Develop of control table: 最後將它變成一個 二階矩陣 ===> 淡江大學資管所侯永昌 33
Rule的轉換過程 淡江大學資管所侯永昌 34
Example 淡江大學資管所侯永昌 35
Discussions § Fuzzy mathematics has provided a range of mathematical tools that helps the investigator formalize these ill-defined descriptions about complex systems into the form of linguistic rules and then eventually into mathematical equations. § At the expense of relaxing some of the demands on the requirements for precision in some nonlinear systems, a great deal of simplification, ease of computation, speed, and efficiency are gained when using fuzzy models. 淡江大學資管所侯永昌 36
類神經網路 淡江大學資管所侯永昌 37
類神經網路 § The development of artificial neural networks began approximately 50 years ago, motivated by a desire to try both to understand the brain and to emulate some of its strengths § Neural nets are basically mathematical models of information processing. § An artificial neural network is characterized by: l its pattern of connections between the neurons (called its architecture) l its method of determining the weights on the connections (called its training, or learning algorithm) l its activation function 淡江大學資管所侯永昌 38
類神經網路 § Neural Networks: A neural network is a technique that seeks to build an intelligent program using models that simulate the working network of the neurons in the human brain. 淡江大學資管所侯永昌 39
類神經網路 § A neuron receives a set of input pulses and sends out another pulse that is a function of the input pulses. 淡江大學資管所侯永昌 40
類神經網路 § xi = signal input ( i = 1, 2, . . , n ) § wi = weight associated with the signal input xi， 代表 不同的重要性 l when wi > 0, input xi acts as an excitatory signal l when wi < 0, input xi acts as an inhibitory signal § t = threshold level（ 門檻值）prescribed by user， input signal 要大於門檻值才會有反應 § F(s) is a nonlinear function; e. g. , a sigmoid function F(s) = 。 Other popular choices for this function are a step function, and a ramp function. 淡江大學資管所侯永昌 41
類神經網路 § Information processing occurs at many simple elements call neurons. § Signals are passed between neurons over connection links. § Each connection link has an associated weight, which multiplies the signal transmitted. § Each neuron applies an activation function (usually nonlinear) to its net input (sum of weighted input signals) to determine its output signal. 淡江大學資管所侯永昌 42
Typical Architectures 淡江大學資管所侯永昌 43
Typical Architectures § Within each layer, neurons usually have the same activation function and the same pattern of connections to other neurons. § single layer vs. multilayer § feedforward vs. recurrent § competitive vs. cooperative 淡江大學資管所侯永昌 44
Setting the Weights § Supervised training： training is accomplished by presenting a sequence of training vectors, or patterns, each with an associated target output vector. § Unsupervised training： sequence of input vectors is a provided, but no target vectors are specified. The net modifies the weights so that the most similar input vectors are assigned to the same output (or cluster) unit. § Fixed-weight nets： weights are set to represent the constraints and the quantity to be maximized or minimized. 淡江大學資管所侯永昌 45
The 1940 s: The Beginning of Neural Nets § Hebb net： l single layer, one pass for training set l earliest and simplest learning rule： two if neurons were active simultaneously, then the strength of the connection between them should be increased. 淡江大學資管所侯永昌 46
The 1950 s and 1960 s: The First Golden Age of Neural Networks § Rosenbelt’s Perceptrons： adjust the weights to reduce the difference between the net output and the desired output. l single layer, iterative weight adjustment if an error occurs. l requires multiple passes l will converge in finite steps if linearly separable 淡江大學資管所侯永昌 47
The 1970 s: The Quiet Years § Minsky and Papert： demonstrate the limitations of perceptrons l single-layer nets can not solve the problem of XOR l lack general method of training a multilayer net 淡江大學資管所侯永昌 48
The 1980 s: Renewed Enthusiasm § Hopfield nets： Backpropagation l initialize weights to small random values l while (stopping condition is false) l for each training pair l Feedforward l backpropagation of errors l update of weights 淡江大學資管所侯永昌 49
The steps to train a neural network 1. random assignment of weights wijk 2. an input x from the training-data set is passed through the neural network， 求出 δ= f(x)actual - f(x)output 3. 利用 back-propagation將 errorδ分配回到 hidden layers 4. 利用每一個 node所分配到的 error，求weight wijk的 修正量 5. step 2 to 4 are iterated until the error value is acceptable 6. step 2 to 5 are repeated for all data in the training-data set (this method makes the neural network simulate the nonlinear relation between the input-output data sets) 7. Checking-data set is used to verify how well the neural network can simulate the nonlinear relationship between the input-output data sets 淡江大學資管所侯永昌 50
Error Propagation 淡江大學資管所侯永昌 51
Weight Adjustment 淡江大學資管所侯永昌 52
Neural Network § Neural Network 適用的場合 l (input， output)資料量很多 l input, output 之間的關係是 unknown or highly nonlinear § Neural Network 與 Expert System 的不同 l Expert system： learn by adding new rules to their knowledge base l Neural networks： learn by adding more data and modifying their overall structure. 淡江大學資管所侯永昌 53
Where are Neural Nets being Used? § § § § Signal processing Control Pattern recognition Medicine Speech production Speech recognition Business 淡江大學資管所侯永昌 54
遺傳演算法 淡江大學資管所侯永昌 55
Genetic Algorithms § Genetic Algorithms: use the concepts of Darwin's theory of evolution l Reproduction l Crossover l Mutation 淡江大學資管所侯永昌 56
Steps to find a solution 1. Different possible solutions to a problem are created， 以 bit string 來代表各種可能的解答 2. These solutions are tested for their performance (how good a solution they provided) 3. A fraction of the good solutions is selected, and the others are eliminated(survival of the fittest， 適者生存 ) 4. The selected solutions undergo the processes of reproduction, crossover, and mutation to create a new generation of possible solutions (which are expected to perform better than the previous generation) 5. steps 2 to 4 are repeated until there is convergence within a generation 淡江大學資管所侯永昌 57
Three genetic operators § reproduction: the process by which strings with better fitness values receive correspondingly better copies in the new generation. § i. e. , we try to ensure that better solutions persist and contribute to better offsprings during successive generations. 淡江大學資管所侯永昌 58
Three genetic operators § crossover: the process in which the strings are able to mix and match their desirable qualities in a random fashion. § Crossover rate 一般約 0. 5 - 1 淡江大學資管所侯永昌 59
Three genetic operators § mutation: the process by which the value of a randomly selected position is changed. It helps to increase the search power. § 例如：如果 selected solution 中第四個 bit 剛好通通是 0，而實際的 solution 中第四 個 bit 應該是 1，則無論如何做 reproduction、 crossover 都無法讓第四個 bit 變成 1， mutation 能讓 bit 0, 1 互換，因此 就成了唯一的 具 § mutation rate 的發生率很低，約 0. 005 bit/generation 淡江大學資管所侯永昌 60
Genetic Algorithms § Genetic Algorithms 的優點為：It searches for a solution from a broad spectrum of possible solutions ( 比較有可能找到 optimal solution), rather than restrict the search to a narrow domain where the results would be normally expected (可能找到只是 local optimal) § Genetic Algorithms 的 作重點在於： l How to code the possible solutions to the problem as finite bit strings ( Chromosomes ) l How to evaluate the fittness of each string 淡江大學資管所侯永昌 61
Optimizing a mathematical function § Given the following table, try to find a line y = C 1 x + C 2 Data number x y 1 2 3 1. 0 2. 0 3. 0 4 6. 0 § 假設 C 1、 2 為實數，而且 -2 C 1、 2 5 C C § Chromosomes: 各以 6 個 bits 來代表 C 1、 2 這兩 C 個係數 § 如何將這兩個係數變成實數呢？ 淡江大學資管所侯永昌 62
Optimizing a mathematical function -2 5 0 63 = -2 + 7*7/63 = -1. 22 § Fitness: § 當 yi’愈接近 yi 代表愈 fit， 為了求 maximum， 因 此用 400 來減 § fitness 愈好 ==> reproduction rate 就愈高 § 第二代之 average 較第一代好，但 maximum 則 未必 (因此需要保持菁英 ) 淡江大學資管所侯永昌 63
淡江大學資管所侯永昌 64
柔性技術的結合 淡江大學資管所侯永昌 65
Summary § Neural Network: learning ability § Fuzzy: reasoning system § Genetic Algorithm: learning ability + finding optimum solutions 淡江大學資管所侯永昌 66
柔性技術的結合 § 利用 GA來找尋合適的 fuzzy rule， 以及 fuzzy set分割的方式 l l l Chuck Karr, 1991. 2, “Genetic algorithms for fuzzy controllers, ” AI Expert, pp. 26 -33. Chuck Karr, 1991. 3, “Applying genetics to fuzzy logic, ” AI Expert, pp. 38 -43. Hisao Ishibuchi, Ken Nozaki, and Naohisa Yamamoto, “Selecting fuzzy rules by genetic algorithm for classification problems, ” pp. 11191124. 淡江大學資管所侯永昌 67
柔性技術的結合 § 利用 GA來找尋 NN的 控制參數或權重 l l David J. Janson and James F. Frenzel, 1993, “Training product unit neural networks with genetic algorithms, ” IEEE Expert, pp. 26 -33. Belinda Choi and Kevin Bluff, 1995, “Genetic optimisation of control parameters of a neural network, ” pp. 174 -177. 淡江大學資管所侯永昌 68
相關的研究 § 侯永昌，王財源，1995. 10, 「應用模糊理論於資 訊系統績效水準評估之研究」， 資訊管理研究 及實務研討會論文集 ，彰化， 304 -312。 pp. § Y. C. Hou, C. M. Chen and W. T. Wu, 1997. 12, “Fuzzy Structural Query Processing in Fuzzy Relational Database, “ International Journal of Information Management and Engineering, Vol. 2, No. 2, pp. 1 -15. § 侯永昌，楊雪花，1998, 「以模糊理論和遺傳演算 法為基礎的中文文件自動分類之研究」， 模糊 系統學刊 ，第四卷，第一期，pp. 45 -57。 淡江大學資管所侯永昌 69
相關的研究 § Y. C. Hou and Y. H. Chang， 2002. 9， ”The New Efficient Hierarchy Combination Encoding method of Evolution Strategies for Production Allocation problems, ” Computers & Industrial Engineering, Vol. 43, Issue 3, pp. 577 -589. § 侯永昌，張雅惠，2003. 06 ，「應用遺傳演算法於 向量量化之新編碼簿設計法」， 電腦學刊 ，第十 五卷，第二期，pp. 16 -28。 § Y. C. Hou and Y. H. Chang，2003. 06, “Dynamic Programming Variant in Evolution Strategies for Production Allocation Problems, ” MIS Review, Vol. 12, pp. 1 -16. 淡江大學資管所侯永昌 70
相關的研究 § 侯永昌，許慶昇，2003. 12，「以機率為基礎之像 素不擴展的多機密影像視覺密碼最佳化模型」， 資訊、科技與社會 ， Vol. 3, No. 2, pp. 19 -37。 § Y. C. Hou and Y. H. Chang，2004. 09, “A New Efficient Mode Genetic Algorithms for the Generalized Plant allocation Problem, ” Journal of Information Science and Engineering, Vol. 20, No. 5, pp. 1019 -1034. (SCIE, EI) § 侯永昌，許慶昇，2005. 01，「以機率為基礎之像 素不擴展的視覺密碼最佳化模型」， 資訊管理 學報 ，第十二卷，第一期，pp. 91 -109。 (TSSCI) 淡江大學資管所侯永昌 71
Скачать презентацию 柔性計算的研究現況與發展 淡江大學 資訊管理系所 侯 永 昌 http mail
dbd32b564b10f62bf5ec3a54684d71ee.ppt