New Concepts for Natural Human Neural Networks and their Embodiment: A New Fuzzy Neural Network and Study Algorithm Dayong Li John R. Birge The University of Michigan Department of Industrial and Operations Engineering Ann Arbor, Michigan 48109-2117 Abstracts: First of all, this paper provides a new abstract brain model (a complex brain model) that is made of real brain and imaginary brain. Several new assumptions about the structure and operation mode of natural neural networks are then proposed. The key point is that knowledge is not stored in physical connections among billions of neurons but in a special unknown way in imaginary brain. On the basis of new philosophical concepts of the neural network, a new fuzzy neural network (FNN) has been introduced with help from fuzzy set theory and parallel network structure. FNN features fast study, no minima problem and zero learning error. Two training algorithms for FNN are also suggested. In the end, some initial tests confirm FNN advantages. Keywords: fuzzy neural network learning algorithm neural network brain model I. INTRODUCTION A. Artificial Neural Network and Biological Neural Network he Artificial Neural Network (ANN) is basically conceived from the biological neural network. It employs many simple but independent neurons and makes them work in an information processing mode of parallel distribution[l]. The special and complex function structure can allow ANN to be trained to approximate any function. This fact has led to great successes of neural networks in both the theoretical research and industrial application communities. B. Recent History Neural networks first became the subject of scientific study about fifty years ago. However until the mid- 1980s the expense of serial computers seriously hampered progress. Since that time, there has been an explosion of interest in the field. It has thrived on the exchange of new ideas and techniques from diverse areas of theory and application. Now neural network technology has become a powerful, new approach to the solution of challenging industrial problems. Neural networks are widely applied because of their powerful abilities to learn. Neural networks can be trained to pick out regularities in a very noisy environment. In principle, what a person's brain can do in information processing is what neural networks can do. The problems to which neural networks can be applied are those for which there is no precise model of any underlying process, or there is considerable variation in the parameters of the problem or of its surroundings. Hence a problem possessing one or other noises with poorly understood intrinsic structure and changing characteristics may be well treated by a neural network approach. There has indeed been great application success in industry. C. Is Information Encoded in Connection Weights? Nearly all the current neural networks are based apparently or latently on the following hypotheses of the natural neural network: * information is encoded in combinations of connection weights of a neural network, * signals are transmitted through physical linkages among different neurons. The training process is thus just the process to adjust limited connection weights to meet an expected training accuracy. The process usually makes the adjustment time too long to be accepted, especially for large-scale problems. One of the best known neural network models is the back-propagation model. It learns by changing its connection weights and the learned knowledge is stored by a rich combinations of these connection weights. Among many drawbacks of the back-propagation (BP) model is that it can not ensure successful learning because of wellknown local minima problems and long training times. Even though much work has been done to overcome these shortcomings [2,3,4,5,6,7], the problems still exist to some extent. Another neural network model is the probabilistic neural network (PNN) [8,9] that is based on a Bayesian decision law. It has several advantages over the back-propagation model, such as fast learning. It employs a one-pass learning algorithm to overcome the main drawbacks of the backpropagation model. However, its attention center is still at adjusting connection weights. The PNN model also suffers from large memory overhead when learning from many training samples. Moreover, it may not be suitable for fuzzy circumstances due to its probabilistic term. Many potential applications involve large data requirements to transfer into training samples studied 1

through neural networks. In these cases, it is hard to ensure that the study process can proceed smoothly to the expected precision. In the case of short-term power system load forecasting problem, for example, historical data for hourly and even annual time limits spans several years. It is hard to imagine a back-propagation model able to treat this large scale problem because the study time increases at least exponentially with problem size. To keep a study feasible, people sacrifice most original data and keep so-called valuable data considered subjectively. Of course, that is not our hope but we must face it often and reluctantly. It undoubtedly affects forecasting performance. The problems mentioned above are due to basic hypotheses about the natural neural network. It is hard to imagine that a real natural neural network works by adjusting linkages among neurons from time to time. In fact, a person does not take several minutes or more time to memorize a word. The problems of current neural networks make us propose some basic questions: * Is information encoded in natural neural networks? * How does the natural system work? These questions lead to new assumptions about natural neural network in this paper. D. Fuzzy Logic and Neural Network Fuzzy logic is a good form for depicting fuzzy conditions. Since many jobs done by a brain are fuzzy or semi-fuzzy, the interest of integrating fuzzy logic and neural networks has been a popular direction for ANN researchers [10,11,12,13,14,15]. Now most research is just to use the fuzzy function to pre-process input or postprocess the output of a neural network. However, the core neural network is still a back-propagation model. The study speed is therefore still slow. It is still hard to keep each study going well. The method still needs effort to determine an appropriate network structure (number of hidden layers, number of neurons of each hidden layer, etc.). Recently, however a few researchers have developed several fuzzy neural networks by defining fuzzy connection weights or even fuzzy neurons [13]. Thus training speed and other concerned performances are increased greatly. E. Research in This Paper First, we uncover some basic assumptions of current neural networks. A short discussion casts doubt on these traditional assumptions. A new complex brain model, composed of real brain and imaginary brain, is suggested as a foundation. From here, some new concepts about natural neural networks are put forward. The major viewpoint is that information may not be encoded in the combination of linkages among neurons. On this basis, several points about designing artificial neural networks are presented. Combining the strengths of fuzzy logic and neural network, we set up a new fuzzy neural network. Two study algorithms, a standard study algorithm and an instant study algorithm, are suggested. Finally, several experiments testify to the characteristics of this new neural network. II. PRESENT CONCEPTIONS FOR HUMAN BRAIN AND NEURAL NETWORK The human brain is too complex to be understood yet. It has been widely accepted that the human brain is the result of millions of years of evolution. It possesses many highly specialized component parts each associated with specific tasks, for example computation, memory and vision. Most of the current conceptions about the human brain and neural network mainly come from anatomy experiments. Some conclusions are from electric experiments on some simple neurons. From many physical or tangible findings[16,17,18], the key points about the human brain and neural network include the following * The most striking feature of human brain is seen in the cortex which has a folded, hemispherical structure. * Regions of the cortex are responsible for vision, auditory senses, voluntary movement and touch sensations. It is also crucial for long term memory. * The central nervous system constitutes the main part of the cortex. * The central nervous system is composed of approximately one hundred billion nerve cells or neurons. * Each nerve cell or neuron possesses a single axon along which it can pass electrical signals to other neurons. Incoming signals are carried by a neuron's dendrites that form a tree-like structure around the neuron. * The neurons send and receive signals through visible links with other neurons. The signal from one neuron reaches another at the junction of axon and dendrite -- the synaptic gap. * Different patterns of electrical firing activity are associated with different brain functions. ~ The brain is both robust and plastic -- able to adapt to new memories and functions. * The brain has the ability to form new connections between neurons. These connections take place at synapses and are mediated by the release of neurotransmitter chemicals. These neurotransmitters can alter the effective strength of the signal that can pass between neurons. 2

* The learning process is to form connections and adjust their strengths. Information is stored or coded in rich combinations of connection weights. * The powerful performances of the brain are supposed from the parallel and distributed structure of the neural network and the simultaneous processing capability of all neurons. Most so-called artificial neural networks are created in consideration of the above mentioned points. One striking feature of these neural networks Is that Information is stored and coded in rich combinations of connections among neurons. The learning process is just to form new connections or adjust connection weights. Practical experiences exhibit problems such as a long time for the learning process and learning stability. These problems are not seen in the biological situation. That is also why this paper casts doubt on the basic conceptions of the brain and neural network. III. SOME NEW ASSUMPTIONS ABOUT HUMAN BRAIN AND NEURAL NETWORK The human brain may be the most complex system known. Though understanding about the human brain in science is still very poor, from some facts in science, religion and Qigong [19], this paper tries to put forward a simple qualitative model about the brain and new concepts about the human brain and possible artificial representations This brain model is called a Complex Brain Model that is made of Real Brain and Imaginary Brain. Real brain is a physical brain that constitutes tangible structures such as cortex, neurons, dendrites, axons, and vessels. A central nervous system in parallel and distribution mode is positioned here. Imaginary Brain is the counterpart of real brain. Imaginary brain is not made of tangible or visible particles but in the form of high energy fields or waves (see Fig. 1). The complex brain model includes the following new assumptions about the human brain: ~ Real brain is in the form of physical structure It is tangible and visible. Imaginary brain is in the form of complex field or some other existence that has not been known by us. * Real brain exists in 3-D space. Imaginary brain exists outside 3-D space. * Real brain's function is to process commands from imaginary brain to control a human's activities or process information gained through sensory organs. universe universe Fig. 1 A Complex Brain Model * Imaginary brain, a special wisdom energy conglomerate, is the real center of information processing and storing. Large amounts of information or knowledge are finally stored here. Many functions of the brain including forecasting, computation and classification stay here. * Many function components in real brain act as control ends like relay systems in a power system. These functional components constitute neurons and connections formed through a lengthy period. * Imaginary brain sends information to real brain or the neural network through waves or fields not tangible wires. * Neurons and connections among neurons mainly act to transmit power or nutriments for neurons or form final control patterns. * The developing process of neurons and connections is the forming process of function components and not the process of memorizing information (generally considered the learning process). * All neurons in the human brain are well organized by imaginary brain. It works as software does in computers. * Control patterns corresponding to function components in real brain are destroyed when real brain dies. However, information and knowledge stored in imaginary brain remain forever. The assumptions mentioned above can be confirmed to some extent. In qigong and religious circles, many believe there is a special high energy and wisdom conglomerate named the true spirit which can leave the real brain for a certain time[19]. The true spirit can still watch, think, 3

judge, memorize, and learn after leaving real brain. The true spirit is similar to what we called imaginary brain. Imaginary brain has rich and profound functions. Powerful performances may actually come from imaginary brain not real brain. It is also found that so-called re-birth people can still think of what they have done in the past life though his body of the past life had been destroyed. That means there is something which can remain stable and keep knowledge or other information. It shows that knowledge or information is not stored in real brain but in something we called imaginary brain. For example, in science circles, Sir Erricle has confidence in his Nobel prize paper that there is something formless and invisible which organizes 10 billion neurons in real brain to work effectively[ 19]. It is not the main point to investigate the human brain exhaustively in this paper. Our goal is to enlighten the design of artificial neural networks. With the assumptions above, we have several principles for designing artificial neural networks. IV. NEW DESIGN PRINCIPLES OF ARTIFICIAL NEURAL NETWORKS After having new and maybe more natural concepts for the human brain and neural network, we put forward the following suggested design principles for designing artificial neural network structure which can produce various, high-performance artificial neural networks ~ Each neuron can send signals and receive signals in a wireless way from any other neurons ~ Knowledge is not stored in connections among neurons. * Each neuron has its own, independent and special signal processing function. Each neuron can store knowledge elements or knowledge groups and have standard input/output expressions as y = f(X,K,T) X: input vector received wirelessly. K: knowledge element T: threshold of a neuron y: neuron output sent wirelessly. * Operations of all neurons are well organized in a distributed and parallel way by a main program embedded in imaginary brain. * Power supply is not considered for the neurons operation. In the following sections, the above mentioned design principles will be used to design a fuzzy neural network (FNN). The main point of FNAN is that information is not memorized in connections among neurons. V. FUZZY NEURAL NETWORK This fuzzy neural network arises from the need to overcome a lengthy learning process and poor convergence of traditional neural networks (typically BP neural networks) and an urgent need to extract fine knowledge from huge amounts of original data. The process is like sifting gold from sands. Its basic ideas come from a fuzzy membership function, fuzzy decisions [20,21,22] and the distributed and parallel- structure of neural networks [1]. Association is realized by not only the integrating capacity of the network but also the fuzzy generalization of the knowledge elements. This is the result of combining neural networks and fuzzy logic. There have been some efforts to realize these combinations [10,16] with good performances. However most of these so-called neural networks still apply the BP neural network as a core program. Fuzzy sets techniques are just used to train input and output data. Training speed and convergence problems have not yet been overcome. Few researchers have recently tried to make connection weights fuzzy [13], though the concentration is still on adjusting connection weights. In this process, learning speed has been greatly enhanced and convergence guaranteed. Considering new concepts for designing neural networks, we do not try to store knowledge in physical links We makes neurons as fuzzy input/output information processing units. Each unit possesses the ability to send and receive signals wirelessly. By connecting these fuzzy neurons properly in a distributed and parallel way, we can construct a high performance fuzzy neural network. These design thoughts may require state-of-the-art hardware implementation but the human brain and neurons in it are complex and fine enough to realize this function. A. Fuzzy Sets and Fuzzy Decision For simplicity, the max-membership decision rule is applied for decision making. Other decision rules can also be used for certain problems. Suppose the domain X, X eR, A, B are fuzzy subsets of X |.IA(x): membership function for fuzzy subset A. gtl,(x): membership function for fuzzy subset B We suppose the following max-membership fuzzy decision law: If [^A(x) > lB(x), then judge x belongs toA If ^IA(X) < JB(X), then judge x belongs to B 4

If gA(x) = ltB(X), then judge x belongs to A and B at the same time. B. Fuzzy Neural Network 1. Fuzzy Neuron We suppose the following standard form for fuzzy neurons (illustrated in Fig. 2). y = f(X,K,T) X: input vector of a neuron K: knowledge element (input part of a sample) T: threshold for a neuron y: output of a neuron (X-K)2 Y=e a (3) We make element K fuzzy by adding a fuzzy membership function (3). This function can be taken as a membership function for knowledge element K. The parameter C represents the degree of fuzziness. Generally, we take it as 1.0. We can adjust it according to the needs of real-world problems. 3) category neuron A category neuron is placed in the knowledge layer. Its role is to produce the degree of membership for one knowledge category. This membership value for the knowledge category is equal to that of the point with the largest membership function value (possibility) in one knowledge category. So we suppose it has the form Xi y = max(xi) i=1 (4) x Note: n is the number of inputs to a category neuron. It shows that there are n knowledge elements under this knowledge category. 4) output neuron An output neuron is placed in the output layer. Its role is to produce a definite output 1 or 0 according to its own threshold t, where t is sent from a threshold neuron in the output layer. In this case, Fig. 2 An illustration for a fuzzy neuron In FNN, we have five types of neurons, input neurons (IN), knowledge neurons (KN), category neurons(CN), output neurons (ON) and a threshold neuron (TN). They are aH derived from the standard form of a fuzzy neuron. 1) input neuron An input neuron is placed in the input layer in FNN. It has only one input. No knowledge is stored in an input neuron. It has no threshold T. We suppose it as y = x (2) So that output equals input for an input neuron. We use the notation x as element of vector X while y is the output for an input neuron. 2) knowledge neuron A knowledge neuron is placed in the front part of the knowledge layer. Its role is just to store knowledge. A knowledge element is generalized and extended around this point through a fuzzy membership function. We suppose it has the form If, if x > t O, if x < t (5) Each output neuron in the output layer'corresponds to one output category. If the output representing a category is 1, then it says the current input vector belongs to this category, otherwise not. 5) threshold neuron There is only one threshold neuron in the output layer. A threshold is to produce a dynamic and changeable threshold for each output neuron. Its function is the same as that of a category neuron in the knowledge layer. It has the form y = max(xi), i=l where n is the input number of a threshold neuron. (6) 2. Fuzzy Neural Network Structure Consider the XOR problem in Table 1 as an example. Fig. 3 is the basic and original FNN network structure 5

before learning the XOR problem. Fig. 4 is the final FNN network structure after 8 samples have been learned. From these two figures, it can be found that FNN has three layers. They are the input layer, the knowledge layer and the output layer. The input layer receives the input vector and transmits it to the knowledge layer. The knowledge layer stores a knowledge and processes it (Note: knowledge is fuzzified and extended here.) The output layer treats (defuzzifies) the output values from the knowledge layer. In the input layer, there are three input neurons (IN) corresponding to three factors of the input vector. The knowledge layer of FNN, which is a supposed place to store knowledge in a fuzzy way in imaginary brain, is self-organized and self-improved. With the learning process going on, the knowledge layer has more and more knowledge elements until the training process stops. In the front part of the knowledge layer, there are eight knowledge neurons (KN) corresponding to 8 stored samples. The number in a neuron represents the number for a knowledge element. Here, all eight samples of the XOR problem are studied and stored. The output layer is to treat (defuzzify) the output values from the knowledge layer by the max-membership decision rule. From Fig. 4, we can find two outputs (yl and y2) and two output neurons (ON). This is because there are two output states (0 and 1) for the XOR problem. It can be also said that there are two categories from the eight samples. Thus there should be two category neurons (CN) in the knowledge layer. Note there is a threshold neuron (TN) in the output layer. It helps to produce final outputs from the output layer. VI. LEARNING ALGORITHMS A learning algorithm for a neural network is a process through which the neural network can realize all the inputoutput mappings of samples with acceptable learning error. Different neural networks have different study algorithms. As most of the current neural networks assume that knowledge is stored in rich combinations of connection weights, the study process is just the process to adjust connection weights. Hence, training a neural network often becomes quite hard. The FNN of this paper makes a different way on the basis of some new concepts of the human brain. The idea is to store knowledge in a knowledge layer that is supposed in a so-called imaginary brain properly. The study process is to select and store most typical knowledge elements from samples in a fuzzy way without work on connections. The FNN can be trained from a zero-knowledge network or a network with learned knowledge. We suppose it develops from an original, basic and zero knowledge network. That is we assume no knowledge elements (or knowledge neurons) in the knowledge layer. We take the XOR problem for example to express this explicitly. We have the original and basic network shown in Fig 3. Fig. 4 is a final structure of FNN when all eight samples have been trained. Yl Y2 OUTPUT LAYER KNOWLEDGE LAYER X1 X2 X3 INPUT LAYER Fig. 3 Basic Network Structure for XOR Problem yl Y2 LAIYER.. --- —--- ----- —.-.,~-{Sh ^ ^^"^ /s.'.... i^ '- -(C;^\ a - KNOWIJIDH QKN Q (-:6 b. ~ IN3 AYE r - -.-. I-NPJITF IAYER Xl X2 X3 Fig. 4 Final Network Structure for XOR Problem Two kinds of study algorithms exist in practical applications. They are called the standard study algorithm and the instant study algorithm, respectively. The standard 6

study algorithm is for the normal learning process that has the capability to choose knowledge as typical as possible and hence to make memory size as small as possible. The instant study algorithm is a one-cycle algorithm. In this case, all the samples are turned into knowledge elements and then the formed knowledge elements placed into FNN directly. Instant study algorithm can be a basis for designing fuzzy neural network databases. A. Standard Study Algorithm For simplicity, the study algorithm has two stages, the learning stage and the test stage, respectively. In the learning stage, samples are chosen one by one. Typical samples are added to the knowledge layer under the relevant categories in the form of (3). In the test stage, the trained FNN is checked to confirm that it can cover all samples with acceptable learning error. First, we input a sample vector and check if the network can produce the desired output for this sample. If yes, we do not adjust the network. Otherwise, we think the trained network cannot cover this sample. This sample is added as a new knowledge element under a related category in the knowledge layer. The new knowledge element is then fuzzified and extended around this point by adding a knowledge neuron with a chosen fuzzy membership function. In this way, the selected function for the knowledge neuron can guarantee that the new knowledge point has the greatest possibility (1.0) among the related categories. Meanwhile, the largest value among all the outputs of output neurons when the vector of this knowledge is input. This process guarantees the trained network can cover the sample. Then we can say this sample has been trained. The complete study algorithm has the following steps and loops. step 0 Input the operation mode to decide to train or test. step 1 Read in the input number and output number. Read in the number of total samples. Define the input vector and output vector. Define the sample array. Define a buffer to store all the samples. Let the learning count = 0. step 2 Let the learning count increase by I. step 3 Training Stage Circulation begins. The circulation ends when all samples have been studied. Then, go to step 4. step 3.1 Input one sample. step 3.2 If the desired output for the sample is a new category, there are no knowledge elements under this category. Set a new category and put a new knowledge element under this category. Go to step 3.1. If the desired output for the sample is not a new category, there is at least one knowledge element under this category. Go to step 3.3. step 3.3 Compute the final output of FNN with equations (2) (3) (4) (5) (6). If the real output is the same as we expect, pass the sample to the stack for later test or training. If not, add a new knowledge element in the form of (3) under the concerned category. step 4 Test Stage Circulation begins. The circulation ends when all the samples have been tested and the total learning error is computed. Go to step 5. step 5 If the total learning error is greater than 0.0, go to step 2. Otherwise, go to step 6. step 6 Stop and output training or test results as the knowledge base. B. Instant Study Algorithm The instant study algorithm is simple. All samples are turned into standard knowledge elements and read as knowledge bases into FNN directly. This is the training process. As each sample has its own concerned knowledge element in FNN, it is definite that the trained FNN can cover all the samples with a learning error of 0.0! VII. OOP C++ CODE FOR FUZZY NEURAL NETWORK To make the source code have more potential improvements in the future applications, we write the program with C++ in object-oriented program style. Five relation Fig. 5 An illustration of class hierarchy in FNN 7

classes are defined. They are class layer, class input layer, class knowledge layer, class output layer, class network layer. Class network layer has the friend relation with class layer and class knowledge layer. So does the class knowledge layer with class network layer. Class input layer, class knowledge layer and class output layer are all derived from class layer. The hierarchy relation can be seen in Fig. 5. VII. TESTS FOR FNN PERFORMANCES The experiments of this part are to test the performances of FNN. There are two problems discussed below. The first program is the XOR problem. That is a small-size but typical problem. The second one is to train and simulate short-term load forecasting problem. The latter is a very practical and large-scale problem. FNN shows its capacity to handle this large-scale problem. A. XOR problem The XOR problem is just to realize a set of input/output mapping relations as shown in Table 1. From Table 1, we see there are eight input/output relations (or samples). So in the input layer, there should be three input neurons. As for the output layer, two states (0 and 1) correspond to two output neurons. The basic and original network for this problem are seen in Fig. 3. At the beginning, there are no knowledge elements in the knowledge layers. Within the learning process, samples 1,2,3,4,5,6,7, and 8 are stored in the knowledge layer in the form of knowledge elements KNi, KN, KN3, KN4, KN5, KN6, KN:, and KN8. See the final network structure in Fig. 4. Table 1 XOR problem However, it is well known that it will take hundreds or even thousands of cycles to finish the learning process with enduring learning error through the back-propagation model. Suppose three more samples in Table 2 need to be trained. Table 2 Three More Samples NO Input Output 9 0.001.001 0.001 0 10 0.02 0.03 0.98 1 11 0.5 0.5 0.5 1 NO Input Output 1 0 0 0 0 2 0 0 1 1 3 0 1 0 1 4 01 1 0 5 1 0 0 1 6 1 0 1 0 7 1 1 0 0 8 1 1 1 1 After training three additional samples, we will find only Sample 11 has been stored as a new knowledge element. Sample 9 and Sample 10 can be covered by the network of Table 1. That occurs because Sample 9 and Sample 10 are very close to Sample 1 and Sample 2. However, Sample 11 is far from any samples in Table 1. So the network considers it typical and takes it as a new knowledge element and puts it under category 1. The above simple example shows another useful performance characteristic in addition to fast training. It can learn knowledge from samples. It can judge which samples have value and which do not. It stores those valuable samples as knowledge and discards those valueless samples which can be covered by the trained network. This performance is very useful in handling large-scale problems which usually have many redundant samples. The situations can be seen in short-term load forecasting signal processing, pattern recognition, knowledge engineering, and data mining. B. Short-term load forecasting problem Short-term load forecasting is to produce daily load forecasting or one week load forecasting hour by hour. It is crucial for optimal system operation. A good-performance short-term load forecasting must consider many influencing factors and system time-lags. There are usually large amounts of yearly original operation data and weather data hour by hour spanning several years. A necessity is the ability to handle this data and abstract fine knowledge from the original data. The short-term load forecasting problem in paper [24,25], includes 3-4 years of original data. After technical handling, we have more than 10 thousand samples. Each In this example, all eight samples are typical. So all eight samples are checked and stored. It is easy to test that the trained FNN can cover eight samples with learning error of 0.0! It takes only one cycle to finish learning. 8

sample has 50 inputs and 1000 outputs. So large a problem is too difficult to be trained by BP or other current networks. FNN is used for this problem with good results. 1) On the training stage: The large-scale short-term load forecasting problem is successfully completed through the standard study algorithm. From this most of the samples are chosen as knowledge elements in FNN. It takes about 14 hours to finish the training process on a Sun Workstation, justifying FNN's ability to handle large-scale problems. It can refine knowledge from original samples directly and quickly. 2) On the simulation and forecasting stage: After being trained, FNN has good simulation and forecasting error. Simulation error can be as low as 0.0%. Average daily forecasting error is about from 1.3% to 2.8%. 3) About multiple scenario forecasting: As FNN is philosophically based on fuzzy set theory, it can produce membership degrees for all the outputs. This characteristic can help produce many forecasting scenarios. It gives us more understanding for the forecasting results and is helpful for unit commitment problems [23]. IX. CONCLUSIONS AND FURTHER POTENTIAL APPLICATIONS In this paper, some new concepts about human brain have been put forward. The brain is assumed composed of real brain and imaginary brain. Knowledge does not exist in the form of connections among neurons but in imaginary brain in a special way. The training process is not the process to adjust connection weights from time to time but to choose most typical samples as knowledge elements. On this basis, several principles for designing artificial neural network have been proposed. With the help of fuzzy set theory and the parallel and distribution structure of neural network, a new fuzzy neural network is created. Two study algorithms, standard learning algorithm and instant learning algorithm, are suggested. With the fuzzy neural network and its training algorithms, the training process is stable and fast even when handling a large-scale problem. The training error can be reduced to 0.0. Two initial tests about the performances of FNN have been conducted. Expected results obtain small errors. The training process is just the evolution process for a network from the basic to the complex which can finally cover all samples. It exclude burdensome work on updating connection weights, concerns about a local minima problem and study speed. Each typical trained and memorized sample can find its own exact position in the network. Other samples can be covered by the network and are discarded. The process is simple and understandable. In China, there is a saying "The most essential, the simplest". One interesting advantage is that an output of a category just represents the degree of membership (possibility) showing the input belonging to this category according to fuzzy set theory. In fact, we use the maximum membership law to defuzzify the outputs in the output layer. That is to say, we can decide not only the exact category to which an input belongs but also its degree of membership of the input vector belonging to this category. That lays the foundation for scenario forecasting which is needed in some cases such as short-term load forecasting problems, This performance makes FNN potentially applicable in practical problem-solving, such as forecasting, knowledge system, neural network database, pattern recognition, noise filtering, data mining and so on. ACKNOWLEDGMENT The authors thank the US National Science Foundation and Electric Power Research institute for their sponsorship under Grant ECS-9216819. REFERENCES (mainly) [1] D. E. Rumehart and J. L. McClelland, Parallel Distributed Processing, vol. I. Cambridge, MA: MIT Press, 1988. [2] Li Dayong, Research On Macro-Decision Of Power System Development And Macro-Forecasting Of Electric Demand, Ph.D. dissertation, Electric Power Research Institute, China, 1993. [3] Wu Xuan, Research On Macro-Energy-Conservation And Forecasting Methods, Ph.D. dissertation, Electric Power Research Institute, China, 1994. [4] Wang Guangsen, Wang Pingyang and Hu Zhaoyi, Fuzzy Control On Back-Propagation Neural Network, Power Network Technology, May 1993. [5 ] R. J. Williams and J. Peng, An Efficient Gradient-Based Algorithm For On-Line Training Of Recurrent Network Trajectories, Neural Computation, vol. 2, no. 4, 1990. [6] Pierre F. Baldi and Kurt Homik, Learning In Linear Neural Networks: A Survey, IEEE Trans on Neural networks, vol. 6. No. 4, July 1995. [7] Barak A. Pearlmutter, Gradient Calculations For Dynamic Recurrent Neural Networks: A Survey, IEEE Trans on Neural networks, vol. 6. No. 5, September 1995. [8] D. F. Specht, Probabilistic Neural Network, IEEE Trans on Neural networks, vol. 3, 1990. [9] D. F. Specht, Probabilistic Neural Network For Classification, Mapping, Or Associated Memory, Proc. IJCNN, Vol. I. 1988 [10] S. HOrikawa et al, On Fuzzy Modeling Using Fuzzy Neural Networks With The Back-Propagation Algorithm, IEEE Trans on Neural Networks, vol. 3,Sept. 1992. [11] B. Kosko, Neural Networks and Fuzzy Systems: A dynamic System Approach to Machine Intelligence, Englewood Cliffs, NJ: Prentice-Hall, 1992. 9

[12] Yan_hwang Kuo et al, A Fuzzy Neural Network Model And Its Hardware Implementation, IEEE Trans On Fuzzy Systems, Vol. 2. No. 3, August 1994. [13] Hon Keung Kwan, A Fuzzy Neural Network Model And Its Application To Pattern Recognition, IEEE Trans on Fuzzy Systems, Vol. 2, No. 3, August 1994 [14] Sushmita Mitra and Sankar K. Pal, Fuzzy Multi-Layer Perception, Inference And Rule Generation, IEEE Trans On Neural Networks, Vol. 6, No. 1, August 1994 [15] Valluru B. Rao and Hayagriva V. Rao, C++ Neural Networks And Fuzzy Logic, MIS Press, 1993 [16] David F. Lindsley and J. Eric Holmes, Basic Neurophysiology, Elsevier Science Publishing Co. Inc., 1984 [17] Armand M. de callatay, Natural An Artificial Intelligence - -Misconception About Brains And Neural Networks, North Holland, 1992. [18] John G. Nicholls, A. Robert Martin and Bruce G. Wallace, From Neuron To Brain, Sinauer Associates, 1992 [19] Feng, Confirm The Philosophies Of Buddhism With Space Science And Nuclear Physics, Tianhua Publishing Inc., 1989 [20] Hans J. Zimmermann, Fuzzy Set Theory -- And Its Application, KLUWER-NIJHOFF PUBLISHING, 1985 [21] A. Kaufnan, H.M. Gupta, Fuzzy Mathematical Models In Engineering And Management Science. [22] Kurt J. Schmucker, Fuzzy Sets, Natural Language Computations And Risk Analysis, Computer Science Press. Inc. 1984. [23] Samer Takriti, John R. Birge, Erik Long, "Intelligent Unified Control Of Unit Commitment And Generation Allocation". Technical Report 94-26, with Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, September 1994 (to appear in IEEE Trans. PWRS). [24] John R. Birge, Dayong Li, Using Fuzzy Neural Network For Short-Term Load Forecasting (to be appeared) [25] John R. Birge and Dayong Li, Fuzzy Neural Network And Simulation Forecasting Model For Short-Term Load Forecasting, Technical Report, with Department of Industrial and Operations Engineering. University of Michigan, Ann Arbor, April 1996. 10