Based on a course given to internal managers at Texas Instruments, this book is an introduction to neural nets for computer science, artificial intelligence and R & D professionals, as well as MIS or DP managers.
"The responsibility for change...lies within us. We must begin with ourselves, teaching ourselves not to close our minds prematurely to the novel, the surprising, the seemingly radical."- Alvin ToefflerThis book originally existed as a half-day seminar, developed during the spring and summer of 1988. The target audience was and still is technical managers and people in other technical fields who want to find out something about neural networks.
We expanded our seminar materials, dropping some things and adding several new sections. The presentation still somewhat parallels our previous work. Many of the graphics used as overhead transparencies in the seminar have been reproduced here. It is difficult in book form, however, to "break" and show software examples. We have tried to overcome this by including numerous examples in the text. Chapter 9 is our own "show-and-tell" chapter.
The style is one in which we let ideas grow. First we mention topics, then we build on them. If you find yourself wanting to know more, you may well find that we pick up on your particular interests again later. The bibliography/reading list in Appendix B provides further references for study. Many of our ideas came from these references as well as from conversations with our seminar students and various people in the field. Other ideas came from our own work with some of the software packages. In general, people working in neural networks are very willing to share their writings, their work, and their ideas.
When we did our initial seminar study in 1988 we kept track of what our own questions were and what organization of topics made sense to us. These questions formed the outline for our twelve chapters. Some of our notes became part of the chapters as well as the appendixes. Many articles were initially too technical for us; some still are. By having started at the beginning ourselves, we feel we are in a good position to piece things together for novices and for the casual reader.
Chapter 1 explores the utility questions: What can you do with neural networks? Why would anyone want to use them? What kinds of things are being done currently? What are some of the issues and problems this technology seeks to address?
Chapter 2 is also brief: What are neural networks? Why have they hit the media so hard in the past couple of years? What is going on in the current technological climate that supports this particular new technology?
Our questions in Chapter 2 lead us to Chapter 3, a review of the developments of the past few decades. Both of us have spent the majority of our efforts over the past few years in pursuing projects related to AI (Artificial Intelligence). We knew there were similarities and differences in the historical developments of these related technologies. We wanted to provide a context.
The biological underpinnings of neural networks were of interest to us for a variety of reasons. We have some rough ideas of how we, as humans, process information and learn. Because this new style of computing is so different from conventional computing, it helps to have a few familiar analogies. Chapter 4 begins with a look at the biological metaphor and some of the terminology. We want to point out here, and again throughout the book, that this metaphor is responsible for a great deal of confusion. Many people consider the name neural network itself to be an unfortunate choice (just as many consider that the name artificial intelligence has not been in the best interest of the field), yet it is the name most widely recognized today in popular writing. There is a definite effort by many people in the field to use nonbiological terms, emphasizing the differences between this new style of computing and the functions of the human brain. Yet, we wonder if some of the original biological terminology won't stick. Even though we begin by talking about the metaphor, the heart of Chapter 4 is a discussion of the components of a typical artificial neural network and how they work together.
Chapter 5 explores some of the characteristics and limitations of these artificial neural networks. What are they like? What are their strengths and weaknesses? Where have they encountered problems?
From our own background in AI, we have been extremely interested in the relationship of these two technologies, as well as how neural networks relate to statistical methods. Some early neural network aficionados came down rather hard on traditional AI - probably an overreaction to being the "step-child" for so many years. We are not a part of that camp. Both AI and neural networks have their strengths and uniqueness. We see the blending of these two as being extremely useful, each contributing their best to the overall effort in many projects. Chapter 6 compares technologies, relates some of the claims of each side, then settles into promoting the amalgam that will surely result from combining the strengths of the various approaches.
Chapter 7 grew out of a single overhead transparency in the original course which asked the question, "How many different ways can you organize a neural network?" This chapter takes the components outlined in the last part of Chapter 4 and develops variations and possibilities. This diversity is responsible for the assortment of neural network paradigms that have already surfaced and those that are being developed. We end Chapter 7 by noting some of the earliest and best-known models.
Perhaps the key feature of a neural network is its ability to adapt, or learn. Chapter 8 examines the question, "How do neural networks learn?" The work learning, as popularly used, is often synonymous with training, though some people make a distinction between the terms. Looking at learning/training aspects of a neural network necessitates drawing again on the components presented in Chapters 4 and 7. Several common learning laws are listed, along with some tables that compare various neural network models with respect to their learning features.
Chapter 9 gets to the question most asked by participants in our early seminars: "How do you move from theory to applications?" We use some of the current software simulations and our own work experience at Texas Instruments to walk you through five detailed case study examples of how to get started. Some things worked well for us, others did not. We learned a lot and often found ourselves excited by our discoveries.
In Chapter 10 we look at the question, "How are neural networks being implemented?" A few general categories are commonly used for comparing implementations, and we look first at those. Although many current neural networks are software simulations, many researchers are putting significant effort into hardware implementations. Both approaches have their niche. We look at commercially available implementations and some of the research categories. Hardware includes special-purpose hardware for conventional computers, new types of computers, chops, optical and holographic implementations. Wetware is a new category. Some researchers are building neural networks using biological materials. There are biochips, for example, with actual cells on an electrode grid.
"What has been happening in neural net research?" Chapter 11 looks at some of the recent research in terms of issues, problems, and directions. Some companies have shared with us stories of specific research questions and projects. Here we have an opportunity to expand on some of the ideas mentioned only briefly earlier.
Chapter 12 wraps it up. We found ourselves asking more questions even in our summarizing. What is intelligence? Are neural networks truly intelligent? How do we separate the hype from the facts? There are plenty of wild-sounding claims. Where do people say this technology is going? And how do we feel about it ourselves?
Appendix A contains our bibliography and reading list. Some of this list is annotated. At the end of Appendix A you can find information on some of the currently available journals and newsletters. Appendix B elaborates on some of the mathematics involved in designing neural networks. Appendix C uses Lotus 1-2-3 to demonstrate the training process for a single artificial neuron.
Obviously we have been interested enough to spend a considerable amount of time on artificial neural networks. Not only are we interested in them, we are excited about them. Enough so that we want to take as much as we can of what we have learned and share it with you. While you are looking for facts and using the keen analytical logic for which our era is well-known, keep an open mind, and maybe even dare to dream a little about a possible new future.
0201633787P04062001
