The Memory System : You Can't Avoid It, You Can't Ignore It, You Can't Fake It

Today, computer-system optimization, at both the hardware and software levels, must consider the details of the memory system in its analysis; failing to do so yields systems that are increasingly inefficient as those systems become more complex. This lecture seeks to introduce the reader to the most important details of the memory system; it targets both computer scientists and computer engineers in industry and in academia. Roughly speaking, computer scientists are the users of the memory system and computer engineers are the designers of the memory system. Both can benefit tremendously from a basic understanding of how the memory system really works: the computer scientist will be better equipped to create algorithms that perform well and the computer engineer will be better equipped to design systems that approach the optimal, given the resource limitations. Currently, there is consensus among architecture researchers that the memory system is "the bottleneck," and this consensus has held for over a decade. Somewhat inexplicably, most of the research in the field is still directed toward improving the CPU to better tolerate a slow memory system, as opposed to addressing the weaknesses of the memory system directly. This lecture should get the bulk of the computer science and computer engineering population up the steep part of the learning curve. Not every CS/CE researcher/developer needs to do work in the memory system, but, just as a carpenter can do his job more efficiently if he knows a little of architecture, and an architect can do his job more efficiently if he knows a little of carpentry, giving the CS/CE worlds better intuition about the memory system should help them build better systems, both software and hardware. Table of Contents: Primers / It Must Be Modeled Accurately / ...\ and It Will Change Soon

Bruce Jacob is an Associate Professor and Director of Computer Engineering in the Dept. of Electrical and Computer Engineering at the University of Maryland, College Park. He received his Ars Baccalaureate, cum laude, in Mathematics from Harvard University in 1988, and his M.S. and Ph.D. in Computer Science and Engineering from the University of Michigan in 1995 and 1997, respectively. In addition to his academic credentials, he has extensive experience in industry: he designed real-time embedded applications and real-time embedded telecommunications architectures for two successful Boston-area startup companies: Boston Technology (now part of Comverse Technology) and Priority Call Management (now part of uReach Technologies). At Priority Call Management he was employee number 2, the system architect, and the chief engineer; he built the first working prototype of the company's product, and he built and installed the first actual product as well. In recognition of Prof. Jacob's research program, he has been honored several times as a University of Maryland "Rainmaker." In memory-systems research, Jacob's cache and memory- management designs demonstrated the viability of software-managed caches in general-purpose systems (he coined the now-common term "software-managed cache" in a 1998 ASPLOS paper). His work in advanced DRAM architectures at Maryland is the first comparative evaluation of today's memory technologies, and he received the CAREER Award from the National Science Foundation for his early work in that area. Honors for his teaching include the departmental George Corcoran Award, the University of Maryland Award for Teaching Excellence, and his 2006 induction as a Clark School of Engineering Keystone Professor. He has published over 50 papers on a wide range of topics, including computer architecture and memory systems, low-power embedded systems, electromagnetic interference and circuit integrity, distributed computing, astrophysics, and algorithmic composition. His recently published book on computer memory systems (Jacob, Ng, and Wang: Memory Systems-Cache, DRAM, Disk, Morgan Kaufmann Publishers, Fall 2007) is large enough to choke a small elephant.