Data Access for Highly-Scalable Solutions: Using SQL, NoSQL, and Polyglot Persistence (Microsoft patterns & practices)

All applications use data, and most applications also need to store this data somewhere. In the world of business solutions, this often meant creating a relational database. However, relational technology is not always the best solution to meet the increasingly complex data-processing requirements of modern business systems, especially when this processing involves storing and retrieving massive amounts of data.

The advent of NoSQL databases has changed the way in which organizations have started to think about the way in which they structure their data. There is no standard definition of what a NoSQL database is other than they are all non-relational. They are less generalized than relational databases, but the driving force behind most NoSQL databases is focused efficiency and high scalability.

The downside of NoSQL is that no single database is likely to be able to support the complete range of business requirements mandated by your applications. How do you select the most appropriate database to use, or should you remain with the relational model? A modern business application is not restricted to using a single data store, and an increasing number of solutions are now based on a polyglot architecture. The key to designing a successful application is to understand which databases best meet the needs of the various parts of the system, and how to combine these databases into a single, seamless solution.

This guide helps you understand these challenges and enables you to apply the principles of NoSQL databases and polyglot solutions in your own environment.

To help illustrate how to build a polyglot solution, this guide presents a case study of a fictitious company faced with building a highly scalable web application capable of supporting many thousands of concurrent users.

John Sharp is a principal technologist at Content Master, part of CM Group Ltd (www.contentmaster.com) , a technical authoring and consulting company. An expert on developing applications with the Microsoft .NET Framework and Windows Azure, John has written several books, including Microsoft Visual C# Step By Step and Microsoft WCF Step By Step. Over the past 25 years, John has produced numerous training courses, white papers, and other technical material covering a range of diverse topics, including C and C++ programming, SQL Server database administration, Java enterprise application development, service-oriented architecture, and Unix programming. He has a degree in Computer Science from Imperial College, University of London.

Douglas McMurtry is a Senior .NET Developer Architect with Ciber working on medium to large scale enterprise applications. Doug has spent all 11 years of his career working with the .NET Framework and spends most of his time in the web stack. Having worked on many different types of applications he specializes in C#, data access, REST and RPC services, and all flavors of ASP.NET.

Andrew Oakley is a Senior Program Manager on the patterns & practices group at Microsoft. He has more than 15 years of enterprise software development and architecture experience, primarily on the .NET platform. He is passionate about delivering to Microsoft customers quality guidance to help them build more stable, scalable, and maintainable software systems.

Mani Subramanian is a software tester on the patterns & practices team. His recent projects include Enterprise Library (Windows Azure, V5), CQRS journey, Unity, Prism 4.1. Areas he has worked on include core.net, Windows Azure and Windows Phone, BizTalk ESB, performance, security, and test management. He has 12+ years of industry experience. Prior to joining p&p, he was a developer for products that enable network security and worked as a project manager and test consultant.

Hanzhong (Hanz) Zhang is a senior software development engineer in test with the Microsoft patterns & practices team, primarily working on Windows Azure, SharePoint, and Enterprise Library. Before joining Microsoft, he focused on the document management system as Senior Software Engineer with Sharp Laboratories of America at Huntington Beach, California. Hanz has more than 15 years of experience with software development, architecture and testing.