The Unicode Standard: 5.0 - Hardcover

Hardcover

ISBN 10: 0321480910 ISBN 13: 9780321480910

Publisher: Addison-Wesley Professional, 2006

Synopsis

"Hard copy versions of the Unicode Standard have been among the most crucial and most heavily used reference books in my personal library for years."

--Donald E. Knuth, The Art of Computer Programming

"For more than a decade, Unicode has been a foundation for many Microsoft products and technologies; Unicode Standard Version 5.0 will help us deliver important new benefits to users."

--Bill Gates, chairman, Microsoft Corporation

"The path W3C follows to making text on the Web truly global is Unicode."

--Sir Tim Berners-Lee, kbe, Web inventor and director of the World Wide Consortium (W3C)

"Without Unicode, Java wouldn't be Java, and the Internet would have a harder time connecting the people of the world."

--James Gosling, Inventor of Java, Sun Microsystems, Inc.

These and other software luminaries recognize that Unicode has become an indispensable tool for supporting an increasingly global marketplace (see inside for more acclaim). A comprehensive system of standards for representing alphabets throughout the world, Unicode is the basis for modern programming-- Windows, XML, Python, PERL, Mac OS, Linux--and every major search engine and browser in operation today.

New to Unicode Version 5.0

• A stable foundation for Unicode Security Mechanisms
• Property data for the Unicode Collation Algorithm and Common Locale Data Repository
• Improvements to the Unicode Encoding Model for UTF-8
• Rigorous stability of case folding and identifiers for improved interoperability and backward compatibility--enabling additional new ways to optimize code
• A systematic framework for improved text processing for greater reliability--covering combining characters, Unicode strings, line breaking, and segmentation

This new edition of Unicode's official reference manual has been substantially updated to document the latest revisions to the Unicode Standard, with hundreds of pages of new information. It includes major revisions to text, figures, tables, definitions, and conformance clauses, and provides clear and practical answers to common questions. For the first time, the book contains the Unicode Standard Annexes, which specify vital processes such as text normalization and identifier parsing.

These improvements are so important that Version 5.0 is the basis for Microsoft's Vista generation of operating systems, and is included in upgrade plans for Google, Yahoo!, and ICU, to name but a few.

This is the one book all developers using Unicode must have.

About this title

About the Author

The Unicode Consortium is a non-profit organization founded to develop, extend and promote use of the Unicode Standard, which specifies the representation of text in modern software products and standards. The membership of the consortium represents a broad spectrum of corporations and organizations in the computer and information processing industry. The Unicode Consortium actively cooperates with many of the leading standards development organizations, such as ISO/IEC JTC1, W3C, IETF, and ECMA. The Consortium is based in Mountain View, CA, with members located around the world.

Excerpt. © Reprinted by permission. All rights reserved.

This book, The Unicode Standard, Version 5.0, together with the Unicode Character Database, is the authoritative source of information on Version 5.0 of the Unicode character encoding standard.

Version 5.0 of the standard is a significant departure from prior versions. It lays out much clearer requirements for supporting Unicode and provides more explicit guidance for implementers to quickly embrace the proliferation of new growth technologies and emerging markets while at the same time meeting users' needs for secure, robust software.

Why Buy This Book

In a major enhancement, Version 5.0 of the Unicode Standard is now available in a smaller, more convenient size while including much more textual content. Most notably, for the first time the book includes all of the Unicode Standard Annexes, which provide specifications for vital processes such as text normalization, bidirectional handling, and identifier parsing.Version 5.0 contains the knowledge gained from many years of worldwide implementation experience and has been enhanced significantly: the text incorporates 15 years of user feedback, provides thorough answers to the many questions users of Unicode have raised, and is much more accessible--with greatly improved figures and tables, and with the text revised for clarity.

• Four-fifths of the figures are new.
• Two-thirds of the definitions are new.
• One-half of the Unicode Standard Annexes are new.
• One-third of the conformance clauses are new.
• One-fourth of the tables are new.

In addition, the text of Version 5.0 reflects advances in the computer implementation of writing systems. It substantially improves the descriptions of rendering Indic scripts to meet the demands of this area of growing market importance--Unicode-based implementations are supported by the government of India, and this book explains how to build them. Version 5.0 also highlights the newly established core CJK subset of characters, IICore, which is critical for rendering and interoperability in the East Asian market.

In short, The Unicode Standard, Version 5.0, enables developers to implement quickly the latest advances for worldwide software users while opening new opportunities in high-growth markets. The changes from Versions 3.0 and 4.0 to Version 5.0 are major and important--this is the one book all Unicode implementers must have.

Why Upgrade to Version 5.0

Version 5.0 of the Unicode Standard brings significant improvements beyond Versions 3.0 and 4.0. The industry has noticed and is quickly moving to Version 5.0--Windows Vista runs on 5.0; ICU, Google, and Yahoo! all have plans to upgrade to 5.0. Internet and W3C protocols are built on Unicode and are continually adapting to the latest versions. The International Standard ISO/IEC 10646 is also synchronized with Version 5.0.

This latest version of the Unicode Standard is the basis for Unicode security mechanisms, the Unicode collation algorithm, the locale data provided by the Common Locale Data Repository, and support for Unicode in regular expressions. Improved expression of the Unicode encoding model makes it much clearer how implementers need to support the representation of Unicode text in UTF-8 and other encoding forms. Character properties have been systematized and greatly extended to help implementers in support of Unicode text processing. The standard has also established principles of stability for casefolding and identifiers, crucial for interoperability and backward compatibility for formal language use and in other contexts that depend on exact usage and matching of identifiers.

Version 5.0 delivers a stable, practical character processing model in sync with today's information technology needs. Unicode now offers:

• Round-trip compatibility with the Chinese standards GB18030 and HKSCS
• The specification of the newly established core CJK subset of characters, IICore
• Refinements to casing and bidirectional behavior to meet industry requirements
• Improved Indic rendering guidelines
• Better guidance on the handling of combining characters, Unicode strings, variation selectors, line breaking, and segmentation

Implementers who want to keep pace with the industry and take advantage of a stable foundation for security, to align with the latest collation and locale data definitions, and, most importantly, to expand their market reach need to upgrade to Version 5.0 as soon as possible.

Detailed Change Information

See Appendix D, Changes from Previous Versions, for detailed information about the changes from previous versions of the standard, including character counts, stability guarantees, and updates to the Unicode Character Database and Unicode Standard Annexes.

Version 5.0 of the Unicode Standard corresponds to ISO/IEC 10646:2003 plus Amendments 1 and 2 to that standard and four characters to support Sindhi from Amendment 3.

Organization of This Book

This book and the Unicode Character Database define Version 5.0 of the Unicode Standard. The book gives the general principles, requirements for conformance, guidelines for implementers, character code charts and names, and the Unicode Standard Annexes.

Concepts, Architecture, Conformance, and Guidelines

The first five chapters of Version 5.0 introduce the Unicode Standard and provide the fundamental information needed to produce a conforming implementation. Basic text processing, working with combining marks, and encoding forms are all described. A special chapter on implementation guidelines answers many common questions that arise when implementing Unicode.

Chapter 1 introduces the standard's basic concepts, design basis, and coverage and discusses basic text handling requirements.Chapter 2 sets forth the fundamental principles underlying the Unicode Standard and covers specific topics such as text processes, overall character properties, and the use of combining marks.

Chapter 3 constitutes the formal statement of conformance. This chapter also presents the normative algorithms for three processes: the canonical ordering of combining marks, the encoding of Korean Hangul syllables by conjoining jamo, and default casing.

Chapter 4 describes character properties in detail, both normative (required) and informative. Tables giving additional character property information appear in the Unicode Character Database.

Chapter 5 discusses implementation issues, including compression, strategies for dealing with unknown and unsupported characters, and transcoding to other standards.

Character Block Descriptions

Chapters 6 through 16 contain the character block descriptions that give basic information about each script or group of symbols and may discuss specific characters or pertinent layout information. Some of this information is required to produce conformant implementations of these scripts and other collections of characters.

Code Charts

Chapter 17 gives the code charts and the Character Names List. The code charts contain the normative character encoding assignments, and the names list contains normative information as well as useful cross references and informational notes.

Han Radical-Stroke Index

Chapter 18 provides a Han radical-stroke index for the IICore subset of CJK ideographs. This index aids in locating specific, common ideographs encoded in the Unicode Standard.

Appendices

The appendices contain detailed background information on important topics regarding the history of the Unicode Standard and its relationship to ISO/IEC 10646.

Appendix A documents the notational conventions used by the standard.

Appendix B provides abstracts of Unicode Technical Reports and lists other important Unicode resources.

Appendix C details the relationship between the Unicode Standard and ISO/IEC 10646.

Appendix D lists the changes to the Unicode Standard since Version 4.0.

Appendix E describes the history of Han unification in the Unicode Standard.

Appendix F reproduces the text of the policies of the Unicode Consortium regarding character encoding stability.

Glossary, References, and Indices

The appendices are followed by a glossary of terms, a bibliography, and two indices: an index to Unicode characters and an index to the text of the book.

Unicode Standard Annexes

The Unicode Standard Annexes are printed in the back of this book, following the indices. These annexes form an integral part of the Unicode Standard. Conformance to a version of the Unicode Standard includes conformance to its Unicode Standard Annexes.

Unicode Standard Annex #9, "The Bidirectional Algorithm," describes specifications for the positioning of characters in mixed-directional text, such as Arabic or Hebrew.

Unicode Standard Annex #11, "East Asian Width," presents the specification of an informative property for Unicode characters that is useful when interoperating with East Asian legacy character sets.

Unicode Standard Annex #14, "Line Breaking Properties," presents the specification of line breaking properties for Unicode characters.

Unicode Standard Annex #15, "Unicode Normalization Forms," describes specifications for four normalized forms of Unicode text.

Unicode Standard Annex #24, "Script Names," specifies an assignment of script names to all Unicode code points.

Unicode Standard Annex #29, "Text Boundaries," describes guidelines for determining default boundaries between certain significant text elements: grapheme clusters ("user-perceived characters"), words, and sentences.

Unicode Standard Annex #31, "Identifier and Pattern Syntax," describes specifications for recommended defaults for the use of Unicode in the definitions of identifiers and in pattern-based syntax.

Unicode Standard Annex #34, "Unicode Named Character Sequences," defines the concept of a Unicode named character sequence and a set of rules constraining possible names applied to character sequences.

Unicode Standard Annex #41, "Common References for Unicode Standard Annexes," contains the listing of references shared by other Unicode Standard Annexes.The 5.0.0 version of each UAX is included on the CD-ROM. All versions, including the most up-to-date versions of all Unicode Standard Annexes, are available on the Unicode Web site:

http://www.unicode.org/reports/

The Unicode Character Database

The Unicode Character Database (UCD) is a collection of data files containing character code points, character names, and character property data. It is described more fully in Section 4.1, Unicode Character Database. All versions, including the most up-to-date version of the Unicode Character Database, are found on the Unicode Web site:

http://www.unicode.org/ucd/

The files for Version 5.0.0 of the Unicode Character Database are also supplied on the CD-ROM that accompanies this book.

Information on versioning and on all versions of the Unicode Standard can be found on the Unicode Web site:

http://www.unicode.org/versions/

Unicode Technical Standards and Unicode Technical Reports

Unicode Technical Reports and Unicode Technical Standards are separate publications and do not form part of the Unicode Standard.

All versions of all Unicode Technical Reports and Unicode Technical Standards are available on the Unicode Web site:

http://www.unicode.org/reports/

The latest available version of each document at the time of publication is included on the CD-ROM. See Appendix B, Unicode Publications and Resources, for a summary overview of important Unicode Technical Standards and Unicode Technical Reports.

On the CD-ROM

The CD-ROM contains additional information, such as sample code, which is maintained on the Unicode FTP site:

ftp.unicode.org

It is also available via HTTP:

http://www.unicode.org/Public/

For the contents of the CD-ROM, see its ReadMe.txt file.

Updates and Errata

Reports of errors in the Unicode Standard, including the Unicode Character Database and the Unicode Standard Annexes, may be reported using the online reporting form:

http://www.unicode.org/reporting.html

A list of known errata is maintained on the Unicode Web site:

http://www.unicode.org/errata/

Any currently listed errata will be fixed in subsequent versions of the standard.

Edition details

• Publisher: Addison-Wesley Professional
• Publication date: 2006
• ISBN 10: 0321480910
• ISBN 13: 9780321480910
• Binding: Hardcover
• Language: English
• Edition number: 5
• Number of pages: 1417
• Editor: Unicode Consortium, Allen Julie D., Becker Joe, Cook Richard, Davis Mark