01Why do computers turn letters into numbers?

We see the letters A, B, and C, but what a computer actually stores and processes is still a string of bits.

So we first need to agree on a set of rules: “Which number represents each character?”

A

Character

A symbol people can see

65

Number

Assigned by a character-encoding rule

01000001

Binary data

Ultimately stored in the computer

02How does ASCII assign numbers to letters?

Early computers mainly handled English text. People created a well-known character-encoding standard called ASCII.

ASCII assigns numbers to English letters, digits, punctuation marks, and some control characters.

A6501000001
B6601000010
C6701000011
a9701100001

For example, the ASCII number for uppercase A is 65. Stored as 8 bits, it can be written as 01000001.

03Are a character and its number the same thing?

No. This is one of the most important ideas in understanding character encoding.

Character

For example, the character “A”

Number / code point

The number assigned to that character

Actual bytes

The specific bytes encoded in a file or in memory

04Are English letters enough?

Of course not. The world also has Chinese, Japanese, Korean, Arabic, many kinds of symbols, and emoji.

That is why Unicode was created. It aims to assign a shared code point to a huge number of characters around the world.

A

U+0041

Latin letter A

中

U+4E2D

Chinese character “中”

你

U+4F60

Chinese character “你”

😀

U+1F600

An emoji

05What does UTF-8 do?

UTF-8 is a way to encode Unicode code points as byte sequences.

One of its features is that different characters can use different numbers of bytes.

AU+0041
1 byte41Written in hexadecimal
中U+4E2D
3 bytesE4 B8 ADThree bytes in UTF-8

06Why can one character take different numbers of bytes?

Because “which character is it?” and “how is it encoded for storage?” are two different questions.

For example, the Unicode code point for A is U+0041, but its stored byte sequence can differ depending on the encoding.

UTF-8

41

1 byte

UTF-16LE

41 00

2 bytes

UTF-32LE

41 00 00 00

4 bytes

07How does garbled text happen?

If a sequence of bytes was saved as UTF-8 but a program reads it using a different encoding, it may appear as strange characters.

Original bytesE4 B8 AD
Interpreted as UTF-8中
Interpreted using the wrong encodingGarbled text

What you have learned

  • To store letters, a computer first maps each character to a number, then stores it as binary data.
  • ASCII was a very important early character encoding; the ASCII number for A is 65.
  • A character, its number (code point), and the bytes actually stored are not the same thing.
  • Unicode assigns shared code points to a huge range of characters. For example, A is U+0041 and “中” is U+4E2D.
  • UTF-8 is one way to encode Unicode code points as specific byte sequences.
  • In UTF-8, ASCII characters usually take 1 byte, while common Chinese characters usually take 3 bytes.
  • The same character may take different numbers of bytes in UTF-8, UTF-16, UTF-32, and other encodings.
  • Garbled text often happens when different character encodings are used to write and read the data.

Next question: English letters can be assigned numbers, but how does a computer recognize and store thousands of Chinese characters? Next article: How Does a Computer Understand Chinese?