Skip to content
DDevToolery

16 April 2025 · 6 min read

Your string length is wrong

Code units, code points and grapheme clusters are three different counts, and JavaScript gives you the least useful one.

Ask JavaScript how long a string is and it answers with the number of UTF-16 code units. For English text that equals the number of characters. For anything else it may not.

"café".length         // 4 — or 5, if é is e + combining accent
"日本語".length         // 3
"🎯".length            // 2
"👨‍👩‍👧‍👦".length          // 11

Three different counts

  • Code units — what .length returns. UTF-16 encodes most characters in one 16-bit unit and everything above U+FFFF in two.
  • Code points — what Unicode assigns numbers to. Spread the string or use Array.from to count these.
  • Grapheme clusters — what a reader would call a character. An emoji family, a flag, or a letter with a combining accent is one grapheme built from several code points.

Where it breaks

A field limited to 100 characters by .length silently allows fewer visible characters for a Japanese user than an English one. Truncating at a code unit boundary can split a surrogate pair, producing an invalid string that renders as a replacement character. Reversing a string by splitting on the empty string mangles every emoji in it.

The bug is rarely noticed in testing, because test data is usually ASCII. It surfaces in production, from users whose names your validation was never written for.

Counting properly

const s = "👨‍👩‍👧‍👦 café";

s.length                                  // code units
[...s].length                             // code points
[...new Intl.Segmenter(undefined, { granularity: "grapheme" })
  .segment(s)].length                     // grapheme clusters

Intl.Segmenter is the correct tool and is now available in every current browser and in Node. Before it existed, everyone used a regex approximation that got flags and skin-tone modifiers wrong.

Normalisation

The same visible text can have more than one encoding. é may be a single code point, or e followed by a combining acute accent. They render identically and compare as unequal, which means a login that works on macOS can fail on Linux for the same typed password.

"café" === "cafe\u0301"                        // false
"café".normalize("NFC") === "cafe\u0301".normalize("NFC")  // true

Normalise to NFC on input — at the boundary, once — and compare afterwards. Doing it at comparison time instead means the stored value and the incoming value can still disagree.

Bytes are a fourth count

A database column declared VARCHAR(20) may count bytes rather than characters, and UTF-8 uses one to four bytes per code point. A twenty-character Japanese string can be sixty bytes and fail to insert. Check whether your column limit is measured in bytes or characters before assuming it matches your validation.

Tools mentioned