A string can look like three characters on a screen while containing four UTF-16 code units. The same text can occupy another number of bytes in a file. Those counts answer different questions. A report builder also needs to distinguish the finished text from the repeated operations used to construct it. Understanding these boundaries prevents broken character slicing and unexpected replacement characters in imported data.
The examples use .NET 8 and C# 12. You will assemble a small report with StringBuilder, count Unicode scalar values using Rune, and explicitly encode UTF-8. A separate exercise rejects malformed byte input. None of these operations assumes that one visible symbol always equals one char.
Strings are immutable and builders are mutable
A C# string is immutable. Concatenation produces a new result, although the compiler and runtime can optimize many simple expressions. A few readable concatenations are usually fine. When a loop repeatedly grows a large string, StringBuilder expresses incremental construction and can avoid many intermediate strings. Append the pieces, then call ToString once when the finished text is needed.
A builder is a mutable object. Sharing the same instance across concurrent operations requires coordination, so prefer a local builder owned by one operation. An initial capacity can reduce growth allocations when a useful estimate exists, but it is not an input limit. A hostile input can still cause excessive output. Bound the number of records and the size of their text independently. Choose a builder because the workflow benefits from it, rather than mechanically replacing every interpolation.
A char is a UTF-16 code unit
.NET strings store UTF-16 code units, and string.Length counts those units. A char represents one code unit. Many common characters fit in one unit, but Unicode characters outside the basic multilingual plane need a surrogate pair. Indexing either half does not produce the complete character. Taking a substring through a pair can create ill-formed text.
Rune represents a Unicode scalar value. Enumerating runes treats a valid surrogate pair as one scalar value, making it useful for character-aware classification. Scalar values are still not the same as user-perceived characters. A letter plus a combining mark can form one visible grapheme while containing two scalar values. Use text-element APIs such as StringInfo when the feature needs grapheme boundaries. First identify whether the requirement counts storage units, scalar values, visible text elements, or encoded bytes.
Encoding turns text into bytes
An encoding defines how characters are represented as bytes. UTF-8 uses a variable number of bytes per scalar value, so the byte length cannot generally be derived from string.Length. When writing a file or sending a message, agree on the encoding with the reader. Decoding arbitrary bytes using the wrong encoding can corrupt text even when the program runs without an exception.
Fallback behavior matters. Some decoders replace malformed input with a replacement character. That is useful for display recovery but can hide a damaged import. new UTF8Encoding(false, true) configures UTF-8 without a generated byte-order mark and requests exceptions for invalid sequences. The constructor options do not magically detect another file's encoding. A file format should name UTF-8 explicitly, and the importer should explain invalid-byte failures without exposing sensitive contents.
Normalization and comparison are separate policies
Unicode can represent some visually similar text through different scalar sequences. Normalization can transform canonically equivalent forms into a consistent representation. It is useful when a contract requires that equivalence, but it does not make every visually similar symbol identical. Do not use normalization as a blanket security solution for identifiers.
Comparison also needs a purpose. For internal protocol tokens, ordinal comparison usually avoids culture-dependent linguistic behavior. For user-facing sorting, an appropriate culture can be correct. Do not normalize, lowercase and truncate a user's name merely to force it into an internal identifier rule. Store the original display text and maintain a separately defined key when needed. If a database participates in comparisons, align the application policy with the relevant collation rather than assuming both systems apply identical rules.
Truncation must preserve the selected boundary
A database limit expressed in bytes needs an encoding-aware check. A UI limit expressed in visible characters needs a text-element policy. A raw Substring(0, limit) satisfies neither policy automatically. It can cut a surrogate pair or separate a base character from a combining mark. Validate length under the intended representation before storing or displaying the result.
When building reports, also keep escaping separate from encoding. UTF-8 encoding does not escape HTML, CSV delimiters, or JSON syntax. Use a serializer or context-appropriate escaping for the destination format, then encode the resulting text. This order prevents a common mistake: correctly encoded bytes can still contain unsafe or structurally invalid markup if the application skipped the format boundary.
Official references
C# StringBuilder, Unicode and Encoding Examples
Each example shows the code, the output it produces and what each part does.
Construct text and compare its representations
The complete program builds two lines and inspects a short string containing one emoji. The output labels deliberately identify what each count measures.
The emoji contributes two UTF-16 code units and four UTF-8 bytes. The other two characters contribute one unit and one byte each. The final line verifies the round trip. Console glyph appearance depends on its font and encoding support, but the labeled numeric counts remain the same.
using System;
using System.Text;
var report = new StringBuilder();
report.AppendLine("Orders: 2");
report.Append("Status: ready");
Console.WriteLine(report.ToString());
string text = "A😀B";
int scalars = 0;
foreach (Rune rune in text.EnumerateRunes())
{
scalars++;
}
var utf8 = new UTF8Encoding(false, true);
byte[] bytes = utf8.GetBytes(text);
Console.WriteLine($"UTF-16 units: {text.Length}");
Console.WriteLine($"Unicode scalars: {scalars}");
Console.WriteLine($"UTF-8 bytes: {bytes.Length}");
Console.WriteLine(utf8.GetString(bytes));Orders: 2
Status: ready
UTF-16 units: 4
Unicode scalars: 3
UTF-8 bytes: 6
A😀BC# StringBuilder, Unicode and Encoding in Real Projects
Worked exercise: reject damaged UTF-8
A small import function receives bytes and returns decoded text only when the bytes form valid UTF-8. The task is to show a valid two-byte sequence and reject an invalid one. Keep the exception handling around decoding so unrelated failures do not become misleading encoding messages. Real importers should also bound the incoming byte count before allocating or decoding.
using System;
using System.Text;
var utf8 = new UTF8Encoding(false, true);
byte[][] inputs =
{
new byte[] { 0xC3, 0xA9 },
new byte[] { 0xC3, 0x28 }
};
foreach (byte[] input in inputs)
{
try
{
string decoded = utf8.GetString(input);
Console.WriteLine("Accepted: " + decoded);
}
catch (DecoderFallbackException)
{
Console.WriteLine("Invalid UTF-8");
}
}Accepted: é
Invalid UTF-8The second sequence starts a multibyte character but follows it with an invalid continuation byte. A permissive decoder could insert a replacement character; this import contract instead rejects the damaged record. That difference should be covered by a test because an ordinary ASCII sample cannot reveal it.
C# StringBuilder, Unicode and Encoding FAQ
Is string.Length the number of characters a user sees?
It counts UTF-16 code units. A supplementary Unicode scalar uses two units, and a visible grapheme can contain multiple scalars. The right count depends on the requirement. Use rune enumeration for scalar-oriented processing and text-element APIs for grapheme-oriented behavior. Test emoji and combining marks when a product promises visible-character limits; ordinary Latin samples can conceal an incorrect implementation.
Is StringBuilder always faster than string interpolation?
No. Simple interpolated expressions are readable and may be efficiently compiled. A builder is especially useful for repeated incremental construction whose final length grows over many operations. Performance depends on the workload, allocations and runtime. Start with clear ownership and bounded output, then measure a representative case if it matters. A builder does not compensate for repeatedly calling ToString inside the same loop.
Why choose a throwing UTF-8 decoder?
It lets an importer distinguish valid text from malformed byte sequences instead of silently substituting replacement characters. That is useful when exact data preservation matters. A display tool may intentionally prefer recovery, but the choice should be explicit. Report the failure safely, preserve the input only under the application's retention rules, and verify the expected encoding before treating every decoding failure as corruption.
Does Unicode normalization remove spoofing risks?
No. Canonical normalization addresses defined equivalences, not every visually similar character across scripts. An identifier policy may need an allowed alphabet, length limits, uniqueness rules and deliberate comparison behavior. Preserve ordinary user text appropriately, and avoid inventing restrictive rules for display names without a requirement. Security-sensitive names should have a documented policy rather than a chain of ad hoc replacements.
C# StringBuilder, Unicode and Encoding Exercises
Answer each one and check it instantly. If you get stuck, open the hint before the answer.
A report loops over thousands of rows and appends text. Which design is appropriate?
C# StringBuilder, Unicode and Encoding Quiz
2 questions to check what you learned.
The string A😀B has Length 4 in .NET. What is being counted?
- Four UTF-8 bytes.
- Four visible text elements.
- Four Unicode scalar values.
- Four UTF-16 code units.
What does UTF8Encoding(false, true) request?
- No generated BOM and exceptions for invalid encoding sequences.
- ASCII-only output with every emoji removed.
- Automatic detection of every source encoding.
- HTML escaping before byte conversion.