A server receives bytes before it receives meaningful text. A photograph, a compressed response, and a JSON document can all arrive through the same socket API, but only the JSON document should be decoded as UTF-8. A Buffer lets a Node program retain those bytes until the application knows how to interpret them. That distinction prevents corrupted text, incorrect content lengths, and accidental changes to signed payloads. This reference assumes Node 24 and uses dependency-free ES modules. Save the complete examples with the .mjs extension and run them with node. The fixtures stay in memory so the printed values do not depend on a file or network connection.
Count bytes rather than characters
JavaScript string length counts UTF-16 code units. A Buffer length counts bytes. Those measurements happen to agree for plain ASCII letters, which can hide the difference during early testing. The word café has four string code units but five UTF-8 bytes because the accented character needs two bytes. An emoji may require two JavaScript code units and four UTF-8 bytes. Neither measurement counts the visible characters a person perceives in every language, since combining marks can form one displayed character from several code points.
Use Buffer.byteLength(text, 'utf8') when an HTTP header, storage limit, or binary protocol asks for the encoded size. Buffer.from(text, 'utf8') performs the encoding and provides the actual bytes. Converting those bytes with toString('hex') displays a representation of the existing bytes; it does not encode the original text as hexadecimal characters inside the buffer. To reconstruct the bytes from that representation, pass the hexadecimal text and the 'hex' encoding to Buffer.from. Base64 follows the same separation between bytes and their printable representation.
Choose allocation and ownership deliberately
Buffer.alloc(size) returns zero-filled memory. It is the clearest choice for a packet whose fields will be written individually, because any unwritten positions have predictable values. Buffer.allocUnsafe(size) avoids initialization and may contain previous memory contents. If such a buffer is sent before every position has been overwritten, an application can reveal unrelated data. A small speed benefit is not a reason to accept that risk in beginner application code. Buffer.from(existingBuffer) creates a copy with its own byte storage.
The ownership question becomes important with subarray. A subarray is a view of the original memory, so modifying a byte through the view modifies the parent. This is useful for parsing a large packet without repeatedly copying its payload. It is surprising when a function retains a view and another function later reuses the source buffer. Make a copy when independent ownership matters. Node Buffer.slice also creates a shared view, unlike the copying behavior many developers expect from ordinary array slice. Use an explicit Buffer.from(view) to make the intended copy visible.
Respect UTF-8 boundaries in streams
A network chunk boundary does not necessarily fall between characters. A two-byte UTF-8 character can arrive with its first byte in one chunk and its second byte in the next. Calling toString('utf8') on each chunk independently can replace the incomplete sequences with replacement characters. Concatenating those already damaged strings cannot restore the missing character. Accumulate all bytes before decoding when the complete input is small and bounded, or use a stateful decoder when processing a stream.
StringDecoder from node:string_decoder retains an incomplete trailing character until a later write provides the remaining bytes. Its end method flushes the final state. TextDecoder is another useful interface, with streaming options and a fatal mode when malformed UTF-8 should be rejected. These APIs solve decoding boundaries; they do not validate JSON structure, enforce an upload limit, or identify the correct encoding automatically. Establish the encoding from the format contract. A binary file should remain a byte sequence, even if a few bytes happen to look like readable letters.
Read binary fields with an explicit format
Binary protocols define both field widths and byte order. A two-byte unsigned big-endian integer stores its most significant byte first. Little-endian stores that byte last. Buffer.readUInt16BE and writeUInt16BE express a big-endian contract directly. Choosing the wrong variant usually produces a valid but incorrect number, which is more difficult to notice than a syntax error. Signed methods interpret a sign bit; unsigned methods do not. Wider values may require BigInt methods when JavaScript Number cannot represent every integer precisely.
Validate available length before reading a field and validate the declared payload length before allocating or waiting for data. A four-byte header that claims a multi-gigabyte payload is not permission to consume that much memory. Keep offsets in bytes, not string positions. When a protocol contains text, decode only its designated text segment after checking the segment boundaries. The worked packet uses a two-byte length followed by UTF-8 text. It is a small framing example, not a complete network protocol: a real parser must also handle incomplete packets, multiple packets, and its own configured maximum length.
Node.js documentation and version notes
Node.js Buffers and UTF-8 Examples
Each example shows the code, the output it produces and what each part does.
Encoding café and framing a length-prefixed packet
The first fixture encodes café and then constructs a packet with a two-byte header. The header says how many payload bytes follow, so its value is five rather than four. A subarray mutation changes the parent byte, whereas the copied buffer keeps the original character. The final decoder receives the accented character in two separate writes. Its first result is empty because the character is incomplete, and the second result contains the complete character. Notice that the decoder has not discarded data while waiting. Running this example on Node 24 prints the exact output shown; no unsafe allocation or platform byte order is involved.
import { Buffer } from 'node:buffer';
import { StringDecoder } from 'node:string_decoder';
const text = 'café';
const payload = Buffer.from(text, 'utf8');
console.log(text.length, payload.length, payload.toString('hex'));
const packet = Buffer.alloc(2 + payload.length);
packet.writeUInt16BE(payload.length, 0);
payload.copy(packet, 2);
console.log(packet.readUInt16BE(0), packet.subarray(2).toString('utf8'));
const original = Buffer.from('cat');
const view = original.subarray(0, 1);
const copied = Buffer.from(view);
view[0] = 98;
console.log(original.toString(), copied.toString());
const decoder = new StringDecoder('utf8');
console.log(JSON.stringify(decoder.write(Buffer.from([0xc3]))));
console.log(JSON.stringify(decoder.write(Buffer.from([0xa9]))));
decoder.end();4 5 636166c3a9
5 café
bat c
""
"é"Node.js Buffers and UTF-8 in Real Projects
Build a length-prefixed packet decoder
Build a reusable decoder for this length-prefixed format. The task is to reject a packet when its header and actual byte count disagree, then return only the decoded payload for a valid packet. Also reject packets shorter than the header before attempting a read. The solution compares the declared length with the remaining byte count instead of trusting either one independently. This makes an accidentally truncated input and an input with trailing bytes fail the same format check. The helper handles one complete packet; it deliberately does not wait for later chunks. That responsibility belongs to a transport parser. The example exercises a valid multibyte payload and a malformed packet, so a solution based on string length would fail.
import { Buffer } from 'node:buffer';
function decodePacket(packet) {
if (packet.length < 2) throw new Error('Missing header');
const size = packet.readUInt16BE(0);
if (size > 1024 || size !== packet.length - 2) {
throw new Error('Invalid payload length');
}
return packet.subarray(2).toString('utf8');
}
const bytes = Buffer.from('é');
const packet = Buffer.alloc(2 + bytes.length);
packet.writeUInt16BE(bytes.length);
bytes.copy(packet, 2);
console.log(decodePacket(packet));
try { decodePacket(Buffer.from([0, 3, 65])); }
catch (error) { console.log(error.message); }é
Invalid payload lengthNode.js Buffers and UTF-8 FAQ
Is Buffer a different kind of string?
No. A Buffer stores numeric byte values, and those values can represent text only under a chosen encoding. The same bytes might instead represent an image header or an integer. A string is appropriate after decoding a known text field; a buffer is appropriate while preserving a binary payload. This distinction matters for signatures because parsing and reserializing a JSON string can change the exact bytes even when the resulting object has the same properties.
Should I use allocUnsafe for every large buffer?
Use it only when code demonstrably overwrites the entire allocation before any byte is read, exposed, or sent. Partial writes and error paths make that guarantee harder than it appears. Buffer.alloc supplies a safer default and documents that unwritten fields start as zero. Measure a real allocation bottleneck before considering a change. An optimization that risks disclosure of stale memory is especially unsuitable for buffers containing user-controlled sizes or data.
Why did a copied-looking slice change my input?
Buffer.slice and Buffer.subarray share memory with their source. They describe a region rather than transferring ownership of its contents. If a function may mutate or retain that region, establish whether sharing is intentional. Buffer.from(region) creates an independent byte copy. Remember that an independent copy also consumes additional memory, so a streaming parser can keep short-lived views while making copies only for data that must outlive the source allocation.
Does valid UTF-8 guarantee a safe request?
It guarantees only an encoding property. Valid UTF-8 can still contain an invalid JSON document, an oversized value, a misleading filename, or a disallowed command. Enforce byte limits before unbounded accumulation, decode according to the protocol, and then validate the resulting structure and values. If malformed text must fail rather than receive replacement characters, choose a decoder with strict error behavior. Keep each validation step tied to the specific format rule it protects.
Node.js Buffers and UTF-8 Exercises
Answer each one and check it instantly. If you get stuck, open the hint before the answer.
The worked packet contains UTF-8 é. What header length must be written?
Node.js Buffers and UTF-8 Quiz
2 questions to check what you learned.
Which operation creates independent storage for an existing Buffer view?
- Buffer.from(view)
- view.subarray(0)
- view.slice(0)
Why use StringDecoder for chunked UTF-8?
- It retains incomplete character bytes between chunks
- It guarantees every network chunk contains complete JSON
- It changes the socket into a text-only transport