A model checkpoint is one of the few files people routinely load without ever looking inside. You download several gigabytes, hand them to a framework, and trust that what came out the other end is what you expected. The weights visualiser opens the file in the browser instead — architecture, real values, distributions — and the interesting part is how little machinery that actually needs.
safetensors is barely a format
The whole spec fits in a sentence: eight bytes of little-endian uint64 giving the
header length, that many bytes of JSON, then the tensor data end to end.
The header names every tensor with its dtype, shape and byte range. Parsing it is a
getBigUint64, a TextDecoder and a JSON.parse. That is the
entire reader. This is why safetensors exists and why it won: there is no code execution surface,
because there is no code — just an offset table.
torch .pt is a ZIP with a pickle in it
The other format people actually have is a torch checkpoint, and that is a different proposition.
A .pt is a ZIP archive; inside it are a data.pkl describing the object
graph and a data/ directory of raw storage blobs. To read it you need two things
browsers do not have: a ZIP reader and a pickle interpreter.
The ZIP part is tractable because torch writes archives stored, not deflated. Scan backwards from the end for the end-of-central-directory signature, walk the central directory to collect names and local header offsets, then compute where each file's bytes begin. No decompression needed — the tensor data can be read in place out of the original ArrayBuffer.
The pickle VM
Pickle is a stack machine, and that is the part that sounds alarming and isn't. Opcodes push values, build tuples and dicts, memoise, and call reduction functions. You do not need all of it — you need the subset torch emits, plus two special cases:
torch._utils._rebuild_tensor_v2— the reduce that reconstructs a tensor. Its arguments give you the storage, the offset into it, and the shape.- Persistent IDs — how torch refers to storage blobs. The id carries the dtype
class name and the key naming the file under
data/.
Resolve those two and you have, for every tensor: a name, a dtype, a shape, and a byte offset into the archive. Which is exactly the same four facts safetensors hands you for free. From that point on both formats flow through identical code.
Never materialising a tensor
An embedding matrix might be 512×4096 floats. Rendering it as a heatmap needs perhaps 80×80 values. Reading the whole thing to display 1% of it would blow out memory for nothing, so the toolkit never reads a whole tensor at all.
Two sampling primitives do all the work. sample() strides through a tensor's bytes at
a computed interval, collecting a few thousand values for a histogram and accumulating min, max,
mean, standard deviation and a near-zero count as it goes. grid() does the 2D version:
given a target of 80 rows by 80 columns it computes a row and column stride and reads only the values
that land on the lattice.
Both read directly out of the DataView at computed offsets. Nothing is copied, no
intermediate array of the full tensor ever exists, and dtype support is a small table of readers —
including hand-written float16 and bfloat16 decoders, since JavaScript has
no native half-precision type. bfloat16 is the easy one: it is the top 16 bits of a float32, so you
shift it back up and reinterpret.
Finding the experts
Mixture-of-experts structure is discovered by name, not declared. Tensor names in practice look
like model.layers.3.block_sparse_moe.experts.5.w1.weight, so a regex for
experts.<n>. groups tensors by layer prefix and expert index, and a second pass
looks for anything matching gate, router or wg under the same
prefix to find the routing matrix. The highest index seen gives the expert count.
It is a heuristic over a naming convention, and a checkpoint that names things differently will simply report no MoE detected rather than mis-detect one.
The synthetic model
With no file loaded, the page builds a seeded Mixtral-shaped MoE — eight experts across four layers — from a small deterministic RNG. It exists so the page is never empty, but it earns its keep as a fixture: experts 0–3 are distinct archetypes and 4–7 are near-duplicates of them, which means the redundancy analysis in the pruner has something real to find. If a change breaks the similarity maths, the synthetic model stops showing four obvious pairs.
Everything above runs in the tab. The file never leaves your machine — not because of a privacy policy, but because there is no server in this architecture to send it to.