Hutter compression guide
This site explains lossless compression using the codecs in the Hutter repository. It covers the underlying concepts, the Rust implementation of each teaching codec, the archive formats, and how to make and test changes. A separate research section follows the project's competitive Hutter Prize candidate. It assumes no prior knowledge of compression.
The repository is an experimental workspace for the Hutter Prize. The Rust codecs work on arbitrary byte files. The current research branch studies a pinned published compressor, but the project has not reproduced a full prize-valid result locally.
Read in order
The course sequence starts with files and bits, then explains how Huffman, LZ77, and predictive codecs represent the same data in different ways. Later chapters trace those ideas through the Rust source. Concepts are defined before they are used; the codebase chapters are separate from the introductory material.
Start here
| If you want to… | Read… |
|---|---|
| Run the program and verify that decompression works | Your first round trip |
| Follow the complete sequence | Course sequence |
| Understand the repository and the Hutter Prize | Project and Hutter Prize |
| Learn the basic ideas from the beginning | What compression does, then bytes and bits |
| Change a small input and see how three methods respond | Interactive examples |
| Understand a specific algorithm | Huffman, LZ77, byte PPM, or the full codec list |
| Understand the project layout and follow the code | Codebase overview, then follow one file |
| Find a module or make a change | Source map and developer onboarding |
| Read the source but are new to Rust | Rust notation used here |
| Understand the competitive candidate and measured Mac/Linux comparison | Competitive research track, then native/Linux case study |
| Plan and save a compression experiment | Experiment playbook |
What the codecs have in common
A compressor reads bytes and writes an archive. A decoder reads that archive and must reproduce the original bytes exactly. Some codecs describe repeated bytes with shorter codes; some point backward to repeated sequences; others predict the next byte or bit from what came before.
The encoder and decoder must use the same model state at each step. When a model learns from a byte, the decoder must update it at the same point as the encoder. The archive walkthrough shows where this happens in the Rust code.
The interactive examples use simplified models to illustrate individual ideas. The algorithm pages distinguish those examples from the repository's actual archive implementations.