"How do you create the shortest message?"
At Yuki's question, Riku answered. "Just make it short?"
"But too short becomes ambiguous," Yuki countered.
Aoi intervened. "That's the problem of optimal coding. Short yet clear."
"Optimal coding?"
"The technology of expressing information with minimum bits. But it must be decodable."
Mira quietly opened her notebook and drew a diagram.
"A: 0 B: 10 C: 110 D: 111"
"This is a prefix-free code," Aoi explained.
"Prefix-free?" Yuki tilted their head.
"No code is a prefix of another. So it can be uniquely decoded."
Riku tried it. "Reading '0110111'... A, C, D?"
"Correct. You can read it uniquely without stopping midway."
Yuki thought of another example. "What if 'A: 0, B: 1, C: 01'?"
"Becomes ambiguous. '01' could be A and B, or C."
"That's why prefix-free is necessary."
Aoi wrote an equation on the whiteboard.
"Kraft inequality: Σ 2^(-li) ≤ 1"
"li is each code's length. If this inequality is satisfied, a prefix-free code exists."
Riku calculated. "In the earlier example, 2^(-1) + 2^(-2) + 2^(-3) + 2^(-3) = 0.5 + 0.25 + 0.125 + 0.125 = 1"
"Exactly 1. This is close to optimal."
Mira drew an additional diagram. The relationship between frequency and code length.
"Assign short codes to frequent symbols, long codes to rare symbols," Aoi explained.
Yuki understood. "This is Huffman coding."
"Yes. Huffman coding constructs the optimal prefix-free code for a given probability distribution."
"How?"
Aoi explained the procedure.
"1. Combine the two lowest probability symbols 2. Treat as a new symbol 3. Repeat until all become one 4. Read codes from the tree structure"
Riku was impressed. "You create codes from a tree."
"A binary tree. Going left is 0, going right is 1."
Mira showed a note. "Average code length ≥ H(X)"
"Average code length cannot be below entropy," Aoi supplemented. "This is Shannon's limit."
Yuki asked. "Is Huffman coding perfect?"
"Nearly perfect. The difference between entropy and average code length is less than 1 bit."
"But there are even better methods," Aoi continued. "Arithmetic coding, or methods that encode multiple symbols together."
Riku wrote in his notebook. "Optimal means no waste."
"Yes. But computational cost must be considered too. Too optimal codes might be computationally heavy."
Yuki remembered. "Another tradeoff."
"Always tradeoffs. Even between theory and practice."
Mira smiled and wrote a message.
"Short message, clear meaning"
"Short message, clear meaning," Aoi translated. "That's the goal of optimal codes."
Yuki was impressed. "Mira's messages are always optimal."
"Mira is a natural Huffman encoder," Riku laughed.
Mira showed a rare smile.
Aoi summarized. "Optimal codes are the art of information theory. A fusion of science and aesthetics."
"Aesthetics..." Yuki repeated.
"Yes. Beauty without waste. That's the appeal of optimal codes."
The four quietly nodded. The shortest message that gets through. That's their goal too.