Short Story ⟡ Informatics

The Journey of Collecting Information Fragments

A story exploring data aggregation, sampling through the lens of information theory.

  • #data aggregation
  • #sampling
  • #inference
  • #incomplete information

"In the end, what does information theory teach us?"

Riku suddenly asked a fundamental question.

Yuki and Aoi looked at each other. They had been learning in the club for weeks, but summarizing in one phrase was difficult.

"The essence of information," Aoi answered carefully. "What 'information' is. How to measure it, how to handle it."

"But what we learned are scattered concepts, right? Entropy, coding, noise..."

"That's the issue," Professor S. entered the room. "Individual theories are actually all connected."

"Professor," the three greeted.

"You are now collecting fragments of information. But you haven't seen the whole picture yet."

Yuki opened her notebook. "Shall we try organizing what we've learned?"

"Good idea," Aoi agreed.

Riku started writing on the whiteboard. "Entropy: measurement of uncertainty."

"Coding: efficient representation of information," Yuki continued.

"Error correction: protection from noise," Aoi added.

Professor S. watched quietly.

"Mutual information: shared information."

"KL divergence: distance between distributions."

"Channel capacity: limits of transmission."

The list grew.

"How do these connect?" Riku asked.

Aoi drew a diagram.

"Source → Encoding → Channel → Decoding → Receiver"

"This is Shannon's communication model. All concepts are positioned somewhere here."

"Entropy is a property of the information source," Professor S. explained.

"Coding is optimized according to entropy," Aoi continued.

"Channels have noise and capacity limits."

"Error correction approaches capacity."

"The receiver infers the original message from received information."

Yuki's eyes sparkled. "Everything is part of the same story."

"Yes. It's information's journey," Professor S. nodded.

"But," Riku said, "what we learned is still just fragments, right?"

"Of course. Information theory is vast. Quantum information, network theory, connections to machine learning..."

"Meaning it never ends?"

"Science has no end. But understanding basics lets you explore on your own."

Aoi supplemented. "What we learned so far is like learning to read maps."

"From now, you can walk on your own feet."

Yuki wrote in her notebook. "The journey of collecting information fragments has only just begun."

Professor S. smiled. "Good understanding. Information theory provides a framework for thinking."

"A perspective to view the world as information flow."

"An attitude to measure and optimize uncertainty without fear."

Riku's face became serious. "I'm getting more interested in learning."

"That feeling is the next step."

"But from where?" Yuki asked.

"Find application fields of interest. Cryptography, compression, machine learning, bioinformatics..."

"Information theory exists everywhere," Aoi said.

"Then I'll look into game information theory," Riku declared.

"I'll study language and information theory," Yuki continued.

"Each journey begins," Professor S. said. "But sometimes, gather here and share fragments."

The three nodded.

"Increase mutual information," Riku laughed.

Outside the window, sunset illuminated the school building. Information fragments gradually gather and eventually form the whole picture. But there's no perfect map. So the journey continues.

Professor S. said as he left, "Information theory is an endless dialogue. You yourselves become part of it."

The three began preparing to search for the next fragment. The journey has only just begun.