Short Story ⟡ Informatics

After School About the Nature of Information

A story exploring information definition, data vs information through the lens of information theory.

  • #information definition
  • #data vs information
  • #knowledge
  • #meaning

"What exactly is information?"

At Yuki's question, Riku answered immediately. "It's data, right?"

"Wrong," Aoi shook their head. "Data and information are different."

"Huh?" Both were surprised simultaneously.

Aoi wrote on the whiteboard.

"Data: Raw facts or numbers Information: Data with meaning added Knowledge: Systematized information"

"Let me give an example. The number '25' is data."

Yuki thought. "That alone has no meaning."

"Right. But 'Tomorrow's temperature is 25 degrees' becomes information. Context is added."

Riku understood. "So the same data has different information amounts depending on who receives it?"

"Sharp. That's subjective information amount," Aoi nodded. "If you already know it, information amount is zero. If you don't know, information amount is high."

Yuki wrote in the notebook. "Information depends on the receiver's knowledge."

"Exactly. So the same message changes value depending on the recipient."

Riku gave an example. "The news 'It will rain tomorrow.' High value to someone without an umbrella. Low value to someone with one."

"Good example. Information value can be measured by uncertainty reduction."

Yuki had a question. "But are lies also information?"

Aoi's face became serious. "Deep question. In information theory, we assume true probability distributions. Lies are attacks on the model."

"Attacks?"

"They disrupt the receiver's probability estimation. So they're enemies of communication."

Riku pondered. "So the nature of information is the amount of uncertainty reduction?"

"That's one definition. Shannon's definition too."

Aoi drew a diagram.

"Initial state: High uncertainty Receive information: Possibilities narrow Final state: Low uncertainty"

"Information amount is that difference."

Yuki got excited. "That's why rare news has more information!"

"Correct. When a low-probability event occurs, the surprise is great. Surprise = information amount."

Riku joked. "People are always surprised by my actions. Does that mean high information amount?"

"In a sense," Aoi laughed. "Unpredictable behavior is high-information."

Yuki asked seriously. "Does information have physical substance?"

"Good question. Information is an abstract concept, but subject to physical constraints. Memory, energy, time. All are necessary."

"In quantum information theory, we study the relationship between information and physical laws."

Riku's eyes lit up. "Maybe information is fundamental to the universe."

"Some physicists think so. The universe as an information processing system."

Yuki looked outside the window. The sun was setting.

"We don't fully understand the nature of information yet."

"True," Aoi admitted. "That's why research continues. What Shannon defined was just one aspect."

"But that aspect is too powerful," Riku said.

"That's why information theory is used so widely. Communication, cryptography, machine learning, physics."

Yuki closed the notebook. "The journey of information has just begun."

"Yes. You're part of that journey too."

The three quietly left the classroom. The journey to discover the nature of information continues.