Short Story ⟡ Informatics

How to Escape the Forest of Uncertainty

An exploration of entropy, uncertainty, and how information theory helps us understand the world.

  • #decision under uncertainty
  • #information gain
  • #entropy reduction
  • #bayesian inference

"I'm confused."

Yuki had a troubled face.

"About what?" Aoi asked.

"Tomorrow's plans. I don't know if it will be sunny or rainy."

Riku laughed. "Just check the weather forecast."

"I did. But 70 percent sunny—how do I decide?"

Aoi became interested. "This is decision-making under uncertainty."

"Uncertainty?"

"A state without complete information. You only know probabilities."

Aoi wrote on the whiteboard.

"Initial state: Sunny 70% Rainy 30% Entropy = -0.7 log(0.7) - 0.3 log(0.3) ≈ 0.88 bits"

"This entropy is the amount of uncertainty."

Yuki understood. "Higher means more uncertain?"

"Yes. If completely certain, entropy is zero. 50-50 is most uncertain, at 1 bit."

Riku asked. "So how do you decide?"

"Add information to reduce entropy," Aoi explained.

"For example, check the weather radar. Look at the morning sky. Listen to the news."

"Each adds information and updates the probability."

Yuki wrote in the notebook. "That's Bayesian inference."

"Precise. Calculate posterior probability from prior probability and new evidence."

Aoi showed an example.

"In the morning, the sky was dark. If this observation has 'dark probability 90% given rain, 20% given sun'..."

Riku attempted the calculation. "Sunny probability decreases?"

"Yes. Updating with Bayes' theorem, sunny probability drops to about 50%."

"The uncertainty changed."

Aoi continued. "There's a concept called information gain. How much entropy decreases with new information."

"Information gain = original entropy - conditional entropy"

Yuki got excited. "Information is a tool to reduce uncertainty!"

"Exactly. Even if you can't eliminate it completely, you can reduce it."

Riku gave another example. "Game choices are the same?"

"Same. Multiple choices exist. Outcomes are uncertain. But you can estimate probabilities through experience and information."

"Then calculate expected value and choose the best."

Aoi drew a diagram.

"Decision tree. Possible outcomes branch from each choice. Assign probability and value to each outcome."

"Choose the option with maximum expected value."

Yuki pondered. "But completely eliminating uncertainty is impossible, right?"

"Impossible. The future is inherently uncertain. But you can improve probabilities with information."

"That's why information gathering is important."

Riku raised his hand. "I always decide by intuition."

"Intuition is also a type of probability estimation," Aoi said. "You unconsciously calculate probabilities from experience."

"But sometimes I'm wrong."

"Yes. That's why conscious probability estimation is useful. Reduces bias."

Yuki made a decision. "Then I've decided about tomorrow's plans."

"What will you do?"

"I'll bet on sunny. But I'll also bring an umbrella."

Aoi smiled. "Wise choice. Bet on the main possibility while hedging risk."

"That's how to deal with uncertainty."

Riku wrote in his notebook. "Uncertainty isn't an enemy, but a companion we walk with."

"That's poetic," Yuki laughed.

"Information theory is the technology of living in an uncertain world," Aoi summarized.

The three looked outside the window. Tomorrow's weather is unknown. But they know how to escape the forest of uncertainty.