Short Story ⟡ Informatics

Divergence Between Expectation and Reality

When expectations don't match reality, KL divergence measures the distance between what we thought and what is.

  • #expected value
  • #variance
  • #prediction error
  • #model update

"I definitely thought I'd get 70 on today's test."

Riku looked dejected at his results.

"What did you get?" Yuki asked.

"55. Talk about disappointing."

Aoi said quietly, "Divergence between expected and observed values. This can also be analyzed information-theoretically."

"Seriously? Not much consolation."

"No, listen," Aoi opened the notebook. "Expected value E[X] is the probability-weighted average."

"So?"

"Your subjective expected value was 70. But calculated from objective probability distribution, it might have been lower."

Yuki supplemented. "So Riku's prediction model was optimistic?"

"Ugh, right on target."

At that moment, Mira quietly approached and wrote in her notebook.

"Variance = E[(X - E[X])²]"

"Variance," Aoi explained. "Spread from expected value. Large variance makes prediction difficult."

"Is my ability high variance?" Riku asked.

"Probably. Your performance fluctuates significantly."

Yuki started calculating. "If your last 5 tests were 50, 65, 40, 70, 55, the average is 56."

"Expected 70 but reality is 56."

"Plus, variance is about 130. Standard deviation about 11.4 points."

Aoi explained. "So 56±11 points is your ability range. 70 points is 1.2 standard deviations above. Possible, but not as frequent as you expect."

Riku's face showed understanding. "I overestimated my ability."

"Humans tend toward optimism bias," Aoi said. "Subjective expected value becomes higher than objective."

Mira showed a new note. "Bayesian update: prior → posterior"

"Bayesian updating," Yuki read. "Update prior distribution with observed data."

"Yes. With this 55-point information, Riku should update his model."

"So have realistic expectations next time?"

"That's learning," Aoi nodded. "Recognize gaps between expectation and reality, modify the model."

Yuki suddenly thought. "But aiming above expected value isn't bad, right?"

"Of course. Goals and expectations are different."

"Goal can be 70. But realistic expectation is around 56. Bridge that gap with effort."

Riku's expression became serious. "I see. Not lowering expectations, but correctly recognizing reality."

"Exactly," Aoi acknowledged. "Information theory seems cold but actually gives hope."

"How so?"

"High variance means large room for improvement. Change practice methods and you can raise expected value."

Mira smiled. A rare expression.

"Divergence between expectation and reality is the driving force of growth," Aoi continued. "If gap is zero, no need to learn."

Yuki wrote in her notebook. "Disappointment = information. Can use next time."

"Too positive?" Riku laughed.

"No, information-theoretically correct," Aoi admitted. "Larger prediction error, stronger learning signal."

Riku stood up. "Alright, I'll retrain my model before the next test."

"Data collection is important too. Measure frequently with quizzes, reduce variance."

"Reduce variance means stabilize?"

"Yes. Control fluctuation in ability."

Mira wrote last. "Consistency > occasional peaks"

"Better than occasional hits, stability matters," Yuki translated.

"Got it. I'll do it steadily."

The four began cleaning up. Between expectation and reality, divergence always exists. But measure it, understand it, correct it. That's growth.