"I think it will rain tomorrow."
Riku said, looking out the window.
"Where does that confidence come from?" Aoi asked.
"The feeling of the clouds, the humidity in the air."
Yuki opened her notebook. "Are prediction and information theory related?"
"Deeply related," Aoi answered. "Prediction accuracy can be measured with information theory."
"How?"
"Using a concept called cross-entropy. It measures the distance between predicted distribution and actual distribution."
Riku tilted his head. "Distribution? Isn't it weather forecasting?"
"Weather forecasting is also probabilistic prediction," Aoi explained. "Like tomorrow's rain probability 70%, clear 30%."
"Oh, true."
Yuki asked. "Then how do we judge if a prediction is good or bad?"
"Compare with actual results," Aoi drew a diagram on the whiteboard.
"Prediction: Rain 70%, Clear 30% Actual: It rained"
"In this case, the prediction was good. Because it assigned high probability to rain."
Riku thought of another case. "Conversely, if we predicted clear 90% and it rained?"
"The prediction is bad. Cross-entropy becomes large."
Yuki began to understand. "So if we assign high probability to what actually happened, it's a good prediction?"
"Exactly. Information-theoretically, a good model minimizes surprise."
"Surprise?"
"When a rare event occurs, surprise is large. For frequent events, surprise is small," Aoi continued.
"A good prediction model estimates events that actually occur as high frequency. So surprise is little."
Riku gave an example. "If I predict my tardiness probability as 90%, I'm not surprised when I'm actually late."
"Unfortunate example, but correct," Aoi smiled wryly.
Yuki wrote in her notebook. "Prediction accuracy = low surprise = low cross-entropy"
"Good organization."
Riku's face became serious. "Then is perfect prediction possible?"
"For truly random phenomena, impossible," Aoi answered. "Unless entropy is zero, perfect prediction cannot be made."
"Entropy... a measure of uncertainty," Yuki recalled.
"Yes. The world has intrinsic uncertainty. So prediction has limits."
Riku looked at the window. "Then tomorrow's weather also cannot be predicted perfectly."
"Cannot. But a good model accurately represents that uncertainty."
Yuki asked. "Accurately representing uncertainty, isn't that contradictory?"
Aoi smiled. "Good question. But it's not contradictory. The prediction 'tomorrow is rain 60%, clear 40%' acknowledges uncertainty. But if the actual probability is also close to that, it's a good prediction."
"Predicting the probability distribution itself."
"Yes. Not point estimation, but distribution estimation. That's the core of modern prediction theory."
Riku pondered. "That's difficult."
"But important," Aoi said. "Machine learning, weather prediction, economic forecasting, all are based on probabilistic prediction."
Yuki summarized. "Prediction is not certainty, but appropriate representation of uncertainty."
"Perfect," Aoi nodded.
Outside the window, clouds flowed by. Tomorrow's weather is unknown. But understanding that uncertainty was the first step of prediction.
Riku said. "So will my prediction be right?"
"We'll know tomorrow," Aoi laughed. "But the quality of the prediction itself cannot be judged by just one result."
"What?"
"Predict many times and see average accuracy. That's the essence of cross-entropy."
Yuki was surprised. "One hit or miss isn't enough?"
"Right. Statistical evaluation is necessary. So for scientific prediction, data accumulation is essential."
The three quietly savored the deep relationship between information theory and prediction.
Whether it will rain tomorrow. That is unknown. But a method to measure prediction quality certainly exists.