Short Story ⟡ Informatics

Small Truths Guided by Probability

Exploring the process of finding truth from data through probability theory and statistical inference.

  • #probability
  • #statistical inference
  • #truth discovery
  • #data analysis

"Does this really work?"

Riku was looking at a health product advertisement.

"What does it say?" Aoi asked.

"'85% of users felt the effect'."

Yuki tilted her head. "The numbers are convincing."

"But can't take it at face value," Aoi warned.

"Why not?"

"The magic of probability. Numbers don't lie, but liars use numbers."

Riku showed interest. "What kind of trick is there?"

Aoi drew a diagram on the whiteboard. "First, sample size."

"Sample size?"

"How many people were asked. 100 people? 10 people?"

"If 85% of 10 people, that's only 8.5... no, 9 people."

"Too few. Statistically meaningless."

Yuki opened her notebook. "Then how many are needed?"

"Depends on purpose," Aoi answered. "But generally, larger is more reliable."

"Law of large numbers."

Riku asked another question. "Then 1000 people is reliable?"

"Even that might be insufficient," Aoi continued. "There might be selection bias."

"Selection bias?"

"Who was asked. What if only people already expecting effects were asked?"

Yuki understood. "Placebo effect might make them think it worked."

"Yes. That's why random sampling is important."

Riku laughed. "The advertisement numbers are getting suspicious."

"Should be suspicious," Aoi acknowledged. "But don't deny everything. Evaluate correctly."

Yuki asked. "How do we evaluate correctly?"

"Statistical testing," Aoi explained. "Judge whether it's coincidence."

"Coincidence?"

"Possibility of happening to be 85%. True effect might be 50%."

Riku was surprised. "85% can happen by luck?"

"If sample is small, it's possible."

Aoi showed a calculation. "Probability of getting 8 heads in 10 coin flips is about 4%."

"4% is quite high."

"Yes. So not statistically significant."

Yuki became serious. "Then to find truth?"

"Large sample, random selection, control group setup," Aoi listed.

"Control group?"

"Comparison target. Measure difference from group doing nothing."

Riku understood. "Scientific method."

"Yes. Probability theory is a tool toward truth."

Yuki thought of another example. "Test scores are the same?"

"Same," Aoi answered. "One score is noise-filled. Measure many times and take average."

"True ability becomes visible."

Riku smiled wryly. "My ability seems confirmed at a low average."

"But what if there's an upward trend?" Aoi pointed out. "That's also statistically meaningful."

"Means growth."

Yuki summarized. "Probability looks at overall patterns, not individual points."

"Exactly," Aoi nodded.

Riku looked at the advertisement again. "Then better not believe this ad?"

"Information is insufficient," Aoi answered. "Sample size, selection method, control group. These aren't stated."

"Lack of transparency."

Yuki pointed out. "But everyone is easily fooled by numbers."

"Yes. Humans judge intuitively. Statistical thinking requires training."

Riku became serious. "Glad I learned in Information Theory Club."

"Understanding probability correctly makes you less fooled."

Yuki wrote in her notebook. "Small truths guided by probability:

  1. Look not just at numbers but their background
  2. Check sample size
  3. Watch for selection bias
  4. Evaluate with statistical testing"

Aoi smiled. "Now you won't be fooled by advertisements."

Riku laughed. "But complete truth can't be found, right?"

"Right. Probability gives not certainty, but confidence level."

"Accumulation of small truths."

Yuki looked at the window. Absolute truth doesn't exist. But probability guides toward answers closer to truth.

That was the attitude of science.