"I have no idea what Mira is thinking."
Yuki murmured in the corner of the library. Mira was reading a book across from her, expressionless as always.
Aoi answered quietly. "That's because the mutual information is low."
"Mutual information?"
"The amount of information shared between two variables. Written as I(X;Y)."
Mira offered her notebook. Somehow, an equation was already written.
"I(X;Y) = H(X) - H(X|Y)"
Aoi explained. "The entropy of X minus the conditional entropy of X after knowing Y."
"That's difficult..."
"Simply put, it's how much the uncertainty of X decreases when you know Y."
Yuki thought. "So observing Mira doesn't reveal Mira's feelings because..."
"The mutual information between observable information Y and internal state X is small."
Mira smiled faintly. Then she wrote a new example.
"Weather and carrying umbrella"
Aoi continued. "On cloudy days, many people carry umbrellas. On sunny days, few do. This correlation is mutual information."
"If they're completely independent?"
"Mutual information is zero. You can't learn anything from each other."
Yuki took notes. "Conversely, if it's completely determined?"
"Mutual information is maximum. Knowing one tells you the other completely."
Mira added more writing. "Friendship = high mutual information?"
Yuki's eyes sparkled. "I see! Being close means you can predict each other's state?"
Aoi nodded. "In a sense, yes. With long-time friends, you can predict their reactions. This is evidence of high mutual information."
"But if you could predict completely?"
"It might be boring. Moderate uncertainty makes relationships interesting."
Mira opened a new page. A diagram was drawn. Two circles overlapping.
"Venn diagram," Aoi explained. "Representing information amounts of X and Y as circles. The overlapping part is mutual information."
"A beautiful diagram."
"Mutual information has an important property. Symmetry. I(X;Y) = I(Y;X)"
"Same even if you swap X and Y?"
"Yes. Unlike KL divergence, order doesn't matter."
Yuki asked. "How is mutual information used?"
Aoi gave an example. "When doing feature selection in machine learning. Choose features X with high mutual information with target variable Y."
"Meaning it's useful for prediction?"
"Exactly. The higher the mutual information, the more that feature contributes to prediction."
Mira wrote. "Communication = transfer of mutual information"
"Communication can also be measured by mutual information," Aoi continued. "The mutual information between transmitted message X and received message Y is the upper limit of channel capacity."
"With noise?"
"Mutual information decreases. In complete noise, transmission and reception become independent, and I(X;Y) = 0."
Yuki thought deeply. "So communication between people too..."
"Can be measured by mutual information. Perfect transmission means maximum mutual information."
Mira wrote more. "Redundancy = low mutual information?"
"Interesting perspective," Aoi acknowledged. "Redundancy means little new information. High mutual information with known information."
Yuki organized her thoughts. "So the amount of shareable surprise is mutual information."
"A poetic expression, but it captures the essence."
Mira spoke for the first time. "With you two, mutual information has been increasing."
Yuki and Aoi were surprised. It was rare for Mira to speak.
"From observing Mira, I've gradually become able to predict her feelings," Aoi said.
"Does that mean we've become close?" Yuki asked happily.
Mira nodded slightly.
Aoi smiled. "Deepening relationships means increasing mutual information. Each other's states begin to correlate."
"But it's better not to be able to predict perfectly."
"Moderate independence is also necessary."
Yuki summarized. "The amount of shareable surprise measures the depth of relationships."
Mira wrote a final equation in her notebook.
"I(person, person) = shared surprise"
The three smiled at each other. Information theory can even express human relationships with equations. But what can't be expressed in equations is also important.
Sunset streamed through the library window. This moment of quiet understanding was also a kind of mutual information.