"Mira, I can never tell what you're thinking."
Yuki murmured suddenly. Mira was quietly reading a book as usual.
Aoi responded. "That connects to the topic of mutual information."
"Mutual information?"
"An indicator measuring how much information two variables share."
Mira wrote an equation in her notebook. "I(X;Y) = H(X) - H(X|Y)"
"Entropy of X minus conditional entropy of X given Y," Aoi explained. "How much X's uncertainty decreases by knowing Y."
Yuki thought. "Even observing Mira's behavior, I can't predict the next action... so mutual information is low?"
"Sharp. If completely independent, mutual information is zero. If completely deterministic, mutual information is maximum."
Mira drew on a new page. Two circles, slightly overlapping.
"Venn diagram," Aoi nodded. "The overlap is mutual information. The amount of shared information."
Yuki wrote an example on the whiteboard.
"Weather and temperature. If cloudy, temperature is often low. Mutual information seems high."
"Correct. Knowing weather improves temperature prediction accuracy."
"Conversely, weather and dice rolls?"
"Independent. Knowing weather doesn't change dice results. Mutual information is zero."
Mira smiled and wrote a new example. "Yuki and Aoi: high I(X;Y)?"
Yuki looked embarrassed. "Us?"
Aoi analyzed calmly. "Considering our conversation patterns, mutual information might be high. When Yuki asks, I explain. It's predictable."
"Is that a good thing?"
"Necessary for communication. If mutual information is too low, conversation doesn't work."
Mira drew another diagram. Three circles overlapping in complex ways.
"Mutual information with three or more variables is more complex," Aoi supplemented. "You can see the network of who shares information with whom."
Yuki was fascinated. "Then what's the mutual information among us three?"
"Interesting question. Mira is taciturn, so direct mutual information seems low. But Mira's hints deepen our understanding. Indirectly, it's high."
Mira nodded. She rarely spoke words. "Indirect mutual information"
"Yes. Information doesn't just flow linearly. Triangular relationships, networks, complex structures."
Yuki wrote in the notebook. "Higher mutual information means deeper relationship?"
"Simply put, yes. But it's not necessarily better," Aoi said carefully. "Completely predictable relationships generate no new information. No surprise."
"Surprise..."
"Right. Appropriate uncertainty enriches communication. Neither complete independence nor complete dependence, but the middle is interesting."
Mira wrote. "Balance: predictability and surprise"
Yuki smiled. "That's why Mira being unpredictable is interesting."
"Mira's behavior has low mutual information, but not zero. That balance makes Mira attractive."
Mira looked down, as if embarrassed.
Aoi continued. "Information theory can quantify relationships. Friendship, communication, everything can be measured by mutual information."
"But maybe some things are better unmeasured," Yuki said.
"True," Aoi laughed. "Mathematics is a tool, not everything."
Mira quietly closed her book. The discussion of mutual information seemed to open her up a little more today too.