Shared Information Between You and Me

"Hey, how much shared information do you think we three have?"

Riku suddenly asked.

"Shared information?" Yuki tilted her head.

Aoi showed interest. "Interesting question. Let's try defining it information-theoretically."

"How do you define it?"

"First, let each person's information be entropy H(X), H(Y), H(Z)."

Riku thought. "I'm X, Yuki is Y, Aoi-senpai is Z?"

"Right. And the amount of information everyone knows is the shared information."

Yuki asked. "How do you calculate it?"

"Consider joint entropy H(X,Y,Z)," Aoi wrote in her notebook. "This is the uncertainty of all three together."

"Isn't it just adding up the three?"

"No. Because there's overlapping information."

Aoi wrote an equation.

"H(X,Y,Z) ≤ H(X) + H(Y) + H(Z)

Equality only when completely independent"

"When there's shared information, joint entropy becomes smaller?"

"Exactly. Because we don't count the same information multiple times."

Riku asked. "Then how much is the shared information?"

"H(X) + H(Y) + H(Z) - H(X,Y,Z) is one definition."

Yuki began to understand. "Subtract joint entropy from the sum of individual entropies."

"Yes. The reduced amount is the shared information."

Riku asked for a concrete example. "Like what?"

"All three are in the Information Theory Club. This is shared information," Aoi explained. "This fact doesn't need to be held independently by three people."

"Once known, everyone knows."

"Right. So entropy is only one portion."

Yuki said, "But there are things only I know."

"That's conditional entropy H(Y|X,Z)," Aoi explained. "Yuki's remaining information after knowing the other two's information."

"Private information."

"Exactly."

Riku pondered. "Does more shared information mean closer relationships?"

"You could say that," Aoi answered carefully. "But individual information is also important."

"Why?"

"If completely shared, one person would be enough."

Yuki understood. "Individuality disappears."

"Right. Optimal is moderate sharing and moderate uniqueness."

Riku laughed. "So balance is important?"

"Information-theoretically too," Aoi nodded.

Yuki asked. "Senpai, is our shared information increasing?"

"I think so. Because we spend a lot of time together."

"How do we increase it?"

"Share experiences. Read the same books, solve the same problems, go to the same places."

Riku added. "Make the same mistakes."

"That's important too," Aoi acknowledged. "Shared experiences make communication efficient."

Yuki summarized. "With lots of shared information, you can communicate with few words."

"Exactly. This also relates to mutual information."

Riku looked outside. "How much shared information can we increase from now?"

"Infinitely," Aoi answered. "As long as there's time."

"But after graduation?"

"Even physically apart, shared information remains."

Yuki said quietly. "As memories."

"Yes. Shared past doesn't disappear."

Riku laughed. "Then let's share lots while we can."

"Agreed," Yuki nodded.

Aoi also smiled. "Shared information between you and me. That's what connects us."

The three nodded. Invisible but certainly existing connection. That's shared information.