"What exactly is a bit?"
While listening to Aoi's explanation, Yuki asked a question that had been on their mind.
"You know it's the smallest unit of data, right?" Aoi confirmed.
"Zero or one, right?"
"That's one way to represent it. But in information theory, it has a deeper meaning." Aoi drew a simple diagram in the notebook. "One bit is the amount of information needed to choose one option from two equally likely choices."
"Equally likely?"
"Meaning 50 percent probability each. For example, a message saying 'it came up heads' after flipping a fair coin carries exactly 1 bit of information."
At that moment, with quiet footsteps, Mira entered the club room. The exchange student always sat silently like an observer.
"Mira, you came again today," Yuki smiled.
Mira nodded slightly and took out her notebook.
Aoi continued. "Now, if I told Yuki 'Today is Tuesday,' how many bits of information is that?"
"Um... choosing one from seven days of the week..."
"log₂(7), about 2.8 bits. But what if Yuki already knew today was Tuesday?"
"The information content is... zero?"
"Correct. Information content depends on the receiver's uncertainty. This is the concept of subjective information."
Mira wrote something in her notebook and showed it to Yuki. "Surprise = -log₂(p)"
"Oh, Mira wrote an equation..." Yuki was surprised.
Aoi took over the explanation. "That's right. The lower the probability p of an event, the greater the surprise—that is, the information content—when it occurs."
"So rarer news has higher information value..." Yuki understood.
"Yes. The news 'The sun rose from the east' has probability close to 1, so information content is nearly zero. But 'Giant meteor approaching' has low probability, so information content is large."
Mira wrote again. "Common events: low information. Rare events: high information."
"But," Yuki pondered, "wouldn't lying give more information content?"
Aoi showed an impressed expression. "Sharp observation. That's why information theory assumes a 'true probability distribution.' Lies are like attacks that disrupt the receiver's probability model."
"Channel attack," Mira murmured quietly.
"Mira knows communication theory well," Aoi said.
Mira nodded but didn't continue explaining. As usual.
Yuki wrote calculations in the notebook. "So telling someone 'I rolled a 6' on a die is -log₂(1/6)... about 2.58 bits."
"Perfect. That's self-information. Meanwhile, entropy is the expected value of that."
"Expected value?"
"The average information content weighted by probability across all possible events. So the overall uncertainty of a die roll is about 2.58 bits."
Mira quietly stood up and wrote on the whiteboard.
"H(X) = E[I(X)] = Σ p(x)・(-log₂ p(x))"
"Entropy is the expected value of individual self-information," Aoi supplemented. "Good summary, Mira."
Mira smiled faintly.
Yuki said excitedly, "So a bit isn't just a unit of data, but a unit for measuring surprise and uncertainty!"
"Yes. The beauty of information theory is that it can convert subjective 'surprise' into objective numbers."
"Talking with you both always brings new surprises."
Aoi laughed. "Maybe we can measure that surprise in bits too."
Mira closed her notebook and quietly stood up. As she left, she handed Yuki a small note.
"Next: conditional probability"
"Conditional probability...?" Yuki murmured.
"Mira always shows us the next step," Aoi said. "Her information content is always high."
Yuki carefully put away the note. Today, too, felt like gaining a new bit.