"The entropy so far was discrete, right?"
Yuki confirmed with Aoi.
"Yes. Dice, coins, finite choices."
"But reality is continuous, right? Time, distance."
Aoi smiled interestingly. "Shall we talk about differential entropy?"
"Differential entropy?"
"Entropy of continuous probability distributions. h(X) = -∫ f(x) log f(x) dx"
Aoi wrote the equation in the notebook.
"Using integration instead of summation."
"But," Yuki thought, "with continuity, there are infinite choices, right?"
"Sharp. That's why differential entropy can be negative."
"Entropy negative?"
"Discrete entropy is always non-negative, but differential entropy is different."
Aoi gave an example.
"Differential entropy of uniform distribution U(0,1) is 0."
"Zero."
"But U(0,0.5) becomes -1."
Yuki was confused. "Why negative?"
"Differential entropy isn't an absolute quantity, but relative."
Mira showed her notebook.
"Differential entropy is not invariant under coordinate transformation"
"Changes with coordinate transformation," Aoi explained. "So just changing units changes the value."
"Strange."
"But entropy differences and mutual information are invariant under coordinate transformation."
Aoi wrote another equation.
"I(X;Y) = h(X) + h(Y) - h(X,Y)"
"Same form as discrete."
Yuki asked. "So is the real world truly continuous?"
"Philosophical," Aoi pondered. "In quantum mechanics, there's a minimum unit called Planck length."
"So ultimately discrete?"
"Maybe. But for many phenomena, treating as continuous is more convenient."
Mira presented another perspective.
"Precision is limited by measurement"
"Measurement precision has limits," Aoi continued. "So continuous distributions are actually treated as finite-precision discrete distributions."
"Quantization."
"Yes. The process of digitizing analog signals."
Aoi drew a diagram. Smooth curve and stair-step approximation.
"Sampling and quantization convert continuous to discrete."
Yuki understood. "But information is lost, right?"
"That's important. Quantization error occurs."
"How much is lost?"
"Determined by bit count. With n bits, 2^n levels of precision."
Mira showed calculations.
"8 bits = 256 levels. 16 bits = 65536 levels"
"Music CDs are 16 bits," Aoi gave an example. "Sufficient precision for human ears."
Yuki had another question. "So infinite-precision measurement is impossible?"
"Theoretically and practically impossible," Aoi answered. "There's Heisenberg's uncertainty principle."
"Uncertainty principle?"
"Quantum mechanics principle that position and momentum can't be measured simultaneously with precision."
"Related to entropy?"
"Deeply related. Uncertainty indicates limits of information."
Mira said quietly. "Universe has fundamental entropy"
"The universe itself has fundamental entropy," Aoi translated.
Yuki looked out the window.
"The world that appears continuous might actually be discrete."
"Or the world that appears discrete might actually be continuous."
"Which is truth?"
Aoi smiled. "No one knows yet."
"But we can choose how to model it."
"Yes. Choose appropriate abstraction for the problem. That's science."
Mira wrote finally.
"Entropy bridges discrete and continuous"
"Entropy connects discrete and continuous," Yuki read aloud.
"Beautiful summary," Aoi said. "Mira always hits the essence."
Dusk approached. Continuous and discrete, finite and infinite.
The world is complex, but understandable through the concept of entropy. Today too, they became a little wiser.