"The really unusual day would be one where nothing unusual happens."This quote above is from the very beginning of the book The Improbability Principle, written by Prof. David J. Hand. It is inarguably a must-read book, if you are interested in how humans perceive probability theory. Basically, the book explains why incredibly unlikely events happen, and keep on happening, like people winning lotteries multiple times, or lightening repeatedly striking the same unfortunate man, etc.
-Persi Diaconis
The book introduces very useful concepts, like the law of inevitability, the law of truly large numbers, the law of selection, the law of probability lever, and the law of near enough, which may occasionally sound counterintuitive due to our cognitive biases. There is actually a really cool website about the book, where you can find the definitions of all these laws.
The book is full of nice examples, where the probability of an event is given, which is probably so small and we interpret it as impossible, but yet it happens, and it is explained by one (or the combination) of the laws I mentioned above. When I was reading the first chapters, it suddenly occurred to me that I was not able to link these laws with the fact that even though the second law is a statistical law, it is never broken. We know that there is a possibility, which is astronomically small, but yet, we call it practically impossible. And the book argues exactly the opposite. It bugged me so much, so much that I actually stopped reading and wrote an e-mail to Prof. Hand in the middle of the night. Lucky for me, he was extremely kind, and answered my e-mail.
Let me explain what confused me more precisely. Considering the law of truly large numbers, it makes sense to think that if there are so much trials such that eventually, an event considered to be rare, will happen. Considering this fact, why don't we (or haven't we) see any examples of an inversion in the thermodynamical arrow at time at macro scale? Atoms are colliding and pushing and pulling each other every second, and although the number of interactions between these atoms result in plenty of trials, we don't see an example where the second law is broken. As a result, the question arises : Is it because there is a limit to what is rare? What is the distinction between surprisingly rare but still observable and practically impossible, in terms of numbers?
And as Prof. Hand wrote back, yes, there is a limit. If you have to wait longer than the age of the universe for something to happen, it is practically impossible.
But how do you make the transition between the probability of an event, and the expected time to see that event happen?
If you consider your trials being discrete and independent, where each trial has a probability of success \(p\), what you can do is to model the number of successful trials as a Poisson random variable. By doing so, you can also calculate the number of trials you need before the first success, which will be an exponential random variable with mean \(1/p\). This makes perfect sense if you apply it to simple examples. For instance, you have a fair coin, and if it turns out to be heads, it is a success. If you have tails, you fail. This makes \(p=1/2\), and expected amount of time for the first success, \(E[T]\), equal to \(1/p = 2\) trials. So if you want to see heads out of a fair coin toss, you have to toss it \(2\) times on average.
Another way of thinking this experiment, which I find more practical to calculate the average waiting time for other events, is the following. By tossing a coin, I do nothing but draw a random number, say \(0\) or \(1\), from a discrete uniform distribution, where \(P[x=0] = P[x=1] = 1/2 \). This means that I can interpret the random variable \(x\), as a state being \(0\) or \(1\) (or being heads or tails), and calculate the average waiting time until my first success by defining the amount of time spent for the transition from a state to another. If it takes \(5\) seconds for me to toss a coin again after the previous one, than it means I have to wait for \(10\) seconds on average to see the heads.
Now lets go back to the second law. Some time ago, I tried to calculate the probability of seeing an event which would violate the second law in this post, in a very roughly manner. It was not even close to reality, but I guess the numbers provided an interpretation of how unlikely it was for such an event to happen, even though the assumptions were too simple to reflect the true dynamics behind the process. What I calculated was the probability of an ink drop containing \(N\) atoms to form out of an homogeneous ink-water solution, and when I read the Prof. Hand's e-mail, it suddenly occurred to me that I had to calculate the amount of time I should wait for my \(N\)-molecule drop to form, and compare it to the age of the universe, so that I can prove that it's really impossible (and get a good night sleep).
As in the case of coin tossing, we can also interpret the drop forming problem as follows. Assume that I have a uniform distribution, which is actually the distribution of positions of each ink molecule in the homogeneous solution. I am drawing \(N\) random variables independently from that distribution, which are actually the molecules that will form the ink drop. Finally, I check whether they are in the same range, or in this example, whether their positions fall in the same shpere in three dimension. If they do, they form the drop. With further calculation, I can also calculate the probability of all these variables to fall in the same range.
In this analogy, every time I sample \(N\) molecules, I actually sample a microstate of this system. If I know the time that it takes for this system to go from one microstate to another, than I can also calculate how much should I wait on average to see that drop forming. As I have calculated previously, probability of an \(N\)-molecule drop to form was equal to \(\left(\frac{2r^2}{R^2h}\right)^N\), and for the given system parameters such as \(R\), \(r\), and \(h\), it was equal to \(p=1.8 \times 10^{-260}\) for \(N=60\).
But how do you determine the time between a microstate and another in a continuous system such as Brownian motion? I guess all you can do is to discretize your system, as I also did for the previous calculations, and chop it into time intervals \(\Delta t\), as small as possible. Lets assume that it takes 1 microsecond, or 1 millionth of a second, (actually, a time step of 1 millisecond is considered fairly enough for Brownian motion simulations) for our system to jump from a microstate to another, i.e., \(\Delta t = 10^-6\) seconds. The average time we need to wait in terms of seconds becomes \begin{align} E[T] &= \frac{1}{p} \times \Delta t, \nonumber \\ &= 10^{260} \times 10^{-6}, \nonumber \\ &= 10^{254}. \nonumber \end{align} And in terms of years? Divide it with \((365 \times 24 \times 60 \times 60)\), and you will get \(3.17 \times 10^{246}\) years. How old is the universe? \(13.8 \times 10^9\) years.
So yes, it is practically impossible. We can all sleep now :)
