The exponential distribution has the quirky property of having no memory. Before we wade into the math and see why, let’s consider a situation where there is memory: drawing cards. Let’s say you have a well-shuffled deck of 52 cards and you draw a single card. What’s the probability of drawing an ace? Since there are 4 aces in a deck of 52 cards, the probability is . We draw our card and it’s not an ace. We set the card aside, away from the deck, and draw again. Now our probability of drawing an ace is . We have a slightly better chance on the 2nd draw. The condition that we have already selected a card that wasn’t an ace changes the probability we draw an ace. This doesn’t happen with the exponential distribution.
Let’s say we have a state-of-the-art widget (version 2.0) that has a lifespan that can be described with an exponential distribution. Further, let’s say the mean lifespan is 60 months, or 5 years. Thanks to the “no memory” property, the probability of the lifespan lasting 7 years is that same whether the widget is new or 5 years old. In math words:
That means if I bought a widget that was 5 years old, it has the same probability of lasting another 7 years as a brand new widget has for lasting 7 years. Not realistic but certainly interesting. Showing why this is the case is actually pretty straight-ahead.
We want to show that for the exponential distribution, .
Recall the cumulative distribution of an exponential distribution is . That’s the probability of an event occurring before a certain time x. The complement of the cumulative distribution is the probability of an event occurring after a certain time:
Also recall the definition of conditional probability:
Let’s plug into the equality we want to prove and see what happens:
There you go. Not too bad.
We can actually go the other direction as well. That is, we can show that if is true for a continuous random variable X, then X has an exponential distribution. Here’s how:
(substitute the cdf expressions)
(using the definition of conditional probability)
(If X > x + y, then X > x)
Now substitute in generic function terminology, say :
Rearranging terms gives us
Now for that equality to hold, the function h(x) has to have an exponential form, where the variable is in the exponent, like this: . Recall that . If , then our equality above works. So we let . That allows to make the following conclusion:
Now let b = ln a. We get . Solving for F(x) we get . Since , b must be negative. So we have . Now we just let and we have the cumulative distribution function for an exponential distribution: .
That’s the memoryless property for you. Or maybe it’s called the forgetfulness property. I can’t remember.