In a previous post, I described how a moment-generating function does what its name says it does. Still, though, it’s not clear why it works. I mean, look at the formula:

$$ M(t)=\sum_{x}e^{tx}f(x)$$

That’s the discrete case. In the continuous case we have:

$$ M(t)=\int_{-\infty}^{+\infty}e^{tx}f(x)dx$$

So you take the usual expected value formula for either discrete or continuous cases, stick \( e^{tx}\) in there, and you get a function that generates moments. I don’t know about you, but that doesn’t make me pop up out of my chair and exclaim, “oh, of course, I see it! How obvious.” Fortunately, it’s not beyond comprehension. To see why it does what it does, let’s take the derivative of the moment-generating function in the discrete case, evaluate it at 0, and see what happens:

$$ \frac{d}{dt}M(t)=\frac{d}{dt}\sum_{x}e^{tx}f(x)$$

$$ =\sum_{x}\frac{d}{dt}e^{tx}f(x)$$

$$ =\sum_{x}e^{tx}xf(x)$$

$$ =EXe^{tX}$$

Now evaluate the derivative at 0:

$$ \frac{d}{dt}M(0)=EXe^{0X} = EX$$

So evaluating the first derivative of the moment-generating function at 0 gives us \(EX\), the first moment about the origin, also known as the mean.

Let’s evaluate the second derivative of the moment-generating function at 0 and see what happens:

$$ \frac{d^{2}}{dt^{2}}M(t)=\frac{d^{2}}{dt^{2}}\sum_{x}e^{tx}f(x)$$

$$ =\sum_{x}\frac{d}{dt}xe^{tx}f(x)$$

$$ =\sum_{x}e^{tx}(0)+xe^{tx}xf(x)$$

$$ =\sum_{x}x^{2}e^{tx}f(x)$$

$$ =EX^{2}e^{tX}$$

Now evaluate the derivative at 0:

$$ \frac{d^{}2}{dt^{2}}M(0)=EX^{2}e^{0X} = EX^{2}$$

That gives us the second moment about the origin, which comes in handy when finding variance, since variance is equal to \( Var(x) = EX^{2}-(EX)^{2}\).

This will keep happening for the third, fourth, fifth and every derivative thereafter, provided the moment-generating function exists. And lest you think that’s just some technicality you never have to worry about, you should know the venerable t and F distributions do not have moment-generating functions. It turns out some of their moments are infinite. So there you go.