Why the moment-generating function works

In a previous post, I described how a moment-generating function does what its name says it does. Still, though, it’s not clear why it works. I mean, look at the formula:

M(t)=\sum_{x}e^{tx}f(x)

That’s the discrete case. In the continuous case we have:

M(t)=\int_{-\infty}^{+\infty}e^{tx}f(x)dx

So you take the usual expected value formula for either discrete or continuous cases, stick e^{tx} in there, and you get a function that generates moments. I don’t know about you, but that doesn’t make me pop up out of my chair and exclaim, “oh, of course, I see it! How obvious.” Fortunately, it’s not beyond comprehension. To see why it does what it does, let’s take the derivative of the moment-generating function in the discrete case, evaluate it at 0, and see what happens:

\frac{d}{dt}M(t)=\frac{d}{dt}\sum_{x}e^{tx}f(x) =\sum_{x}\frac{d}{dt}e^{tx}f(x) =\sum_{x}e^{tx}xf(x) =EXe^{tX}

Now evaluate the derivative at 0:

\frac{d}{dt}M(0)=EXe^{0X} = EX

So evaluating the first derivative of the moment-generating function at 0 gives us EX, the first moment about the origin, also known as the mean.

Let’s evaluate the second derivative of the moment-generating function at 0 and see what happens:

\frac{d^{2}}{dt^{2}}M(t)=\frac{d^{2}}{dt^{2}}\sum_{x}e^{tx}f(x) =\sum_{x}\frac{d}{dt}xe^{tx}f(x) =\sum_{x}e^{tx}(0)+xe^{tx}xf(x) =\sum_{x}x^{2}e^{tx}f(x) =EX^{2}e^{tX}

Now evaluate the derivative at 0:

\frac{d^{}2}{dt^{2}}M(0)=EX^{2}e^{0X} = EX^{2}

That gives us the second moment about the origin, which comes in handy when finding variance, since variance is equal to Var(x) = EX^{2}-(EX)^{2}.

This will keep happening for the third, fourth, fifth and every derivative thereafter, provided the moment-generating function exists. And lest you think that’s just some technicality you never have to worry about, you should know the venerable t and F distributions do not have moment-generating functions. It turns out some of their moments are infinite. So there you go.

One thought on “Why the moment-generating function works

  1. Pingback: » The moment-generating function Statistics you can Probably Trust

Leave a Reply

Your email address will not be published. Required fields are marked *