{"id":11,"date":"2011-08-06T14:54:07","date_gmt":"2011-08-06T14:54:07","guid":{"rendered":"http:\/\/www.clayford.net\/statistics\/?p=11"},"modified":"2023-08-14T21:51:31","modified_gmt":"2023-08-15T01:51:31","slug":"why-the-moment-generating-function-works","status":"publish","type":"post","link":"https:\/\/www.clayford.net\/statistics\/why-the-moment-generating-function-works\/","title":{"rendered":"Why the moment-generating function works"},"content":{"rendered":"

In a previous post<\/a>, I described how a moment-generating function does what its name says it does. Still, though, it’s not clear why it works. I mean, look at the formula:<\/p>\n

$$ M(t)=\\sum_{x}e^{tx}f(x)$$<\/p>\n

That’s the discrete case. In the continuous case we have:<\/p>\n

$$ M(t)=\\int_{-\\infty}^{+\\infty}e^{tx}f(x)dx$$<\/p>\n

So you take the usual expected value formula for either discrete or continuous cases, stick \\( e^{tx}\\) in there, and you get a function that generates moments. I don’t know about you, but that doesn’t make me pop up out of my chair and exclaim, “oh, of course, I see it! How obvious.” Fortunately, it’s not beyond comprehension. To see why it does what it does, let’s take the derivative of the moment-generating function in the discrete case, evaluate it at 0, and see what happens:<\/p>\n

$$ \\frac{d}{dt}M(t)=\\frac{d}{dt}\\sum_{x}e^{tx}f(x)$$<\/p>\n

$$ =\\sum_{x}\\frac{d}{dt}e^{tx}f(x)$$<\/p>\n

$$ =\\sum_{x}e^{tx}xf(x)$$<\/p>\n

$$ =EXe^{tX}$$<\/p>\n

Now evaluate the derivative at 0:<\/p>\n

$$ \\frac{d}{dt}M(0)=EXe^{0X} = EX$$<\/p>\n

So evaluating the first derivative of the moment-generating function at 0 gives us \\(EX\\), the first moment about the origin, also known as the mean.<\/p>\n

Let’s evaluate the second derivative of the moment-generating function at 0 and see what happens:<\/p>\n

$$ \\frac{d^{2}}{dt^{2}}M(t)=\\frac{d^{2}}{dt^{2}}\\sum_{x}e^{tx}f(x)$$<\/p>\n

$$ =\\sum_{x}\\frac{d}{dt}xe^{tx}f(x)$$<\/p>\n

$$ =\\sum_{x}e^{tx}(0)+xe^{tx}xf(x)$$<\/p>\n

$$ =\\sum_{x}x^{2}e^{tx}f(x)$$<\/p>\n

$$ =EX^{2}e^{tX}$$<\/p>\n

Now evaluate the derivative at 0:<\/p>\n

$$ \\frac{d^{}2}{dt^{2}}M(0)=EX^{2}e^{0X} = EX^{2}$$<\/p>\n

That gives us the second moment about the origin, which comes in handy when finding variance, since variance is equal to \\( Var(x) = EX^{2}-(EX)^{2}\\).<\/p>\n

This will keep happening for the third, fourth, fifth and every derivative thereafter, provided the moment-generating function exists<\/em>. And lest you think that’s just some technicality you never have to worry about, you should know the venerable t<\/em> and F<\/em> distributions do not have moment-generating functions. It turns out some of their moments are infinite. So there you go.<\/p>\n","protected":false},"excerpt":{"rendered":"

In a previous post, I described how a moment-generating function does what its name says it does. Still, though, it’s… Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9],"tags":[],"class_list":["post-11","post","type-post","status-publish","format-standard","hentry","category-expectation"],"_links":{"self":[{"href":"https:\/\/www.clayford.net\/statistics\/wp-json\/wp\/v2\/posts\/11","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.clayford.net\/statistics\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.clayford.net\/statistics\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.clayford.net\/statistics\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.clayford.net\/statistics\/wp-json\/wp\/v2\/comments?post=11"}],"version-history":[{"count":2,"href":"https:\/\/www.clayford.net\/statistics\/wp-json\/wp\/v2\/posts\/11\/revisions"}],"predecessor-version":[{"id":845,"href":"https:\/\/www.clayford.net\/statistics\/wp-json\/wp\/v2\/posts\/11\/revisions\/845"}],"wp:attachment":[{"href":"https:\/\/www.clayford.net\/statistics\/wp-json\/wp\/v2\/media?parent=11"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.clayford.net\/statistics\/wp-json\/wp\/v2\/categories?post=11"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.clayford.net\/statistics\/wp-json\/wp\/v2\/tags?post=11"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}