about me
art
biz
Chess
corrections
economics
EconoSchool
Finance
friends
fun
game theory
games
geo
mathstat
misc
NatScience
... more
Profil
Logout
Subscribe Weblog

 
In his latest entry Steffen (Mit dem Kopf voran) posted the following quote from Enrico Fermi:
"I remember my friend Johnny von Neumann used to say, 'with four parameters I can fit an elephant and with five I can make him wiggle his trunk.'" A meeting with Enrico Fermi, Nature 427, 297; 2004.
I know that there exists a fool-proof method for sculpting an elephant (get a huge block of marble and chip away everything that doesn't look like an elephant) but fitting an elephant with four parameters is an impossible task. Actually, the question, "How many paramters does it take to fit an elephant?" has already been answered by Wel in 1975: He started with an idealized drawing (A) defined by 36 points and showed that one could be satisfied with the fit of a 30 term model:elephant
The point of the elephant-story* is what every student is told in his first econometrics lecture: The residual sum can't increase by adding more explanatory variables (and therefore more parameteres) to a linear regression model. As a rule: The more explanatory variables, the better the fit. The problem that arises when one keeps adding regressors is that the precision of the estimators will be very poor, i.e. overfitted models (models with too much paramters) have estimated (and actual) sampling variances that are needlessly large (i.e. with new data your model will perform poorly).

For a linear model containing predictors x1,x2, ...,xk with estimators of coefficients b0, b1,...,bk, if the regressor xk+1 is added to the model producing new estimators of coefficients b*0, b*1,...,b*k+1, then for i = 0,...,k

Var(b*i) ≥ Var(bi)

Ergo: If a model is made more complex, the variance of model predictions will increase (since the parameters are estimated less precisely).

On the other hand if important regressors are excluded the predicted dependend variables ("y hat") will systematically deviate from the observed variables (y) and the residuals (which contain the omitted variable) will be unnecessarily large. If the assumed model is not correct due to an important predictor being excluded then the bias could be reduced by including that predictor.

The figure shown below shows the trade-off between (squared) bias (solid line) and variance versus the number of estimable parameters in the model. All model selection methods implicitly employ some notion of this trade-off:biasvariancetradeoff
Keep in mind that the best approximating model need not occur exactly where the two curves intersect. Conclusion: Whether a predictor is worth adding to a model (for a fixed vector of regressors, x) depends on whether the reduction in squared bias is greater than the increase in variance.

Model too simple ⇔ high bias/low variance

Model too complex ⇔ low bias/high variance

~~~

Note for Advanced Readers: Mean Squared Error (MSE):

Assume z = h(x,θ)+v, where z is the dependend variable, h(.) is some function, x is is the vector of regressors, v is noise and θ is a vector of model parameters. The MSE of the model at a fixed x can be decomposed as:
variancebiastradeoff

*If one shows up with a model with lots of parameters opponents will argue that one has used enough parameters to fit an elephant.

Graphics taken from Model Selection and Multi-Model Inference,
Kenneth Burnham & David Anderson
maskodok meinte am 15. Jan, 01:30:
Interesting topic for a blog. I have been searching the Internet for fun and came upon your website. Fabulous post. Thanks a ton for sharing your knowledge! It is great to see that some people still put in an effort into managing their websites. I'll be sure to check back again real soon.
Mobil Sedan COrolla,Idrpoker.com agen Texas poker Online Indonesia Terpercaya, Mobil Sedan COrolla, Cipto Junaedy 
ARL (guest) meinte am 23. Feb, 05:11:
A great experience Jasa SEO 
asda (guest) antwortete am 27. Feb, 11:58:
This machine or device can be used to treat pains associated with muscle spasms, headaches, back pain, and other chronic or acute pains. Its use not only gives the user instant relief from pain but its regular use can help manage extreme flare-ups of arthritis pain. The machines are used to reduce swelling and muscle stiffness in the localized area.

A sonogram is a useful tool in obstetrics and gynecology. It has no known ill effects on the human body, so it is the most commonly used imaging modality for obstetric ultrasound. It is used to evaluate the uterus and ovaries because X-rays could negatively affect fertility. It is also used to compliment mammography. Mammograms are formed by the reaction of X-rays to the different densities of breast tissue. Ultrasound confirms if a mass seen on mammography is cystic or solid and if there is any blood flow in the mass. 
kimcil (guest) antwortete am 7. Apr, 12:24:
I am preparing a research paper and collecting information on this topic. Your post is one of the better that I have read. Thank you for putting this information into one post.
alfamart official partner merchandise fifa piala dunia brazil 2014
Unit Link Terbaik di Indonesia Commonwealth Life Investra Link 
Ahmad (guest) antwortete am 15. Apr, 14:57:
Almost everyone who drives a car is required by law to carry auto insurance. Month to month (or perhaps every six months, depending on how you choose to pay your premium) this can seem like an inconvenience on which you don’t see any return. steamautospa 
Ahmad (guest) antwortete am 15. Apr, 14:57:
Car shopping is one thing people seem to struggle with. The reason is due to the fact that many people aren’t sure what they should check out in a car. You all need to be a good negotiator and business savvy. chrusciel-transport 
Ahmad (guest) antwortete am 15. Apr, 14:58:
Although there are many internet marketing tactics that you can use, a lot of what you will actually be able to do depends on your technology. Make sure you take into consideration the type of item you are trying to market. bbs-businessschool 
begedir antwortete am 4. May, 14:58:
I am preparing a research paper and collecting information on this topic. Your post is one of the better that I have read. Thank you for putting this information into one post. belajar grammar bahasa inggris , ingin hamil , kata kata untuk ibu 
BlackMen (guest) antwortete am 20. May, 15:14:
BlackMen
I am preparing a research paper and collecting information on this topic. Your post is one of the better that I have read. Caravans West 
If the assumed (guest) antwortete am 7. Jul, 12:49:
If the assumed
If the assumed mannequin is nay regulate payable to an essential predictor because excluded therefore the bias could be reduced by including that predictor. freedebtconsolidationquotes.com 
octavinsu meinte am 1. Mar, 06:45:
Dewa Poker
I comparable what you guys are positive too. Such smart work and reporting! Keep up the superb works guys I've incorporated you guys to my blogroll. I think it'll improve the value of my website :). agen bola terpercaya,agen bola online,agen bola sbobet,agen ibcbet Online 
octavinsu meinte am 1. Mar, 06:45:
Sedu
Hi my loved one! I wish to say that this post is amazing, nice written and include almost all important infos. I would like to look more posts like this . jasa seo,jasa seo murah,jasa seo profesional,jasa seo murah bergaransi,jasa seo terpercaya,Jasa SEO Indonesia,Pakar SEO 
Agen Bola Terpercaya (guest) meinte am 1. Mar, 09:16:
Dewa Poker
I was just searching for this information for a while. After 6 hours of continuous Googleing, at last I got it in your site. I wonder what is the lack of Google strategy that do not rank this kind of informative web sites in top of the list. Normally the top web sites are full of garbage. 
Women Hairstyles 2014 (guest) meinte am 24. Mar, 18:28:
I was very encouraged to find this site. I want to thank you for this special read. I definitely enjoyed every bit of it and I ve marked you to check out new stuff you write. 
iGamble247 (guest) meinte am 14. Apr, 22:10:
i would like to bookmark the page so i can come here again to read you, as you have done a wonderful job. live casino online 
robin11223 (guest) meinte am 30. Apr, 12:12:
Excellent
This is a great inspiring article.I am pretty much pleased with your good work.You put really very helpful information...
Siri 
waar234 meinte am 1. May, 13:44:
william
back pain, and other chronic or acute pains. Its use not only gives the user instant relief from pain but its regular use can help manage extreme flare-ups of arthritis pain. The machines are used to reduce swelling and muscle stiffness in the localized area. iPad screen replacement 
ashleydanielle (guest) meinte am 5. May, 17:00:
I think
I can see that you are an expert at your field! I am launching a website soon, and your information will be very useful for me.. Thanks for all your help and wishing you all the success in your business.
60 days to 1k bonus 
BlackMen (guest) antwortete am 24. May, 09:30:
BlackMen
failing to qualify for the 1964 race, driving a Brabham for Rob Walker. For his two Grand Prix starts, he drove for Team Lotus. He scored no championship points.Buy Traffic 
ashleydanielle (guest) meinte am 7. May, 11:15:
I think
Yes i am totally agreed with this article and i just want say that this article is very nice and very informative article.I will make sure to be reading your blog more. You made a good point but I can't help but wonder, what about the other side? !!!!!!THANKS!!!!!!

EON contact number 
ashleydanielle (guest) meinte am 7. May, 14:28:
I think
I have been reading out many of your articles and i must say pretty good stuff. I will surely bookmark your site.
SAP planning 
Jackson Gordon  (guest) meinte am 10. May, 09:25:
william
The reason is due to the fact that many people aren’t sure what they should check out in a car. You all need to be a good negotiator and business savvy. 
ttlwu meinte am 12. May, 14:37:
Thank you very much for writing such an interesting article on this topic. This has really made me think and I hope to read more.
Escortnaturalfeeling.de 
berita bola terkini hari ini (guest) meinte am 13. May, 12:04:
Klik http://tinyurl.com/n5cqtzh berita capres jokowi 2014 di situs berita hari ini Liputan6.com
Klik http://tinyurl.com/yzty94u komisi pemilihan umum kpu di situs berita hari ini Liputan6.com
Klik http://tinyurl.com/nu4qhdh capres prabowo subianto di situs berita hari ini Liputan6.com
Klik http://tinyurl.com/lh8owza berita pemilu 2014 indonesia di situs berita hari ini Liputan6.com
Klik http://tinyurl.com/pzopl89 Capres partai gerindra di situs berita hari ini Liputan6.com
Klik http://tinyurl.com/nmgc58c berita persib bandung di situs berita hari ini Liputan6.com
Klik http://tinyurl.com/n44kfj7 berita bola terbaru hari ini di situs berita hari ini Liputan6.com
Klik http://tinyurl.com/mjrvjqz jadwal liga champion eropa di situs berita hari ini Liputan6.com
Klik http://tinyurl.com/jwh3hcu piala dunia brazil 2014 di situs berita hari ini Liputan6.com
Klik http://tinyurl.com/yzty94u berita indonesia super league isl 2014 di situs berita hari ini Liputan6.com 
ttlwu meinte am 15. May, 12:46:
I think
I just couldn't leave your website before telling you that I truly enjoyed the top quality info you present to your visitors? Will be back again frequently to check up on new posts.
free credit report gov 
stylishlampshades.com (guest) meinte am 17. May, 15:04:
williamss
This is a great inspiring article.I am pretty much pleased with your good work.You put really very helpful information... 
BlackMen (guest) antwortete am 26. Jun, 08:02:
BlackMen
I am pretty much pleased with your good work.You put really very helpful information... buy active instagram followers 
plauku priauginimas klaipeda (guest) meinte am 17. May, 16:12:
william
: Whether a predictor is worth adding to a model (for a fixed vector of regressors, x) depends on whether the reduction in squared bias is greater than the increase in variance. 
ttlwu meinte am 18. May, 11:11:
I think
Its a great pleasure reading your post.Its full of information I am looking for and I love to post a comment that "The content of your post is awesome" Great work.
http://rebelmouse.com/ 
BlackMen (guest) antwortete am 26. Jun, 12:00:
BlackMen
The machines are used to reduce swelling and muscle stiffness in the localized area.Winifred Henry 
بفركس نصب (guest) meinte am 19. May, 10:44:
william
This continued until 1938 when Theremin mysteriously disappeared from his New York apartment. It is widely believed that Russian officials extradited Theremin, but many feel that he fled the states of his own volition, either out of fear of his cover being blown or pure homesickness. 
ttlwu meinte am 21. May, 15:57:
I think
This article gives the light in which we can observe the reality. This is very nice one and gives indepth information. Thanks for this nice article.
samantha sanderson penguin method guide 
ttlwu meinte am 22. May, 15:02:
I think
Thanks a lot for sharing us about this update. Hope you will not get tired on making posts as informative as this.
bail bond la puente 
Plombier Paris (guest) meinte am 23. May, 13:27:
elephant-story* is what every student is told in his first econometrics lecture: The residual sum can't increase by adding more explanatory variables (and therefore more parameteres) to a linear regression model. As a rule: The more explanatory variables, the better Plombier Paris 
ttlwu meinte am 24. May, 11:44:
I think
Thanks for posting this info. I just want to let you know that I just check out your site and I find it very interesting and informative. I can't wait to read lots of your posts.
https://www.rebelmouse.com/thevenusfactorreview/ 
ttlwu meinte am 24. May, 15:43:
I think
I am very much pleased with the contents you have mentioned. I wanted to thank you for this great article.
blood pressure solution ken burge 
ttlwu meinte am 25. May, 13:42:
I think
It was a very good post indeed. I thoroughly enjoyed reading it in my lunch time. Will surely come and visit this blog more often. Thanks for sharing.
www.rebelmouse.com/purereikihealingmasteryreviews 
ttlwu meinte am 29. May, 12:28:
I think
I wanted to thank you for this great read!! I definitely enjoying every little bit of it I have you bookmarked to check out new stuff you post.
www.rebelmouse.com/purereikihealingmasteryreviews 
ttlwu meinte am 31. May, 13:57:
I think
this is really nice to read..informative post is very good to read..thanks a lot!
lean moms fast fat loss success program pdf 
ttlwu meinte am 5. Jun, 16:48:
I think
I think you did an awesome job explaining it. Sure beats having to research it on my own. Thanks
lift weights faster by jen sinkler method 
ttlwu meinte am 7. Jun, 17:28:
I think
Your content is valid and informative in my personal opinion. You have really done a lot of research on this topic. Thanks for sharing it.
girlfriend activation system reviews: www.archive.org/details/girlfriend-activation-system-review/ 
ttlwu meinte am 8. Jun, 15:33:
I think
hi was just seeing if you minded a comment. i like your website and the thme you picked is super. I will be back.
fibroids miracle 
ttlwu meinte am 11. Jun, 15:25:
I think
Positive site, where did u come up with the information on this posting?I have read a few of the articles on your website now, and I really like your style. Thanks a million and please keep up the effective work.
magasin tout pour bebe 
MaleXem.com (guest) meinte am 12. Jun, 07:33:
MaleXem.com
impossible task. Actually, the question, "How many paramters does it take to fit an elephant?" has already been answered by Wel in 1975: He started with an idealized drawing MaleXem.com 
ttlwu antwortete am 14. Jun, 16:20:
I think
I have recently started a blog, the info you provide on this site has helped me greatly. Thanks for all of your time & work.
texbvba.be 
ttlwu meinte am 12. Jun, 15:39:
I think
It’s really a nice and helpful piece of info. I’m glad that you just shared this helpful info with us. Please keep us informed like this. Thanks for sharing.
www.kateshouse.be 
ttlwu meinte am 15. Jun, 15:30:
I think
I can see that you are an expert at your field! I am launching a website soon, and your information will be very useful for me.. Thanks for all your help and wishing you all the success in your business.
www.tvv.be 
ttlwu meinte am 17. Jun, 17:16:
I think
Thanks you very much for sharing these links. Will definitely check this out..
Baumdienst 
cavedog fitness (guest) meinte am 17. Jun, 19:15:
defined by 36 points and showed that one could be satisfied with the fit of a 30 term model:cavedog fitness 
Unimat (guest) meinte am 18. Jun, 08:39:
impossible task. Actually, the question, "How many paramters does it take to fit an elephant?Unimat 
ttlwu meinte am 21. Jun, 16:55:
I have read your blog it is very helpful for me. I want to say thanks to you. I have bookmark your site for future updates.
girlfriend activation system review 
ttlwu meinte am 22. Jun, 12:59:
irthkklsnfs

Thanks for your post. I’ve been thinking about writing a very comparable post over the last couple of weeks, I’ll probably keep it short and sweet and link to this instead if thats cool. Thanks. LTE Geschwindigkeit

 
ttlwu meinte am 23. Jun, 13:02:
irthkklsnfs
Thanks for this post, I am a big fan of this website would like to go on updated.
visit website 
ttlwu meinte am 24. Jun, 16:57:
irthkklsnfs
This is very educational content and written well for a change. It's nice to see that some people still understand how to write a quality post.!
datasoftrecovery 
ttlwu meinte am 25. Jun, 13:49:
This is such a great resource that you are providing and you give it away for free. I love seeing blog that understand the value. Im glad to have found this post as its such an interesting one! I am always on the lookout for quality posts and articles so i suppose im lucky to have found this! I hope you will be adding more in the future...
trick photography and special effects guide 
ttlwu meinte am 25. Jun, 17:42:
i think
It is imperative that we read blog post very carefully. I am already done it and find that this post is really amazing.
https://www.rebelmouse.com/kenburgepaleoburnreview/ 
Fashion Magazine (guest) meinte am 26. Jun, 09:21:
Fashion Magazine
This post is very amusing. I was extremely jolly when I finished reading this post. I don't know what part makes this extremely cheerful but I sure as hell was festive after finishing this. Please keep giving us posts like this! Fashion Magazine 
Electro Golf Trolley (guest) meinte am 26. Jun, 13:09:
Elektro Golf Caddys (Trolleys) zu günstigen Preisen - Top Qualität.Electro Golf Trolley 
ttlwu meinte am 26. Jun, 13:15:
irthkklsnfs
Nice to read your article! I am looking forward to sharing your adventures and experiences.
desire system method 
tsflw meinte am 1. Jul, 14:37:
I know your expertise on this. I must say we should have an online discussion on this. Writing only comments will close the discussion straight away! And will restrict the benefits from this information.
www.slaapcomfortcenter.be 
slideshare (guest) meinte am 4. Jul, 11:14:
slideshare
Path, the private social network, said it had finally closed a $25 million Series C round, adding a new investor, Indonesia’s Bakrie Global Group. slideshare 
thscp1122 meinte am 6. Jul, 11:56:
Great post, and great website. Thanks for the information!
photographe mariage toulouse 
thscp1122 meinte am 7. Jul, 10:10:
Great post, and great website. Thanks for the information!
what does it mean when your ex doesnt text you back 
thscp1122 meinte am 7. Jul, 11:12:
Your website is really cool and this is a great inspiring article. Thank you so much.
how to get a girlfriend and keep her 
ttlwu meinte am 7. Jul, 15:03:
Your content is nothing short of brilliant in many ways. I think this is engaging and eye-opening material. Thank you so much for caring about your content and your readers.
learn more at Rebel Mouse 
monu (guest) antwortete am 18. Jul, 06:27:
tape
And they do plenty of that in Sex Tape. Reuniting with Jake Kasdan, who helmed their 2011 comedy hit Bad Teacher, the pair play Annie and Jay—a married couple with young kids who, in an attempt to inject some spice in their nonexistent sex life, decide to film a wild-and-wacky three-hour sex tape. http://www.jojobaa.com 
ASGAS (guest) antwortete am 23. Jul, 11:25:
Once
Without doubt Once i may be conventional fortify they'll presented any person some sort of write-up alongside leads to furthermore discover. Cellulite Cream 
ANU (guest) antwortete am 24. Jul, 08:17:
classs
Once i fissure wise many cogitates to assist swiftness classs what you are doing any kind of mundane location. Impressive treat. Once i weakness for getting thinking about regularly this web site together with Now i am affected! That's a time period. chromium supplement 
tsflw meinte am 8. Jul, 12:19:
Thank you again for all the knowledge you distribute,Good post. I was very interested in the article, it's quite inspiring I should admit. I like visiting you site since I always come across interesting articles like this one.Great Job, I greatly appreciate that.Do Keep sharing! Regards,
3 credit reports 
thscp1122 meinte am 8. Jul, 15:21:
Your blog provided us with valuable information to work with. Each & every tips of your post are awesome. Thanks a lot for sharing. Keep blogging,
megashare 
ttlwu meinte am 9. Jul, 14:05:
Very good written article. It will be supportive to anyone who utilizes it, including me. Keep doing what you are doing – can’r wait to read more posts.
Royal Vegas Casino 
thscp1122 meinte am 12. Jul, 11:38:
This is such a great resource that you are providing and you give it away for free. I love seeing websites that understand the value of providing a quality resource for free. It is the old what goes around comes around routine.
truth about cellulite 
ttlwu meinte am 12. Jul, 12:27:
I am very happy to discover your post as it will become on top in my collection of favorite blogs to visit.
text your ex back 2.0 michael fiore reviews 
http://advancedseotutorials.com/ (guest) meinte am 15. Jul, 15:17:
A real fine article this is. To be honest, I never expected you to deliver on such consistent rates but you've proven me wrong and delivered all good blogs. Great article. Thanks for sharing folks. THANK YOU!!!! http://advancedseotutorials.com/ 
tsflw meinte am 16. Jul, 13:54:
Its a great pleasure reading your post.Its full of information I am looking for and I love to post a comment that "The content of your post is awesome" Great work. venus factor system 
montecarloatl meinte am 16. Jul, 17:27:
joe
cool elephant. I always wanted one. I saw one at a party the Monte Carlo Productions casino theme party atlanta 
atlfreedomlimo1 antwortete am 17. Jul, 20:11:
jay
i love elephants and atlanta freedom limousine rental 
tsflw meinte am 17. Jul, 14:17:
Wonderful article, thanks for putting this together! This is obviously one great post. Thanks for the valuable information and insights you have so provided here. hypothyroidism revolution by tom brimeyer 
page jaune (guest) meinte am 18. Jul, 22:41:
idealized drawing (A) defined by 36 points and showed that one could be satisfied with the fit of a 30 term model: 
tsflw meinte am 19. Jul, 14:36:
Its a great pleasure reading your post.Its full of information I am looking for and I love to post a comment that "The content of your post is awesome" Great work. https://www.rebelmouse.com/oldschoolnewbodyreviewf4x/ 
cheap nike air rifts (guest) meinte am 20. Jul, 11:56:
sculpting an elephant (get a huge block of marble and chip away everything that doesn't look like cheap nike air rifts 
izdelava spletne trgovine (guest) meinte am 28. Jul, 07:28:
Great post
e are a group of volunteers and opening a new scheme in our community. Your website offered us with valuable info to work on. You have done a formidable job and our entire community will be grateful to you 
izdelava spletne trgovine (guest) meinte am 28. Jul, 07:30:
Great infors
Thanks, I've just been searching for information approximately this topic for a while and yours is the greatest I've found out so far. But, what in regards to the conclusion? Are you positive in regards to the source?
http://izdelava-spletnihstrani.weebly.com/
https://sites.google.com/site/naprednaizdelavaspletnihstrani/
https://sites.google.com/site/naprednaizdelavaspletnihstrani/izdelava-spletne-trgovine
http://freshxdesign-spletne-storitve.blogspot.com/
http://profesionalnespletnestrani.wordpress.com/
http://izdelavaspletnih-strani.webs.com/
http://dzetodzeto.wix.com/spletnestrani
http://goo.gl/j0M3fl
http://bit.ly/freshxdesign