Quantitative finance collector
C++ Matlab VBA/Excel Java Mathematica R/Splus Net Code Site Other
Apr 28

Stata, Fast and Slow: Why Running Many Small Regressions in a Large Dataset Takes So Long; and What to Do About It

Posted by abiao at 15:08 | Default | Comments(4) | Reads(1991)

Quotation
Stata is fast, often very fast. However, when performing regressions on small sub-samples within a large host dataset (more than 1 million observations) performance can deteriorate by many orders of magnitude. For example, an OLS regression on a sub-sample of 100 consecutive observations takes 3.6 seconds in a host dataset with 1 billion observations, but only 3.8 milliseconds in a host dataset with 1000 observations. The difference in performance is due to the mechanism regress uses to mark estimation samples. This performance deterioration has practical implications in finance research, where many variables of interest are themselves estimated via millions of individual OLS regressions within large panel datasets. I suggest an approach that circumvents this issue by using a simple Mata implementation of regress which I call fastreg. As a test, I estimate daily Fama and French 3-factor betas for individual stocks in the CRSP database from 1923 to 2013 using a 250-day rolling window. In this setting fastreg is approximately 367 times faster than regress. The code for fastreg ado is included in the Appendix and is open-source licensed under the GNU GPL.


http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2423171


jack Email
2019/08/08 23:04
Now i am definitely content to uncover this page in addition to performed delight in examining practical articles or blog posts placed in this article. This thoughts on the article author seemed to be brilliant, appreciate it with the write about.  East Grinstead Accountants
Vincent Williams
2019/08/28 01:30
Excellent article. Very interesting to read. I really love to read such a nice article. Thanks! keep rocking.  busines class to Dubai
Video transcriber Email Homepage
2019/09/11 21:56
Please continue this great work and I look forward to more of your awesome blog posts.  Video transcriber
Tyler Walsh Email
2019/09/19 21:26
Superbly written article, if only all bloggers offered the same content as you, the internet would be a far better place..  자동차 보험
Pages: 1/1 First page 1 Final page
Add a comment
Emots
Enable HTML
Enable UBB
Enable Emots
Hidden
Remember
Nickname   Password   Optional
Site URI   Email   [Register]