Quantitative Finance Collector

R Optimization Test

Mon, 23 Jul 2012 10:12:36 +0000

I have tested several R optimization functions before: nlm, optim(Nelder-Mead), optim(BFGS), optim(SANN), nlminb, optim (L-BFGS-B) for a eight-parameter Vasicek interest rate model, overall I find that for my setting, nlminb is the best and all R functions finish within seconds. For detail please read the old post at R optimization function test

Pat at Portfolio Probe recently had a wonderful test on some heuristic optimization methods, including simulated annealing, traditional genetic algorithm, evolutionary algorithms. By using R packages and functions - Rmalschains, GenSA, genopt, DEoptim, soma, rgenoud, GA, NMOF and SANN method of optim, he finds that the Rmalschains and GenSA packages are standing out. Nice one, original article is at A comparison of some heuristic optimization methods.
Tags - r , optimization
Read the full post at R Optimization Test.

Week in Review 021211 R Language

Fri, 02 Dec 2011 10:08:47 +0000

Happy last month of 2011. I will fly to Sydney to present a paper at the 24th Australasian Finance & Banking Conference on next Thursday, so we may not have a review next week. However, feel free to contact me @a_biao for sharing any useful post. This week's review is highly concentrated on R language.

R-code for the algorithm of Ait-Sahalia: the Closed-Form expansion for the transition densities of diffusions by Professor Yacine Aït-Sahalia facilitates the Maximum Likelihood Estimation, the related papers and Matlab package can be downloaded directly at his website at Ait-Sahalia, but in case you are a R user, this is what you need.

R Memory Issue: insights on R memory issue, most of us have met it more or less.

Regression via Gradient Descent in R: detailed simple example demonstrating how to run a regression via Gradient descent in R: principle and codes.

Amelia II: A Program for Missing Data: An excellent R package for multiple imputation of missing data. I had a post introducing its first version at missing data imputation.

The Art of R Programming: A Tour of Statistical Software Design: the book title tells it, you can't miss it as a R user.

R Cheat Sheets: still having trouble remembering the exact commands in R? here is an excellent collection of R cheet sheets.

How to interpret Johansens' test results: simple while detailed examples guiding you through the basic knowledge how to interpret Johansenss test results for cointegration analysis.

Improving Trend-Following Strategies With Counter-Trend Entries: minor adjustments to strategies that can both improve their backtest performance and also reduce the real costs of trading.
Tags - r , week , review
Read the full post at Week in Review 021211 R Language.

Necessity to Explain CDS with A Regime Switching Model

Thu, 07 Jul 2011 09:56:38 +0000

Examining the determinants of credit default swap (CDS) spreads is a hot topic, CDS spread has displayed siginificant regime switching behaviour since the break of credit crisis, which can be seen from the old graph in the post Credit Default Spread and Historical Volatility

There are sound reasons to believe that CDS spreads keep high in the period of turbulence while stay stably low during most of quiet periods. To investigate if there is possible regime switch phenomenon, I run a three year rolling panel regression using CDSs of over 250 reference entities on several widely accepted explanatory variables including: leverage, volatility, treasury yield and the spread of three month Libor and repo rates, where the last variable is used to proxy liquidity risk. The coefficients for each variable is plotted below

the coefficients of leverage and treasury yields are changing but without clear regime pattern, on the contrary, the volatility, especially the liquidity effects are suggesting there may exist regime switching and the necessity to employ a Markov regime switch model to explain CDS spreads.

PS: a matlab markov regime switching package can be found here; the panel regression is done with the R package PLM at http://cran.r-project.org/web/packages/plm/vignettes/plm.pdf
Tags - regime , cds
Read the full post at Necessity to Explain CDS with A Regime Switching Model.

Using R in Excel

Wed, 01 Jun 2011 08:20:20 +0000

Got to know a very cool tool to use R in Excel named RExcel, basically it provides an integration solution such that users can get data, run command in Excel the same way as in R, which is presumably good and convenient to present results to your colleagues.

Check yourself a demo video at http://rcom.univie.ac.at/RExcelDemo/

The package can be downloaded free at http://rcom.univie.ac.at/download.html
Tags - r , excel
Read the full post at Using R in Excel.

Download R 2.13.0 for Windows

Fri, 15 Apr 2011 09:10:08 +0000

Some of you may hear that the new release of R 2.13.0 is out, with some Windows-specific changes to R 2.13.0:
file.exists() and unlink() have more support for files > 2GB;
A few more file operations will now work with >2GB files.
which could help us to relieve the worry of handling large datasets in R. An exciting post shows us how to speed up R code up to 4 times by using the new R compiler package.

Download R 2.13.0 for Windows at http://cran.r-project.org/bin/windows/base/, to help you upgrade or install R for windows, here are a few useful articles:
http://cran.r-project.org/bin/windows/base/rw-FAQ.html#Does-R-run-under-Windows-Vista_003f
http://www.r-statistics.com/2011/04/how-to-upgrade-r-on-windows-7/
http://stackoverflow.com/questions/1401904/painless-way-to-install-a-new-version-of-r
Tags - r
Read the full post at Download R 2.13.0 for Windows.

How to Vectorize Nested Loop in R?

Sat, 12 Mar 2011 14:24:53 +0000

Could any R expert here help me to vectorize my for loop? Thanks in advance for your favor. The reason I am in trouble is the variable inside my "for" function are updated after each loop, which makes me feel difficult to use lapply, sapply or whatever.

Simplifed codes are listed below:

for (i in 1:N) { #N could be a large number, AdjS and VarS are initially given and updated for each i
        PredS <- F %*% AdjS
        PredY <- H %*% PredS
        PredError <- (Y[i,] - t(PredY))
        VarY <- (H %*% VarS) %*% t(H)
        InvVarY <- solve(VarY)
        KG <- (VarS %*% t(H)) %*% InvVarY
        AdjS <- PredS + PredError
        VarS <- (diag(3) - KG %*% H) %*% VarS
        ll[i] <- PredError %*% InvVarY %*% t(PredError)
  }

The point is how to vectorize the for loop while allowing AdjS and VarS to be updated. I appreciate your help.
Tags - r , vectorize
Read the full post at How to Vectorize Nested Loop in R?.

Issues of R Client Library For The Google Prediction API

Thu, 16 Dec 2010 22:30:53 +0000

The Google Prediction API is a black-box system for building predictive models, it provides pattern-matching and machine learning capabilities. So Google algorithms automatically creates a model from the training models given a set of training data and makes prediction under this model given a set of explanatory variables, read http://code.google.com/apis/predict/docs/getting-started.html for an overview.

I am eager to make my hands dirty after the release of R client library for the Google Prediction API @ http://code.google.com/p/google-prediction-api-r-client/, however, as both my office's computer and my laptop are under Windows, there are few issues for the R API package:
1, the original usage example had mistake. I always had a problem when I wanted to train my local data, my.model <- PredictionApiTrain(data="MYPATH/MYFILE.csv"), it says
Error in PredictionApiTrain(data = "test.csv") : 'remote.file' should be character.
At the beginning of PredictionApiTrain() there are lines:

if (missing(remote.file) || !is.character(remote.file))
stop("'remote.file' should be character")

However, remote.file is an argument without default value, function (data, remote.file, verbose = FALSE). Now the example has been changed to my.model <- PredictionApiTrain (data="MYPATH/MYFILE.csv",remote.file="gs://MYBUCKET/MYOBJECT");

2, even when I use PredictionApiTrain
(data="MYPATH/MYFILE.csv",remote.file="gs://MYBUCKET/MYOBJECT"),
trained model my.model is expected, however, R returns
Error in system(command = command, intern = TRUE) : gsutil not found
I did install gsutil as described at http://code.google.com/apis/storage/docs/gsutil.html, and can list my bucket using: python gsutil ls gs://MYBUCKET/MYOBJECT.
The comment I got this afternoon is:

Quotation

We recommend to run this R client under Linux since this version doesn't support gsutil under Windows using Python.
CygWin is a more recommended environment for running gsutil under Windows. And we will add this feature in future release soon.

It seems we have to wait for a while before testing this promising application, luckily the ball is moving forward. Please share with us if you find a way out. I am looking forward to it and will update once I successfully try CygWin.
Tags - r , prediction
Read the full post at Issues of R Client Library For The Google Prediction API.

Life Is Short, Use Python

Wed, 24 Nov 2010 12:00:58 +0000

Life is short, use Python

I started to play with Python two weeks ago due to the limitation of R in terms of handling large data, then a friend of mine suggested me to try Python since I had to do data massage frequently, "Python is the best choice, trust me", he said. Although I was unwilling to learn another new software, I couldn't bear with the low efficiency of R (or of my work) for large data. You may realize my learning curve as: Excellent free CSV splitter --> MySQL+RMySQL package --> Several R packages including bigmemory and ff. But to be honest, none of them satisfies me either because of the limitation of the method (slow + malfunction) or of my own computer (short of memory).

I am shocked by python's extreme power and easy-to-use design after nearly two weeks, dealing with a 10GB CSV had never become so easy. More importantly, you can access R from Python almost seamlessly with the package RPY. To get started, I would like to recommend the following readings to all Python newbies like me:
1, commands dictionary Matlab vs R vs Python;
2, free ebook Dive Into Python;
3, a text book Machine Learning: An Algorithmic Perspective by Prof. Stephen Marsland.

The third book is especially useful for data analysis, as there are lots of Python code examples in the book, the code and dataset are available to download @ the author's website http://www-ist.massey.ac.nz/smarsland/MLBook.html, take a look before deciding to add it to your shelf.
Tags - python , r
Read the full post at Life Is Short, Use Python.

Handling Large Datasets in R

Tue, 26 Oct 2010 23:52:20 +0000

Handling large dataset in R, especially CSV data, was briefly discussed before at Excellent free CSV splitter and Handling Large CSV Files in R. My file at that time was around 2GB with 30 million number of rows and 8 columns. Recently I started to collect and analyze US corporate bonds tick data from year 2002 to 2010, and the CSV file I got is 6.18GB with 40 million number of rows, even after removing biases data as in Biases in TRACE Corporate Bond Data.

How to proceed efficiently? Below is an excellent presentation on handling large datasets in R by Ryan Rosario at http://www.bytemining.com/2010/08/taking-r-to-the-limit-part-ii-large-datasets-in-r/, a short summary of the presentation:
1, R has a few packages for big data support. The presentation covers the following: bigmemory and ff; and also some uses of parallelism to accomplish the same goal using Hadoop and MapReduce;
2, the data used in the presentation is 11GB comma-separated values with 120 million rows, 29 columns;
3, For datasets with size in the range 10GB, bigmemory and ff handle themselves well;
4, For larger datasets, use Hadoop;

Taking R to the Limit (High Performance Computing in R), Part 2 -- Large Datasets, LA R Users' Group 8/17/10

View more presentations from Ryan Rosario.

BTW, determining the number of rows of a very big file is tricky, you don't have to load the data first and use dim(), which easily leads to short of memory. One way of doing it is readLines(), for example:

data <- gzfile("yourdata.zip",open="r")
MaxRows <- 50000
TotalRows <- 0
while((LeftRow <- length(readLines(data,MaxRows))) > 0 )
TotalRows <- TotalRows+LeftRow
close(data)

Tags - data , csv
Read the full post at Handling Large Datasets in R.

R API to Interactive Brokers Trader Workstation

Mon, 25 Oct 2010 21:14:05 +0000

Interactive Brokers via Matlab was mentioned at the old post Matlab trading code, IBrokers: R API to Interactive Brokers Trader Workstation is the R package I realize for algo trading API. Should you are also interested, you can watch the following short video about algo trading in R. You may watch from 15:00 directly.

The IBrokers package can be found at http://cran.r-project.org/web/packages/IBrokers/index.html
Tags - trading
Read the full post at R API to Interactive Brokers Trader Workstation.

Liquidity Premium vs Liquidity of Corporate Bonds

Thu, 14 Oct 2010 16:18:20 +0000

All else equal, investors should require higher returns on assets whose liquidity is lower, in other words, investors demand a higher expected return, and hence larger liquidity premium, by holding a less liquidity asset. Risk & return co-exist.

Is this really true for corporate bonds? I run a simple regression using R to test my data, where US corporate bonds are downloaded from TRACE (Trade Reporting and Compliance Engine), CDS data from Datastream, Treasury / Swap interest rate from Federal Reserve Bank, the total number of bonds in my sample is 2409 from year 2004 ~ 2010. Liquidity of a corporate bond is measured as in the paper Corporate Yield Spreads and Bond Liquidity by Chen, Lesmond, and Wei (2007), Journal of Finance,

where alpha1 & alpha2 represent bid & ask spread, respectively, by using maximum likelihood estimation we could estimate the transaction cost alpha2-alpha1 for each bond, obviously the higher the transaction cost, the lower the liquidity.

Then I estimate the long-term liquidity premium by

here y, r, lambda and l are corporate bond yield, risk free rate, credit risk premium and liquidity premium. Finally I rank the corporate bonds by their liquidity premium, and scaled the ranking to be between 0 ~ 1, the higher the number, the lower the liquidity premium. What we would expect is a negative relationship between liquidity & liquidity premium, that is to say, investors would expect a lower liqudity premium by holding a larger liquidity bond, and vice versa.

Below is a simple univariate regression of liquidity on liquidity premium, where swap.liquidity is the liquidity premium estimated using swap rate as risk free. Not only the liquidity premium is significant enough, but also its coefficient sign is intuitive and same as expected

the regression result shows the liquidity premium is highly significant even after controlling for those typical bond characteristics such as coupon rate, issue amount, age, maturity and the rating, where rating is expressed as from 1 to 7, with 1 being the highest rank "AAA". The larger issue amount, the lower transaction cost (and the higher liquidity); also the higher rating, the lower transaction cost. Overall there is a 140 basis point different in transaction costs between the lowest and highest premium.

I think by now we can draw a conclusion based on this empirical analysis: investors do demand a higher return for less liquidity corporate bond in the market.
Tags - bond , liquidity
Read the full post at Liquidity Premium vs Liquidity of Corporate Bonds.

R Optimization Function Test

Fri, 01 Oct 2010 06:13:07 +0000

Using Kalman Filter for CIR interest rate model parameter estimation was introduced at my previously post Kalman Filter finance, soon after that I got a few comments saying the final results are unstable and highly depend on the initial values, that's true, local vs global minimum is never ending.

This post is therefore a sample test of the optimization functions in R as I started to move from Matlab to R recently, and R allows us to choose the method we'd like to use for minimization.
Purpose: to estimate the parameters for Vasicek interest rate model;
Function to be minimized: similar as in Kalman Filter finance, where a CIR model is used instead;
Number of Parameters: 8
Data: two years time series of 3 month, 6 month, 1 year and 5 year US interest rate;
R function to be tested: nlm, optim(Nelder-Mead), optim(BFGS), optim(SANN), nlminb, optim (L-BFGS-B)

The true values are listed in the paper "estimating and testing exponential-affine term structure models by kalman filter", abs.tol and rel.tol are set to be 1e-6 wherever possible, derivative is not given and unconstrained optimization is prefered

I intentionally set the starting values far away from the true ones in order to see which one is able to find closest answers, obviously nlminb outperforms all others and returns almost the true values, I change the starting values randomly and still, nlminb is the best and all R functions finish within seconds.

Check several alternative R optimization packages if you are not satisfied.

Tags - r , optimization , filter
Read the full post at R Optimization Function Test.

Missing Data in R

Sun, 12 Sep 2010 20:59:51 +0000

Probably all of us have met the issue of handling missing data, from the basic portfolio correlation matrix estimation, to advanced multiple factor analysis, how to impute missing data remains a hot topic. Missing data are unavoidable, and more encompassing than the ubiquitous association of the term, irgoring missing data will generally lead to biased estimates. The following ways are often applied to handle the problem:
1, simple deletion strategies: including pairwise deletion and listwise deletion, the former may lead to inconsistent results, for example, non positive definite correlation or covariance matrices, while the cumulation of deleted cases may be enormous except in the case of very few missing values for the latter method;
2, so called "Working around" strategies, for example, the Full Information Maximum Likelihood (FIML) integrates out the missing data when fitting the desired model;
3, imputation strategies, these are the most widely used methods both in academia and industry, replacing missing value with an estimate of the actual value of that case. For instance, ‘hot-deck’ imputation consists of replacing the missing value by the observed value from another, similar case from the same dataset for which that variable was not missing; mean imputation consists of replacing the missing value by the mean of the variable in question; expectation Maximization (EM) arrives at the best point estimates of the true values, given the model (which itself is estimated on the basis of the imputed missings); regression-mean imputation replaces the missing value by the conditional regression mean, and multiple imputation, rather than a single imputed value, multiple ones are derived from a prediction equation.

I came across an easy-to-use missing data imputation named Amelia II developed by professor Gary King from Harvard university, as its webpage introduces: Amelia II "multiply imputes" missing data in a single cross-section (such as a survey), from a time series (like variables collected for each year in a country), or from a time-series-cross-sectional data set (such as collected by years for each of several countries). Amelia II implements bootstrapping-based algorithm that gives essentially the same answers as the standard IP or EMis approaches, is usually considerably faster than existing approaches and can handle many more variables....Unlike...other statistically rigorous imputation software, it virtually never crashes.

Amelia II was developed based on R language, so users have to install R before running it, installation of Amelia is staightforward: download and run the exe file, that's it. For me, the beauty of Amelia II is its friendly interface, I don't even need to run R software myself. Double clicking Amelia II shows the following

as you can see from the input and output menus, it supports csv files, simply importing a csv file with missing data returns a csv with imputed data, amazing, isn't it?

Downloading the software and help documents at http://gking.harvard.edu/amelia/.
Tags - r
Read the full post at Missing Data in R.

R Reshape Package

Mon, 06 Sep 2010 08:08:10 +0000

Some of you may know this R reshape package already, I have started to play with it after the post Handling Large CSV Files in R. It is really an excellent one worthing a new post to introduce formally.

What is reshape package? reshape: Flexibly reshape data, Reshape lets you flexibly restructure and aggregate data using just two functions: melt and cast. Therefore basically it allows us to massage, re-organize our data as the hierarchy we need with only two steps: first melt the data into a form suitable for easy casting, then cast a molten data frame into the reshaped or aggregated form you want. Sounds tongue twisters? A small example will help you feel clearer.

Suppose you have a matrix of bond data

you are interested in the total amount of bonds of each rating, of each industry, or of each time to maturity, how to proceed? you may be thinking of lapply, sapply or even for loop, that's OK but at the cost of efficiency (coding time & running time) and possible error (personally I often have to modify twice for my sapply code to work, sad...).

It becomes much easier with the R Reshape package,
first, melt the data, newdata <- melt(data, id=c("RATING", "TIME_TO_MATURITY", "INDUSTRY_CODE", "BOND_TYPE"));
second, cast the data based on your needs, for instance, to get the total amount of each industry, cast(newdata, INDUSTRY_CODE ~ variable, sum) returns you a data.frame like

That's it, easy to use, efficient, right? Download the R Reshape package at http://cran.r-project.org/web/packages/reshape/index.html
Tags - r , package
Read the full post at R Reshape Package.

Handling Large CSV Files in R

Mon, 09 Aug 2010 12:40:10 +0000

A follow-up of my previous post Excellent Free CSV Splitter. I asked a question at LinkedIn about how to handle large CSV files in R / Matlab. Specifically,

Quotation

suppose I have a large CSV file with over 30 million number of rows, both Matlab / R lacks memory when importing the data. Could you share your way to handle this issue? what I am thinking is:
a) split the file into several pieces (free, straightforward but hard to maintain);
b) use MS SQL/MySQL (have to learn it, MS SQL isn't free, not straightforward).

A useful summary of suggested solution:
1, 1) import the large file via "scan" in R;
2) convert to a data.frame --> to keep data formats
3) use cast --> to group data in the most "square" format as possible, this step involves the Reshape package, a very good one.

2, use the bigmemory package to load the data, so in my case, using read.big.matrix() instead of read.table(). There are several other interesting functions in this package, such as mwhich() replacing which() for memory consideration, foreach() instead of for(), etc. How large can this package handle? I don't know, the authors successfully load a CSV with size as large as 11GB.

3, switch to a 64 bit version of R with enough memory and preferably on linux. I can't test this solution at my office due to administration constraint, although it is doable, as mentioned in R help document,

Quotation

64-bit versions of Windows run 32-bit executables under the WOW (Windows on Windows) subsystem: they run in almost exactly the same way as on a 32-bit version of Windows, except that the address limit for the R process is 4GB (rather than 2GB or perhaps 3GB)....The disadvantages are that all the pointers are 8 rather than 4 bytes and so small objects are larger and more data has to be moved around, and that far less external software is available for 64-bit versions of the OS.

Search & trial.

Tags - r , csv
Read the full post at Handling Large CSV Files in R.

Excellent R Code Format Package

Sun, 04 Jul 2010 16:38:19 +0000

I have been looking for this type of package for several days, and luckily found it today. Unquestionable R is powerful, however, R programming is unfriendly as far as I concern, mainly due to the lack of format shortcut, which makes the R codes rather ugly. (It is an absolute advantage of Matlab, for example, ctrl+R for comment, ctrl+T for uncomment, ctrl+I for smart indent, etc.)

FormatR is the package for tidying R source code, although it is less convenient to use than the straightforward shortcuts in Matlab, this package is good enough for me, what is it for? as the title suggest:

Quotation

formatR: format R code automatically, farewell to ugly R code

Below is a comparison before and after using FormatR.
Before:

After:

Download the package at http://cran.r-project.org/web/packages/formatR/index.html
Tags - r
Read the full post at Excellent R Code Format Package.

Simple Dummy R GUI Generator

Thu, 01 Jul 2010 11:28:30 +0000

Imagine you finish a dirty coding project and want to present to your boss who is not in a good mood (may not be occasionally), how are you going to start? Show him your hundreads of lines code, point to the lines, explain what the arguments and outputs are? No, it is not a smart way since you are supposed to introduce in a few short sentences. Generating a GUI is probably the quickest / easiest way of understanding what this code does. As the old saying goes: a picture is worth a thousand words, so in a same logic, a GUI is worth n thousand words.

However, generating GUI is by no means easy as I know the pain when creating the Matlab-GUI equity derivative calculator. It becomes even worse in R language, to be honest, I hate the graph plotting in R, terribly unflexible compared with in Matlab. Luckily I came across a good R GUI package named "fgui", it does as its description: Rapidly create a GUI interface for a function you created by automatically creating widgets for arguments of the function.

Very nice indeed, after playing for half an hour, it is simple to use, especially when what you need is just a basic GUI demonstrating to others a rough idea. One line code is enough.

For a simple example, I create a European option pricer with Black Scholes formula,

EuropeanOption <- function (s, k, r, t, vol, CallOption) {
  d1 <- (log(s/k)+(r+0.5*vol^2)*t)/(vol*sqrt(t))
  d2 <- d1-vol*sqrt(t)
  if (CallOption){
    return (s*pnorm(d1)-k*exp(-r*t)*pnorm(d2))
  } else {
    return (k*exp(-r*t)*pnorm(-d2)-s*pnorm(-d1))
  }
}

then generating a GUI for this function is as simple as adding the following code

res <- gui(EuropeanOption, argOption=list(CallOption=c("TRUE","FALSE")))

It returns a GUI looks like

where you are able to set inputs and get outputs. Nice. More advanced GUI is possible by adding more lines.

Interested readers shall download the package "fgui" at http://cran.r-project.org/web/packages/fgui/index.html
Tags - r , gui
Read the full post at Simple Dummy R GUI Generator.

R Sapply Problem

Wed, 16 Jun 2010 19:07:38 +0000

Any expert in R please educates me. I have got a problem about the sapply (or lapply), it made me headache for over two hours.

As "for loop" is very slow in R, we should try best to avoid using it, and to use vectorization instead. sapply is designed for this, for example, instead of:

for (i in 1:10) {
z[i] <- mean(x[1:i])
}

we could use

z <- sapply(1:10, function(i, x) {mean(x[1:i])}, x)

It went well, but what if besides computing z, I need to update another variable, for example, with loop, it is

temp <- 3
for (i in 1:10) {
x[i] <- x[i]-temp
z[i] <- mean(x[1:i])
temp <- x[i]-z[i]
}

in this case, temp is changing every step (it doesn't have to be a function of z[i]). How to vectorize that and use sapply then? since sapply can't return two variables z and temp.

~~I tried to define a matrix and store z in the first column and temp in the second column and return the matrix, however, failed.~~

Many thanks.

PS, NO, still not correct, working on it...
ah, I worked it out, it can be done by passing z itself to sapply, that's good.
the following is a sapply example returning the same result for "for loop" and "sapply".
sapply.example <- function(nsim = 10){
  x <- rnorm(nsim)
  y <- list()
  z.for <- array(0, nsim)
  temp <- 3
  for (i in 1:nsim) {
    x[i] <- x[i]-temp
    z.for[i] <- mean(x[1:i])
    temp <- x[i]-z.for[i]
  }
  y$z.for <- z.for

  z.sapply <- array(0,2*nsim)
  z.sapply[1] <- 3
  z.sapply <- sapply(seq(1,2*nsim,by=2), function(i,x,z.sapply) {
        temp <- z.sapply[i]
        z.temp <- mean(x[1:((i+1)/2)])
        temp <- x[((i+1)/2)]-z.temp
        z.sapply[i] <- temp
        z.sapply[i+1] <- z.temp
        z.sapply[i:(i+1)]
      }, x, z.sapply, simplify =TRUE)
  y$z.sapply <- z.sapply[seq(2,2*nsim, by=2)]
  y
}

Tags - r
Read the full post at R Sapply Problem.

R for Matlab Users

Tue, 15 Jun 2010 14:37:52 +0000

My favorite software is Matlab, but partly because R is free, more and more people & companies choose to use R as a major working language. Nothing wrong with that, I am at the moment changing some of my Kalman Filter Matlab codes to R.

One bothering issue is each software has its own coding rules, for example, in Matlab we use a(1,1) but in R we use a[1,1]; in Matlab we have ones(3,2) but in R we dont have such a command but matrix(1,3,2), etc. (I am always wondering why they can't be designed in a similar way). It does bring me trouble sometimes, luckily I came across a web page similar as cheat-sheets, it lists those widely used commands in R and corresponding commands in Matlab, very convenient indeed.

Search before trial & error: http://mathesaurus.sourceforge.net/octave-r.html

PS:
Walking Randomly suggests another excellent PDF manual consisting of 47 pages, fantastic! http://www.math.umaine.edu/~hiebeler/comp/matlabR.pdf
Tags - r , matlab
Read the full post at R for Matlab Users.

An Interest Rate R Package Plan

Mon, 07 Jun 2010 09:26:05 +0000

I recently have done some empirical studies on zero-coupon bond modelling and pricing, and plan to create an interest rate R package in order to make it re-usable, as I find there are only two R package on it, one is http://cran.r-project.org/web/packages/termstrc/index.html, the other one is http://cran.r-project.org/web/packages/YieldCurve/index.html, both of them don't cover short interest rate model, unfortunately.

Below is a short plan, help me add it by leaving a comment, thanks.

Tags - r , package , rate
Read the full post at An Interest Rate R Package Plan.

How to Create an R Package in Windows

Fri, 04 Jun 2010 13:53:13 +0000

There are many reasons to create an R package, such as codes protection, convenient usage, etc. However, creating an R package in Unix is not hard, it IS in Windows, as R is designed in a Unix environment which includes a set of compilers, programming utilities, and text-formatting routines while Windows lacks those. I used to build an R library in Unix and thought it was relatively as the same convenient as in Windows, but it turned out I had to spend several hours on it. Silly me.

Below are the selected steps to create an R package in Windows I summarize:
1, you have to download Rtools at http://www.murdoch-sutherland.com/Rtools/.

2, depending on your needs, you may download and install LaTex, Microsoft HTML Help Workshop and the Inno Setup installer, available at http://www.miktex.org, http://msdn.microsoft.com/en-us/library/ms669985.aspx, and http://www.jrsoftware.org/, respectively. For example, Microsoft HTML Help Workshop is for making HTML Help documents, LaTex is for a nicer outlook, especially when your documents have math equations.

3, double click Rtools.exe to install it and its accompanying tools: minGW, perl. Rtools automatically recognizes the paths of those relevant softwares and add them to the environment variables of your computer.

4, check and change your PATH “environment variables” of your computer if incorrect. To set the path, right click on the “My Computer” icon on your desktop, choose properties and click on the “advanced” tab, click the environmental variables button, click on the path variable and select the edit button. Check carefully whether all the paths are correct, special attention needs to be given to the path of R software.

5, double check whether all tools are installed correctly by opening a “command prompt” window and typing the following commands: R; gcc –help; perl –help; Tex –help, respectively. You should see a list of options, otherwise re-install or check the path if you see “… is not a recognized as an internal or external command …”

6, start R and run the command package.skeleton( ) with suitable arguments, read the help http://stat.ethz.ch/R-manual/R-patched/library/utils/html/package.skeleton.html for detail. For instance, I write a sample code MonteCarloPi.R to estimate PI by Monte Carlo simulation, we use package.skeleton(name = "MonteCarloPi", code_files = “MonteCarloPi.R”), where MonteCarloPi.R includes function we want to add to the package MonteCarloPi. This function creates the directory tree for the package containing:
DESCRIPTION <<man <<< for documentation
R <<< for R function definitions
src <<< for low-level source code (this is optional for other programming files, such as your c++ codes)
data <<< for package datasets (this is optional)

7, remove the Read-and-delete-me file and modify other files in the above directory with notepad or other tool, for instance, to add the author information and Help explanation.

8, open a “command prompt” window, change the directory to where your package is, type the command “R CMD build MonteCarloPi” to build the package, this will generate a file called MonteCarloPi_1.0.tar.gz. You must run the command in a “command prompt” window instead of in R, otherwise you are expected to see “Error: unexpected symbol in "R CMD"”.

9, test the package by typing “R CMD check MonteCarloPi_1.0.tar.gz”, go back to step 8 if you see errors.

10, now you are able to install the package typing “R CMD INSTALL MonteCarloPi_1.0.tar.gz”. Congratulations if you see “…done…” and you will notice there is one more sub-folder “MonteCarloPi” in your R library folder.

#MonteCarloPi.R -- PI estimation by Monte Carlo simulation

MonteCarloPi <- function(nsim){
  x <- 2*runif(nsim)-1
  y <- 2*runif(nsim)-1
  inx <- which((x^2+y^2)<=1)
  return ((length(inx)/nsim)*4)
}

Quotation

Reference:
http://cran.r-project.org/doc/manuals/R-exts.html

Tags - r , package
Read the full post at How to Create an R Package in Windows.

Missing data imputation

Sat, 12 Dec 2009 20:53:28 +0000

RQuantlib

Mon, 23 Nov 2009 18:58:05 +0000

Quantlib is a free library for modeling, trading, and risk management in real-life providing a comprehensive software framework for quantitative finance, it is written in C++, which might be inconvenient for some users. JQuantLib aiming at Java-fans is naturally developed, correspondently, RQuantlib connects GNU R software with QuantLib.

The installation is straintforward, I tried it on my Windows, the source code is at http://cran.r-project.org/web/packages/RQuantLib/index.html, which is self-contained and does not even require a QuantLib (or Boost) installation. Nothing more to say, following the process, users are able to use the library immediately.

So far the function and option types supported by RQuantlib are limited, vanilla and a few popular exotic options, for example, American option, Asian option, Barrier, Bermudan, Binary option, as well as a range of fixed-income functions, mainly on Convertible bond valuation. Hopefully it will grow quickly.

Detailed reference manual is also available at http://cran.r-project.org/web/packages/RQuantLib/index.html.
Tags - quantlib
Read the full post at RQuantlib.

Test cointegration with R

Sat, 22 Aug 2009 04:04:50 +0000

Cointegrated pairs of securities are crucial for mean reversion trading portfolio construction, Play with cointegration has several good papers to start with. Should you want to test pairs of securities for cointegration using R, here is an excellent webpage with data, code and detailed example, cheers.

http://quanttrader.info/public/testForCoint.html
Tags - cointegration
Read the full post at Test cointegration with R.

Nonparametric High-Dimensional Time Series Analysis

Wed, 13 May 2009 21:43:59 +0000

Functional Gradient Descent (FGD) is a method of nonparametric time series analysis, useful in particular for estimating conditional mean, variances and covariances for very high-dimensional time series. FGD is a kind of hybrid of nonparametric statistical function estimation and numerical optimization. In fact, the idea of FGD comes from the fact that boosting can be viewed as an optimization algorithm in function space. This method employs an iterative refitting of generalized residuals, based on a given statistical procedure called base learner, to approximate the first two conditional moment functions of a multivariate process. An appealing feature of this expansion is that it is a nonlinear nonparametric model that directly nests the Gaussian diagonal VAR model, the Gaussian GARCH model and the multivariate CCC-GARCH as simple, starting special cases. The FGD model is fitted using conventional maximum likelihood together with a cross-validation strategy that determines the appropriate number of additive terms in the final expansions.

Interested ppl shall download the Splus codes and data at http://www.raffonline.altervista.org/fgd/
PS: to be honest, this is not the area i am family with at all, download at your own risk
Tags - nonparametric
Read the full post at Nonparametric High-Dimensional Time Series Analysis .

Maximum likelihood estimation in R

Sun, 18 Jan 2009 17:58:12 +0000

Maximum likelihood estimation can be implemented like Quasi-maximum likelihood in Matlab, You can also write an R function which computes out the likelihood function. As always in R, this can be done in several different ways.

One issue is that of restrictions upon parameters. When the search algorithm is running, it may stumble upon nonsensical values - such as a sigma below 0 - and you do need to think about this. One traditional way to deal with this is to "transform the parameter space". As an example, for all positive values of sigma, log(sigma) ranges from -infinity to +infinity. So it's safe to do an unconstrained search using log(sigma) as the free parameter.

For detail about methodology and sample codes see http://www.mayin.org/ajayshah/KB/R/documents/mle/mle.html.
Tags - mle
Read the full post at Maximum likelihood estimation in R.

Extra moments measure

Tue, 16 Dec 2008 21:57:00 +0000

The following functions are intended to replicate calculations for taking higher moments of hedge fund returns into account in analyzing particular investments. Most of the formulae are taken from various EDHEC research papers.

# All returns are assumed to be on a monthly scale!

functions including:

# moment.third
# moment.fourth
# CoSkewness
# CoKurtosis
# BetaCoVariance
# BetaCoV (wrapper for BetaCoVariance)
# SystematicBeta (wrapper for BetaCoVariance)
# BetaCoSkewness
# BetaCoS (wrapper for BetaCoSkewness)
# SystematicSkewness (wrapper for BetaCoSkewness)
# BetaCoKurtosis
# BetaCoK (wrapper for BetaCoKurtosis)
# SystematicKurtosis (wrapper for BetaCoKurtosis)
# VaR
# VaR.Beyond
# VaR.column
# VaR.CornishFisher
# VaR.Marginal
# modifiedVaR (wrapper for VaR.CornishFisher)

http://braverock.com/brian/R/extra_moments.R
Tags - moment , portfolio
Read the full post at Extra moments measure.

Functions for portfolio analysis

Thu, 11 Dec 2008 15:07:17 +0000

Functions include:
1. efficient.portfolio      compute minimum variance portfolio subject to target return
2. globalMin.portfolio      compute global minimum variance portfolio
3. tangency.portfolio      compute tangency portfolio
4. efficient.frontier      computer Markowitz bullet

http://faculty.washington.edu/ezivot/econ483/portfolio.ssc

Tags - markowitz , splus
Read the full post at Functions for portfolio analysis .

Convert Splus to R

Wed, 10 Dec 2008 18:57:26 +0000

Suppose you have got used to Splus and want to switch to R software (why bother to change? R is free while Splus is not, fair enough?), what can you do? since there are many functions in S-PLUS that are missing in R, one way is to understand the functions and write your owns, working N hours without sleep (N>?). however, you can avoid doing like that if you are as headche as me whenever you think of this solution. There is a package named Splus2R, which is to facilitate the conversion of S-PLUS packages to R packages, this package provides some missing S-PLUS functionality in R.

I have not tested the package, though, will update later. Here is downloading link: http://cran.r-project.org/web/packages/splus2R/index.html.
Tags - splus , r
Read the full post at Convert Splus to R.

Multivariate dependence with copulas

Mon, 17 Nov 2008 20:03:56 +0000

Classes (S4) of commonly used copulas including elliptical (normal and t), Archimedean (Clayton, Gumbel, Frank, and Ali-Mikhail-Haq), extreme value (Husler-Reiss and Galambos), and other families (Plackett and Farlie-Gumbel-Morgenstern). Methods for density, distribution, random number generation, bivariate dependence measures, perspective and contour plots. Functions for fitting copula models. Independence tests among random variables and random vectors. Serial independence tests for univariate and multivariate continuous time series. Goodness-of-fit tests for copulas based on multipliers and on the parametric bootstrap.

R package can be downloaded at http://cran.r-project.org/web/packages/copula/index.html
Tags - copula
Read the full post at Multivariate dependence with copulas.

Modeling Financial Time Series with S-PLUS

Wed, 12 Nov 2008 21:36:34 +0000

Although S-plus is the most terrible language I have ever used in terms of debugging (I have to say that, no offense to S-plus fans, as my colleagues said, it is hard to understand it is still existed in 21 centuary), I found the S-plus scripts accompanying the book Modeling Financial Time Series with S-PLUS, covering:

Time Series Manipulation, Time Series Concepts, Unit Root Tests, Modeling Extreme Values, Time Series Regression, Univariate GARCH, Long Memory, Rolling Analysis, Systems of Regression Eqations, VAR Models, Cointegration, Factor Models, Term Structure, Copulas, Generalized Method of Moments, etc.

For detail please download at http://faculty.washington.edu/ezivot/MFTS2ndEditionScripts.htm
Tags - s-plus
Read the full post at Modeling Financial Time Series with S-PLUS.

Quantitative Risk Management R package

Wed, 05 Nov 2008 20:43:48 +0000

I shared an Econometric tools for performance and risk analysis package in R, today I introduce another Quantitative Risk Management R package, which is accompanying the book Quantitative Risk Management: Concepts, Techniques and Tools by Alexander J. McNeil, Rudiger Frey and Paul Embrechts, a nice book written by one of my professors. In this book special care is given to Copula analysis, Extreme value thoey, credit risk analysis, etc. Given the fact it was ranked by one of the top 10 most technical books of the year 2007, i bet you will learn a lot from it.

R-language version can be downloaded at http://cran.r-project.org/web/packages/QRMlib/index.html and S-PLUS library to accompany book is at http://www.ma.hw.ac.uk/~mcneil/book/index.html.
Tags - risk
Read the full post at Quantitative Risk Management R package.

Rmetrics - Basics of Option Valuation

Wed, 22 Oct 2008 19:14:49 +0000

Open Source Software for Financial Engineering and Computational Finance

Rmetrics is the premier open source solution for teaching financial market analysis and valuation of financial instruments. With hundreds of functions build on modern methods Rmetrics combines explorative data analysis, statistical modeling and rapid model prototyping. The Rmetrics Packages are embedded in R building an environment which creates for students a first class system for applications in statistics and finance.

Download at http://cran.cnr.berkeley.edu/web/packages/fOptions/index.html
Tags - r , option
Read the full post at Rmetrics - Basics of Option Valuation.

Econometric tools for performance and risk analysis

Mon, 13 Oct 2008 19:31:47 +0000

Quotation

Library of econometric functions for performance and risk analysis of financial portfolios. This library aims to aid practitioners and researchers in using the latest research in analysis of both normal and non-normal return streams.

We created this library to include functionality that has been appearing in the academic literature on performance analysis and risk over the past several years, but had no functional equivalent in R. In doing so, we also found it valuable to have wrapper functions for functionality easily replicated in R, so that we could access that functionality using a function with defaults and naming consistent with common usage in the finance literature. The following sections cover Performance Analysis, Risk Analysis (with a separate treatment of VaR), Summary Tables of related statistics, Charts and Graphs, a variety of Wrappers and Utility functions, and some thoughts on work yet to be done.

http://braverock.com/brian/R/PerformanceAnalytics/html/PerformanceAnalytics-package.html
Tags - econometrics , performance , r
Read the full post at Econometric tools for performance and risk analysis.

evolutionary algorithm optimization

Thu, 04 Sep 2008 15:27:42 +0000

In the post Optimization packages dozens of optimization routines can be downloaded, here I am going to share a special optimization method: evolutionary algorithm.

Evolutionary algorithms (EAs) are search methods that take their inspiration from natural selection and survival of the fittest in the biological world. EAs differ from more traditional optimization techniques in that they involve a search from a "population" of solutions, not from a single point. Each iteration of an EA involves a competitive selection that weeds out poor solutions. The solutions with high "fitness" are "recombined" with other solutions by swaping parts of a solution with another. Solutions are also "mutated" by making a small change to a single element of the solution. Recombination and mutation are used to generate new solutions that are biased towards regions of the space for which good solutions have already been seen.

This R package provides the DEoptim function which performs Differential Evolution Optimization (evolutionary algorithm), for detail check http://cran.r-project.org/web/packages/DEoptim/index.html.
wiki(Evolutionary algorithm)
Tags - optimization
Read the full post at evolutionary algorithm optimization .

Visualize Copulas

Wed, 20 Aug 2008 14:09:55 +0000

In those Copula codes you can get a rough idea what copula is, how to estimate and simulate it, how to test its performance, etc., to help you visualize what on earth the copula should look like, below R code draws plots of some widely used copulas.

http://www.math.tu-berlin.de/~mkeller/index.php?target=rcode
Tags - copula
Read the full post at Visualize Copulas.

Process Simulation in R

Tue, 12 Aug 2008 07:50:09 +0000

Simple demonstration codes for process simulation in R, including Brownian motion simulation, Poisson process simulatio, Euler scheme simulation for Geometric Brownian motion, the mean-reverting process, and the process with two 'attractors', etc.

http://www.math.ku.dk/~rolf/teaching/mfe04/MiscInfo.html#Code
Tags - simulation
Read the full post at Process Simulation in R.

R-code for Vasicek estimation

Fri, 08 Aug 2008 06:51:47 +0000

A short-rate model is usually calibrated to some initial structures in the market, typically the initial yield curve, the caps volatility surface, the swaptions volatility surface, and possibly other products, thus determining the model parameters. Vasicek, Cox Ingersoll Ross (CIR), Dothan, for instance, are among the frequently-used short-rate models. The strength of Vasicek model is analytical bond prices and analytical option prices can be obtained and easily calculatied, however, negative short rates are also possible with positive probability.

R code can be downloaded at http://www.math.ku.dk/~rolf/teaching/mfe04/MiscInfo.html#Code

wiki(Vasicek model)
Tags - vasicek , cox ingersoll ross
Read the full post at R-code for Vasicek estimation.

download option price data from Yahoo

Tue, 29 Jul 2008 07:57:15 +0000

This R program can be used to download option price data from Yahoo to a data frame and to plot the corresponding implied-volatility smiles.

http://www.math.tu-berlin.de/~mkeller/index.php?target=rcode
Tags - download , data , option
Read the full post at download option price data from Yahoo.

Quantile Regression

Tue, 29 Jul 2008 07:48:16 +0000

Quantile regression is a statistical technique intended to estimate, and conduct inference about, conditional quantile functions. Just as classical linear regression methods based on minimizing sums of squared residuals enable one to estimate models for conditional mean functions, quantile regression methods offer a mechanism for estimating models for the conditional median function, and the full range of other conditional quantile functions. By supplementing the estimation of conditional mean functions with techniques for estimating an entire family of conditional quantile functions, quantile regression is capable of providing a more complete statistical analysis of the stochastic relationships among random variables.

http://www.econ.uiuc.edu/~roger/research/rq/rq.html
wiki(Quantile regression)
Tags - regression
Read the full post at Quantile Regression.