RCpp

[IlyaKEightSix]

So I was wondering if anyone had any experience with the RCpp package in R, created by Dirk Eddelbuettel. From what I hear, it greatly speeds up R if you know how to implement it at its bottlenecks (i.e. loop iteration in state-dependent instances).

I don't suppose this is a complete substitute for having several years of C++ development experience, but when it comes to doing trading research and such, would knowledge of this package (which essentially allows select porting of C++ functionality into R) allow one to be productive at a rate comparable to C++ guys?

 

[alain]

So I was wondering if anyone had any experience with the RCpp package in R, created by Dirk Eddelbuettel. From what I hear, it greatly speeds up R if you know how to implement it at its bottlenecks (i.e. loop iteration in state-dependent instances).

I don't suppose this is a complete substitute for having several years of C++ development experience, but when it comes to doing trading research and such, would knowledge of this package (which essentially allows select porting of C++ functionality into R) allow one to be productive at a rate comparable to C++ guys?

Here are my 2 cents. Programming in R will make you an order of magnitude more productive than using C++ for the tasks that you are usually going to face. RCpp allows you to speed up your R code but that is not going to make you more productive (unless I don't understand what you mean by "being productive").

 

[IlyaKEightSix]

Erm...I suppose what I meant is that so many job openings have a want for C++ (for whatever godforsaken reason), and I wonder if RCpp would allow me to do the same thing with R that all those C++ openings want done, at the speed they want it done at.

In short, I like programming in R. I'd like to have an easier time finding jobs. Is RCpp a shortcut to transferring R skills to firms that want C++, or is it no substitute?

 

[Polter]

Productivity at a task depends on what the task is (this follows from a certain technical result known as the duh theorem):

- some things requiring / naturally encodable as for loops (say, writing an iterative/FDM PDE solver -- or pretty much any iterative method) are as easy to get up and running in C++ as in most other languages (OTOH, I find vectorizing these and employing idiomatic style with apply & friends in R to be a PITA making it a definitively lower-productivity choice for these tasks -- subjectively / to me, that is*) -- note also that often parallelization of the loops popular in number crunching (over regular, static grids) is (relatively speaking) a no-brainer using something like OpenMP -- a no-brainer as in one line of code (certain conditions apply, false sharing, yada yada -- sure, and none of that gets any easier in general anyway)

Similarly, some tasks requiring connecting multiple solutions together, say, interactive-UI-over-(pre/post)processed-data-given-network-connectivity tasks (say, an Excel sheet with Greeks surface plots for products (re)priced on data coming from the Bloomberg API -- with an interface smooth enough to be useful for a trader) are doable (as in: regularly done) in C++, while I wouldn't attempt to employ R for these.

* - well, to be frank, everyone speaks subjectively, some just don't admit it and pretend to talk about some imaginary "average developer"; "average developer" doesn't exist (paraphrasing Lady Thatcher: "there is no such thing as an average developer"), so I wouldn't put too much stock into opinions of this sort, signal-to-noise ratio threshold being what it is and all that.

- some things requiring an activity spanning a range of wide tasks for which packages already exist in a given language are going to be accomplished fast given that language -- it's worth noting R has an enormous number of packages (check out CRAN Task Views) -- if you do have the right package and its features cover your needs, (re)writing it would be a waste of time -- hence, this is a definite higher-productivity choice;

In general: right tool for the right job.
(Corollary: if you only need one tool, you might have a bit of a monotonous job.)

As for Rcpp -- it's definitively easier to pick up than any of the "old-school" approaches, like:
- http://stat.ethz.ch/R-manual/R-devel/library/base/html/CallExternal.html
- http://stat.ethz.ch/R-manual/R-devel/library/base/html/Foreign.html

So, given a choice, don't waste your time on these and only focus on Rcpp if you need R <-> C++ interoperability.

On the other hand, you still need some knowledge of C++ -- if anything, you're even going to have to visit some "dark corners" that you'd otherwise be able to avoid as a "pure C++" guy, like foreign function interface(s) interplaying with (not always compatible) data (you could spend your whole life using R without learning what S-expressions are -- not so with Rcpp) interchange (pass-by-value vs. pass-by-reference will cost you -- you can't just count on inlining, (N)RVO, or move like in C++11) and subtleties involved (an example from another combination, MATLAB<->C++, i.e., MEX: due to MATLAB not being thread-safe, making the multithreading-and-C++-and-MATLAB combination work is hell -- and it's MATLAB that's the weakest link here, the multithreading-and-C++ part is fun and joy in comparison). Given that the very reason you even want to bring Rcpp into the equation is performance, these things will matter. This is all doable (and much easier than the old interop. ways), just something to keep in mind.

Another subtlety is the compiler, or rather compilers -- if you happen to be on Windows, you could spend your whole life using Microsoft Visual C++ and not even being aware of anything else even existing (hopefully you'd only write code for yourself and keep it on a private computer forever disconnected from the Internet, but I digress).
However, Rcpp developers are on the other side of the equation (this time with the spending whole life with GCC and not caring about MSVC even existing part being true) -- http://stackoverflow.com/questions/8605170/calling-r-functions-from-vc -- so, GCC it is. As you can learn from the FAQ you gotta use Rtools, which bundles pre-4.6.3 GCC. Given that GCC 4.6.3 was released on March 1, 2012 (that would be over a year ago), I wouldn't count on much in terms of C++11 support. Although it's partially there -- http://gcc.gnu.org/projects/cxx0x.html -- so, there's that.

That being said, Rcpp at least takes some of the pain away -- in my opinion Rcpp is much higher level (read: better) than what MEX has to offer for MATLAB (MEX is more comparable to a traditional .Call, messing around with pointers, etc.).

--

TL;DR: learn both (and Rcpp, too)

 

[Jose T]

That was a great post! Thanks

 

[Yike Lu]

There is a lot of C++ because a lot of trading systems and APIs are implemented in C++, and the majority of such jobs are along those lines.

If you need intraday capability, the chance of building your system in pure R (or even Rcpp) is negligible. Reliability, error recovery, threading, GUI interaction, etc aren't what they need to be.

I like the language and like using it, but I would not use it for live intraday trading at scale. Rcpp doesn't help because it's not just the speed, but the whole environment and its assumptions.

One could use it for a non-critical, non-realtime analytics server at the seconds to minutes scale. But you'd have to think really hard about error handling.

 

[Lyosha]

I know people that used RCpp extensively (true R patriots, that looked away from any shortcomings for "the cause"). Like with many half-baked open source ideas, they spent much of their days going around the bugs in it that ultimately noone was going to fix (and they had no control over) and having dreams about how they are saving time over using C++.

Due to the extreme instability of generated code you would be absolutely insane to use this in a production environment.

I would absolutely never even run a backtest using RCpp.

IMO stop trying to find shortcuts and take control over your systems.

 

[Yike Lu]

IMO stop trying to find shortcuts and take control over your systems.

The good shortcuts tend to be expensive.

 

[Lyosha]

The good shortcuts tend to be expensive.

"the good" anything tends to be expensive lol