Matlab

[John]

By the they, are there any thoughts about having a course in SAS/S-Plus? Half semester in SAS, half in S-Plus.
Part of the course could be devoted to learning SAS and databases, which seems to be a very important.

I think this would make a great elective.

Regarding these two, SAS is tailored-made for handling large set of data. It is used, e.g., to perform mortgage/MBS prepayment forecast. S-Plus, on the other hand, is more like Matlab, in that it allows one to do quick-and dirty modeling prototyping and development, in addition to statistical analysis.

 

[Andy Nguyen]

Someone mentioned to me that most hedge funds use Matlab. I'm not sure if it's true or any article regarding this. I've seen a few examples agreeing to this but it's certainly not indicative of the whole industry.

 

[John]

Someone mentioned to me that most hedge funds use Matlab. I'm not sure if it's true or any article regarding this. I've seen a few examples agreeing to this but it's certainly not indicative of the whole industry.

S-Plus is also very popular among the HFs, as matter of fact its finmetrics is, sh...., perfect for stat arb trading...

 

[Andy Nguyen]

S-Plus is also very popular among the HFs, as matter of fact its finmetrics is, sh...., perfect for stat arb trading...

I'll ask the stat arb guys to see what they use.

John,
How long did you wait to download the SPLUS student version ? i don't think I ever got the email from them to download.

 

[John]

I'll ask the stat arb guys to see what they use.

John,
How long did you wait to download the SPLUS student version ? i don't think I ever got the email from them to download.

I was given the download & registration information right after I have faxed my student credential. It was a quick turnaround as far as I remember.

However, it does not come with the optional finmetrics module, which was a bit of dissappointment. Perhaps you can ask your company to purchase a license, it is definitely worth it. (I have purchased the old student version of S+ 6.1 (no license expiration) and and s+finmetrics 1.0 from a while ago and it works like a charm)

 

[Andy Nguyen]

I looked at the SPLUS just now and they have several modules and I take it only the FinMetrics module is of any use to us ?

I can find out about the software part. It's never been a problem for me
Can you give us some examples of where you used SPlUS ? is your data on a remote server or local file ? I see they have SPLUS server as well but I don't think i will ever get to that, mostly will have to use local computer to do number crunching.

 

[John]

I looked at the SPLUS just now and they have several modules and I take it only the FinMetrics module is of any use to us ?

I can find out about the software part. It's never been a problem for me
Can you give us some examples of where you used SPlUS ? is your data on a remote server or local file ? I see they have SPLUS server as well but I don't think i will ever get to that, mostly will have to use local computer to do number crunching.

data could be residing anywhere, S-Plus is very flexible in this regarding. The following is from S-Plus 6.1 online help file:

Import File Type

Select the format of the file you wish to import:

• <LI class=mvd-P-Bullet>ASCII files (space-, whitespace-, tab-, comma-, and user-defined-delimited)
  <LI class=mvd-P-Bullet>dBase files (.DBF II, II+, III, IV files)
  <LI class=mvd-P>Epi Info files (.rec)
  <LI class=mvd-P>Fixed Format ASCII files (.fix, .fsc)
  <LI class=mvd-P-Bullet>FoxPro files (use same import filter as dBase files)
  <LI class=mvd-P-Bullet>Gauss (.DAT files - automatically reads the related DHT file)
  <LI class=mvd-P-Bullet>Lotus Worksheets (.WKS,.WK1,.WK3,.WK4 and.WRK)
  <LI class=mvd-P-Bullet>Matlab (.MAT)
  <LI class=mvd-P-Bullet>Microsoft Access files (.MDB)
  <LI class=mvd-P-Bullet>Microsoft Excel (versions 2.1 through Excel '97 .XLS* files)
  <LI class=mvd-P>Microsoft SQL Server files (.sql)
  <LI class=mvd-P>Minitab Workbook files (.mtw)
  <LI class=mvd-P-Bullet>Paradox Databases (.DB, through Version 7)
  <LI class=mvd-P-Bullet>Quattro Pro Worksheets (.WQ1)
  <LI class=mvd-P-Bullet>SAS files
  • <LI class=mvd-P-Bullet>Windows/OS2 (.SD2)
    <LI class=mvd-P-Bullet>Version 7/8 - Windows, Sun, and Linux (.SAS7BDAT, .SD2)
    <LI class=mvd-P>HP IBM and Sun UNIX (.SSD01)
  • DEC UNIX (.SSD04)
  <LI class=mvd-P-Bullet>SAS Transport files (version 6.x. TPT, XPT). Some special export options may need to be specified in your SAS program. We suggest using the SAS Xport engine (not PROC CPORT) to read and write these files.
  <LI class=mvd-P>SigmaPlot files (.jnb)
  <LI class=mvd-P-Bullet>SPSS files (.SAV)
  • Windows OS/2, DEC, HP IBM, Sun UNIX, DEC UNIX
  <LI class=mvd-P-Bullet>SPSS Export files (.POR)
  <LI class=mvd-P-Bullet>S-PLUS (.SDD)
  <LI class=mvd-P-Bullet>STATA files (.DTA Versions 2.0 and higher)
• Systat files (double precision, .SYD, or single precision, .SYS)

Note

The following ODBC file formats can be imported by selecting File Import Data From ODBC Connection:

• <LI class=mvd-P>Informix (.ifx)
• Oracle databases (.ora)

Note: All imports from Oracle files are done using ODBC, so the various ODBC components must be properly installed. When exporting to Oracle, table names and column names must be in UPPERCASE.
​

• SYBASE SQL server (.syb)

 

[Andy Nguyen]

Thanks for bringing this up, John. I overlooked SPLUS. Maybe it has exactly what I need and lack from both SAS, Matlab.

Read this on their site

Trading Strategies

Insightful Risk Solutions

In fast-paced markets, leading traders require rapid access to the right information to make important market-driven decisions. Inundated with volatile financial data, traders require answers they can rely upon quickly and constantly. Insightful’s S-PLUS and econometric libraries, S+FinMetrics and S+NuOPT provide rigorous technical and fundamental analysis using sophisticated models to predict future behavior.

You need advanced risk analysis of current and future exposure to optimize your financial performance. When you partner with Insightful you can quickly implement, monitor and adjust proprietary trading strategies to meet dynamic market conditions and improve your risk/return profiles. Insightful has provided advanced analytics and consulting to the world’s top financial institutions for20 years. We can help you create exactly the solution that your organization requires with our modern, cutting-edge methods and flexible, extensible solutions that can be easily integrated with your existing systems.

Insightful Solutions deliver:
World-class time-series analysis and forecasting:
Multivariate GARCH models
State-space time-series modeling
Co-integration models
Trading signal generation
Rolling window analysis
System Back-testing framework
State of the art risk management capabilities
Multi-factor models
Robust statistics
Extreme-value distributions

The part about trading signal generation is of interest to me. I'll play with it on Tues. Maybe we can open a SPLUS thread ?

 

[John]

Thanks for bringing this up, John. I overlooked SPLUS. Maybe it has exactly what I need and lack from both SAS, Matlab.

glad to contribute. BTW Professor Ciresi is our in-house expert in this subject matter I have learned a lot from him in the past.

 

[Jmuller]

SPlus vs. Matlab vs. SAS for large data

I see that you folks have been discussing the relative merits of
Matlab, SPlus and SAS in this thread.

I would like to get some advice and opinions on math/stat software
for dealing with large data sets in a Windows 32 bit desktop environment.

If this is not the site or thread for such a discussion forgive me.
Maybe you can suggest where I should post.

If you are open to it ... great!

Let's say "large data set" is anything too large to fit into RAM.
i.e., maybe it fits into virtual memory but a regression takes a long, long
time due to OS paging.

Note: we are also considering solutions beyond the "Windows 32 bit desktop".
such as move to 64-bit and some kind of server-based solution, but
for now we want to see what we can get without either of those moves.

I give my initial impressions from our search below.
Would love to hear if others have advice and experience on this topic.

best regards,
- john muller

R (Note: I have been using R for years and love it.)
R seems to have some attempts at dealing with large data sets,
e.g. SQLiteDf, filehash, g.data
I not really tried any of these yet.

Matlab. (Note: most of the folks at my office are Matlab users)
With Matlab it seems there are 2 main options: move to 64 bit or
use the DCT (Distributed Computing Toolbox).

Star-P. This is a Unix server-based distributed computing solution so
in a sense it competes with Matlab's DCT. Seems interesting,
and especially so for me since they claim they will also support R
sometime next year. We are considering this as a server-based solution.

SAS. Not sure if the desktop version of SAS can do anything special for us
but if and when we consider server solutions this seems like a standard
choice. I don't particularly like the SAS language, but if it meets our needs
I can live with it.

SPlus. On the surface, these folks have created exactly what we are looking for,
the bdFrame (big data frame for S and R users).
The bdFrame is stored on disk and processed in blocks, so effectively they
are doing the paging rather than the OS. You cannot use all the modeling
functions with bdFrames but you can do the most common things
(generalized linear models, clustering, principal components)
but that might be a deal breaker for some folks.
Not sure if the FinMetrics or NuOpt packages support bdFrames now
and if not if there is a plan for that.
I am experimenting with this now and so far it seems good.
It's not rocket science, but it saves me from managing all this
writing to disk myself.

 

[Andy Nguyen]

Let's say "large data set" is anything too large to fit into RAM.
i.e., maybe it fits into virtual memory but a regression takes a long, long time due to OS paging.

Our large data set would be in the size of close to 50GB each so using RAM is out of the question. I don't know which of the packages will be better fit for doing number crunching on the remote data. Most software will use ODBC driver to connect and ODBC is one of the slowest.

I haven't tried SPLUS yet but it seems promising. Whatever package we use, a seamless integration with Excel is a must.

To take full advantage, it seems you need a dedicated server that runs SAS or SPLUS.
Would love to read your stories about setting up SPLUS. Sure will have questions.

 

[kean]

SAS

I know SAS remote call function using ODBC will work for 50GB but I don't know about S-PLUS. Why don't you call Insightful to find out? I personally like SPlus more than SAS in terms of its statistical analysis function.

Both softwares have no problem integrating with Excel.

Our large data set would be in the size of close to 50GB each so using RAM is out of the question. I don't know which of the packages will be better fit for doing number crunching on the remote data. Most software will use ODBC driver to connect and ODBC is one of the slowest.

I haven't tried SPLUS yet but it seems promising. Whatever package we use, a seamless integration with Excel is a must.

To take full advantage, it seems you need a dedicated server that runs SAS or SPLUS.
Would love to read your stories about setting up SPLUS. Sure will have questions.

 

[lindsay]

In my experience, in both undergraduate and graduate school, it appears that 99% of us have used MATLAB almost exclusively over anything else. But I'm from applied maths, so that may explain why.

 

[vrsantoshkumar]

Hi

I would suggest to go to udemy and learn the best of matlab course online and you can get a discount 74% OFF coupon to Learn Matlab Udemy course here is Couponcode: MATLAB25 and given below is the website
Learn Matlab - Udemy

 

[Daniel Duffy]

In my experience, in both undergraduate and graduate school, it appears that 99% of us have used MATLAB almost exclusively over anything else. But I'm from applied maths, so that may explain why.

Matlab is a useful tool..

It does not teach you programming skills. Just saying..