dimple_timeseries.Rmd

---
title: rCharts, dimple, and time series
author: Timely Portfolio
github: {user: timelyportfolio, repo: rCharts_dimple, branch: "gh-pages"}
framework: bootplus
layout: post
mode: selfcontained
ext_widgets: {rCharts: "libraries/dimple"}
highlighter: prettify
hitheme: twitter-bootstrap
lead : >
  dimplejs and rCharts Tutorial with US Treasury Yield Data
---

# rCharts Smiles With dimple

<style>
/*
path.line {
  fill: none;
  stroke: #666;
  stroke-width: 1.5px;
}
*/
.axis {
  shape-rendering: crispEdges;
}

.axis text {
  font: 10px sans-serif;
  margin: 0;
}

.x.axis line {
  stroke: #000;
}

.x.axis path {
  display: none;
}
</style>

```{r echo=F, warning= F, message=F}
opts_chunk$set(message = FALSE, warning = FALSE, error = FALSE, tidy = FALSE, cache = FALSE,results = 'asis' )
```


Even this early in it release, [dimplejs](http://dimplejs.org) is such a powerful and promising [d3](http://d3js.org) library that we decided to quickly make it available in [rCharts](http://rcharts.io/site).  The first test was to recreate all (48) of the examples provided by [dimplejs](http://dimplejs.org/examples_index.html) with `rCharts`.  Once we [completed those](http://timelyportfolio.github.io/rCharts_dimple/gallery/), we felt fairly satisfied that we had good coverage of the `dimplejs` API (easily one of the best [documented](https://github.com/PMSI-AlignAlytics/dimple/wiki)).  We are aware of a couple missing items, but I really wanted to throw some real financial time series at it to see how well it might work in my real life as a portfolio manager.  If you are not familiar with `rCharts`, you might want to see this [Quickstart](http://ramnathv.github.io/rCharts/).

---
### Get the Data

Since bonds are a hot topic now, I thought some US Treasury Yield data from the St. Louis Federal Reserve (FRED) would make a nice subject.  I will start very basic and build up to a still simple but slightly more complicated plot.  As always, we first need data so we grab the data with `quantmod` `getSymbols()` and then merge it into one big `ust.xts` object.

```{r}
require(quantmod)
require(rCharts)
#now get the US bonds from FRED
USbondssymbols <- paste0("DGS",c(1,2,3,5,7,10,20,30))

ust.xts <- xts()
for (i in 1:length( USbondssymbols ) ) {
  ust.xts <- merge( 
    ust.xts,
    getSymbols( 
      USbondssymbols[i], auto.assign = FALSE,src = "FRED"
    )
  )
}
```
---
### Transform Our Data

Then we will define a little `xtsMelt` function to easily transform our wide `xts` data into long form.

```{r}
xtsMelt <- function(data) {
  require(reshape2)
  
  #translate xts to time series to json with date and data
  #for this behavior will be more generic than the original
  #data will not be transformed, so template.rmd will be changed to reflect
  
  
  #convert to data frame
  data.df <- data.frame(cbind(format(index(data),"%Y-%m-%d"),coredata(data)))
  colnames(data.df)[1] = "date"
  data.melt <- melt(data.df,id.vars=1,stringsAsFactors=FALSE)
  colnames(data.melt) <- c("date","indexname","value")
  #remove periods from indexnames to prevent javascript confusion
  #these . usually come from spaces in the colnames when melted
  data.melt[,"indexname"] <- apply(matrix(data.melt[,"indexname"]),2,gsub,pattern="[.]",replacement="")
  return(data.melt)
  #return(df2json(na.omit(data.melt)))
}
```

Now let's use our new `xtsMelt` function on our data.  You might notice that we use `na.omit` to remove the pesky NAs that sometimes haunt FRED data.

```{r}
ust.melt <- na.omit( xtsMelt( ust.xts["2012::",] ) )

ust.melt$date <- format(as.Date(ust.melt$date))
ust.melt$value <- as.numeric(ust.melt$value)
ust.melt$indexname <- factor(
  ust.melt$indexname, levels = colnames(ust.xts)
)
ust.melt$maturity <- as.numeric(
  substr(
    ust.melt$indexname, 4, length( ust.melt$indexname ) - 4
  )
)
ust.melt$country <- rep( "US", nrow( ust.melt ))
```

---
### Our First Plot - A Single Line

Getting the data was fairly easy, now let's plot.  I hope you see how easy it is to get an interactive `dimplejs` chart from `R`.  Without my comments, the code would all fit nicely on one line.  We can even use a `r` formula to define our `x` and `y` as shown `value~date`.

```{r}
#simple line chart of 10 year
d1 <- dPlot(
  value ~ date,  #or x="date", y="value"
  #dimplejs allows filtering but will lessen data to write
  #if we subset in R
  data = subset(ust.melt,maturity==10),  #get all data for 10y maturity
  type = 'line'
)
d1
```
<div id ="chart1"></div>
```{r echo = F}
d1$print('chart1')
```


---
### Fix the x axis

Uh oh, our x axis does not look too pretty.  However, `rCharts` is extensible and modular, so let's quickly jump to a little more advanced concept.  Currently `dimplejs` does not support dates on the x axis as well as I would like.  `dimplejs` is built on `d3`, so let's fix with a little [help](http://bl.ocks.org/mbostock/1166403) from the `d3` master [Mike Bostock](http://bost.ocks.org/mike/).  I built a custom [layout template](http://timelyportfolio.github.io/rCharts_dimple/assets/chart_d3dateaxis.html) to remove the `dimple` x-axis and replace with a `d3` version featuring much better date/time support.  To access it, we can set it with a little hack (I have a little inside information that this will soon become much easier).

```{r}
d2 <- d1
d2$field(
  'templates',
  modifyList(d2$templates, list(id = "chart2", script = 'http://timelyportfolio.github.io/rCharts_dimple/assets/chart_d3dateaxis.html') )
)
d2
```
<div id = "chart2"></div>
```{r echo = F}
d2$print('chart2')
```

---
### Line Chart with all Maturities

Sorry for the little advanced topic.  The author of dimplejs says better date handling is on his TO-DO list.

Most portfolio managers/analysts will want more than just the US 10y.  Let's plot all the maturities with just one little `groups` addition in our `dPlot`.  I also anticipate that we will need a legend to identify our maturities by color so we will also add a legend.

```{r}
#simple line chart of all maturities
d3 <- dPlot(
  value ~ date,  #or x="date", y="value"
  groups = 'maturity',
  data = ust.melt,  #get all maturities so remove subset from above
  type = 'line'
)
d3$legend( x = 60, y = 10, width = 620, height = 20,
  horizontalAlign = "right")
d3$field(
  'templates',
  modifyList(d3$templates, list(id = "chart3", script = 'http://timelyportfolio.github.io/rCharts_dimple/assets/chart_d3dateaxis.html') )
)
d3
```
<div id ="chart3"></div>
```{r echo = F}
d3$print('chart3')
```

---
### Line Chart by Maturity by Date

I am still really proud of my chart displayed in this [post](http://timelyportfolio.blogspot.com/2013/05/even-more-jgb-yield-charts-with-r.html).  I bet we can do something like that but better since we will have interactivity.  Will it be hard?  Of course not, we just change our `x` in `dPlot()` and then sort with `d4$xAxis(grouporderRule="date")`.


```{r}
#simple line chart of all maturities
d4 <- dPlot(
  x = c("maturity","date"),
  y = "value",
  groups = 'maturity',
  data = ust.melt,
  type = 'line'
)
d4$xAxis( grouporderRule = "date" ) #sort by date
d4$legend( x = 60, y = 10, width = 620, height = 20,
  horizontalAlign = "right")
d4
```
<div id ="chart4"></div>
```{r echo = F}
d4$print('chart4')
```

---
### Yield Curve Storyboard

Another way to look at yields would be as a yield curve.  Generally, this means remove time, but with `dimplejs` `storyboard` feature we can see the history of the yield curve.  Daily would be a little tedious, so let's do monthly 2013.  Watch closely.

```{r}
#get monthly for 2013
ust.melt <- xtsMelt(ust.xts[endpoints(ust.xts,"months"),]["2013::",])

ust.melt$date <- format(as.Date(ust.melt$date),"%m/%d/%Y")
ust.melt$value <- as.numeric(ust.melt$value)
ust.melt$indexname <- factor(
  ust.melt$indexname, levels = colnames(ust.xts)
)
ust.melt$maturity <- as.numeric(
  substr(
    ust.melt$indexname, 4, length( ust.melt$indexname ) - 4
  )
)
ust.melt$country <- rep( "US", nrow( ust.melt ))

d5 <- dPlot(
  value ~ maturity,
  data = ust.melt,
  type = "line"
)
d5$xAxis( orderRule ="maturity" )
d5$set( storyboard = "date" )
d5
```
<div id ="chart5"></div>
```{r echo = F}
d5$print('chart5')
```

---
### Other dimplejs Chart Types

Telling a story with your data has never been so easy.  Add a little text content describing the change and next you will be presenting at [Eyeo](http://eyeofestival.com/).

Just to make sure we cover some of the other plot types, let's draw our other two `dimplejs` options--area, bar, and bubble.

```{r}
d6 <- dPlot(
  x = "date",
  y = "value",
  groups = "indexname",
  data = ust.melt,
  type = "area"
)
d6$xAxis( orderRule = "date" )
d6
```
<div id ="chart6"></div>
```{r echo = F}
d6$print('chart6')
```

And although a stacked 100% bar does not make a lot of sense with this data, here is how we might do that by changing `type = "bar"` and `d7$yAxis( type = "addPctAxis" )`.

```{r}
d7 <- dPlot(
  x = "date",
  y = "value",
  groups = "indexname",
  data = ust.melt,
  type = "bar"
)
d7$xAxis( orderRule = "date" )
d7$yAxis( type = "addPctAxis" )
d7
```
<div id ="chart7"></div>
```{r echo = F}
d7$print('chart7')
```

Bubble also might not be the best way to plot yields, but here is an example.

```{r}
d8 <- dPlot(
  x = c("indexname","date"),
  y = "value",
  z = "value",
  groups = "indexname",
  data = ust.melt,
  type = "bubble"
)
d8$xAxis( grouporderRule = "date", orderRule = "maturity" )
d8$zAxis( type = "addMeasureAxis", overrideMax = 10 )
d8
```
<div id ="chart8"></div>
```{r echo = F}
d8$print('chart8')
```

---
### More rCharts, slidify, and dimplejs

The beauty of [rCharts](http://rcharts.io/site) is its ability to harness the ingenuity from the entire ecosystem inside and outside of `r`.  Whole libraries, such as [dimplejs](http://dimplejs.org), [nvd3](http://nvd3.org), [rickshaw](http://code.shutterstock.com/rickshaw/), [highcharts](http://highcharts.com), [morris](http://www.oesmith.co.uk/morris.js/), and [polycharts](http://www.polychartjs.com/), or even specific custom [visualizations](http://rcharts.io/site/gallery.html) can quickly be incorporated into your workflow and publishing.  If desired, these can be delivered in various reproducible formats with [slidify](http://slidify.org).  I strongly encourage you to check out [rCharts](http://rcharts.io/site), [slidify](http://slidify.org), and [dimplejs](http://dimplejs.org).  They have changed my world.


---
### Thanks
- [Ramnath Vaidyanathan](http://github.com/ramnathv) for his amazing dedication, responsiveness, and help with [rCharts](http://rcharts.io/site) and [slidify](http://slidify.org).
- [John Kiernander](https://twitter.com/jkiernander) for `dimplejs`[http://dimplejs.org] and his seemingly infinite supply of helpful and inspirational [examples](http://dimplejs.org/advanced_examples_index.html).
- [Mike Bostock](http://bost.ocks.org/mike/example/) for everything.