Fast Tube by Casper

To investigate using Poisson regression via the GLM framework consider a small data set on failure modes (here).

> failure.df = read.table("twomodes.dat", header = TRUE)
> failure.df
  Mode1 Mode2 Failures
1  33.3  25.3       15
2  52.2  14.4        9
3  64.7  32.5       14
4 137.0  20.5       24
5 125.9  97.6       27
6 116.3  53.6       27
7 131.7  56.6       23
8  85.0  87.3       18
9  91.9  47.8       22

The machinery is run in two modes and the objective of the analysis is to determine whether the number of failures depends on how long the machine is run in mode 1 or mode 2 and whether there is an interaction between the time in each mode to increases or decreases the number of failures.

The response for this set of data is the number of failures (count) so a Poisson regression model is considered.

> fmod1 = glm(Failures ~ Mode1 * Mode2, data = failure.df, family = poisson)
> summary(fmod1)
 
Call:
glm(formula = Failures ~ Mode1 * Mode2, family = poisson, data = failure.df)
 
Deviance Residuals: 
       1         2         3         4         5         6         7         8         9  
 0.91003  -1.15601  -0.28328  -0.10398   0.03526   0.84825  -0.49211  -0.57298   0.64821  
 
Coefficients:
              Estimate Std. Error z value Pr(>|z|)    
(Intercept)  2.105e+00  4.481e-01   4.698 2.63e-06 ***
Mode1        7.687e-03  4.285e-03   1.794   0.0729 .  
Mode2        4.703e-03  1.163e-02   0.405   0.6858    
Mode1:Mode2 -1.978e-05  1.037e-04  -0.191   0.8487    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 
 
(Dispersion parameter for poisson family taken to be 1)
 
    Null deviance: 16.996  on 8  degrees of freedom
Residual deviance:  3.967  on 5  degrees of freedom
AIC: 55.024
 
Number of Fisher Scoring iterations: 4

The model output does not provide any support for an interaction between the number of time spent in the two different modes of operation. If we remove the interaction term and re-fit the model, using the update function, we get:

> fmod2 = update(fmod1, . ~ . - Mode1:Mode2)
> summary(fmod2)
 
Call:
glm(formula = Failures ~ Mode1 + Mode2, family = poisson, data = failure.df)
 
Deviance Residuals: 
     Min        1Q    Median        3Q       Max  
-1.21984  -0.44735  -0.05893   0.68351   0.87510  
 
Coefficients:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept) 2.175168   0.255456   8.515  < 2e-16 ***
Mode1       0.007015   0.002429   2.888  0.00387 ** 
Mode2       0.002549   0.002835   0.899  0.36852    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 
 
(Dispersion parameter for poisson family taken to be 1)
 
    Null deviance: 16.9964  on 8  degrees of freedom
Residual deviance:  4.0033  on 6  degrees of freedom
AIC: 53.06
 
Number of Fisher Scoring iterations: 4

This output suggests that the time of operation in mode 1 is important for determining the number of faults but the time of operation in mode 2 is not important. One last step gives us:

> fmod3 = update(fmod2, . ~ . - Mode2)
> summary(fmod3)
 
Call:
glm(formula = Failures ~ Mode1, family = poisson, data = failure.df)
 
Deviance Residuals: 
     Min        1Q    Median        3Q       Max  
-1.43194  -0.56958  -0.00745   0.66742   0.82231  
 
Coefficients:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept) 2.237196   0.243053   9.205  < 2e-16 ***
Mode1       0.007705   0.002264   3.403 0.000667 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 
 
(Dispersion parameter for poisson family taken to be 1)
 
    Null deviance: 16.9964  on 8  degrees of freedom
Residual deviance:  4.8078  on 7  degrees of freedom
AIC: 51.865
 
Number of Fisher Scoring iterations: 4

The diagnostic plots are shown below which do not indicate any major problems with the final model, especially given the small number of data points.

Four diagnostic plots for a Poisson regression model based on total failures

Other useful resources are provided on the Supplementary Material page.

Fast Tube by Casper

As an example consider a data set on the number of views of the you tube channel ramstatvid. A short snippet of the data is shown here:

> head(yt.views)
        Date Views
1 2010-05-17    13
2 2010-05-18    11
3 2010-05-19     4
4 2010-05-20     2
5 2010-05-21    23
6 2010-05-22    26

The ggplot function is used by specifying a data frame and the aes maps the Date to the x-axis and the number of Views to the y-axis.

ggplot(yt.views, aes(Date, Views)) + geom_line() +
  scale_x_date(format = "%b-%Y") + xlab("") + ylab("Daily Views")

The axis labels for the Date variable are created with the scale_x_date function where the format is specified as a Month/Year combination with the %b and %Y formatting strings. The graph that is produced is shown here:

Time Series Plot Example with ggplot2 package

Other useful resources are provided on the Supplementary Material page.

As an initial example we will consider the English legend Sir Ian Botham who played 102 test matches for England between his debut in 1977 until his final game in 1992.

The first obvious breakdown is to consider how Botham performed against the six countries that he played against during his test career. A summary of his statistics are shown here:

 Opposition Matches Bat Inns Runs NO Bowl Inns Wicket Catch
  Australia      36       49 1673  2       66     148    57
      India      14       16 1201  0       23      59    14
New Zealand      15       22  846  2       28      64    14
   Pakistan      14       20  647  1       18      40    14
  Sri Lanka       3        3   41  0        6      11     2
West Indies      20       37  792  1       27      61    19

Botham only played three matches against Sri Lanka so it is difficult to properly assess his performance against them. If the above table is stored in a data frame itb.opp then we can create a histogram of the total runs (or wickets) by opposition country:

ggplot(itb.opp, aes(Opposition, Runs)) + geom_bar() + xlab("Country") +
  ylab("Total Runs")

This code produces the following graph:

IT Botham Total Runs by Opposition

The total wickes graph is produced by the next code:

ggplot(itb.opp, aes(Opposition, Wicket)) + geom_bar() + xlab("Country") +
  ylab("Total Wickets")

IT Botham Total Wickets by Opposition

We may now want to delve deeper into the performance against different nations to take into account the number of games or innings where Botham batted or bowled. The traditional way to assess performance is to calculate batting and bowling averages and we can do this by opposition which provides the following data frame:

> itb.opp.sum
 Opposition Discipline  Average
  Australia    Batting 29.35088
      India    Batting 70.64706
New Zealand    Batting 42.30000
   Pakistan    Batting 32.35000
  Sri Lanka    Batting 13.66667
West Indies    Batting 21.40541
  Australia    Bowling 27.65541
      India    Bowling 26.40678
New Zealand    Bowling 23.43750
   Pakistan    Bowling 31.77500
  Sri Lanka    Bowling 28.18182
West Indies    Bowling 35.18033

This can be converted into a dot plot so we can see whether Botham had a high batting average than bowling average, which is often taken to be one of the signs of an all-rounder.

ggplot(itb.opp.sum, aes(Average, Opposition, colour = Discipline)) +
  geom_point()+ xlab("Average") + ylab("")

The graph is shown here:

IT Botham Batting and Bowling Averages by Opposition

We can see the differences in performance based on the opposition. Botham’s performance against the West Indies, by far the strongest team during most of his international career, were worse than against the other countries. However, his averages were far from embarassing when compared to other players at the time. The graph also shows that Botham enjoyed batting and bowling against India.

We can divide this data further based on whether the matches were played in England or outside of England and this data is shown here:

> itb.opp.ha.sum
  Opposition Venue Discipline  Average
   Australia  Away    Batting 30.22581
       India  Away    Batting 61.55556
 New Zealand  Away    Batting 50.44444
    Pakistan  Away    Batting 16.00000
   Sri Lanka  Away    Batting 13.00000
 West Indies  Away    Batting 14.17647
   Australia  Home    Batting 28.30769
       India  Home    Batting 80.87500
 New Zealand  Home    Batting 35.63636
    Pakistan  Home    Batting 34.16667
   Sri Lanka  Home    Batting 14.00000
 West Indies  Home    Batting 27.55000
   Australia  Away    Bowling 28.44928
       India  Away    Bowling 25.53333
 New Zealand  Away    Bowling 27.44444
    Pakistan  Away    Bowling 45.00000
   Sri Lanka  Away    Bowling 21.66667
 West Indies  Away    Bowling 39.50000
   Australia  Home    Bowling 26.96203
       India  Home    Bowling 27.31034
 New Zealand  Home    Bowling 20.51351
    Pakistan  Home    Bowling 31.07895
   Sri Lanka  Home    Bowling 30.62500
 West Indies  Home    Bowling 31.97143

A dot plot is created from this data with a separate panel for each of the six opposition countries and the averages divided into batting and bowling performances. The coloured dots in the graph indicated whether the average is for matches at home or away.

ggplot(itb.opp.ha.sum, aes(Average, Discipline, colour = Venue)) +
  geom_point() + facet_wrap( ~ Opposition) +
  xlab("Batting Average") + ylab("")

This graph is shown below:

IT Botham Batting and Bowling Averages by Country and Home/Away

We can see that the difference between home and away peformance is, in general, not very large for bowling averages but in some cases there is a noticeable difference in batting averages. When looking at Botham’s performances against the West Indies his statistics at home are much better than his away performance, suggesting that his main struggles against the strong West Indies team were in the Caribbean. This might be due to his swing bowling being more suitable to English conditions compared to pitches in the West Indies.

To round off this brief look at the career of IT Botham let us consider some other important statistics, in particular games where he performed with the bat and ball.

• Overall Botham scored 14 hundreds and 22 fifties out of 161 innings so he reached fifty runs every five innings or so.
• He also took 27 five wicket hauls and 17 four wicket hauls so he took four or more wickets every four innings or so.
• He took 120 catches.

Individual matches of excellence include five games with a century and at least five wickets:

Year  Opposition       Ground Venue Runs Wicket
1978 New Zealand Christchurch  Away  133      8
1978    Pakistan       Lord's  Home  108      8
1980       India       Mumbai  Away  114     13
1981   Australia        Leeds  Home  199      7
1984 New Zealand   Wellington  Away  138      6

These performances and others show why Botham was considered such a great player as he produced some sustained periods of excellent all-round cricket rather than having one discipline more dominant for a long period of time.

To illustrate this type of graph we will consider some surface elevation data that is available in the geoR package. The data set in this package is called elevation and stores the elevation height in feet (as multiples of ten feet) for a grid region of x and y coordinates (recorded as multiples of 50 feet). To access this data we load the geoR pacakage and then use the data function:

require(geoR)
data(elevation)

For some packages we need the call to the data function to make a set of data available for our use. The elevation object is not a data frame so our first step is to create our own data frame to be used to create the level plots using the different graphics packages.

elevation.df = data.frame(x = 50 * elevation$coords[,"x"],
  y = 50 * elevation$coords[,"y"], z = 10 * elevation$data)

We extract the x and y grid coordinates and the height values, multiplying them by 50 and 10 respectively to convert to feet for the graphs. Rather than trying to plot the individual values we need to create a surface to cover the whole grid region as the points themselves are too sparse. We make use of the loess function to fit a local polynomial trend surface (using weighted least squares) to approximate the elevation across the whole region. The function call for a local quadratic surface is shown below:

elevation.loess = loess(z ~ x*y, data = elevation.df,
  degree = 2, span = 0.25)

The next stage is to extract heights from this fitted surface at regular intervals across the whole grid region of interest – which runs from 10 to 300 feet in both the x and y directions. The expand.grid function creates an array of all combinations of the x and y values that we specify in a list. We choose a range every foot from 10 to 300 feet to create a fine grid:

elevation.fit = expand.grid(list(x = seq(10, 300, 1), y = seq(10, 300, 1)))

The predict function is then used to estimate the surface height at all of these combinations of x and y coordinates covering our grid region. This is saved as an object z which will be used by the base graphics function:

z = predict(elevation.loess, newdata = elevation.fit)

The lattice and ggplot2 expect the data in a different format so we make use of the as.numeric function to convert from a table of heights to a single column and append to the object we create based on all combinations of x and y coordinates:

elevation.fit$Height = as.numeric(z)

The data is now in a format that can be used to create the level plots in the various packages.

Base Graphics

The function image in the base graphics package is the function we use to create a level plot. This function requires a list of x and y values that cover the grid of vertical values that will be used to create the surface. These heights are specified as a table of values, which in our case was saved as the object z during the calculations on the local trend surface.

The text on the axis labels are specified by the xlab and ylab function arguments and the main argument determines the overall title for the graph. The function call below creates the level plot:

image(seq(10, 300, 1), seq(10, 300, 1), z,
  xlab = "X Coordinate (feet)", ylab = "Y Coordinate (feet)",
  main = "Surface elevation data")
box()

After the image function is used we call the box function mainly for aesthetic purposes to ensure there is a line surrounding the level plot. The graph that is created is shown below:

Base Graphics Level Plot

The default colour scheme used by the base graphics produces an attractive level plot graph where we can easily see the variation in height across the grid region. It is basically a fancy version of a contour plot where the regions between the contour lines are coloured with different shades indicating the height in those regions.

Lattice Graphics

The lattice graphics package provides a function levelplot for this type of graphical dispaly. We use the data stored in the object elevation.fit to create the graph with lattice graphics.

levelplot(Height ~ x*y, data = elevation.fit,
  xlab = "X Coordinate (feet)", ylab = "Y Coordinate (feet)",
  main = "Surface elevation data",
  col.regions = terrain.colors(100)
)

The formula is used to specify which variable to use for the three axes and a data frame where the values are stored – as there are three dimensions it is the z axis that is specified on the left hand side of the formula. The axes labels and title are specified in the same way as the base graphics.

The range of colours used in the lattice level plot can be specified as a vector of colours to the col.regions argument of the function. We make use of the terrian.colors function to create this vector which a range of 100 colours which are less striking than those used above with the base graphics. The level plot that we can is shown here:

Lattice Graphics Level Plot

This is in general similar to the base graphics display but the actual plot region is a different shape that makes things look slightly different.

ggplot2

The ggplot2 package also provides facilities for creating a level plot making use of the tile geom to create the desired graph. The function ggplot forms the basis of the graph and various other options are used to customise the graph:

ggplot(elevation.fit, aes(x, y, fill = Height)) + geom_tile() +
  xlab("X Coordinate (feet)") + ylab("Y Coordinate (feet)") +
  opts(title = "Surface elevation data") +
  scale_fill_gradient(limits = c(7000, 10000),low = "black",high = "white") +
  scale_x_continuous(expand = c(0,0)) +
  scale_y_continuous(expand = c(0,0))

This large number of options that are added to the graph change various settings. The choice of colours for the heights used on graph is selected by the scale_fill_gradient function with colours ranging from black to white. The scale_x_continuous and scale_y_continuous options are used to stretch the tiles to cover the whole grid region covering up the default gray background – this makes the graph more visually appealing. The graph that is produced is shown here:

ggplot2 Level Plot

The graph from ggplot2 is visually as impressive as the other graphs – there is more smoothing between the colours which blurs some of the lines on the other graphs because of the type of colour gradient that was selected.

This blog post is summarised in a pdf leaflet on the Supplementary Material page.

The box and whisker plot is an effective way to investigate the distribution of a set of data. For example, skewness can be identified from the box and whisker as the display does not make any assumptions about the underlying distribution of the data. The extreme values at either end of the scale are sometimes included on the display to show how far they extend beyond the majority of the data.

To illustrate creating box and whisker plots we consider UK meteorological data that has been collected on a monthly basis at Southampton, UK between 1950 and 1999 and is publicly available. This data is available from the UK Met Office and we will compare the range of temperatures recorded in each month of the year over this period by creating box and whisker plots with the different packages.

The data is assumed to have been imported into R and stored in a data frame called soton.df. An extract of the data is shown here:

    Year Month Max.Temp Min.Temp Frost  Rain
1   1950   Jan      7.7      2.8     7  20.1
2   1950   Feb     10.3        4     4 127.0
3   1950   Mar     13.0      4.5     2  39.4
4   1950   Apr     13.6      4.7     0  62.0
5   1950   May     17.9      7.8     0  32.2

Base Graphics

Fast Tube by Casper

The base graphics approach makes use of the boxplot function to create box and whisker plots. In this situation the function can be used with a formula rather than specifying two separate vectors of data – we can specify a data frame to point towards a source of data to be used in the graph. For the temperature data we use this code:

boxplot(Max.Temp ~ Month, data = soton.df,
  xlab = "Month", ylab = "Maximum Temperature",
  main = "Temperature at Southampton Weather Station (1950-1999)"
)

The horizontal and vertical axes labels are specified using the xlab and ylab arguments respectively and the title of the plot is created using the main argument. The box and whisker plot is shown here:

Base Graphics Box and Whisker Plot

The function boxplot makes it easy to create a reasonably attractive box and whisker plot. The variation in the distribution of temperatures across the year can be seen from the graph.

Lattice Graphics

Fast Tube by Casper

In the lattice graphics package there is a function bwplot which is used to create box and whisker plots. The function call also uses a formula to specify the x and y variables to use on the graph. The function call arguments are identical to the boxplot function in base graphics:

bwplot(Max.Temp ~ Month, data = soton.df,
  xlab = "Month", ylab = "Maximum Temperature",
  main = "Temperature at Southampton Weather Station (1950-1999)"
)

The variable Month is categorical so a separate box and whisker summary is created for each month separately. The lattice version of the graph is shown here:

Lattice Graphics Box and Whisker Plot

This is very similar to the box and whisker plot created by base graphics with a similar level of effort required. The main difference is the use of a circle rather than a line to identify the location of the median of the data.

ggplot2

Fast Tube by Casper

In the ggplot2 package there is a general function ggplot that is used to create graphs of any type. We make use of the boxplot geom to create a box and whisker plot following the standard approach. The first step is to specify a data frame to use to create the graph and then map the columns of this data frame, via the \texttt{aes} argument, to the different axes or other aesthetics (such as colour or symbol shape). The particular geom is used to specify the type of plot that we want to create. Our final step is to add on the various axes labels and an overall title to the graph.

ggplot(soton.df, aes(Month, Max.Temp)) + geom_boxplot() +
  ylab("Maximum Temperature") +
  opts(title = "Temperature at Southampton Weather Station (1950-1999)")

The ggplot2 version of box and whisker plots is shown here:

ggplot2 Graphics Box and Whisker Plot

The distinctive gray background used by ggplot2 is an obvious visual difference compared to the default clear background used in the other two approaches. The boxes themselves have a cleaner look in this graph than the other two methods and the overall look is slick.

This blog post is summarised in a pdf leaflet on the Supplementary Material page.

To illustrate creating a scatter plot we will use a simple data set for the population of the UK between 1992 and 2009. This data is saved in a data frame uk.df using the following command:

uk.df = data.frame(Year = 1992:2009,
  Population = c(57770, 57933, 58096, 58258, 58418, 58577,
  58743, 58925, 59131, 59363, 59618, 59894, 60186, 60489,
  60804, 61129, 61461, 61796)
)

For this example the data is recorded in thousands to make the graph easier to read and there is no benefit or noticeable improvement to be seen by using greater detail.

Base Graphics

Fast Tube by Casper

In the base graphics system the general purpose plot function can be used to create a scatter plot for the UK population data set that we created. The first two arguments to the plot function are the x and y variables respectively. The following code will create a scatter plot, including various labels:

plot(uk.df$Year, uk.df$Population,
  xlab = "Year", ylab = "Total Population (Thousands)",
  main = "UK Population (1992-2009)", pch = 16)

The labels for the x and y axes are specified via the xlab and ylab arguments to the plot function and the main argument specifies the title for the plot.

Base Graphics Histogram

The graph itself is plain and functional which solid circles indicating the population (in thousands) for each of the years covered by the data.

Lattice Graphics

Fast Tube by Casper

The lattice graphics package provides a function xyplot specifically to create scatter plots and the function is used in a similar way to the base graphics approach. The first argument to the function is a formula describing the relationship to be plotted on the graph, with the y variable preceding the x variable as we are used to when describing mathematical fomula such as y=a+bx. The data frame is specified with the data argument to simplify the expression in the formula. The code used is as follows:

xyplot(Population ~ Year, data = uk.df,
  xlab = "Year", ylab = "Total Population (Thousands)",
  main = "UK Population (1992-2009)",
  scales = list(x = list(at = seq(1992, 2009, 2)))
)

The axis labels and the overall title for the graph are specified in the same way as the base graphics system. We indulge in some fine tuning of the labels on the x axis via the scales argument – here we indicate that every second year should be included on the label starting in 1992 and running until 2009. The lattice graph is shown here for comparison with the graphs created using the other two packages:

Lattice Graphics Scatter Plot

There are very few visual differences between the lattice and base graphics. In lattice graphics an object is created that can be edited to add or remove components and then printed to the screen. This approach is more flexible than the base graphics where the components are painted on top of each other and the use of themes in lattice will make it easier to keep a consistent look to all graphs in a document.

ggplot2

Fast Tube by Casper

In the ggplot2 package the ggplot function is used to create graphs of all types rather than having a separate function defined for each type of graph. The first argument is adata frame with the data to be plotted and the aes argument specifies the aesthetics associated with the graph such as the point symbol, size or colour. In this case the Year variable appears on the x axis and the Population variable on the y axis. The code to create the scatter plot is shown here:

ggplot(uk.df, aes(Year, Population)) + geom_point() +
  xlab("Year") + ylab("Total Population (Thousands)") +
  opts(title = "UK Population (1992-2009)")

The geom_point specifies the type of graph to create (a scatter plot in this situation and this highlights the flexibility of the ggplot2 package as changing the geom will create a new type of graph) and the labels for the graph are created by adding them to the graph with the xlab, ylab and opts functions. The graph is shown below:

ggplot2 Scatter plot

This graph is not greatly different to the scatter plot created using the base and lattice packages. The default theme in the ggplot2 package has a gray background with white grid lines that allows easy visual recognition of graphs created using this package.

This blog post is summarised in a pdf leaflet on the Supplementary Material page.

The shape of the histogram is determined by the width and number of regions that divided up the data. A histogram provides an indication the following features of a set of data: the general shape, symmetry or skewness of data and modality (uni-, bi- or multi-modal). There are some situations where a different type of graph would be preferable but histograms are useful for describing the general features of the distribution of a set of data.

To illustrate creating a histogram we consider data from the AFL sports league in Australia and the total number of points scored by the home team in each fixture. If we assume that the data is in a comma separated text file, called afl_2003_2007.csv, then we would import that data using the following command saving the results in a data frame:

afl.df = read.csv("afl_2003_2007.csv")

Edit: The data is available as AFL Data Set. Change the file extension manually to csv or change the command to reflect the different file name.

Base Graphics

Fast Tube by Casper

In base graphics the function hist is used to create a histogram with the first argument being the name of the vector that contains the data to be plotted. The x-axis is given a label using the xlab argument and the main argument is used to add a title to the graph. Code to create a histogram of home points is shown below:

hist(afl.df$Home.Total, xlab = "Home Points",
  main = "Histogram of Points Scored at Home\nAFL 2003-2007")

The default option is to display bars representing the frequency of data values in each of the ranges and the overall look of the graph is basic as shown here:

Base Graphics Histogram

The default algorithm for selecting number of bins to use for the histogram usually makes a sensible selection but this can be specified if required.

Lattice Graphics

Fast Tube by Casper

In the lattice graphics package there is a function histogram and we make use of the formula to specify a single variable for the number of points scored by the home team. The specification for the axis labels and graph title are the same as for the base graphics package. The equivalent graph is created using the following code:

histogram( ~ Home.Total, data = afl.df, xlab = "Home Points",
  main = "Histogram of Points Scored at Home\nAFL 2003-2007")

Here the default option is the work with proportions of the total number of data points rather than counts so the shape of the distribution is slightly different when compared to the base graphics plot. The lattice version is shown below:

Lattice Graphics Histogram

The main other difference is the choice of colour for the bars in the histogram and these can be adjusted by changing the global theme for lattice.

ggplot2

Fast Tube by Casper

The ggplot2 library uses a general purpose graphics function called ggplot to create graphs of all types and the geom specifies the type of display to create, in this case a histogram. Components that make up the graph are added sequentially to build up the whole plot and in the example below we add axis labels and a main title.

ggplot(afl.df, aes(Home.Total)) + geom_histogram() +
  xlab("Home Points") + ylab("Frequency") +
  opts(title = "Histogram of Points Scored at Home\nAFL 2003-2007")

The default theme for ggplot2 is distinctive and the histogram is shown in the graph below:

ggplot 2 Histogram

The default number of bins is larger compared to base and lattice graphics which provides a rough distribution in this particular case. The online ggplot2 manual is a good source of information about customising graphs created using this approach.

This blog post is summarised in a pdf leaflet on the Supplementary Material page.

In this post we will considered creating a dot plot using the base graphics, lattice graphics and ggplot2 approaches. To illustrate creating a dot plot we used data from the FAO website on the total irrigation area for Africa, Latin America, North America and Europe. We create a data frame using the following code:

irrigation.df = data.frame(
  Region = rep(c("Africa", "Latin America", "North America", "Europe"), 4),
  Year = factor(c(rep(1980, 4), rep(1990, 4), rep(2000, 4), rep(2007, 4))),
  Area = c(9.3, 12.7, 21.2, 18.8, 11.0, 15.5, 21.6, 25.3,
    13.2, 17.3, 23.3, 26.7, 13.6, 17.3, 23.8, 26.3)
)

Base Graphics

Fast Tube by Casper

In the base graphics system we build up the dotplot with a series of commands. The first function call creates the graph region based on the data set but we do not plot any data by setting the type = “n” argument. The axis labels for the horizontal and vertical scales are set along with the title in the initial function call:

plot(irrigation.df$Area, irrigation.df$Region, xlab = "Area",
  ylab = "Region", main = "Irrigation Area by Region", type = "n")

To add the points with separate colours for each of the four years we use the points function and subset to the particular year by testing a condition on the year. The col argument is used with a text string to specify the colour for the symbols for the given year:

points(irrigation.df$Area[irrigation.df$Year == 1980],
  irrigation.df$Region[irrigation.df$Year == 1980], col = "black", pch = 16)
points(irrigation.df$Area[irrigation.df$Year == 1990],
  irrigation.df$Region[irrigation.df$Year == 1990], col = "blue", pch = 16)
points(irrigation.df$Area[irrigation.df$Year == 2000],
  irrigation.df$Region[irrigation.df$Year == 2000], col = "red", pch = 16)
points(irrigation.df$Area[irrigation.df$Year == 2007],
  irrigation.df$Region[irrigation.df$Year == 2007], col = "green", pch = 16)

The code is rather long winded compared to the using the other two graphics packages. We can add a legend to the graph so that the years can be identified:

legend(10, 4, legend = c("1980", "1990", "2000", "2007"),
  col = c("black", "blue", "red", "green"), pch = 16)

The placement of the legend uses the x and y coordinates within the graph to position the box. All the code above produces the following graph:

Base Graphics Dot Plot

The graph is basic but we can consider the changes over time for the four regions. One downside is that the regions have been labelled with numbers rather than text strings.

Lattice Graphics

Fast Tube by Casper

The lattice graphics package has a function dotplot that is used to create dot plots. The first argument to the function is a formula describing the variables to use for the horizontal and vertical axes. We also specify the data frame to use for the graph and which column to determine different symbols and/or colours to highlight groupings within the plot:

dotplot(Region ~ Area, data = irrigation.df, groups = Year,
  main = "Irrigation Area by Region")

The lattice variant of the graph is shown here:

Lattice Graphics Dot Plot

The graph is simple and very similar to the one produced using the base graphics with the advantage that the R code is not as complicated.

ggplot2

Fast Tube by Casper

The ggplot function is used to create the dot plot where we first specify the name of the data frame with the information to be displayed and then use the aes argument to list the variables to plot on the horizontal and vertical axes. The colour argument determines the variable to use for assigning colours to (usually) a categorical variable.

ggplot(irrigation.df, aes(x = Area, y = Region, colour = Year)) +
  geom_point() + opts(title = "Irrigation Area by Region")

The ggplot2 version of the dot plot is shown below:

ggplot2 Dot Plot

This graph is very similar to the ones produced using the other graphics packages but has the distinctive background and legend style that is used as the default option in ggplot2.

This blog post is summarised in a pdf leaflet on the Supplementary Material page.

As an example consider a company that regularly has to ship parcels between its various (five for this example) sub-offices and has the option of using three competing parcel delivery services, all of which charge roughly similar amounts for each delivery. To determine which service to use, the company decides to run an experiment shipping three packages from its head office to each of the five sub-offices. The delivery time for each package is recorded and the data loaded into R:

delivery.df = data.frame(
  Service = c(rep("Carrier 1", 15), rep("Carrier 2", 15),
    rep("Carrier 3", 15)),
  Destination = c(rep(c("Office 1", "Office 2", "Office 3",
    "Office 4", "Office 5"), 9)),
  Time = c(15.23, 14.32, 14.77, 15.12, 14.05,
  15.48, 14.13, 14.46, 15.62, 14.23, 15.19, 14.67, 14.48, 15.34, 14.22,
  16.66, 16.27, 16.35, 16.93, 15.05, 16.98, 16.43, 15.95, 16.73, 15.62,
  16.53, 16.26, 15.69, 16.97, 15.37, 17.12, 16.65, 15.73, 17.77, 15.52,
  16.15, 16.86, 15.18, 17.96, 15.26, 16.36, 16.44, 14.82, 17.62, 15.04)
)

The data is then displayed using a dot plot for an initial visual investigation of any trends in delivery time between the three services and across the five sub-offices. The colour aesthetic is used to distinguish between the three services in the plot.

ggplot(delivery.df, aes(Time, Destination, colour = Service)) + geom_point()

This code produces the following graph:

Graph of the delivery time for different services and destintions

The graph shows a general pattern of service carrier 1 having shorter delivery times than the other two services. There is also an indication that the differences between the services varies for the five sub-offices and we might expect the interaction term to be significant in the two-way ANOVA model. To fit the two-way ANOVA model we use this code:

delivery.mod1 = aov(Time ~ Destination*Service, data = delivery.df)

The * symbol instructs R to create a formula that includes main effects for both Destination and Service as well as the two-way interaction between these two factors. We save the fitted model to an object which we can summarise as follows to test for importance of the various model terms:

> summary(delivery.mod1)
                    Df  Sum Sq Mean Sq  F value    Pr(>F)    
Destination          4 17.5415  4.3854  61.1553 5.408e-14 ***
Service              2 23.1706 11.5853 161.5599 < 2.2e-16 ***
Destination:Service  8  4.1888  0.5236   7.3018 2.360e-05 ***
Residuals           30  2.1513  0.0717                       
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

We have strong evidence here that there are differences between the three delivery services, between the five sub-office destinations and that there is an interaction between destination and service in line with what we saw in the original plot of the data. Now that we have fitted the model and identified the important factors we need to investigate the model diagnostics to ensure that the various assumptions are broadly valid.

We can plot the model residuals against fitted values to look for obvious trends that are not consistent with the model assumptions about independence and common variance. The first step is to create a data frame with the fitted values and residuals from the above model:

delivery.res = delivery.df
delivery.res$M1.Fit = fitted(delivery.mod1)
delivery.res$M1.Resid = resid(delivery.mod1)

Then a scatter plot is used to display the fitted values and residuals where the colour asthetic highlights which points correspond to the three competing delivery services:

ggplot(delivery.res, aes(M1.Fit, M1.Resid, colour = Service)) + geom_point() +
  xlab("Fitted Values") + ylab("Residuals")

The xlab() and ylab() are used to change the text on the axis labels. The residual diagnostic plot is:

Diagnostic Residual Plot for Delivery Time Model

There are no obvious patterns in this plot that suggest problems with the two-way ANOVA model that we fitted to the data.

As an alternative display we could separate the residuals into destination sub-offices, where the facet_wrap() function instructs ggplot to create a separate display (panel) for each of the destinations.

ggplot(delivery.res, aes(M1.Fit, M1.Resid, colour = Service)) +
  geom_point() + xlab("Fitted Values") + ylab("Residuals") +
  facet_wrap( ~ Destination)

To produce the following alternative residual plot:

Diagnostic Residual Plot for Delivery Time Model by Destination

No obvious problems in this diagnostic plot.

We could also consider dividing the data by delivery service to get a different view of the residuals:

ggplot(delivery.res, aes(M1.Fit, M1.Resid, colour = Destination)) +
  geom_point() + xlab("Fitted Values") + ylab("Residuals") +
  facet_wrap( ~ Service)

This creates the following graph:

Diagnostic Residual Plot for Delivery Time Model by Service

Again there is nothing substantial here to lead us to consider an alternative analysis.

Lastly we consider the normal probability plot of the model residuals, using the stat_qq() option:

ggplot(delivery.res, aes(sample = M1.Resid)) + stat_qq()

The quantile plot is:

Normal Probability Plot for Delivery Time Model

This plot is very close to the straight line we would expect to observe if the data was a close approximation to a normal distribution. To round off the analysis we look at the Tukey HSD multiple comparisons to confirm that the differences are between delivery service 1 and the other two competing services:

> TukeyHSD(delivery.mod1, which = "Service")
  Tukey multiple comparisons of means
    95% family-wise confidence level
 
Fit: aov(formula = Time ~ Destination * Service, data = delivery.df)
 
$Service
                        diff        lwr       upr     p adj
Carrier 2-Carrier 1 1.498667  1.2576092 1.7397241 0.0000000
Carrier 3-Carrier 1 1.544667  1.3036092 1.7857241 0.0000000
Carrier 3-Carrier 2 0.046000 -0.1950575 0.2870575 0.8856246

Even with the multiple comparison post-hoc adjustment there is very strong evidence for the differences that we have consistenly observed throughout the analysis.

We can use ggplot to visualise the difference in mean delivery time for the services and the 95% confidence intervals on these differences. We create a data frame from the TukeyHSD output by extracting the component relating to the delivery service comparison and add the text labels by extracting the row names from the data frame.

delivery.hsd = data.frame(TukeyHSD(delivery.mod1, which = "Service")$Service)
delivery.hsd$Comparison = row.names(delivery.hsd)

We then use the geom_pointrange() to specify lower, middle and upper values based on the three pairwise comparisons of interest.

ggplot(delivery.hsd, aes(Comparison, y = diff, ymin = lwr, ymax = upr)) +
  geom_pointrange() + ylab("Difference in Mean Delivery Time by Service") +
  coord_flip()

The coord_flip() is used to make the confidence intervals horizontal rather than vertical on the graph. This can be confusing for creating the axis labels as we specify the label where it would appear prior to the filp of coordinates. In the example above we add text to the y axis but this now appears on the x axis in the final graph:

Plot of Confidence Intervals for Mean Differences using Tukey HSD

Fast Tube by Casper

Fast Tube by Casper

In one-way ANOVA the data is sub-divided into groups based on a single classification factor and the standard terminology used to describe the set of factor levels is treatment even though this might not always have meaning for the particular application. There is variation in the measurements taken on the individual components of the data set and ANOVA investigates whether this variation can be explained by the grouping introduced by the classification factor.

As an example we consider one of the data sets available with R relating to an experiment into plant growth. The purpose of the experiment was to compare the yields on the plants for a control group and two treatments of interest. The response variable was a measurement taken on the dried weight of the plants.

The first step in the investigation is to take a copy of the data frame so that we can make some adjustments as necessary while leaving the original data alone. We use the factor function to re-define the labels of the group variables that will appear in the output and graphs:

plant.df = PlantGrowth
plant.df$group = factor(plant.df$group,
  labels = c("Control", "Treatment 1", "Treatment 2"))

The labels argument is a list of names corresponding to the levels of the group factor variable.

A boxplot of the distributions of the dried weights for the three competing groups is created using the ggplot package:

require(ggplot2)
 
ggplot(plant.df, aes(x = group, y = weight)) +
  geom_boxplot(fill = "grey80", colour = "blue") +
  scale_x_discrete() + xlab("Treatment Group") +
  ylab("Dried weight of plants")

The geom_boxplot() option is used to specify background and outline colours for the boxes. The axis labels are created with the xlab() and ylab() options. The plot that is produce looks like this:

Initial inspection of the data suggests that there are differences in the dried weight for the two treatments but it is not so clear cut to determine whether the treatments are different to the control group. To investigate these differences we fit the one-way ANOVA model using the lm function and look at the parameter estimates and standard errors for the treatment effects. The function call is:

plant.mod1 = lm(weight ~ group, data = plant.df)

We save the model fitted to the data in an object so that we can undertake various actions to study the goodness of the fit to the data and other model assumptions. The standard summary of a lm object is used to produce the following output:

> summary(plant.mod1)
 
Call:
lm(formula = weight ~ group, data = plant.df)
 
Residuals:
    Min      1Q  Median      3Q     Max 
-1.0710 -0.4180 -0.0060  0.2627  1.3690 
 
Coefficients:
                 Estimate Std. Error t value Pr(>|t|)    
(Intercept)        5.0320     0.1971  25.527   <2e-16 ***
groupTreatment 1  -0.3710     0.2788  -1.331   0.1944    
groupTreatment 2   0.4940     0.2788   1.772   0.0877 .  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 
 
Residual standard error: 0.6234 on 27 degrees of freedom
Multiple R-squared: 0.2641,     Adjusted R-squared: 0.2096 
F-statistic: 4.846 on 2 and 27 DF,  p-value: 0.01591

The model output indicates some evidence of a difference in the average growth for the 2nd treatment compared to the control group. An analysis of variance table for this model can be produced via the anova command:

> anova(plant.mod1)
Analysis of Variance Table
 
Response: weight
          Df  Sum Sq Mean Sq F value  Pr(>F)  
group      2  3.7663  1.8832  4.8461 0.01591 *
Residuals 27 10.4921  0.3886                  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

This table confirms that there are differences between the groups which were highlighted in the model summary. The function confint is used to calculate confidence intervals on the treatment parameters, by default 95% confidence intervals:

> confint(plant.mod1)
                       2.5 %    97.5 %
(Intercept)       4.62752600 5.4364740
groupTreatment 1 -0.94301261 0.2010126
groupTreatment 2 -0.07801261 1.0660126

The model residuals can be plotted against the fitted values to investigate the model assumptions. First we create a data frame with the fitted values, residuals and treatment identifiers:

plant.mod = data.frame(Fitted = fitted(plant.mod1),
  Residuals = resid(plant.mod1), Treatment = plant.df$group)

and then produce the plot:

ggplot(plant.mod, aes(Fitted, Residuals, colour = Treatment)) + geom_point()

which produces this graph:

We can see that there is no major problem with the diagnostic plot but some evidence of different variabilities in the spread of the residuals for the three treatment groups.