To create the league table we need to count various things such as the number of games played, number of wins/draws/losses, goals scored etc. This information is available in the results object that is loaded from a csv file in the function as it stands.

To facilitate these calculations we create a data frame with a row for each team in the division and then calculate the statistics required – this was a reason for ordering the factors in the HomeTeam and AwayTeam columns of the results table. The data frame is created with the code below:

tmpTable = data.frame(Team = teams,
    Games = 0, Win = 0, Draw = 0, Loss = 0,
    HomeGames = 0, HomeWin = 0, HomeDraw = 0, HomeLoss = 0,
    AwayGames = 0, AwayWin = 0, AwayDraw = 0, AwayLoss = 0,
    Points = 0,
    HomeFor = 0, HomeAgainst = 0,
    AwayFor = 0, AwayAgainst = 0,
    For = 0, Against = 0, GoalDifference = 0)

There are a number of slots that are may be redundant in a league table but are used for intermediate calculations, such as HomeWin and AwayWin that are combined to find the total number of victories for a team.

The number of games played by each team home and away are counted using the table command for the two columns respectively.

tmpTable$HomeGames = as.numeric(table(tmpResults$HomeTeam))
tmpTable$AwayGames = as.numeric(table(tmpResults$AwayTeam))

The labels created by the table command are discarded using the as.numeric function to retain only the number of games. The table command is also used to count the number of wins, draws and losses at home and away for each team. The commands are shown here:

tmpTable$HomeWin =
    as.numeric(table(tmpResults$HomeTeam[tmpResults$FTR == "H"]))
tmpTable$HomeDraw =
    as.numeric(table(tmpResults$HomeTeam[tmpResults$FTR == "D"]))
tmpTable$HomeLoss =
    as.numeric(table(tmpResults$HomeTeam[tmpResults$FTR == "A"]))
 
tmpTable$AwayWin =
    as.numeric(table(tmpResults$AwayTeam[tmpResults$FTR == "A"]))
tmpTable$AwayDraw =
    as.numeric(table(tmpResults$AwayTeam[tmpResults$FTR == "D"]))
tmpTable$AwayLoss =
    as.numeric(table(tmpResults$AwayTeam[tmpResults$FTR == "H"]))

Note that we subset on the values in the FTR column, which is full-time result, and then count. The subsetting is reversed when looking at the away fixtures because a victory for the team is now an away win rather than a home win.

This information is then combined to get total games played, won etc.

tmpTable$Games = tmpTable$HomeGames + tmpTable$AwayGames
tmpTable$Win = tmpTable$HomeWin + tmpTable$AwayWin
tmpTable$Draw = tmpTable$HomeDraw + tmpTable$AwayDraw
tmpTable$Loss = tmpTable$HomeLoss + tmpTable$AwayLoss

The total points is calclated by multiplying the number of wins, draws and losses by the number of points awarded for each match outcome.

tmpTable$Points = winPoints * tmpTable$Win +
    drawPoints * tmpTable$Draw + lossPoints * tmpTable$Loss

The next set of calculations are to count the number of goals scored, goals conceeded and goal difference. The tapply function is used for these calculations.

tmpTable$HomeFor =
    as.numeric(tapply(tmpResults$FTHG, tmpResults$HomeTeam, sum, na.rm = TRUE))
tmpTable$HomeAgainst =
    as.numeric(tapply(tmpResults$FTAG, tmpResults$HomeTeam, sum, na.rm = TRUE))
 
tmpTable$AwayFor =
    as.numeric(tapply(tmpResults$FTAG, tmpResults$AwayTeam, sum, na.rm = TRUE))
tmpTable$AwayAgainst =
    as.numeric(tapply(tmpResults$FTHG, tmpResults$AwayTeam, sum, na.rm = TRUE))

The tapply function applies the sum to the number of goals scored at home or away, and the number of goals conceeded by each team in the division. These are then combined to create totals home and away:

tmpTable$For =
    ifelse(is.na(tmpTable$HomeFor), 0, tmpTable$HomeFor) +
    ifelse(is.na(tmpTable$AwayFor), 0, tmpTable$AwayFor)
tmpTable$Against =
    ifelse(is.na(tmpTable$HomeAgainst), 0, tmpTable$HomeAgainst) +
    ifelse(is.na(tmpTable$AwayAgainst), 0, tmpTable$AwayAgainst)

The ifelse statement is used to handle situations where a team hasn’t played a home and/or away fixture yet. The goal difference is easy to calculate:

tmpTable$GoalDifference = tmpTable$For - tmpTable$Against

Now that all of the statistics have been calculated we sort the table based on the number of points, goal difference and finally alphabetically. There might be different ways that we can order the teams but this is what we will use for the time being:

tmpTable =
  tmpTable[order(- tmpTable$Points, - tmpTable$GoalDifference, tmpTable$Team),]

The ordering might look odd but we want to ranking from highest to lowest points and goal difference but then in ascending alphabetical order for the teams.

The whole function is now:

football.process.v2 = function(datafile, country, divname, season, teams, winPoints = 3, drawPoints = 1, lossPoints = 0)
{
## Validation Function Arguments
 
if (missing(datafile))
{
stop("Results csv file not specified.")
}
 
if (missing(country))
{
warning("Country of league not specified.")
country = ""
}
 
if (missing(divname))
{
warning("Name of league division not specified.")
divname = ""
}
 
## Import Results
 
tmpResults = read.csv(datafile)[,c("Date","HomeTeam","AwayTeam","FTR","FTHG","FTAG")]
 
if (missing(teams))
{
warning("Team names not specified - extracted from results data.")
teams = sort(unique(c(as.character(tmpResults$HomeTeam), as.character(tmpResults$AwayTeam))))
}
 
tmpResults$HomeTeam = factor(tmpResults$HomeTeam, levels = teams)
tmpResults$AwayTeam = factor(tmpResults$AwayTeam, levels = teams)
 
## Create Empty League Table
 
tmpTable = data.frame(Team = teams,
Games = 0, Win = 0, Draw = 0, Loss = 0,
HomeGames = 0, HomeWin = 0, HomeDraw = 0, HomeLoss = 0,
AwayGames = 0, AwayWin = 0, AwayDraw = 0, AwayLoss = 0,
Points = 0,
HomeFor = 0, HomeAgainst = 0,
AwayFor = 0, AwayAgainst = 0,
For = 0, Against = 0, GoalDifference = 0)
 
## Count Number of Games Played
 
tmpTable$HomeGames = as.numeric(table(tmpResults$HomeTeam))
tmpTable$AwayGames = as.numeric(table(tmpResults$AwayTeam))
 
## Count Number of Wins/Draws/Losses
 
tmpTable$HomeWin = as.numeric(table(tmpResults$HomeTeam[tmpResults$FTR == "H"]))
tmpTable$HomeDraw = as.numeric(table(tmpResults$HomeTeam[tmpResults$FTR == "D"]))
tmpTable$HomeLoss = as.numeric(table(tmpResults$HomeTeam[tmpResults$FTR == "A"]))
 
tmpTable$AwayWin = as.numeric(table(tmpResults$AwayTeam[tmpResults$FTR == "A"]))
tmpTable$AwayDraw = as.numeric(table(tmpResults$AwayTeam[tmpResults$FTR == "D"]))
tmpTable$AwayLoss = as.numeric(table(tmpResults$AwayTeam[tmpResults$FTR == "H"]))
 
tmpTable$Games = tmpTable$HomeGames + tmpTable$AwayGames
tmpTable$Win = tmpTable$HomeWin + tmpTable$AwayWin
tmpTable$Draw = tmpTable$HomeDraw + tmpTable$AwayDraw
tmpTable$Loss = tmpTable$HomeLoss + tmpTable$AwayLoss
tmpTable$Points = winPoints * tmpTable$Win + drawPoints * tmpTable$Draw + lossPoints * tmpTable$Loss
 
## Count Goals Scored and Conceeded
 
tmpTable$HomeFor = as.numeric(tapply(tmpResults$FTHG, tmpResults$HomeTeam, sum, na.rm = TRUE))
tmpTable$HomeAgainst = as.numeric(tapply(tmpResults$FTAG, tmpResults$HomeTeam, sum, na.rm = TRUE))
 
tmpTable$AwayFor = as.numeric(tapply(tmpResults$FTAG, tmpResults$AwayTeam, sum, na.rm = TRUE))
tmpTable$AwayAgainst = as.numeric(tapply(tmpResults$FTHG, tmpResults$AwayTeam, sum, na.rm = TRUE))
 
tmpTable$For = ifelse(is.na(tmpTable$HomeFor), 0, tmpTable$HomeFor) +
ifelse(is.na(tmpTable$AwayFor), 0, tmpTable$AwayFor)
tmpTable$Against = ifelse(is.na(tmpTable$HomeAgainst), 0, tmpTable$HomeAgainst) +
ifelse(is.na(tmpTable$AwayAgainst), 0, tmpTable$AwayAgainst)
 
tmpTable$GoalDifference = tmpTable$For - tmpTable$Against
 
## Sort Table
## By Points
## By Goal Difference
## By Team Name (Alphabetical)
 
tmpTable = tmpTable[order(- tmpTable$Points, - tmpTable$GoalDifference, tmpTable$Team),]
 
tmpTable = tmpTable[,c("Team", "Games", "Win", "Draw", "Loss", "Points", "For", "Against", "GoalDifference")]
 
## Return Division Information
 
tmpSummary = list(Country = country, Division = divname, Season = season, Teams = teams,
Results = tmpResults, Table = tmpTable)
 
invisible(tmpSummary)
}

There are other functionality that we might want to add to the function.

The first step is to consider what control options should be available as part of the function and here is a list of some arguments that will be used for this implementation of a football result data processing function:

• The name of a csv data file from the football-data.co.uk website.
• A text string to specify the country and division for the data.
• A text string specifying the season.
• A list of teams in the division (optional), which could be used to test for data entry errors in the data file.
• The number of points for a win, draw or loss. This might seem a strange option initially but different leagues might award different points for the three outcomes.

Some of this information might appear optional but is included so that we can write a custom print function at a later date to display a meaningful summary of the object (list) that will be created by the function.

The first part of our function is concerned with checking the various values provided to the function arguments. Our skeleton function is as follows:

football.process.v1 = function(datafile, country, divname, season,
  teams, winPoints = 3, drawPoints = 1, lossPoints = 0)
{
 
}

Here we have specified default options for three of the arguments with the most likely number of points for each match outcome, i.e. 3 points for a win and 1 point for a draw.

To illustrate the working of the result processing function we will use a small exert from the start of the 2010/2011 English Premiership season which is shown below:

Div,Date,HomeTeam,AwayTeam,FTHG,FTAG,FTR,HTHG,HTAG,HTR,Referee
E0,14/8/2010,Aston Villa,West Ham,3,0,H,2,0,H,M Dean
E0,14/8/2010,Blackburn,Everton,1,0,H,1,0,H,P Dowd
E0,14/8/2010,Bolton,Fulham,0,0,D,0,0,D,S Attwell
E0,14/8/2010,Chelsea,West Brom,6,0,H,2,0,H,M Clattenburg
E0,14/8/2010,Sunderland,Birmingham,2,2,D,1,0,H,A Taylor
E0,14/8/2010,Tottenham,Man City,0,0,D,0,0,D,A Marriner
E0,14/8/2010,Wigan,Blackpool,0,4,A,0,3,A,M Halsey
E0,14/8/2010,Wolves,Stoke,2,1,H,2,0,H,L Probert
E0,15/8/2010,Liverpool,Arsenal,1,1,D,0,0,D,M Atkinson
E0,16/8/2010,Man United,Newcastle,3,0,H,2,0,H,C Foy
E0,21/8/2010,Arsenal,Blackpool,6,0,H,3,0,H,M Jones
E0,21/8/2010,Birmingham,Blackburn,2,1,H,0,0,D,M Oliver
E0,21/8/2010,Everton,Wolves,1,1,D,1,0,H,L Mason
E0,21/8/2010,Stoke,Tottenham,1,2,A,1,2,A,C Foy
E0,21/8/2010,West Brom,Sunderland,1,0,H,0,0,D,K Friend
E0,21/8/2010,West Ham,Bolton,1,3,A,0,0,D,A Marriner
E0,21/8/2010,Wigan,Chelsea,0,6,A,0,1,A,M Dean
E0,22/8/2010,Fulham,Man United,2,2,D,0,1,A,P Walton
E0,22/8/2010,Newcastle,Aston Villa,6,0,H,3,0,H,M Atkinson
E0,23/8/2010,Man City,Liverpool,3,0,H,1,0,H,P Dowd
E0,28/8/2010,Blackburn,Arsenal,1,2,A,1,1,D,C Foy
E0,28/8/2010,Blackpool,Fulham,2,2,D,0,1,A,M Oliver
E0,28/8/2010,Chelsea,Stoke,2,0,H,1,0,H,M Atkinson
E0,28/8/2010,Man United,West Ham,3,0,H,1,0,H,M Clattenburg
E0,28/8/2010,Tottenham,Wigan,0,1,A,0,0,D,P Dowd
E0,28/8/2010,Wolves,Newcastle,1,1,D,1,0,H,S Attwell
E0,29/8/2010,Aston Villa,Everton,1,0,H,1,0,H,M Jones
E0,29/8/2010,Bolton,Birmingham,2,2,D,0,1,A,K Friend
E0,29/8/2010,Liverpool,West Brom,1,0,H,0,0,D,L Probert
E0,29/8/2010,Sunderland,Man City,1,0,H,0,0,D,M Dean

This is stored in a file E0test.csv so that we can use the read.csv function to import the results data and then process it.

The first series of commands that we add to the function are for checking various function arguments specified by the user to ensure that they are sensible. First up we check whether a results data file has been specified as we cannot do any processing without any results. The simple test is for whether a file name has been specified:

if (missing(datafile))
{
    stop("Results csv file not specified.")
}

It might be sensible to check whether the object datafile is actually a character string specifying a file, but this hasn’t been done for now. We then check whether the country name and division have been specified and set them to blank strings if they haven’t been set by the user.

if (missing(country))
{
    warning("Country of league not specified.")
    country = ""
}
 
if (missing(divname))
{
    warning("Name of league division not specified.")
    divname = ""
}

Next up we import the data file and only save the columns of interest (at this point of the development of the function at least. There are many more columns of information that we need in the raw data from the website,

tmpResults =
    read.csv(datafile)[,c("Date","HomeTeam","AwayTeam","FTR","FTHG","FTAG")]

The square brackets are used to subset on a part set of columns and only save these. Then we check whether the team names have been specified by the user and if not extract them from the data provided:

if (missing(teams))
{
    warning("Team names not specified - extracted from results data.")
    teams = sort(unique(c(as.character(tmpResults$HomeTeam),
        as.character(tmpResults$AwayTeam))))
}

The sort function is used to order the team names alphabetically which is the order often used in league tables, especially when no games have been played. We then convert the columns HomeTeam and AwayTeam into factors, which allows teams that haven’t played a fixture yet to be included in the table.

tmpResults$HomeTeam = factor(tmpResults$HomeTeam, levels = teams)
tmpResults$AwayTeam = factor(tmpResults$AwayTeam, levels = teams)

To round off the first part of creating the result processing function we create a list object to return at the end of the function.

tmpSummary = list(Country = country, Division = divname,
    Season = season, Teams = teams, Results = tmpResults)

The function so far:

football.process.v1 = function(datafile, country, divname, season, teams, winPoints = 3, drawPoints = 1, lossPoints = 0)
{
## Validation Function Arguments
 
if (missing(datafile))
{
stop("Results csv file not specified.")
}
 
if (missing(country))
{
warning("Country of league not specified.")
country = ""
}
 
if (missing(divname))
{
warning("Name of league division not specified.")
divname = ""
}
 
## Import Results
 
tmpResults = read.csv(datafile)[,c("Date","HomeTeam","AwayTeam","FTR","FTHG","FTAG")]
 
if (missing(teams))
{
warning("Team names not specified - extracted from results data.")
teams = sort(unique(c(as.character(tmpResults$HomeTeam), as.character(tmpResults$AwayTeam))))
}
 
tmpResults$HomeTeam = factor(tmpResults$HomeTeam, levels = teams)
tmpResults$AwayTeam = factor(tmpResults$AwayTeam, levels = teams)
 
## Return Division Information
 
tmpSummary = list(Country = country, Division = divname, Season = season, Teams = teams,
Results = tmpResults)
 
invisible(tmpSummary)
}

We then test this function with the data file shown above. First up we create our own list of teams in the English Premiership for 2010/2011 and specify some of the other function arguments while using the defaults for points.

> E0teams.1011 = c("Arsenal", "Aston Villa", "Birmingham", "Blackburn",
+ "Blackpool", "Bolton", "Chelsea", "Everton", "Fulham", "Liverpool",
+ "Man City", "Man United", "Newcastle", "Stoke", "Sunderland",
+ "Tottenham", "West Brom", "West Ham", "Wigan", "Wolves")
> print(football.process.v1("E0test.csv", "England", "Premiership",
    "2010-2011", E0teams.1011))
$Country
[1] "England"
 
$Division
[1] "Premiership"
 
$Season
[1] "2010-2011"
 
$Teams
 [1] "Arsenal"     "Aston Villa" "Birmingham"  "Blackburn"   "Blackpool"  
 [6] "Bolton"      "Chelsea"     "Everton"     "Fulham"      "Liverpool"  
[11] "Man City"    "Man United"  "Newcastle"   "Stoke"       "Sunderland" 
[16] "Tottenham"   "West Brom"   "West Ham"    "Wigan"       "Wolves"     
 
$Results
        Date    HomeTeam    AwayTeam FTR FTHG FTAG
1  14/8/2010 Aston Villa    West Ham   H    3    0
2  14/8/2010   Blackburn     Everton   H    1    0
3  14/8/2010      Bolton      Fulham   D    0    0
4  14/8/2010     Chelsea   West Brom   H    6    0
5  14/8/2010  Sunderland  Birmingham   D    2    2
6  14/8/2010   Tottenham    Man City   D    0    0
7  14/8/2010       Wigan   Blackpool   A    0    4
8  14/8/2010      Wolves       Stoke   H    2    1
9  15/8/2010   Liverpool     Arsenal   D    1    1
10 16/8/2010  Man United   Newcastle   H    3    0
11 21/8/2010     Arsenal   Blackpool   H    6    0
12 21/8/2010  Birmingham   Blackburn   H    2    1
13 21/8/2010     Everton      Wolves   D    1    1
14 21/8/2010       Stoke   Tottenham   A    1    2
15 21/8/2010   West Brom  Sunderland   H    1    0
16 21/8/2010    West Ham      Bolton   A    1    3
17 21/8/2010       Wigan     Chelsea   A    0    6
18 22/8/2010      Fulham  Man United   D    2    2
19 22/8/2010   Newcastle Aston Villa   H    6    0
20 23/8/2010    Man City   Liverpool   H    3    0
21 28/8/2010   Blackburn     Arsenal   A    1    2
22 28/8/2010   Blackpool      Fulham   D    2    2
23 28/8/2010     Chelsea       Stoke   H    2    0
24 28/8/2010  Man United    West Ham   H    3    0
25 28/8/2010   Tottenham       Wigan   A    0    1
26 28/8/2010      Wolves   Newcastle   D    1    1
27 29/8/2010 Aston Villa     Everton   H    1    0
28 29/8/2010      Bolton  Birmingham   D    2    2
29 29/8/2010   Liverpool   West Brom   H    1    0
30 29/8/2010  Sunderland    Man City   H    1    0

Other useful resources are provided on the Supplementary Material page.

As an initial example we will consider the English legend Sir Ian Botham who played 102 test matches for England between his debut in 1977 until his final game in 1992.

The first obvious breakdown is to consider how Botham performed against the six countries that he played against during his test career. A summary of his statistics are shown here:

 Opposition Matches Bat Inns Runs NO Bowl Inns Wicket Catch
  Australia      36       49 1673  2       66     148    57
      India      14       16 1201  0       23      59    14
New Zealand      15       22  846  2       28      64    14
   Pakistan      14       20  647  1       18      40    14
  Sri Lanka       3        3   41  0        6      11     2
West Indies      20       37  792  1       27      61    19

Botham only played three matches against Sri Lanka so it is difficult to properly assess his performance against them. If the above table is stored in a data frame itb.opp then we can create a histogram of the total runs (or wickets) by opposition country:

ggplot(itb.opp, aes(Opposition, Runs)) + geom_bar() + xlab("Country") +
  ylab("Total Runs")

This code produces the following graph:

IT Botham Total Runs by Opposition

The total wickes graph is produced by the next code:

ggplot(itb.opp, aes(Opposition, Wicket)) + geom_bar() + xlab("Country") +
  ylab("Total Wickets")

IT Botham Total Wickets by Opposition

We may now want to delve deeper into the performance against different nations to take into account the number of games or innings where Botham batted or bowled. The traditional way to assess performance is to calculate batting and bowling averages and we can do this by opposition which provides the following data frame:

> itb.opp.sum
 Opposition Discipline  Average
  Australia    Batting 29.35088
      India    Batting 70.64706
New Zealand    Batting 42.30000
   Pakistan    Batting 32.35000
  Sri Lanka    Batting 13.66667
West Indies    Batting 21.40541
  Australia    Bowling 27.65541
      India    Bowling 26.40678
New Zealand    Bowling 23.43750
   Pakistan    Bowling 31.77500
  Sri Lanka    Bowling 28.18182
West Indies    Bowling 35.18033

This can be converted into a dot plot so we can see whether Botham had a high batting average than bowling average, which is often taken to be one of the signs of an all-rounder.

ggplot(itb.opp.sum, aes(Average, Opposition, colour = Discipline)) +
  geom_point()+ xlab("Average") + ylab("")

The graph is shown here:

IT Botham Batting and Bowling Averages by Opposition

We can see the differences in performance based on the opposition. Botham’s performance against the West Indies, by far the strongest team during most of his international career, were worse than against the other countries. However, his averages were far from embarassing when compared to other players at the time. The graph also shows that Botham enjoyed batting and bowling against India.

We can divide this data further based on whether the matches were played in England or outside of England and this data is shown here:

> itb.opp.ha.sum
  Opposition Venue Discipline  Average
   Australia  Away    Batting 30.22581
       India  Away    Batting 61.55556
 New Zealand  Away    Batting 50.44444
    Pakistan  Away    Batting 16.00000
   Sri Lanka  Away    Batting 13.00000
 West Indies  Away    Batting 14.17647
   Australia  Home    Batting 28.30769
       India  Home    Batting 80.87500
 New Zealand  Home    Batting 35.63636
    Pakistan  Home    Batting 34.16667
   Sri Lanka  Home    Batting 14.00000
 West Indies  Home    Batting 27.55000
   Australia  Away    Bowling 28.44928
       India  Away    Bowling 25.53333
 New Zealand  Away    Bowling 27.44444
    Pakistan  Away    Bowling 45.00000
   Sri Lanka  Away    Bowling 21.66667
 West Indies  Away    Bowling 39.50000
   Australia  Home    Bowling 26.96203
       India  Home    Bowling 27.31034
 New Zealand  Home    Bowling 20.51351
    Pakistan  Home    Bowling 31.07895
   Sri Lanka  Home    Bowling 30.62500
 West Indies  Home    Bowling 31.97143

A dot plot is created from this data with a separate panel for each of the six opposition countries and the averages divided into batting and bowling performances. The coloured dots in the graph indicated whether the average is for matches at home or away.

ggplot(itb.opp.ha.sum, aes(Average, Discipline, colour = Venue)) +
  geom_point() + facet_wrap( ~ Opposition) +
  xlab("Batting Average") + ylab("")

This graph is shown below:

IT Botham Batting and Bowling Averages by Country and Home/Away

We can see that the difference between home and away peformance is, in general, not very large for bowling averages but in some cases there is a noticeable difference in batting averages. When looking at Botham’s performances against the West Indies his statistics at home are much better than his away performance, suggesting that his main struggles against the strong West Indies team were in the Caribbean. This might be due to his swing bowling being more suitable to English conditions compared to pitches in the West Indies.

To round off this brief look at the career of IT Botham let us consider some other important statistics, in particular games where he performed with the bat and ball.

• Overall Botham scored 14 hundreds and 22 fifties out of 161 innings so he reached fifty runs every five innings or so.
• He also took 27 five wicket hauls and 17 four wicket hauls so he took four or more wickets every four innings or so.
• He took 120 catches.

Individual matches of excellence include five games with a century and at least five wickets:

Year  Opposition       Ground Venue Runs Wicket
1978 New Zealand Christchurch  Away  133      8
1978    Pakistan       Lord's  Home  108      8
1980       India       Mumbai  Away  114     13
1981   Australia        Leeds  Home  199      7
1984 New Zealand   Wellington  Away  138      6

These performances and others show why Botham was considered such a great player as he produced some sustained periods of excellent all-round cricket rather than having one discipline more dominant for a long period of time.

These tolerance limits, taken from the estimated interval, are limits within which a stated proportion of the population is expected to occur. The function normtol.int from the tolerance package can be used to calculate a tolerance interval for data from a normal distribution.

The function arguments include the data itself in a vector denoted x. The confidence level associated with the tolerance interval is specified by alpha, where alpha is the difference between 100% and the confidence level – alpha is 0.05 for 95% confidence. The argument P is the proportion of the data to be included in the tolerance interval. The side argument determines whether a one-sided or two-sided interval is required.

Consider a simulated set of data from a manufacturing process loaded into R, stored as vector object obs, as follows:

obs = c(102.17, 102.45, 106.23, 98.16, 100.82, 101.40, 90.51, 102.51, 97.93,
  96.98, 101.74, 104.34, 103.50, 94.72, 102.80, 103.92, 97.43, 102.76, 100.03,
  107.12, 104.96, 105.32, 87.06, 97.89, 100.23)

A 95% tolerance interval for 90% of data of this type, based on the 25 observations above is created with this code:

> normtol.int(x = obs, alpha = 0.05, P = 0.90, side = 2)
  alpha   P    x.bar 2-sided.lower 2-sided.upper
1  0.05 0.9 100.5192      90.07606      110.9623

The alpha and P are as noted above and the average of the data is reported along with the lower and upper tolerance intervals in this case as we asked for a two-sided interval. This can be easily changed to cover 95% rather than 90% of the data:

> normtol.int(x = obs, alpha = 0.05, P = 0.95, side = 2)
  alpha    P    x.bar 2-sided.lower 2-sided.upper
1  0.05 0.95 100.5192      88.07543      112.9630

The package tolerance can create intervals for other data distributions.

The lattice library has a function bwplot that can be used to create a box and whisker plot for a some data and using the standard mechanism individual plots can be produced for different factors levels to divide up the data into meaningful groups.

As an example we can use the olive oil data to produce a box and whisker summary of the palmitic variable for each of the areas in the data:

library(lattice)
bwplot(Area ~ palmitic, data = olive.df)

The first argument is a formula to describe the variables to include in the plot and because Area is a factor the function interprets this to mean that we want a separate summary for each of the levels of this factor. The graph produced looks like this:

Olive Oil Data Box and Whisker Summary

This type of plot can also be useful when exploring residuals from a fitted model.

The histogram is a commonly used graphical display used to summarised univariate data and it provides a visual indication of the location and variation in the data. Histograms are constructed by dividing the data into ranges and count the number of data points that occur in each range and the height of the bar is based on this information.

We can create a histogram using either the base graphics or lattice graphics in R. The function hist is part of the base graphics and the first argument we specify in the function call is the actual data to be used in the histogram. An example of creating a histogram would use the following code:

hist(olive.df$palmitic, xlab = "Palmitic", main = "Histogram")

In this example we have also specified a label for the x-axis as well as the main title. The resulting graph looks like this:

Demonstration of using a histogram to summarise data

We can make use of the histogram function in the lattice library to create this plot and the syntax that we use is slightly different.

histogram( ~ palmitic, data = olive.df)

The first argument is a model formula that specifies that data to be used for the histogram as the independent variable component of the formula and the data argument is used to specify a data frame in which the function will look for the data. The histogram looks slightly different using this library:

Demonstration of using a histogram to summarise data

There are other types of graph that can be used to summarise univariate data which include the bow and whisker plot, density plot, strip plot or dot plot. These will be covered in subsequent posts either using the base graphics system or lattice graphics.

There are four functions that are defined for each distribution that is available within R. These functions are:

• The density function – name starts with a d.
• The cumulative density function – name starts with a p.
• The quantile function – name starts with a q.
• Random number generation – name starts with a r.

Both discrete and continuous distributions are available in R. Distributions that we can access include: Beta, Binomial, Chi-squared, F, Logistic, Normal, Poisson, Student’s t and Weibull.

There is a base name for each of the distributions and we use the suffix letter mentioned above to access the requisite information. If we consider the Normal distribution as an example then dnorm is the function that will provide the density:

> dnorm(1.96, mean = 0, sd = 1)
[1] 0.05844094

The mean and sd arguments are used to specify a particular pair of parameters for the Normal distribution. The cumulative distribution function is pnorm and the syntax is very similar to the dnorm function:

> pnorm(1.96, mean = 0, sd = 1)
[1] 0.9750021

The default option is for the function to return the cumulative probability for values less than the specified figure. The quantile function, qnorm, allows us to work back from probabilities to values on the original data scale:

> qnorm(0.95, mean = 0, sd = 1)
[1] 1.644854

As with the previous functions we can definition the parameters of the distribution where required. The last option of interest is the function that allows us to generate random samples for a particular distribution, which in the case of the Normal distribution is rnorm. Here we specify the number of samples to be drawn from the distribution along with the parameters of the distribution:

> rnorm(n = 20, mean = 0, sd = 1)
 [1] -1.1322606 -2.8320170 -0.5768220  1.0569513  1.0824524  1.4925396 -0.3010086 -0.4345893  2.6813322
[10]  0.3774106  1.7226911  0.5922038  0.0770510  1.4015955 -0.9998051  0.1924921  0.7181194  1.0107967
[19]  1.3224979 -0.1511634

The previous command samples twenty observations from the standard Normal distribution.

There is a function called apply that can be used to run a specific function on each of the rows or columns individually. For example we could calculate row or column means or variances using the apply or we could define a more complicated function that is more appropriate for the statistics that we want to calculate. If we take a look at the Olive oil data used in some of the other posts we might be interested in calculating variable (columns in this case) means and we would use this code:

apply(olive.df[,c("palmitic", "palmitoleic", "stearic", "oleic", "linoleic",
  "linolenic", "arachidic", "eicosenoic")], 2, mean)

The first thing we do is indicate which columns that we are interested in as the Region and Area are not important for these mean calculations – the square brackets are used to specify a subset of our data frame and we provide a vector of column names after the comma. The output from this function call is:

   palmitic palmitoleic     stearic       oleic    linoleic   linolenic   arachidic  eicosenoic 
 1231.74126   126.09441   228.86538  7311.74825   980.52797    31.88811    58.09790    16.28147

We could quite easily adjust this function call to use a different function on the data. Let’s say that we are interested in the maximum values for each variable then we would replace mean with max:

apply(olive.df[,c("palmitic", "palmitoleic", "stearic", "oleic", "linoleic",
  "linolenic", "arachidic", "eicosenoic")], 2, max)

which returns:

   palmitic palmitoleic     stearic       oleic    linoleic   linolenic   arachidic  eicosenoic 
       1753         280         375        8410        1470          74         105          58

There are other associated functions – tapply, lapply and sapply – that perform on a similar routine on different types and format of data which will be discussed in subsequent posts.

Fast Tube by Casper

The main two functions that are used to produce contingency tables are table and xtabs. We can use these two functions to get one, two or higher dimension tables that summarise the number of records that correspond to the combination of variables used to create the table.

The simplest case is where we are interested into a summary based on a single variable and the syntax is straightforward. The function table takes a single argument that corresponds to a vector of data. For example, if we are working with a data frame based on an unbalanced design and wanted to count the number of observations corresponding to each treatment we might run some code like:

table(temp.design3$Treatment)

which would produce a simple summary table:

A B C D 
7 7 3 7

If there was a second factor in the data set corresponding to different plots, labelled 1 to 4, then we could generate a two dimensional contingency table by adding a second argument to the function call like:

table(temp.design3$Treatment, temp.design3$Plot)

and the output would be of the form:

    1 2 3 4
  A 2 2 2 1
  B 2 2 2 1
  C 1 1 1 0
  D 2 2 2 1

The function xtabs can be used to create the same contingency tables but the function works using a formula in a similar vein to the modelling functions. So to get the one dimensional table we would write code similar to this:

xtabs(~ Plot, data = temp.design3)

which would summarise the data by plot:

Plot
1 2 3 4 
7 7 7 3

Note that the output is slightly different to using the table function. The two dimensional table would be created like this:

xtabs(~ Treatment + Plot, data = temp.design3)

and the output would be:

         Plot
Treatment 1 2 3 4
        A 2 2 2 1
        B 2 2 2 1
        C 1 1 1 0
        D 2 2 2 1

These functions can be extended to higher dimensions and the output is based on 2×2 tables for each combination of the other variables.

Fast Tube by Casper

In this post we will consider the numerical and tabular summaries of data of various types, e.g. numeric, categorical etc. The function mean calculates the average value of a vector of numeric data. As an example we could run the following:

> mean(CO2$conc)
[1] 435

The $ indicates that we are interested in a specific column from the CO2 data frame. A trimmed mean can be calculated by specifying the percentage of data at either end (minimum and maximum) of the data range. For the previous example we could get a 10% trimmed mean with this code:

> mean(CO2$conc, trim=0.05)
[1] 423.1579

If there was missing data then the function will return NA so to get around this we can instruct the mean function to ignore missing data by the na.rm argument

> mean(CO2$conc, na.rm = TRUE)

Other functions of interest are min, max and range which compute the values that we would expect based on the names of these functions:

> min(CO2$conc)
[1] 95
> max(CO2$conc)
[1] 1000
> range(CO2$conc)
[1]   95 1000

The range function returns a vector with two elements corresponding to the minimum and maximum values of the data respectively.

The variance and standard deviation are other useful statistics that would be calculated for a data variable – to get the standard deviation we use the square root function on the value returned by the variance function:

> var(CO2$conc)
[1] 87571.08
> sqrt(var(CO2$conc))
[1] 295.9241