First step is to install the package then make it available for use in the current session:

require(FrF2)

A basic call to the main functino FrF2 specifies the number of runs in the fractional factorial design (which needs to be a multiple of 2) and the number of factors. For example a three factor design would have a total of eight runs if it was a full factorial but if we wanted to go with four runs then we can generate the design like this:

> FrF2(4, 3)
   A  B  C
1  1 -1 -1
2 -1  1 -1
3 -1 -1  1
4  1  1  1
class=design, type= FrF2

The default output labels the factors A, B, C and so on and the factor levels are -1 and +1 for the two levels of each factor. We can change the level names to low and high using the default.levels function argument:

> FrF2(4, 3, default.levels = c("low", "high"))
     A    B    C
1 high high high
2  low high  low
3 high  low  low
4  low  low high
class=design, type= FrF2

The factors can be specified as a list of names rather than the number of factors via the factor.names argument:

> FrF2(4, factor.names = c("One", "Two", "Three"),
  default.levels = c("low", "high"))
   One  Two Three
1  low high   low
2 high high  high
3  low  low  high
4 high  low   low
class=design, type= FrF2

These are the basics and there are other features for greater control over the confounding between factors and their interactions that is introduced by using a fractional factorial design.

For a BIBD there are v treatments repeated r times in b blocks of k observations. There is a fifth parameter lambda that records the number of blocks where every pair of treatment occurs in the design.

We first load the crossdes package in our sessions:

require(crossdes)

The function find.BIB is used to generate a block design with specific number of treatments, blocks (rows of the design) and elements per block (columns of the design).

Consider an example with five treatments in four blocks of three elements. We can create a block design via:

> find.BIB(5, 4, 3)
     [,1] [,2] [,3]
[1,]    1    3    4
[2,]    2    4    5
[3,]    2    3    5
[4,]    1    2    5

This design is not a BIBD because the treatments are not all repeated the same number of times in the design and we can check this with the isGYD function. For this example:

> isGYD(find.BIB(5, 4, 3))
 
[1] The design is neither balanced w.r.t. rows nor w.r.t. columns.

This confirms what we can see from the design.

Let us instead consider a design with seven treatments and seven blocks of three elements to see whether we can create a BIBD with these parameters:

> my.design = find.BIB(7, 7, 3)
> my.design
     [,1] [,2] [,3]
[1,]    1    2    5
[2,]    3    4    5
[3,]    1    3    6
[4,]    2    3    7
[5,]    2    4    6
[6,]    1    4    7
[7,]    5    6    7
> isGYD(my.design)
 
[1] The design is a balanced incomplete block design w.r.t. rows.

In this situation we are able to generate a valid BIBD experiment with the specified parameters.

The least complicated scenario is where we would have a single (nuisance) factor that we want to control for in the experiment. The statistical model used to describe the data collected in such an experiment could be written in the form:

where there are v treatments in b blocks and the number of units in each block does not have to be the same and is denoted using the k subscript.

In a complete block design all treatments occur the same number of times in every block, usually one replicate of all treatments per block. There will be situations where the number of treatments is too large for all of them to be included in every block of the design. In these situations an incomplete block design would be used for running an experiment.

A special type of design is the balanced incomplete block design (BIBD), where the v treatments are investigated by allocating them to b blocks of equal size k. We have that k is less than t and b and k are chosen so that b * k is a multiple of v. All of the treatments occur exactly r times in the design and every pair of treatments occur together in lambda of the b blocks.

Two-way analysis of variance (ANOVA) is used to analyse data collected from an experiment using a block design, as discussed elsewhere in this post.

As an example consider a company that regularly has to ship parcels between its various (five for this example) sub-offices and has the option of using three competing parcel delivery services, all of which charge roughly similar amounts for each delivery. To determine which service to use, the company decides to run an experiment shipping three packages from its head office to each of the five sub-offices. The delivery time for each package is recorded and the data loaded into R:

delivery.df = data.frame(
  Service = c(rep("Carrier 1", 15), rep("Carrier 2", 15),
    rep("Carrier 3", 15)),
  Destination = c(rep(c("Office 1", "Office 2", "Office 3",
    "Office 4", "Office 5"), 9)),
  Time = c(15.23, 14.32, 14.77, 15.12, 14.05,
  15.48, 14.13, 14.46, 15.62, 14.23, 15.19, 14.67, 14.48, 15.34, 14.22,
  16.66, 16.27, 16.35, 16.93, 15.05, 16.98, 16.43, 15.95, 16.73, 15.62,
  16.53, 16.26, 15.69, 16.97, 15.37, 17.12, 16.65, 15.73, 17.77, 15.52,
  16.15, 16.86, 15.18, 17.96, 15.26, 16.36, 16.44, 14.82, 17.62, 15.04)
)

The data is then displayed using a dot plot for an initial visual investigation of any trends in delivery time between the three services and across the five sub-offices. The colour aesthetic is used to distinguish between the three services in the plot.

ggplot(delivery.df, aes(Time, Destination, colour = Service)) + geom_point()

This code produces the following graph:

Graph of the delivery time for different services and destintions

The graph shows a general pattern of service carrier 1 having shorter delivery times than the other two services. There is also an indication that the differences between the services varies for the five sub-offices and we might expect the interaction term to be significant in the two-way ANOVA model. To fit the two-way ANOVA model we use this code:

delivery.mod1 = aov(Time ~ Destination*Service, data = delivery.df)

The * symbol instructs R to create a formula that includes main effects for both Destination and Service as well as the two-way interaction between these two factors. We save the fitted model to an object which we can summarise as follows to test for importance of the various model terms:

> summary(delivery.mod1)
                    Df  Sum Sq Mean Sq  F value    Pr(>F)    
Destination          4 17.5415  4.3854  61.1553 5.408e-14 ***
Service              2 23.1706 11.5853 161.5599 < 2.2e-16 ***
Destination:Service  8  4.1888  0.5236   7.3018 2.360e-05 ***
Residuals           30  2.1513  0.0717                       
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

We have strong evidence here that there are differences between the three delivery services, between the five sub-office destinations and that there is an interaction between destination and service in line with what we saw in the original plot of the data. Now that we have fitted the model and identified the important factors we need to investigate the model diagnostics to ensure that the various assumptions are broadly valid.

We can plot the model residuals against fitted values to look for obvious trends that are not consistent with the model assumptions about independence and common variance. The first step is to create a data frame with the fitted values and residuals from the above model:

delivery.res = delivery.df
delivery.res$M1.Fit = fitted(delivery.mod1)
delivery.res$M1.Resid = resid(delivery.mod1)

Then a scatter plot is used to display the fitted values and residuals where the colour asthetic highlights which points correspond to the three competing delivery services:

ggplot(delivery.res, aes(M1.Fit, M1.Resid, colour = Service)) + geom_point() +
  xlab("Fitted Values") + ylab("Residuals")

The xlab() and ylab() are used to change the text on the axis labels. The residual diagnostic plot is:

Diagnostic Residual Plot for Delivery Time Model

There are no obvious patterns in this plot that suggest problems with the two-way ANOVA model that we fitted to the data.

As an alternative display we could separate the residuals into destination sub-offices, where the facet_wrap() function instructs ggplot to create a separate display (panel) for each of the destinations.

ggplot(delivery.res, aes(M1.Fit, M1.Resid, colour = Service)) +
  geom_point() + xlab("Fitted Values") + ylab("Residuals") +
  facet_wrap( ~ Destination)

To produce the following alternative residual plot:

Diagnostic Residual Plot for Delivery Time Model by Destination

No obvious problems in this diagnostic plot.

We could also consider dividing the data by delivery service to get a different view of the residuals:

ggplot(delivery.res, aes(M1.Fit, M1.Resid, colour = Destination)) +
  geom_point() + xlab("Fitted Values") + ylab("Residuals") +
  facet_wrap( ~ Service)

This creates the following graph:

Diagnostic Residual Plot for Delivery Time Model by Service

Again there is nothing substantial here to lead us to consider an alternative analysis.

Lastly we consider the normal probability plot of the model residuals, using the stat_qq() option:

ggplot(delivery.res, aes(sample = M1.Resid)) + stat_qq()

The quantile plot is:

Normal Probability Plot for Delivery Time Model

This plot is very close to the straight line we would expect to observe if the data was a close approximation to a normal distribution. To round off the analysis we look at the Tukey HSD multiple comparisons to confirm that the differences are between delivery service 1 and the other two competing services:

> TukeyHSD(delivery.mod1, which = "Service")
  Tukey multiple comparisons of means
    95% family-wise confidence level
 
Fit: aov(formula = Time ~ Destination * Service, data = delivery.df)
 
$Service
                        diff        lwr       upr     p adj
Carrier 2-Carrier 1 1.498667  1.2576092 1.7397241 0.0000000
Carrier 3-Carrier 1 1.544667  1.3036092 1.7857241 0.0000000
Carrier 3-Carrier 2 0.046000 -0.1950575 0.2870575 0.8856246

Even with the multiple comparison post-hoc adjustment there is very strong evidence for the differences that we have consistenly observed throughout the analysis.

We can use ggplot to visualise the difference in mean delivery time for the services and the 95% confidence intervals on these differences. We create a data frame from the TukeyHSD output by extracting the component relating to the delivery service comparison and add the text labels by extracting the row names from the data frame.

delivery.hsd = data.frame(TukeyHSD(delivery.mod1, which = "Service")$Service)
delivery.hsd$Comparison = row.names(delivery.hsd)

We then use the geom_pointrange() to specify lower, middle and upper values based on the three pairwise comparisons of interest.

ggplot(delivery.hsd, aes(Comparison, y = diff, ymin = lwr, ymax = upr)) +
  geom_pointrange() + ylab("Difference in Mean Delivery Time by Service") +
  coord_flip()

The coord_flip() is used to make the confidence intervals horizontal rather than vertical on the graph. This can be confusing for creating the axis labels as we specify the label where it would appear prior to the filp of coordinates. In the example above we add text to the y axis but this now appears on the x axis in the final graph:

Plot of Confidence Intervals for Mean Differences using Tukey HSD

Fast Tube by Casper

Fast Tube by Casper

In the analysis we made use of the linear model function lm and the analysis could be conducted using the aov function. The code used to fit the model is very similar:

> plant.mod2 = aov(weight ~ group, data = plant.df)
> summary(plant.mod2)
            Df  Sum Sq Mean Sq F value  Pr(>F)  
group        2  3.7663  1.8832  4.8461 0.01591 *
Residuals   27 10.4921  0.3886                  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

The output from using the summary function of the fitted model object shows the analysis of variance table with the p-value showing evidence of differences between the three groups. In R we can investigated the particular groups where there are differences using Tukey’s multiple comparisons:

> TukeyHSD(plant.mod2)
  Tukey multiple comparisons of means
    95% family-wise confidence level
 
Fit: aov(formula = weight ~ group, data = plant.df)
 
$group
                          diff        lwr       upr     p adj
Treatment 1-Control     -0.371 -1.0622161 0.3202161 0.3908711
Treatment 2-Control      0.494 -0.1972161 1.1852161 0.1979960
Treatment 2-Treatment 1  0.865  0.1737839 1.5562161 0.0120064

The multiple comparison tests highlight that the difference is due to comparing treatments 1 and 2. These 95% confidence intervals for the differences shown above can be plotted:

plot(TukeyHSD(plant.mod2))

which gives

The post-hoc adjustments are recommended as we are testing after looking at the data rather than undertaking a pre-planned analysis.

Fast Tube by Casper

Fast Tube by Casper

In one-way ANOVA the data is sub-divided into groups based on a single classification factor and the standard terminology used to describe the set of factor levels is treatment even though this might not always have meaning for the particular application. There is variation in the measurements taken on the individual components of the data set and ANOVA investigates whether this variation can be explained by the grouping introduced by the classification factor.

As an example we consider one of the data sets available with R relating to an experiment into plant growth. The purpose of the experiment was to compare the yields on the plants for a control group and two treatments of interest. The response variable was a measurement taken on the dried weight of the plants.

The first step in the investigation is to take a copy of the data frame so that we can make some adjustments as necessary while leaving the original data alone. We use the factor function to re-define the labels of the group variables that will appear in the output and graphs:

plant.df = PlantGrowth
plant.df$group = factor(plant.df$group,
  labels = c("Control", "Treatment 1", "Treatment 2"))

The labels argument is a list of names corresponding to the levels of the group factor variable.

A boxplot of the distributions of the dried weights for the three competing groups is created using the ggplot package:

require(ggplot2)
 
ggplot(plant.df, aes(x = group, y = weight)) +
  geom_boxplot(fill = "grey80", colour = "blue") +
  scale_x_discrete() + xlab("Treatment Group") +
  ylab("Dried weight of plants")

The geom_boxplot() option is used to specify background and outline colours for the boxes. The axis labels are created with the xlab() and ylab() options. The plot that is produce looks like this:

Initial inspection of the data suggests that there are differences in the dried weight for the two treatments but it is not so clear cut to determine whether the treatments are different to the control group. To investigate these differences we fit the one-way ANOVA model using the lm function and look at the parameter estimates and standard errors for the treatment effects. The function call is:

plant.mod1 = lm(weight ~ group, data = plant.df)

We save the model fitted to the data in an object so that we can undertake various actions to study the goodness of the fit to the data and other model assumptions. The standard summary of a lm object is used to produce the following output:

> summary(plant.mod1)
 
Call:
lm(formula = weight ~ group, data = plant.df)
 
Residuals:
    Min      1Q  Median      3Q     Max 
-1.0710 -0.4180 -0.0060  0.2627  1.3690 
 
Coefficients:
                 Estimate Std. Error t value Pr(>|t|)    
(Intercept)        5.0320     0.1971  25.527   <2e-16 ***
groupTreatment 1  -0.3710     0.2788  -1.331   0.1944    
groupTreatment 2   0.4940     0.2788   1.772   0.0877 .  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 
 
Residual standard error: 0.6234 on 27 degrees of freedom
Multiple R-squared: 0.2641,     Adjusted R-squared: 0.2096 
F-statistic: 4.846 on 2 and 27 DF,  p-value: 0.01591

The model output indicates some evidence of a difference in the average growth for the 2nd treatment compared to the control group. An analysis of variance table for this model can be produced via the anova command:

> anova(plant.mod1)
Analysis of Variance Table
 
Response: weight
          Df  Sum Sq Mean Sq F value  Pr(>F)  
group      2  3.7663  1.8832  4.8461 0.01591 *
Residuals 27 10.4921  0.3886                  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

This table confirms that there are differences between the groups which were highlighted in the model summary. The function confint is used to calculate confidence intervals on the treatment parameters, by default 95% confidence intervals:

> confint(plant.mod1)
                       2.5 %    97.5 %
(Intercept)       4.62752600 5.4364740
groupTreatment 1 -0.94301261 0.2010126
groupTreatment 2 -0.07801261 1.0660126

The model residuals can be plotted against the fitted values to investigate the model assumptions. First we create a data frame with the fitted values, residuals and treatment identifiers:

plant.mod = data.frame(Fitted = fitted(plant.mod1),
  Residuals = resid(plant.mod1), Treatment = plant.df$group)

and then produce the plot:

ggplot(plant.mod, aes(Fitted, Residuals, colour = Treatment)) + geom_point()

which produces this graph:

We can see that there is no major problem with the diagnostic plot but some evidence of different variabilities in the spread of the residuals for the three treatment groups.

Blocking is a technique used in design of experiments methodology to deal with the systematic differences to ensure that all the factors of interest and interactions between the factors can be assessed in the design. When blocking occurs one or more of the interactions is likely to be confounded with the block effects but a good choice of blocking should hopefully ensure that it is a higher order interaction that would be challenging to interpret or not be expected to be important that is confounded.

The conf.design package in R is described by its author as a small library contains a series of simple tools for constructing and manipulating confounded and fractional factorial designs. The function conf.design can be used to construct symmetric confounded factorial designs.

A very simple example would be a three factor experiment where each factor has low and high settings (levels). If we wanted to divide the experiment into two blocks of four experimental units then we could confounded the block effect with the three way interaction between the factors. The following code would create the required design plan:

conf.design(rbind(c(1,1,1)), p=2, treatment.names = c("F1","F2","F3"))

The first argument is a matrix, with a single row in this case as there are only two blocks, which specifies the levels of the factors for the effect to be confounded with the blocks. The output from this function call is:

  Blocks F1 F2 F3
1      0  0  0  0
2      0  1  1  0
3      0  1  0  1
4      0  0  1  1
5      1  1  0  0
6      1  0  1  0
7      1  0  0  1
8      1  1  1  1

This shows two blocks, labelled 0 and 1, and the settings of the experiments to run in each block. In the first block the four factor combinations would be:

• F1 low, F2 low, and F3 low.
• F1 high, F2 high, and F3 low.
• F1 high, F2 low, and F3 high.
• F1 low, F2 high, and F3 high.

The remaining four combinations are use in the second block of experiments.

In many cases each factor takes only two levels, often referred to as the low and high levels, the design is known as a 2^k experiment. Given a three factor setup where each factor takes two levels we can create the full factorial design using the expand.grid function:

expand.grid(Factor1 = c("Low", "High"), Factor2 = c("Low", "High"),
  Factor3 = c("Low", "High"))

which creates the following design:

  Factor1 Factor2 Factor3
1     Low     Low     Low
2    High     Low     Low
3     Low    High     Low
4    High    High     Low
5     Low     Low    High
6    High     Low    High
7     Low    High    High
8    High    High    High

We could also make use of the gen.factorial function from the AlgDesign package. In this function we use a vector to specify the number of levels for each of the variables, the number of variables and possibly the names of the variables.

To create the full factorial design for an experiment with three factors with 3, 2, and 3 levels respectively the following code would be used:

gen.factorial(c(3,2,3), 3, center=TRUE,
  varNames=c("F1", "F2", "F3"))

The center option makes the level settings symmetric which is a common way of representing the design. The full design is:

   F1 F2 F3
1  -1 -1 -1
2   0 -1 -1
3   1 -1 -1
4  -1  1 -1
5   0  1 -1
6   1  1 -1
7  -1 -1  0
8   0 -1  0
9   1 -1  0
10 -1  1  0
11  0  1  0
12  1  1  0
13 -1 -1  1
14  0 -1  1
15  1 -1  1
16 -1  1  1
17  0  1  1
18  1  1  1

In these scenarios computer generated designs, the optimal designs of a given size, can be identified from a list of candidate factor combinations. The library AlgDesign in R has facilities for optimal design searches based on the Federov exchange algorithm. An optimality criterion has to be selected by the investigator, currently D, A or I, and this criterion is minimise by searching for an optimal subset of a given size from the candidate design list.

Given the total number of treatment runs for an experiment and a specified model, the computer algorithm chooses the optimal set of design runs from a candidate set of possible design treatment runs. This candidate set of treatment runs usually consists of all possible combinations of various factor levels that one wishes to use in the experiment.

First stage, as always, is to make the package available for use:

library(AlgDesign)

For illustrative purposes consider a four factor experiment, where the factors have 4, 3, 2, and 2 levels each respectively. Using the expand.grid function we can create a data frame of all possible combinations of the factor settings:

cand.list = expand.grid(Factor1 = c("A", "B", "C", "D"),
  Factor2 = c("I", "II", "III"),
  Factor3 = c("Low", "High"),
  Factor4 = c("Yes", "No"))

The random number seed is set so that the algorithm can run:

set.seed(69)

The function optFederov calculates an exact or approximate algorithmic design for one of three criteria, using Federov’s exchange algorithm. The first argument to the function is a formula for the intended model for the data and the data argument specifies the list of candidate points:

optFederov( ~ ., data = cand.list, nTrials = 13)

In this example all of the factors in the candidate list appear in the model with a linear term. Quadratic or cubic terms can be included in this formula. The argument nTrials specifies the number of design points to select from the candidate list. The output from this function is:

$D
[1] 0.226687
 
$A
[1] 7.022811
 
$Ge
[1] 0.718
 
$Dea
[1] 0.676
 
$design
   Factor1 Factor2 Factor3 Factor4
3        C       I     Low     Yes
6        B      II     Low     Yes
12       D     III     Low     Yes
16       D       I    High     Yes
19       C      II    High     Yes
21       A     III    High     Yes
25       A       I     Low      No
26       B       I     Low      No
29       A      II     Low      No
35       C     III     Low      No
39       C       I    High      No
44       D      II    High      No
46       B     III    High      No
 
$rows
 [1]  3  6 12 16 19 21 25 26 29 35 39 44 46

This provides details of the values of the optimality criteria for the design points selected from the candidate list, the row numbers and the levels for the factors for the chosen design points.

When undertaking sample size or power calculations for a prospective trial or experiment we need to consider various factors. There are two main probabilities of interest that are tied up with calculating a minimum sample size or the power of a specific test, and these are:

• Type I Error: The probability that the test accepts the null hypothesis, H_0, given that the null hypothesis is actually true. This quantity is often referred to as alpha.
• Type II Error: The probability that the test rejects the null hypothesis, H_0, given that the null hypothesis is not true. This quantity is often referred to as beta.

A decision needs to be made about what difference between the two groups being compared should be considered as corresponding to a meaningful difference. This difference is usually denoted by delta.

The base package has functions for calculating power or sample sizes, which includes the functions power.t.test, power.prop.test and power.anova.test for various common scenarios.

Consider a scenario where we might be buying batteries for a GPS device and the average battery life that we want to have is 400 minutes. If we decided that the performance is not acceptable if the average is more than 10 minutes (delta) lower than this (390 minutes) then we can calculate the number of batteries to test:

power.t.test(delta = 10, sd = 6, power = 0.95, type = "one.sample",
  alternative = "one.sided")

For this example we have assumed a standard deviation of 6 minutes for batteries (would either be assumed or estimated from previous data) and that we want a power of 95% in the test. Power is defined as 1 – beta, the Type II error probability. The default option for this function is for 5% probability of alpha, a Type I error. The test will involve only one group so we are considering a one-sample t test and only a one sided alternative is relevant as we do not mind if the batteries perform better than required.

The output from this function call is as follows:

     One-sample t test power calculation 
 
              n = 5.584552
          delta = 10
             sd = 6
      sig.level = 0.05
          power = 0.95
    alternative = one.sided

So we would need to test at least 6 batteries to obtain the required power in the test based on the other parameters that have been used.