MUSHRA Experiment
=================

In this exercise you will analyze the results of the MUSHRA test that you have done today. The variable *responses_stimulus* describes the test stimuli/conditions. The conditions *"Anchor35"* and *"Anchor70"* are the low-pass filtered anchors processed from the reference signal. *condition="reference"* is the hidden reference. Additionally, the results do contain the results of three audio algorithms which influence the overall quality (variable *responses_score*).

#  Exploratory statistics of the data

1. load the data from the webserver file using the the command *read.csv(...)*

```{r message=FALSE, results='hide'}
temporaryFile <- tempfile()
download.file("https://work.audiolabs.uni-erlangen.de/webmushra/download/aplab/mushra.csv", destfile=temporaryFile, method="curl")
mushra = read.csv(temporaryFile, header=T, sep=";")
remove(temporaryFile)
```

Graphical analysis of the data

2. Plot histograms of the *responses_score*

```{r}

```

3. Generate box plots to show differences the *responses_score* over the different *responses_stimulus* conditions. Use the "formula" syntax in R.

```{r}

```

4. Detect possible outliers by filtering out participants (identified by their `questionaire_uuid`) who do not reliably detect the hidden reference. Use the first rule as stated in the MUSHRA recommendation (4.1.2). Provide the R code and show a boxplot after the filter operation. Generate a new (cleaned) subset *mushra_clean* from the original data.

  1. Find invalid responses that match the MUSHRA post-screening conditions
  2. Discard all responses of the corresponding users by matching the `questionaire_uuid` of all responses to the `questionaire_uuid` of the invalid responses.

    Hint: The value match operator `%in%` (see ?match) can be helpful. Example: `a %in% b` returns `TRUE` for the elements of `a` being present within `b` and `FALSE` otherwise:  
    `> c(2,6,7,3,1) %in% c(1,3,5)`

    `[1] FALSE FALSE FALSE  TRUE  TRUE`

```{r}



```

4. Generate a subset *mushracs* of the data that only includes the 3 different audio systems. Report if *responses_score* is normal distributed by providing both graphical analysis using Q-Q plots and statistical tests of normality. Check each codec seperately! Plot another boxplot with the final cleaned data.

```{r}












```


# Hypothesis testings

5. When several different audio systems are to be compared we want to answer some specific questions. E.g.: Does the audio system have an impact on the overall audio quality? Which system gives the best overall audio quality? Transform these simple questions into statistical hypotheses while keeping in mind that we want to do ANOVA for statistical evaluation!

```{r}


```

# ANOVA

6. Besides normal distribution what are the other main assumptions that should be met to allow an ANOVA analysis? Check these assumptions where applicable. Name the tests you use and interprete the test report outputs with your own words.
Why is Pearson's chi-squared test for independence of observations not applicable to the data set?
Make a statement about independence of observations accros the groups of `mushracs$responses_stimulus` by only considering the MUSHRA experiment design.
```{r}






```


7. Run ANOVA according to the design of experiment. Effect sizes are recommended to be reported along with the levels of significance. Report the output of the data.
If possible check your additional (bonus) hypothesis. Use the R Package **aov** to run the ANOVA.


```{r}









```

```

```
8. (bonus) Investigate the influence of the participants of the experiment by inserting the corresponding participants' identifyer (`questionaire_uuid`) into the model and report the results in your own words!

```{r}

```


9. Multiple Comparisons

By using multiple comparisons try to address the question which audio system is the best.

Apply post hoc tests in order to establish the significance of difference between means where the dependent factor (or factor interaction) is significant in the ANOVA. Provide the output in tabular form and in your own words. A number of different post hoc tests are available with different levels of discrimination, e.g. Fisher's Least Significant Difference (LSD), Tukey's Honestly Significant Difference (HSD), etc. In the cases that factor interactions (e.g. 2- or 3- way) are found to be significant, these should be plotted to provide a thorough overview of the data. In such cases plotting only main effects will provide an overview of the data with interaction effect confounded.

```{r}



```

```
There is (no) significant...
```

10. (bonus) identify items which are problematic for specific coders. Look for statistcal significance.

```{r}

```