Statistical Evaluation

A statistical evaluation can be performed for all test scenarios. Doing this will not only provide you with an overview of how often questions were answered correctly or incorrectly, but also with detailed information on the frequency distribution of all answers that were submitted. In this way, it is possible to draw conclusions about the participants’ general learning progress, detect concrete problem areas and perform a quality analysis of the individual questions, answer options and defined evaluation (e.g. of the solution options).

For the statistical evaluation, all data will be made anonymous. In the case of closed question types (e.g. choice interactions), the frequency distribution of the selected predefined answers will be indicated as a percentage value. In the case of open question types (e.g. text entry interactions), all answers submitted by the participants will be listed according to their frequency. Correct answers will be highlighted in green. If a survey is to be statistically evaluated, answers will no longer be marked as correct or incorrect and only the corresponding frequencies will be displayed.

Characteristic values for difficulty and selectivity

For tests with at least 10 results, additional statistical values on the difficulty and selectivity of the used questions will be available. Additional information will be displayed for both values if thresholds are exceeded.

Difficulty

The difficulty is calculated from the ratio of the average score achieved for the question and the maximum score possible. It is given as an integer percentage and shows how many test participants have answered the question correctly.

High values indicate questions that tend to be answered correctly by participants (easy questions). Low values indicate questions that are more likely to be answered incorrectly or are awarded fewer points (more difficult questions). If a question has a noticeably low or high difficulty (<20 %, >80 %), a note will appear.

Selectivity

Selectivity measures the correlation between participants’ question and test results. It indicates how well or badly a question is reflected in the overall test result.

The calculation of this correlation includes a part-whole correction (deduction of the affected item’s score from the test score). In addition, for each question, only those results for which participants have submitted an answer will be used for the calculation. Unanswered questions will not be included in the calculation.

If the selectivity is below 0.3, then this question is considered to have a noticeably low selectivity, as it does not differentiate sufficiently in the answers between those participants with weak and strong test results. Negative selectivity may hint at a question that has been understood exactly opposite to the way intended.