Expert assessment

Last modified by Hendrik Zschocke on 11.01.2024

In addition to the configurable assessment options in the Assessment tab, extensive analyses can be used to assess participant responses via expert functions. Their application allows, for example, the implementation of adaptive question formats with individual error analyses and, based on this, individualised user feedback.

Content

Related topics

Example resources

Access to answer options

Access to the answer selected by the participant and the correct solution to the question can be realised, for example, through Maxima queries in the Variables tab. For more information on how to create and use variables, see the Variables help page.

Beispiel - LR und CR per MAXIMA_de.png

Return values of answer options

The return values type of the answer options and thus of the system variables LEARNERRESPONSE and CORRECTRESPONSE depends on the question type:

Question type

Type

Example return value

System variable

  • Choice interaction, hotspot interaction, hottext interaction – one answer can be selected (single choice)

single ID

ID_1

LEARNERRESPONSE

CORRECTRESPONSE

  • Inline choice (one text box)

ID_1

LEARNERRESPONSE_GAP_1 CORRECTRESPONSE_GAP_1

  • Choice interaction, hotspot interaction, hottext interaction – more than one answer can be selected (multiple choice)
  • Order interaction

list of IDs

ID_1, ID_2, ID_3

LEARNERRESPONSE

CORRECTRESPONSE

  • Match interaction, matrix interaction

ID value pairs in a list

[ID_1, IDT_1], [ID_2, IDT_2], [ID_3, IDT_3]

LEARNERRESPONSE

CORRECTRESPONSE

  • Text entry interaction (all formats)
  • Extended text interaction
  • Programming interaction

specific values

 

 

ID display in the editor

The identifiers of the answers to a question can be displayed in the editor. Activate this function in your user profile settings.

Editor - Anzeige der IDs_de.png

Return value preparation with auxiliary functions

The functions shown below are used to prepare LEARNERRESPONSES and CORRECTRESPONSES in the MAXIMA backend if the identifiers are each present in the form of ID_x (or IDT_y) with a natural number x.

SimpleLR({LEARNERRESPONSE})

  • The ID_ and IDT_ strings are removed, i. e. each ID is reduced to the numerical values x,y.
  • If LEARNERRESPONSE is a list of IDs (single/multiple choice, hotspot interaction, order interaction, etc.), the output is a corresponding list of numerical IDs, e. g. NumLR(ID_3,ID_4,ID_2) -> [3,4,2].
  • If LEARNERRESPONSE is a list of ID pairs (matrix interaction, match interaction, etc.), the output consists of the list of source elements and the list of associated target elements, e. g. NumLR([ID_3,IDT_1],[ID_4,IDT_3],[ID_2,IDT_2]) -> [[3,4,2], [1.3,2]].

SimpleLR(0,{LEARNERRESPONSE})

  • Like SimpleLR({LEARNERRESPONSE}), but the output is sorted in ascending order by (source) IDs.

SimpleLR(n,{LEARNERRESPONSE})

  • If LEARNERRESPONSE is a list of ID pairs: Output is sorted in ascending order by (source) IDs. Additionally, for missing source IDs 1<=x<=n, the pairs [x,0] are added, completing the list for 1<=x<=n.

Choices(LR)

  • Input: LR – the LEARNERRESPONSE simplified by SimpleLR
  • Output: LEARNERRESPONSE as a binary matrix, rows=source elements, columns=target elements

Choices(m,LR), Choices (0,n,LR), Choices(m,n,LR)

  • Like Choices(LR), additionally filled up with or reduced to m rows or m columns if necessary.

Additional auxiliary functions:

List2Matrix(LR)

Converts the list simplified by NumLR into a one- or two-line matrix

Matrix2List(M)

Converts matrix to list

ransposeList(LR)

Switches between the list displays

  • Two lists: [[source elements], [target elements]]
  • List of pairs: [[source1, target1], [souce2, target2], …]

Function to assess a formula comparison

The following standard comparison is used when assessing formula gaps:

is(equal(ev({LEARNERRESPONSE}),ev({CORRECTRESPONSE})))

However, it has several disadvantages:

  • mandatory use of decimal point instead of comma
  • rounding/tolerance cannot be adjusted
  • mandatory use of MAXIMA syntax, especially %e, %pi, %i and log instead of ln

The comparison can be extended using the function

CompareLR(”{LEARNERRESPONSE}“,“{CORRECTRESPONSE}“)

or

CompareLR(”{LEARNERRESPONSE}“,“{CORRECTRESPONSE}“,tol)

 This offers the following advantages:

  • also accepts e, pi, i, j, ln
  • accepts both decimal point and comma
  • performs the comparison "abs(LR-CR)<=tol" if tol>0 is specified
  • otherwise performs the comparison "is(equal(ev(LR),ev(CR)))" and if necessary (i. e. if the result is unknown) additionally the checks/formula comparisons using radcan and trigsimp

Note: {LEARNERRESPONSE} and {CORRECTRESPONSE} must be committed in quotation marks (""), i. e. as strings!

Testdurchführung - Beispiel Formelvergleich_de.png

Example using the auxiliary functions

All described auxiliary functions are shown in the attached MAXIMA-functions_for_LR-Processing.zip example test. Download the file and import it into your question bank to reproduce the examples.

In the example test, you will find the auxiliary functions in the Variables tab under "Post-processed variables".

Variablen - Maxima ID Beispiele_de.png

The variables created in this way can be used in the question feedback (among others).

Feedback - Übergabe der Maxima ID Beispiele_de.png

The feedback can be displayed directly in the test run or the assessment documents.

Testdurchführung - Beispiel_de.png