You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: app/docs/dev.md
+32-6Lines changed: 32 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -27,7 +27,7 @@ The returned dictionary will be referred to as the `result` in this documentatio
27
27
### Overview
28
28
The overall flow of the evaluation procedure can be described as follows:
29
29
30
-
1. The function uses the paramters given in `params` to determine the context of the evaluation. What context means will be discussed in more detail in a later section.
30
+
1. The function uses the parameters given in `params` to determine the context of the evaluation. What context means will be discussed in more detail in section TODO: Add section name here.
31
31
2. After the context is determined the response, answer and criteria (either supplied via `params` or from the context) are analysed an necessary information is stored for future use in a dictionary with frozen valuues, i.e. a dictionary where new items can be added but existing items cannot be changed.
32
32
3. The feedback generating procedure supplied by the context is used to generate feedback based on the contents of the frozen value dictionary.
33
33
4. If all criteria are found to be satisfied the response is considered correct, i.e. the `is_correct` field in the result is set to true and the feedback string and list of tags generated by the feedback generation procedure are added to their respective fields.
@@ -36,21 +36,47 @@ The overall flow of the evaluation procedure can be described as follows:
36
36
37
37
### Context
38
38
39
-
The context is a data structure that contains at least the following seven pieves of information:
40
-
-`default_criteria` A list of strings that describe the default criteria used if none are supplied by the input parameters.
39
+
The context is a data structure that contains at least the following seven pieces of information:
40
+
-`default_parameters` A dictionary where the keys are parameter names and the values are the default values that the evaluation function will use unless another value is provided together with the response. The only required field is the
41
41
-`expression_parse` function that parses expressions (i.e. the `response` and `answer` inputs) into the form used by the feedback generation procedure.
42
42
-`expression_preprocess` function that performs string manipulations that makes ensures that correctly written input expressions follows the conventions expected by `expression_parse`.
43
-
-`expression_preview` is a function that generates a string that can be turned into a human-readable represenation of how the evaluation function interpreted the response.
43
+
-`expression_preview` is a function that generates a string that can be turned into a human-readable representation of how the evaluation function interpreted the response.
44
44
-`feedback_procedure_generator` function that generates a function for each criteria that can be used to evaluate if the criteria is satisfied or not. The output from this function should be a list of tags that the feedback string generator can use to produce human readable feedback.
45
45
-`feedback_string_generator` function that takes tags and outputs human readable feedback strings.
46
46
-`generate_criteria_parser` function that generates a parser that can be used to turn the criteria (given in string form) into a form that the feedback generation procedure can use to determine if they are correct or not.
47
47
48
48
The context can also contain other fields if necessary.
49
49
50
-
**Remark:** The current implementation uses a dictionary rather than a dedicated class for ease of iteration during the prototyping phase.
50
+
**Remark:** The current implementation uses a dictionary rather than a dedicated class for ease of iteration during the initial development phase.
51
+
52
+
There are currently two different contexts:
53
+
-`symbolic` Handles comparisons of various symbolic expressions. Defined in `context\symbolic.py`.
54
+
-`physical_quantity` Handles comparisons of expressions involving units. Defined in `context\physical_quantity.py`.
55
+
56
+
**Remark:** Handwritten expressions are sent as latex, which requires extra preprocessing before the right context can be determined in some cases. It should be considered whether a new context, perhaps called `handwritten`, should be created for this purpose.
51
57
52
58
**TODO** Describe currently available contexts
59
+
#### `symbolic` - Comparison of symbolic expressions
60
+
61
+
**Remark:** The `symbolic` context should probably be split into several smaller contexts, the following subdivision is suggested:
62
+
-`numerical`: Comparison of expressions that can be evaluated to numerical values (e.g. expressions that are already numerical values or expressions only containing constants). Focuses on identifying if numerical values are greater than, less than, proportional to the expected answer or similar.
63
+
-`symbolic`: Comparison of symbolic expressions that cannot be reduced to numerical values.
64
+
-`equality`: Comparison of mathematical equalities (with the extra complexities that come with equivalence of equalities compared to equality of expressions).
65
+
-`inequality`: Same as `equality` except for mathematical inequalities (which will require different choices when it comes to what can be considered equivalence). It might be appropriate to combine `equality` and `inequality` into one context (called `statements` or similar).
66
+
-`collection`: Comparison of collections (e.g. sets, lists or intervals of the number line). Likely to consist mostly of code for handling comparison of individual elements using the other contexts, and configuring what counts as equivalence between different collections.
67
+
68
+
#### `physical_quantity` - Comparison of expressions that involve units
Copy file name to clipboardExpand all lines: app/docs/user.md
+16-8Lines changed: 16 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,10 @@ Note that this function is designed to handle comparisons of mathematical expres
8
8
9
9
### Optional parameters
10
10
11
-
There are twelve optional parameters that can be set: `complexNumbers`, `convention`, `criteria`, `elementary_functions`, `feedback_for_incorrect_response`, `multiple_answers_criteria`, `physical_quantity`, `plus_minus`/`minus_plus`, `specialFunctions`, `strict_syntax`, `strictness`, `symbol_assumptions`.
11
+
There are 15 optional parameters that can be set: `atol`, `complexNumbers`, `convention`, `criteria`, `elementary_functions`, `feedback_for_incorrect_response`, `multiple_answers_criteria`, `physical_quantity`, `plus_minus`/`minus_plus`, `rtol`, `specialFunctions`, `strict_syntax`, `strictness`, `symbol_assumptions`.
12
+
13
+
## `atol`
14
+
Sets the absolute tolerance, $e_a$, i.e. if the answer, $x$, and response, $\tilde{x}$, are numerical values then the response is considered equal to the answer if $|x-\tilde{x}| \leq e_aBy default `atol` is set to `0`, which means the comparison will be done with as high accuracy as possible. If either the answer or the response aren't numerical expressions this parameter is ignored.
12
15
13
16
## `complexNumbers`
14
17
@@ -55,16 +58,19 @@ If unset, `physical_quantity` will default to `false`.
55
58
56
59
If `physical_quantity` is set to `true` the answer and response will interpreted as a physical quantity using units and conventions decided by the `strictness` and `units_string` parameters.
57
60
58
-
**Remark:** Setting `physical_quantity` to `true` will also mean that comparisons will be done numerically. If neither the `atol` nor `rtol` parameters are set, the evaluation function will choose a relative error based on the number of sigificant digits given in the answer.
61
+
**Remark:** Setting `physical_quantity` to `true` will also mean that comparisons will be done numerically. If neither the `atol` nor `rtol` parameters are set, the evaluation function will choose a relative error based on the number of significant digits given in the answer.
59
62
60
63
61
64
When `physical_quantity` the evaluation function will generate feedback based on the flowchart below.
62
65
63
66
**TODO:** Generate new flowchart for updated physical quantity feedback generation procedure.
64
67
68
+
## `rtol`
69
+
Sets the relative tolerance, $e_r$, i.e. if the answer, $x$, and response, $\tilde{x}$, are numerical values then the response is considered equal to the answer if $\left|\frac{x-\tilde{x}}{x}\right| \leq e_r$. By default `rtol` is set to `0`, which means the comparison will be done with as high accuracy as possible. If either the answer or the response aren't numerical expressions this parameter is ignored.
70
+
65
71
## `strictness`
66
72
67
-
Constrols the conventions used when parsing physical quantities.
73
+
Controls the conventions used when parsing physical quantities.
68
74
69
75
**Remark:** If `physical_quantity` is set to `false`, this parameter will be ignored.
70
76
@@ -80,7 +86,9 @@ Controls what sets of units are used. There are three values `SI`, `common` and
80
86
81
87
If `SI` is chosen then only units from the tables `Base SI units` and `Derived SI units` (below) are allowed (in combinations with prefixes). If `common` is chosen then all the units allowed by `SI` as well as those listed in the tables for `Common non-SI units`. If `imperial` is chosen the base SI units and the units listed in the `Imperial units` table are allowed.
82
88
83
-
**Remark:** The different settings can also be combine, e.g. `SI common imperial` will allow all units.
89
+
**Remark:** The different settings can also be combined, e.g. `SI common imperial` will allow all units.
90
+
91
+
The default setting is to allow all units, i.e. `units_string` is set to `SI common imperial`.
84
92
85
93
### Notation and definition of units
86
94
@@ -317,7 +325,7 @@ In the example set there is a response area with `complexNumbers` set to `true`
317
325
318
326
Any response that is mathematically equivalent to $2+i$ will be accepted, e.g. `2+I`, `2+(-1)^(1/2)`, `conjugate(2-I)`, `2sqrt(2)e^(I*pi/4)+e^(I*3*pi/2)` or `re(2-I)-im(2-I)*I`.
319
327
320
-
**Note:** If the particular way that the answer is written matter, e.g. only answers on cartesian form should be accepted, then that requires further configuration, see the example *Syntactical comparison*.
328
+
**Note:** If the particular way that the answer is written matter, e.g. only answers on Cartesian form should be accepted, then that requires further configuration, see the example *Syntactical comparison*.
321
329
322
330
### Using `constant` and `function` assumptions
323
331
@@ -340,9 +348,9 @@ By default `compareExpressions` assumes that symbols are independent of each oth
340
348
Taking the ratio of the given answer and the example response gives:
341
349
$$ \frac{\frac{\partial^2 T}{\partial x^2}+\frac{\dot{q}}{k} - \frac{1}{\alpha}\frac{\partial T}{\partial t}}{\alpha k \frac{\partial^2 T}{\partial x^2}+ \alpha \dot{q} - k \frac{\partial T}{\partial t}} = \alpha k $$
342
350
343
-
By default $\alpha$ and $k$ are assumed to be variables so the ratio is not seen as a constant. This can be fixed by addding`('alpha','constant') ('k','constant')` to the `symbol_asssumptions` parameter.
351
+
By default $\alpha$ and $k$ are assumed to be variables so the ratio is not seen as a constant. This can be fixed by adding`('alpha','constant') ('k','constant')` to the `symbol_asssumptions` parameter.
344
352
345
-
The make it simpler and more intuitive to write valid reponses we add the following input symbols:
353
+
The make it simpler and more intuitive to write valid responses we add the following input symbols:
Typically `compareExpressions` only checks if the response is mathematically equivalent to the answer. If we want to require that the answer is written in a cetain way, e.g. Cartesian form vs. exponential form of a complex number or standard form vs factorized form of a polynomial, further comparsions need to be done. There are some built in standard forms that can be detected, as well as a method that tries to match the way that the response is written in a limited fashion. Either method can be activated either by setting the flag `syntactical_comparison` to `true`, or by using the criteria `response written as answer`.
388
+
Typically `compareExpressions` only checks if the response is mathematically equivalent to the answer. If we want to require that the answer is written in a certain way, e.g. Cartesian form vs. exponential form of a complex number or standard form vs factorized form of a polynomial, further comparisons need to be done. There are some built in standard forms that can be detected, as well as a method that tries to match the way that the response is written in a limited fashion. Either method can be activated either by setting the flag `syntactical_comparison` to `true`, or by using the criteria `response written as answer`.
0 commit comments