SEB Config Keys, R Logs, and Clever Shortcuts

Generating the SEB Config Key

OPAL checks whether the correct SEB configuration is used by comparing a configuration key. Previously, I copied this key from the SEB Configurator. This worked, but it was not ideal. The SEB GUI is slow, opening the configuration and waiting for the updated key takes several seconds, and every manual copy step is another place for an avoidable exam-day error.

We now have a small Python script that computes the Config Key directly from the .seb file:

./seb_config_key.py methodenlehre.seb

This is especially useful when changing the configuration repeatedly while testing. Instead of opening the SEB Configurator, waiting for it to load, navigating to the key, and copying it manually, the key can be regenerated instantly from the command line.

The exact algorithm was not obvious from looking at the .seb XML file alone. SEB does not simply hash the raw file bytes. Instead, it parses the configuration as a plist, sets sebConfigPurpose to the value used for starting an exam, serializes the resulting settings dictionary in SEB’s deterministic order, skips originatorVersion, and then computes an SHA hash.

The slightly tricky part is the ordering. SEB uses .NET’s StringComparer.InvariantCulture, which does not sort exactly like Python’s default string ordering. Once this was replicated, the Python script produced the same key as SEB.

This is a small change, but operationally important. After changing the SEB configuration, the checklist becomes:

• edit methodenlehre.seb,
• run ./seb_config_key.py methodenlehre.seb,
• copy the generated key into the SEB/OPAL setting,
• run the local SEB/R tests,
• commit the configuration and the regenerated key note.

This is faster and less error-prone than opening the SEB Configurator only to retrieve the key manually.

The Next Step: Updating OPAL Automatically

What would be even better in the future is updating the SEB Config Key in OPAL automatically. As far as I know, OPAL does not currently expose a convenient API for this setting. Ideally, however, the workflow would be:

1. edit methodenlehre.seb,
2. generate the Config Key,
3. update the corresponding OPAL exam setting through an API,
4. commit the change.

If such an API existed, it would remove the last manual step from the process. That would be a real improvement because the most dangerous exam-setup mistakes are often not conceptual mistakes but copy-paste mistakes, forgotten updates, or small inconsistencies between files and LMS settings.

Improved R Logging

We also inspected the R history logs from an exam setting. The good news is that there were basically no signs of tampering attempts. I did not see commands such as system(), shell(), source(), file reads, package installation, or anything similar that would suggest students tried to escape the intended use of R.

Most commands were exactly what one would expect in a course on research methods and statistics in psychology:

• computing means and variances,
• using pnorm(), qnorm(), and qt(),
• calculating confidence intervals by hand,
• doing simple probability arithmetic,
• checking standard deviations and z-values.

There were also the usual mistakes. Some students mixed up pnorm() and qnorm(), used mad() although they probably wanted the mean absolute deviation, or struggled with sample versus population variance. This is useful information. The logs are not only a security tool; they are also a didactic diagnostic tool.

The logging itself has also improved. The original version used an R task callback and logged evaluated commands. This is useful, but it misses an important class of events: not everything a student types becomes a valid R expression. Syntax errors, for example, may never reach the task callback.

The new logging therefore keeps two files per session. The regular .Rlog records evaluated commands, runtime errors, and quit events. A separate raw .Rhistory file stores the literal console input. The raw history files are placed in a history subfolder so that they do not clutter the main log directory.

For example, if a student enters:

1 + 1
)
stop("bad")
q("no")

the evaluated-command log contains entries of this form:

2026-07-10 13:37:30.620115 | OK | 1 + 1
2026-07-10 13:37:30.662878 | ERROR_MESSAGE | Error: unexpected ')' in ")"
2026-07-10 13:37:30.674524 | ERROR_MESSAGE | Error: bad
2026-07-10 13:37:30.674967 | QUIT | q(save = "no")
2026-07-10 13:37:30.675183 | SESSION_END | R session ended

while the raw history file preserves the actual input:

1+1
)
stop("bad")
q("no")

This distinction matters. The .Rlog is better for quick inspection, because it classifies commands and errors. The .Rhistory file is better for reconstructing what was actually typed, including failed syntax. Together, they provide a much more useful audit trail than either file would alone.

The Interesting Case: binom.test()

The most interesting entry was not a tampering attempt at all. It was a student using R well:

binom.test(19, 123, conf.level = 0.95)

This was a faster and more direct solution to a binomial confidence interval exercise. Instead of calculating the interval step by step, the student simply used the exact test function built into R.

This is not cheating in the technical sense. If R is available, and if binom.test() is available, then this is a perfectly reasonable statistical move. In fact, it is probably closer to how one should work outside an exam. The issue is that the question was apparently too easy to bypass with a single specialized function.

That changes the problem. We do not primarily need to worry about students trying to hack the system. We need to worry about writing questions that still measure understanding when a statistical computing environment is available.

One obvious response would be to restrict R more aggressively. But I am not sure this is the best direction. If the goal is to teach students to use statistical software responsibly, then using a built-in function should not automatically be treated as a problem.

Instead, this particular question type should be changed. For binomial confidence intervals, the most promising direction is to ask for backwards planning parameters rather than only asking students to compute an interval.

For example, instead of “calculate the confidence interval for 19 successes in 123 trials”, we could ask which sample size n is required to achieve a certain target width under plausible assumptions about the expected proportion.

This is harder to answer with one specialized function call. R can still help, but the student has to understand the relation between sample size, uncertainty, confidence level, expected proportion, and interval width. That is much closer to the kind of understanding we actually want to test.

Another attractive option would be to ask for a short explanation of the output. For example: “Use R if you want, but explain why the exact interval is asymmetric.” However, this would require reliable grading of free-text answers. We are working on LLM-assisted grading for this kind of response, but this will still take some time before I would rely on it in an exam.

Lessons

The main lesson is that logging changes the conversation. Before looking at the logs, it is easy to imagine dramatic misuse scenarios. After looking at the logs, the picture is calmer and more useful: students mostly used R as a calculator. The rare surprising case was not malicious, but competent.

For future exams, I would therefore focus on two things:

• keep logging because it provides both security evidence and teaching feedback,
• design questions for a world in which students have access to statistical functions.

The last point is the most important. Digital exams should not merely reproduce paper exams with a locked-down browser. If students have R, even in a restricted setup, the exam has to ask for understanding that survives contact with R.