The measured dataset is in Polish. Case titles, sources, questions, answers and reference explanations stay in Polish so they match the actual model runs. Downloads contain the original Polish dataset and project.
Catch the AI mistake
The answer sounds convincing. Does it match the source? Read it, decide and check the explanation.
Data is not ready yet
This lab needs a complete set of cases and model results. The first case will appear once they are included.
Does this answer meet the evaluation rule?
JEV returns a probability; Luna reports a score from 0 to 1. We have not checked either model for calibration on this dataset.
All cases reviewed.
This measures agreement with the stated rule for this dataset, not your overall ability or model quality.
This will reveal the answers.
The comparison uses every case, including ones you have not reviewed. You will see model results and their mistakes.
How many mistakes get through?
A higher threshold stops more answers. See how many needed stopping and how many were correct.
An answer passes when its model score is equal to or greater than the threshold. The reference rule stays the same.
We have not checked model calibration on this dataset. The same threshold may mean different levels of risk for different models. We only count cases in this set.