Multimodal AI Exam Evaluation
Marks scanned exam papers against the mark scheme and hands them back annotated.
2026 · Client project
Overview
Students upload a past exam paper, as a PDF or a few phone photos. A vision model marks every answer against that exam's mark scheme, and the student gets their own paper back, highlighted in red, orange and green, with feedback on each question and topics to revise.
Calling a model on an image is the easy part. The hard part is making marks consistent and checkable across uneven scans, stopping the model from awarding marks for answers it can't see, and making sure a failed run never costs a student a credit. I built the AI grading pipeline and backend engineering with a small team at KNYSYS.
How a paper gets graded
Upload
A 12-page biology paper, photographed on a phone.
Quality check
Every page gets a sharpness score. Blurry photos are rejected with a clear message before any AI runs.
Prepare pages
Each page becomes two images: a smaller one for grading and the full-resolution original for annotation.
Grade
A vision model marks every answer against the mark scheme. Papers over 8 pages are graded in overlapping 6-page chunks, then merged.
Validate
The output must fit a strict schema, or it goes back to the model with the errors.
- marks ≤ max marks
- low confidence → human review
- answer not visible → 0 marks
Annotate
An image model highlights each answer on the original page.
- Q1a correct
- Q2 partial
- Q3b incorrect
Result
The student's own paper, marked and highlighted, with feedback per question, a model answer and topics to revise.
If grading still fails after 3 retries, the attempt is marked failed and the student's credit is refunded.
What made it hard
Students need to trust the marks, and teachers need to be able to check them. Four things made that hard:
- Uneven input: phone photos, skewed pages and faint handwriting.
- Long papers: one model call over a 30-page paper is slow and less reliable.
- Model mistakes: a vision model can award marks for an answer that isn't on the page.
- Paid attempts: every paper uses a credit, so a crash must never eat one.
Architecture
- Upload PDFs or photos and merge them into one PDF
- Reject blurry pages
- Render page images at two resolutions
- Grade with GPT-5.2, falling back to Gemini 2.5 Pro, in chunks for long papers
- Validate against the schema and retry with the errors
- Annotate each page with an image model
- Save results, email the student and serve signed download links
Key features
- Vision grading through LiteLLM and Instructor, with validation errors fed back to the model for a retry
- Automatic fallback from GPT-5.2 to Gemini 2.5 Pro
- Chunked grading for papers over 8 pages: 6-page windows with a 1-page overlap, merged into one result
- Guards in the schema: marks can't exceed the maximum, answers below 0.7 confidence are flagged for review, and answers not visible on the page get zero marks
- Raw model output stored with every result for debugging and review
- A study panel with a model answer, topics to revise and matched learning resources
- Background processing on Celery with time limits, up to 3 retries, cleanup of partial results and credit refunds on failure
- Plan quotas, upload rate limits and private S3 storage with signed links
Built with
What I learned
- Rules that matter belong in code, not the prompt. A vision model will sometimes award marks for an answer that isn't on the page, or give more marks than a question is worth. So the model has to say whether it can actually see each answer. The schema then enforces the rules: unseen answers get zero, no score can exceed the maximum, and anything that breaks the schema goes back to the model with the errors attached.
- Uncertainty should be visible. The model rates its confidence for every question, and anything under 0.7 is flagged for a human to review instead of quietly becoming a final mark. The raw model output is stored with every result, so when a student questions a grade, there's a record of exactly what the model saw and said.
- Most of the engineering is for the runs that go wrong. Blurry photos are rejected before a paid model call. Long papers are graded in overlapping chunks, so one huge request can't sink the whole paper. Failed jobs retry up to three times with partial results cleaned up first, and if a paper still fails, the student's credit is refunded. Students pay per paper, so a failed run should never cost them anything.