A
Direct grading
Grade the submission as it is.
The model receives the student code and a human-designed rubric, then scores each criterion directly.
- Student code
- Apply rubric
- Assign score
Research journey · 2025—present
From grading code to trusting the rubric.
My research asks where generative AI can genuinely support programming education—and where human judgment still needs to lead.
01peer-reviewed paper
02connected research stages
2025IEEE CASCON publication
01
The starting question
What happens after a program fails the tests?
Traditional unit tests are useful, but they usually reduce an answer to pass or fail. In an introductory programming course, a partially correct solution may still demonstrate meaningful understanding. My first study explored whether an AI system could read that code, apply a rubric, and award partial credit in a way that remains close to human teaching assistants.
Published · IEEE CASCON 2025
Evaluating Generative AI for CS1 Code Grading: Direct vs Reverse Methods
Ahmad Memon · Abdallah Mohamed
IEEE International Conference on Collaborative Advances in Software and Computing · pp. 178—183
B
Reverse grading · proposed method
Fix first, then infer the grade.
The model repairs the code, classifies the fixes, and estimates a score from their number and complexity.
- Student code
- Repair & classify
- Infer score
A focused view of the results
Promising alignment—plus a clear reason to keep people involved.
The study compared GPT-4 grading against human teaching-assistant scores using synthetic CS1 Java submissions grouped as poor, moderate, and good. The chart shows the original 10-point rubric results.
Poor
Human TA2.27
Direct AI3.20
Reverse AI4.25
Moderate
Human TA5.68
Direct AI6.12
Reverse AI6.73
Good
Human TA7.83
Direct AI7.54
Reverse AI8.99
The experiment used controlled, synthetic submissions. These findings are an early signal—not evidence for fully automated classroom grading.
02
The question moved upstream
If AI uses a rubric to grade, who checks the rubric?
The published study showed how strongly grading behavior depends on clear criteria and careful prompt design. That led to the next stage of my research: not simply generating rubrics faster, but creating an auditable way to decide when an AI-generated rubric is safe to use as a draft—and when an instructor should intervene.
Ongoing research · Unpublished manuscript
Trusting AI-Generated Rubrics in CS1
This work treats trust as a workflow, not a feeling. AI drafts are compared with instructor-authored reference rubrics using coverage, point-weight similarity, and semantic alignment. A threshold then routes the draft toward acceptance or human review.
01QuestionDefine the CS1 task
→
02AI draftGenerate a structured rubric
→
03AlignmentCompare with a human reference
→
04DecisionAccept or escalate
Controlled pilot · Early findings
30CS1 programming questions
0.80illustrative review threshold
The threshold is a policy choice, not a universal definition of rubric quality. It is designed to concentrate human attention where mismatch risk appears higher.
66.7%accepted as drafts
33.3%flagged for review
The escalated cases repeatedly exposed differences in granularity, terminology, and scoring structure—the areas where instructor judgment matters most.
The thread through both studies
AI should make educational decisions easier to inspect—not harder to question.
Assist, don’t replace
Use AI for the first pass while preserving human control over ambiguous and high-impact decisions.
Measure alignment
Evaluate how closely outputs match real expectations instead of trusting confident-looking answers.
Design for escalation
Treat review as a normal part of a trustworthy system, not as evidence that automation has failed.