No single similarity score cleanly separates copied work from independent work. In a set of 4,100 CS1 submissions across Python, Java, C++, and JavaScript, only the top 10% of pairwise similarity scores crossed 70%, while the median stayed below 36% in every language. The full code similarity score distribution shifts with language, assignment scaffold, and class level.
If you grade intro programming courses, you have probably stared at a similarity report and wondered whether 64% is high or normal. The answer depends on more than the number. After three semesters of running the same assignments through MOSS and through Codequiry's code plagiarism checker, I found that a score that would be routine in Java can be a serious red flag in Python. If you treat one fixed threshold as universal, you will either flag too many honest students or miss copied work.
What 4,100 Submissions Show About Language-Level Differences
The dataset came from 14 sections of CS1 and CS2 at a public Midwestern university. The eight assignments were identical across sections within a given semester, and the starter scaffolding was also identical. I looked at pairwise similarity scores after excluding empty submissions, late resubmissions, and one section that switched languages mid-semester. The remaining 4,100 submissions split roughly as follows: 1,600 Python, 1,100 Java, 940 C++, and 460 JavaScript.
Pairwise similarity is not an average across the class. It compares each student submission against all other submissions in the same assignment group. A student with no close neighbors still generates a list of maximum similarity values. Those maxima are what most instructors actually see when they open a report, so that is the distribution that matters in practice.
Here is what the maximum pairwise similarity distributions looked like across the four languages for two comparable assignments: a loop-based list/array manipulation task and a simple class definition task.
| Language | Median similarity | 75th percentile | 90th percentile | Typical false-positive zone |
|---|---|---|---|---|
| Python | 18% | 42% | 61% | 70% and above |
| Java | 27% | 55% | 74% | 80% and above |
| C++ | 32% | 58% | 76% | 82% and above |
| JavaScript | 22% | 48% | 67% | 75% and above |
Two things stand out. First, the medians are much lower than many instructors expect. A pair of honest students in Python often sits between 10% and 25% because the language provides many idiomatic ways to solve the same problem. Second, the dangerous zone starts earlier in Python than in C++ or Java. In C++, mandatory boilerplate, header includes, and syntax constraints push honest work into the 30% to 60% band before any copying happens.
Why Java and C++ Produce Higher Baselines Than Python
Boilerplate code drives a large share of the difference. A Java class with a public main method cannot avoid repeating the same declaration structure. Consider the standard starting point for a Java assignment:
public class Student {
public static void main(String[] args) {
System.out.println("Hello, world");
}
}Every submitted file with that structure begins with roughly five identical tokens before any student logic appears. If the assignment is only 35 lines long, those five lines alone can account for 15 to 20% of the raw token similarity. C++ is even more constrained because students must include the same headers, declare the same return types, and often wrap code in the same namespace conditions.
#include <iostream>
int main() {
std::cout << "Hello" << std::endl;
return 0;
}Python has far less mandatory structure. A student can solve a vowel-counting problem with a loop, with a string method chain, with a list comprehension, or with a one-line generator expression. Those solutions may be semantically equivalent but tokenically very different. That is why Python medians sit lower. It also means that when two Python submissions reach 70% similarity, the explanation is rarely just shared boilerplate.
C++ also introduces a second effect: compiler-driven uniformity. Students in CS1 tend to compile against the same GNU or Clang toolchain, include the same standard library headers, and follow the same instructor-provided coding convention. Those shared constraints compress independent solutions toward the same token stream even when no copying occurs.
Reading the Code Similarity Score Distribution by Assignment Type
Language is only one variable. The assignment scaffold matters just as much. An assignment where students fill in a single method inside a provided class will produce a higher baseline than one where students write a full program from scratch. I grouped the 4,100 submissions into three scaffold levels and recalculated the distributions.
Full-program assignments, where students write the entire file, had the lowest medians. In Python the median maximum similarity was 14%. Java came in at 23%. C++ was 28%. This is the cleanest comparison because most of the code is student-authored.
Fill-in-the-method assignments, where students completed a function or method inside a large starter file, had dramatically higher medians. Java jumped to 41% median, C++ to 47%, and Python to 29%. The starter code dominates the token stream. A student who completes a 15-line method inside a 120-line class starts with roughly 85% identical structure before writing a character of original code.
GUI or framework-based assignments had the highest and least reliable baselines. In JavaScript assignments built around a small DOM manipulation starter, the median maximum similarity reached 38%, and the 90th percentile sat at 81%. Much of that similarity is framework scaffolding and event-listener boilerplate, not copied logic.
A 72% similarity score in a Java exercise with a three-line starter file can mean nothing, while a 58% score in a Python list-comprehension assignment can mean copied logic that was then reordered.
If you are using a plagiarism checker for code that only returns a single number, you are missing the scaffold context. A useful report must show what part of the match comes from starter code, what part comes from common idioms, and what part is distinctive enough to warrant manual review.
Boilerplate, Starter Code, and the 80% Trap
One of the most common mistakes in intro-course plagiarism review is applying a fixed 80% threshold to every assignment. The data above explains why that rule fails. In a C++ fill-in-the-method assignment, honest students can exceed 80% pairwise similarity against four or five classmates simply because the starter file was identical and the method was short.
Here is an example from a Java fill-in-the-method assignment. The starter file included this method stub:
public int countHighValues(List<Integer> values, int threshold) {
// TODO: Count how many values exceed the threshold
return 0;
}Each student replaced the comment with a loop or a stream expression. The submitted files stayed highly similar because the surrounding 80 lines were identical. The meaningful distinguishing code often occupied less than 10% of the file. When I compared the extracted method bodies separately, the similarity scores dropped sharply, and only two suspicious pairs remained above 60%. The rest were honest variations.
That is why token-level similarity alone is not enough. A detector that can isolate student-authored regions from starter regions, or that uses an AST comparison to focus on structural logic rather than raw tokens, returns a much more actionable signal. Codequiry performs this separation by first identifying the shared starter code and excluding it from the similarity calculation where appropriate, which prevents the 80% trap from consuming a TA's entire weekend.
Refactoring-Resistant Similarity and Why Manual Review Still Matters
Students who intentionally copy rarely submit unchanged files. They rename variables, reorder methods, change loops to for-each loops, and alter whitespace. A text diff or simple token match can miss those changes. Token-based tools with normalization and AST comparison catch a meaningful subset of them, but no tool eliminates the need for human review.
Consider this original Java snippet:
for (int i = 0; i < items.size(); i++) {
if (items.get(i) > 10) {
count++;
}
}And this refactored version:
for (int value : items) {
if (value > 10) {
++count;
}
}Raw token similarity drops because i becomes value, get(i) disappears, and count++ becomes ++count. An AST comparison still recognizes the loop traversal, the condition, and the increment as structurally equivalent. That structural match is the strongest plagiarism signal in refactored code.
But refactoring also creates false positives. Two students who independently learned the same standard loop pattern will produce structurally similar code. A manual review needs to look for the specific choices that reveal copying: identical comments, identical unusual variable names, identical off-by-one bugs, the same unnecessary temporary variables, or the same misunderstood API call. Those details appear in a good side-by-side report but not in a single score.
When I review flagged pairs, I ignore the percentage at first and read the highlighted fragments. If the distinctive lines match despite renaming, I mark it copied. If only the generic loop structure matches and the comments, error handling, and naming differ, I clear it. This two-stage process caught 31 confirmed cases across 4,100 submissions, with only 9 false positives that required follow-up. That is far better than a threshold-only approach.
When AI-Generated Code Enters the Similarity Distribution
AI-generated code complicates the picture. A student who submits ChatGPT-generated code may show low peer similarity because the output is not copied from a classmate, but it can still show high web-source overlap if the model reproduced a Stack Overflow pattern. The code similarity score distribution does not capture that kind of originality concern by itself.
Across the same 4,100 submissions, I saw 64 files with peer similarity below 25% but web-source similarity above 45%. Manual review revealed that most of those were strings of code lifted from tutorial sites or generated by a model trained on those sites. Some were honest reuse with citation, some were unattributed, and a few were entirely LLM-generated.
The signal differs from traditional plagiarism. Peer similarity is low because no two students asked the model the same way. Web similarity is moderate because the model often reproduces known tutorials. The comments may be unusually polished, the variable names generic, and the code structure monotonic. Those patterns are not visible in a pairwise similarity score. Pairing similarity checks with an AI code detector gives instructors a clearer picture of whether a submission came from the student or from a model.
When I combined Codequiry's peer similarity, web-source matching, and AI detection into one review pass, the number of false-positive cases that needed a second look dropped. The reason is simple: no single signal is enough. Peer similarity catches copying from classmates. Web matching catches copying from tutorials. AI detection catches synthetic code that may be original against the peer set but not written by the student. A review workflow that uses all three avoids the blind spots that come from relying on any one percentage.
Setting Language-Aware Review Thresholds
Based on the distributions above, I no longer use one cutoff. I set three review bands for each language and scaffold level.
For Python full-program assignments, I review anything above 55% and treat anything above 70% as high priority. For Java and C++ full-program assignments, I start review at 65% and treat 80% and above as high priority. For fill-in-the-method assignments, I push those thresholds up by 10 to 15 points because the starter code raises the baseline.
The point is not to replace judgment with numbers. It is to use the numbers to find the right pairs to review. If your current workflow flags 40% of submissions, the thresholds are too low and you are burning teaching-assistant time on boilerplate. If it flags only one or two pairs in a 150-student section, the thresholds are too high and you are likely missing copied work.
I also stopped comparing similarity scores across assignments. A 60% score in week 2 is not the same as a 60% score in week 9. Early assignments with short programs and dense starter code produce naturally high values. Later assignments with larger student-written codebases produce lower values even when copying occurs. Normalize within the assignment group, not across the semester.
A Practical Review Workflow for TAs and Instructors
The workflow that saved me the most time looks like this. First, run the assignment through Codequiry's code plagiarism checker and wait for the peer, web, and AI signals to come back in one report. Second, filter out pairs whose similarity is entirely explained by starter code or standard headers. Third, sort the remaining pairs by a combination of peer similarity, web overlap, and AI probability rather than by peer similarity alone. Fourth, manually review the top 10 to 20 flagged pairs.
This approach caught all 31 confirmed plagiarism cases in my dataset and reduced the review load from over 900 flagged pairs to 73 pairs worth reading. The difference came from excluding starter code, comparing refactored structures, and adding web and AI signals to the weighted filter.
The lesson from 4,100 submissions is that similarity scores are useful but context-dependent. Learn the distribution for each language and assignment type before setting thresholds. Then pair similarity with structural analysis, web-source matching, and AI detection. A thoughtful instructor using the right tools will miss less and falsely accuse far less than one who relies on a single fixed number.
Frequently Asked Questions
What similarity score indicates copied code in a CS1 assignment?
There is no universal cutoff. In Python full-program assignments, 55% to 70% is worth review. In Java or C++ assignments with substantial starter code, honest submissions can reach 80% before any copying occurs. The threshold should rise as scaffolding increases and fall for large student-written programs.
Do similarity thresholds differ across programming languages?
Yes. Python tends to produce lower baselines because the language allows many idiomatic solutions. Java and C++ produce higher baselines because of mandatory boilerplate, headers, and strong syntactic constraints. JavaScript varies with the amount of framework code in the starter file.
Can AI-generated code raise a student's similarity score?
Usually not against peers, because different students prompt different outputs. AI-generated code