Check Whether AI Is Replacing Your Learning

Finishing a problem with AI does not by itself show that you learned the method. After the session, close the chat and test four things: can you explain the method, reproduce the key step, transfer it to a different example, and verify the result?

The check is diagnostic. A failed step is not a verdict about your ability or about AI in general. It tells you which part of the learning process still needs work.

Assistance and independent performance are different outcomes

AI can improve performance while it is available. The learning question is what remains when the assistance is removed.

A randomized field experiment by Bastani and colleagues studied GPT-4-based assistance during high-school mathematics practice. Students using a standard chat interface performed better during assisted practice, but their later unassisted exam performance was worse than the control group’s; a tutor design with teacher-developed guardrails reduced that negative effect. The result is specific to that setting, subject, population, model, and instructional design. It does not establish that every use of generative AI harms learning. See the published PNAS study.

Other designs can produce different results. A study by Pardos and Bhandari found learning gains from ChatGPT-generated mathematics help that were comparable to gains from human tutor-authored help within its experiment. See the PLOS ONE study.

Taken together, these studies support a narrow practical conclusion: do not infer learning from tool-assisted completion alone. Check what you can do independently in the conditions that matter for your course.

Start with one skill, not the whole assignment

Name the specific capability the AI-assisted session was supposed to support:

Skill: Trace how a loop variable changes and explain why the loop stops.
AI assistance used: One explanation and one worked example.
Independent condition: No chat, but blank paper and the language reference are allowed.

Course rules determine which references, tools, and collaboration are permitted. Match the check to the real learning objective. If memorization is not required, do not turn the exercise into a memory contest.

The AI tutoring guide explains how to preserve a useful reasoning step during the interaction. This article begins after that interaction ends.

Run four checks

1. Explain

State the method in your own words without reopening the AI answer.

For a loop, explain:

• what state changes;
• what remains true during each iteration;
• what condition stops the loop; and
• what result should exist afterward.

Do not grade the explanation by how polished it sounds. Check whether it names the mechanism accurately enough to guide the next step.

If the explanation is vague, return to the course source or request a smaller explanation focused on the missing concept. Avoid asking for another complete solution.

2. Reproduce

Start from a blank page or file and recreate the central step.

For the loop example, write only the initialization, condition, and state update. You do not need to recreate an entire assignment. The goal is to find out whether the mechanism can be produced without copying.

If you know what the code should do but cannot produce the syntax, consult the permitted language reference. That result points to a syntax retrieval problem, not necessarily a conceptual one.

3. Transfer

Apply the same method to a meaningfully different example.

If the AI example counted upward from 0 to 4, trace a loop that processes a three-item list or counts downward to a boundary. Change the structure that matters, not only the variable names.

Transfer matters because memorizing one worked example can look like understanding. A different input, context, or representation tests whether you can identify the underlying method.

Keep the transfer task small and separate from graded work unless the course explicitly permits AI-assisted practice on that material.

4. Verify

Predict the result, then compare it with evidence:

• trace the state by hand;
• run an expected case;
• run a boundary case;
• compare terminology with the textbook or official documentation; or
• ask an instructor or teaching assistant to review the reasoning.

Verification checks the method and your result. Confidence is not evidence, whether it comes from the student or the AI.

For a factual output rather than a learning diagnosis, use Test an AI-Generated Claim with the Scientific Method.

Interpret the first failed step

Use the earliest failure to choose the next action:

These are working interpretations, not diagnoses of a learning disorder or proof of mastery. One short check provides evidence about one skill under one set of conditions.

An illustrative loop check

Suppose AI helped explain this pattern:

total = 0
for value in values:
    total += value

Close the chat and try:

1. Explain what total represents before and after each iteration.
2. Write the pattern from a blank file.
3. Adapt it to count values that satisfy a condition instead of summing them.
4. Predict and test empty input and a three-item input.

If step 3 fails, do not request the completed adapted code immediately. State the difference between “sum values” and “count matching values,” then ask for one hint about which state should change.

Record only the evidence you need

Write a short note titled AI independence check — [skill]:

Explain: demonstrated | needs work
Reproduce: demonstrated | needs work
Transfer: demonstrated | needs work
Verify: demonstrated | needs work

Evidence:
Next learning step:
Course or accessibility condition:

The note should help you decide what to practice next. Do not turn every AI conversation into a large assessment record.

Accessibility needs can change the appropriate medium and permitted support. Independent understanding does not require a single way of speaking, writing, recalling, or interacting. Use the course’s approved accommodations and evaluate the intended skill rather than an unrelated barrier.

Common mistakes

• Reopening the chat during every step. The check needs a clear independent condition.
• Testing the exact worked example. Change the relevant structure or input.
• Treating recall as the only evidence. Use permitted references when the course tests application rather than memorization.
• Calling one failure proof that AI harmed learning. The failure only locates missing evidence.
• Calling one success mastery. Retention and broader transfer require more than one short check.
• Ignoring course policy. Practice material and AI help still need to fit the assignment boundary.

Do this now

Choose one skill from your most recent AI-assisted session. Close the chat and run the explain, reproduce, transfer, and verify checks. Stop at the first failed step and choose one smaller next action.

Log what you learned

The independence-check note is the learning record. Save the observed evidence and next step. Before the next AI request, write down what you already know, what remains uncertain, and what you tried.

Further reading

• Bastani et al., “Generative AI without guardrails can harm learning”
• Pardos and Bhandari, “ChatGPT-generated help produces learning gains equivalent to human tutor-authored help on mathematics skills”