Compare Studies Before Calling Them Contradictory

Write a comparison paragraph that identifies comparable conditions and leaves unresolved disagreement explicit.

By Ian Fang Beginner 15 minutes
A student compares two studies side by side and discovers that their measurement timing differs.

Before describing two studies as contradictory, check whether they examined the same question under comparable conditions. Compare the participants, intervention or exposure, comparison group, outcome, and timing. A difference in any of these may change what the results mean.

Your outcome is a short synthesis: where the studies agree, where they differ, and what remains unresolved. You do not need to force a winner.

State the question both studies would need to answer

“Does this study method work?” hides several questions. Does it help immediate recall, retention after a delay, transfer to an unfamiliar problem, or confidence? Compared with what activity, and for whom?

Write the question you want to answer before comparing the papers. Then read each source independently. First check what each source actually supports; a misleading summary can create a disagreement absent from the papers themselves.

The UNC Writing Center recommends organizing a literature review around connected ideas rather than presenting an isolated summary of each source. Its literature-review guide provides broader writing guidance. The comparison below is a small preparation step, not a complete literature-review method.

Compare the conditions that could change the answer

Keep the following questions beside the papers. Write answers only where they help explain the comparison; an existing reading note is sufficient.

  • Who or what was studied, and how were participants or cases selected?
  • What intervention, exposure, or practice was examined, and what was the comparison?
  • What outcome was measured, how was it measured, and when?
  • What features of the design limit the conclusion?

If a paper does not report a needed detail, mark it unknown. Do not infer a shared condition because both titles use the same broad term.

An apparent disagreement with different outcomes

The following descriptions are synthetic teaching examples, not real studies or evidence for a study strategy.

Feature Synthetic study A Synthetic study B
Activity compared Rereading versus practice recall Rereading versus practice recall
Material A short passage A short passage
Outcome timing Immediately after practice After a delay
Reported direction Higher recall after rereading Higher recall after practice recall

These descriptions differ on outcome timing. They cannot establish that one activity is better for every purpose. They also do not, by themselves, prove that timing caused the difference: other method details remain unspecified.

A defensible comparison paragraph is:

The synthetic studies report different directions of performance, but they measure recall at different times. They therefore do not answer the same outcome question. The descriptions do not establish which activity would produce better performance under a shared testing schedule.

Notice the final sentence. Finding a plausible reason for disagreement is not the same as testing that explanation.

What if the conditions appear to match?

Now imagine that both synthetic descriptions measure delayed recall. The results still differ. You can no longer explain the difference simply by pointing to timing.

Read the full methods and results. Check the implementation, materials, sample, comparison activity, measurement, and reported uncertainty. Determine whether the papers use the same data or independent samples. Two articles built from one dataset do not automatically provide two independent replications.

If the available information does not resolve the difference, say so:

Both reports address delayed recall under apparently similar conditions, but their results differ. The information reviewed here does not establish why.

That is an honest synthesis. Avoid inventing a methodological flaw to protect your preferred result.

Do not count papers as votes

The number of papers on each side is not enough to establish a conclusion. Studies may differ in relevance, design, precision, and independence. A review may discuss the same studies you already counted separately.

Likewise, a result described as statistically significant in one study and not significant in another does not by itself establish a statistically significant difference between the studies. Gelman and Stern explain this error in their paper on comparing statistical significance. If your argument depends on that comparison, use the statistical methods required by your course and seek appropriate help. This workflow does not calculate a pooled effect or replace a systematic review.

For a student paragraph, it may be enough to describe the evidence boundary and explain why a broader conclusion remains open. Do not claim comprehensive coverage from a convenient pair of papers.

Write around the comparison

Lead the paragraph with the shared question or relevant distinction. Describe each source’s contribution, then state what the comparison supports. Keep source citations beside the corresponding claims.

When you return later, you should be able to reopen the passages behind that synthesis. Keep the source locations connected to your notes so a polished paragraph does not lose its evidence trail.

Do this now

Choose two sources you already describe as disagreeing. Check whether they measure the same outcome at the same time.

Log what you learned

Save the comparison in your existing draft or reading note. State the most consequential difference you found, or record that the disagreement remains unresolved. Preserve unknown details instead of filling them with an explanation you have not tested.