Source reliability and adversarial verification

AI research tools make it trivial to produce citation-heavy output. The danger: confident-looking reports built on shaky sources. Source reliability discipline is what separates useful AI research from reputation risk.

The hierarchy of sources

From most to least reliable (generally):

1. Primary research — original peer-reviewed papers, original datasets.
2. Systematic reviews and meta-analyses — synthesis of primary research.
3. Reputable secondary sources — published books, credible industry reports.
4. Expert commentary — individual experts writing in their domain.
5. Mainstream journalism from quality outlets.
6. Blog posts from credible practitioners.
7. Anonymous or low-credibility web content.
8. AI-generated content (including other AI tools' output).

AI research tools often retrieve from tiers 3-6. Understand where your sources sit.

The reliability checks

For any source you're considering citing:

• Authorship. Real person or institution? Verifiable? Have they written in this area before?
• Publication. Who published this? Peer-reviewed journal? Vendor whitepaper? Personal blog?
• Date. When written? In fast-moving fields, 2-year-old sources may be stale.
• Citations in it. Does the source cite its claims? Or is it making confident claims without basis?
• Corroboration. Do other credible sources say the same thing?

Not every source needs to be peer-reviewed primary research. But you should know where each source sits.

Adversarial verification

The technique: actively try to disprove a claim before accepting it.

• Search for counter-evidence. If a source says X, search specifically for "X is wrong" or "arguments against X."
• Find the opposing view. On contested topics, read the best counter-arguments.
• Check the underlying data. If a source makes a statistical claim, can you find the dataset?

If 20 minutes of adversarial searching finds nothing, the claim is probably sound. If it finds substantive disagreement, your synthesis must reflect that.

Common AI-research failure modes

The "sounds right" trap. A report confidently states something; no effort to verify. Later, an expert points out it's wrong. Damage to credibility.

Source laundering. AI summarizes a summary of a blog post that misrepresented a study. Four levels removed from the primary source.

Confirmation bias amplification. You search for evidence of what you believe; AI finds it; you conclude. Missing: the evidence against.

Stale consensus. AI cites a 2019 paper as current. In a fast field, 2019 is ancient history.

The verification protocol

For any research that'll inform a real decision or be published:

1. List every claim you'll cite or build on.
2. Identify the source for each.
3. Evaluate source reliability per the hierarchy above.
4. Trace to primary source when feasible.
5. Verify the claim against the primary source directly.
6. Adversarial check on high-stakes claims.
7. Document the process (which sources, which verifications).

This is 2-4x the work of naive AI research. It's also the difference between work that holds up and work that embarrasses you.

When to stop

Diminishing returns applies. For a personal-use research brief, lighter verification is fine. For a public report or a high-stakes decision, full protocol.

Calibrate to the stakes.

Credibility markers of good AI-assisted research

• Clear citations to primary or tier-2 sources.
• Explicit acknowledgment of uncertainty.
• Discussion of counter-arguments where relevant.
• Dated claims.
• Documented methodology — how was the research done?

If your output has all of these, it's defensible. If it lacks several, be cautious about shipping it.

The professional responsibility

When you publish AI-assisted research under your name, you vouch for it. "AI wrote this, don't blame me" isn't a defense. If it's wrong, you're wrong.

This standard is appropriate. AI tools are assistants; the human authoring the work bears responsibility.