FUURAA AI Knowledge Library · Research synthesis guide

How to review AI-generated research syntheses

Use six evidence gates to turn a fluent synthesis into a predefined question, reproducible search, auditable selection, source-level extraction, bounded synthesis and dated conclusion. Apply the guide to research briefs, policy reviews, industry reports, evidence maps and due diligence.

Enter the six evidence gates ↓Open the failure matrix

Published23 August 2026Evidence statusMethod synthesis grounded in primary review reporting, search, bias, provenance and generative-AI risk sourcesScopeResearch briefs, policy reviews, industry reports, evidence maps and due diligence

Many citations do not make a complete synthesis

A research synthesis depends on how evidence enters, changes and affects the conclusion—not on how fluent the prose sounds.

AI can accelerate discovery, screening, extraction, translation and summarisation, but every step can change the evidence set. Review must preserve question and protocol, search coverage, selection reasons, exact sources, extraction locations, bias appraisal, synthesis method and expiry to distinguish “many sources” from “a supported conclusion.”

Applicability boundaryThis is a public research method, not a FUURAA product-capability claim or an assessment of any model, provider, search tool, review or publisher. It does not replace domain expertise, statistical review, peer review or a formal systematic review, and is not medical, legal, investment, procurement, certification, audit or compliance advice.

Six rejectable evidence gates

Each gate must answer a decision question, produce minimum evidence and stop synthesis or narrow the conclusion when material unknowns remain.

Freeze the question, protocol and review type

Decision question: What exact question, population or context, concepts, outcomes, eligibility rules, review type and planned synthesis existed before sources were selected?
Minimum evidence: Dated protocol, question framework, inclusion and exclusion criteria, outcome definitions, planned subgroup or synthesis rules, review team and amendment log.
Stop condition: Stop when the question or eligibility rules appear to have been rewritten after seeing favourable studies, or the review type is unnamed.

Reconstruct the search and its coverage

Decision question: Which databases, platforms, registries, websites, languages, date ranges, search strings and supplementary methods could have found eligible evidence?
Minimum evidence: Complete search strings by platform, run dates, limits, grey-literature and citation-search methods, peer review of the search, result counts and deduplication rules.
Stop condition: Stop when the search cannot be rerun, relies on one opaque retrieval tool, omits material languages or sources without reason, or has no end date.

Audit selection, identity and source status

Decision question: Can every record move from discovery through deduplication, screening, full-text review and inclusion with explicit reasons and current source status?
Minimum evidence: Flow counts, record identifiers, duplicate and version relationships, excluded full texts with reasons, reviewer decisions, conflict resolution, corrections and retractions.
Stop condition: Stop when included studies cannot be reconciled with search results, selection was silently changed by AI, or preprints and final reports are double-counted.

Verify extraction and appraisal against source text

Decision question: Were study characteristics, methods, results and limitations extracted from exact locations and appraised with a suitable domain-specific method?
Minimum evidence: Structured extraction record, page or table locations, multiple reports per study, outcome definitions, human verification, risk-of-bias judgements and supporting quotations.
Stop condition: Stop when generated summaries replace source data, translations change meaning, appraisal labels lack reasons, or study reports are mistaken for independent studies.

Challenge the synthesis before accepting its fluency

Decision question: Are studies sufficiently comparable for the chosen synthesis, and are heterogeneity, missing evidence, weighting, contradictions and sensitivity visible?
Minimum evidence: Synthesis rationale, grouping rules, effect or theme definitions, heterogeneity and uncertainty, risk-of-bias integration, sensitivity analyses, contrary findings and excluded comparisons.
Stop condition: Stop when unlike populations, interventions, tasks, metrics or dates are compressed into one claim, or a narrative majority replaces explicit weighting and uncertainty.

Bound conclusions, provenance and expiry

Decision question: Which included evidence supports each conclusion, how certain and transferable is it, and what new evidence, correction or context change triggers an update?
Minimum evidence: Claim-to-study map, evidence certainty, applicability limits, conflicts, generation and human-review provenance, checked date, search end date, expiry and update trigger.
Stop condition: Stop when conclusions exceed the evidence set, AI contribution is undisclosed, source dependencies are hidden, or no dated recheck condition exists.

Minimum synthesis failure matrix

Check these conditions deliberately to expose search, selection, extraction and synthesis bias behind fluent prose.

01
The research question narrows only after unfavourable studies appear
Record affected studies, synthesis steps and conclusions, preserve the narrowest statement that remains, and specify whether search, screening, extraction or synthesis must be repeated.
02
One database or AI search tool supplies nearly all included evidence
Record affected studies, synthesis steps and conclusions, preserve the narrowest statement that remains, and specify whether search, screening, extraction or synthesis must be repeated.
03
Non-English, unpublished or negative evidence disappears without explanation
Record affected studies, synthesis steps and conclusions, preserve the narrowest statement that remains, and specify whether search, screening, extraction or synthesis must be repeated.
04
A preprint and final article are counted as two independent studies
Record affected studies, synthesis steps and conclusions, preserve the narrowest statement that remains, and specify whether search, screening, extraction or synthesis must be repeated.
05
AI extraction changes a denominator, effect direction or uncertainty
Record affected studies, synthesis steps and conclusions, preserve the narrowest statement that remains, and specify whether search, screening, extraction or synthesis must be repeated.
06
Risk-of-bias labels are present but never affect the synthesis
Record affected studies, synthesis steps and conclusions, preserve the narrowest statement that remains, and specify whether search, screening, extraction or synthesis must be repeated.
07
Incompatible tasks, populations or metrics are merged into one confident statement
Record affected studies, synthesis steps and conclusions, preserve the narrowest statement that remains, and specify whether search, screening, extraction or synthesis must be repeated.
08
A later correction, retraction or major new study does not reopen the review
Record affected studies, synthesis steps and conclusions, preserve the narrowest statement that remains, and specify whether search, screening, extraction or synthesis must be repeated.

Minimum research-synthesis review record

Let the next reviewer reconstruct why evidence was found, included, transformed and allowed to affect the conclusion.

01review question, type, publisher, date and evidence state
02protocol, eligibility criteria, planned synthesis and amendments
03databases, platforms, search strings, languages and search dates
04supplementary search, source counts, deduplication and coverage limits
05screening flow, excluded full texts, reasons and reviewer resolution
06study identity, versions, reports, corrections and retractions
07extracted data, exact locations, transformations and human verification
08appraisal method, risk-of-bias judgements and supporting reasons
09synthesis method, heterogeneity, missing evidence and sensitivity
10claim map, certainty, applicability, AI provenance, expiry and update trigger

Common evidence states

Bind conclusion status to the complete process and date—not citation count or textual fluency.

Supported

The protocol, reproducible search, selection flow, source-level extraction, appraisal, synthesis and dated conclusion can be reconstructed and independently checked.

Conditional

The synthesis is reviewable but conclusions hold only for named databases, languages, study designs, populations, outcomes, dates or analytical choices.

Mixed

Included studies, risk-of-bias assessments, subgroups or synthesis methods lead to materially different conclusions that should remain visible.

Insufficient

Question, search, selection, extraction, appraisal or synthesis evidence is missing; fluent prose and many citations do not substitute for a reviewable process.

FUURAA analysisCitation verification asks whether one source supports one sentence. Synthesis review must also ask why these sources were selected, how they were weighted, and how missing or conflicting evidence changes the conclusion. FUURAA recommends preserving every synthesis conclusion as question and protocol × search coverage × selection and extraction × appraisal and synthesis × cut-off date. AI can accelerate reading; it must not make selection an invisible black box.

Primary sources and non-transfer boundaries

These methods constrain reporting, search, selection, bias, provenance and AI risk; none independently proves a synthesis reliable.

Sources rechecked 23 August 2026. Each retains its publication timing, role in this method and non-transfer boundary.

29 March 2021PRISMA 2020 Statement

Provides a 27-item reporting framework and flow diagrams for explaining why a systematic review was done, what was identified and selected, and how findings were synthesised.

BoundaryPRISMA is a reporting guideline; complete reporting does not by itself prove that the search, appraisal, synthesis or conclusions are unbiased or correct.

Open primary source ↗26 January 2021PRISMA-S — Reporting Literature Searches in Systematic Reviews

Adds sixteen items for reporting databases, platforms, search strategies, dates, supplementary methods, peer review and record counts reproducibly.

BoundaryA reproducibly reported search can still be too narrow, outdated or poorly matched to the review question; reporting is not proof of coverage.

Open primary source ↗Version 6.5.1; chapter updated March 2025Cochrane Handbook — Chapter 4: Searching for and Selecting Studies

Details systematic approaches to sources, search planning, reference management, documentation, eligibility decisions and study selection.

BoundaryThe chapter is designed for Cochrane intervention reviews; its methods must be adapted explicitly before transfer to other domains or review types.

Open primary source ↗1 January 2016; online 16 June 2015ROBIS — A Tool to Assess Risk of Bias in Systematic Reviews

Structures review-level bias assessment around eligibility, study identification and selection, data collection and appraisal, synthesis and interpretation.

BoundaryROBIS was developed mainly for healthcare reviews of interventions, diagnosis, prognosis and aetiology; it is not a universal quality score.

Open primary source ↗W3C Recommendation, 30 April 2013W3C PROV-DM — The PROV Data Model

Defines entities, activities, agents, derivation, revision and quotation relationships for reconstructing source selection, extraction, translation and synthesis transformations.

BoundaryProvenance can reveal how a synthesis was produced, but a complete trail does not make its sources, transformations or conclusions reliable.

Open primary source ↗26 July 2024; updated 8 April 2026NIST AI 600-1 — Generative AI Profile

Extends AI risk management to generative-AI risks including confabulation, information integrity, monitoring and human verification.

BoundaryThe cross-sector profile does not prescribe systematic-review conduct, assess an individual synthesis or certify generated research content.

Open primary source ↗

Continue checking

Move from a complete synthesis back to individual citations, AI claims and evidence records.

Verify generated citations Evaluate an AI claim Build an evidence record Enter AI Evidence Atlas