Methodological Compliance for The Use of Loon AI® in Evidence Synthesis: How Loon's Confidence Calibration Meets NICE, CDA-AMC, and ISPOR Auditing Requirements

Technical Brief for Market Access and HEOR professionals.

Introduction: The Mandate for Trustworthy AI in HTA Submissions

For HEOR professionals, the adoption of Large Language Models (LLMs) in evidence synthesis presents a paradox: immense efficiency gains versus compliance uncertainty. While AI can revolutionize manual Systematic Literature Review (SLR) processes—demonstrably reducing human review timelines from over 6,000 person-hours to approximately 85 hours for complex reviews—generic models are plagued by uncalibrated confidence and the risk of generating factually incorrect data (hallucination).¹

This fundamental technical instability is the source of regulatory anxiety. Global HTA bodies, including the National Institute for Health and Care Excellence (NICE), the Canadian Drug Agency (CDA-AMC), and ISPOR, have established rigorous frameworks to govern AI use. This brief outlines how the Loon AI® confidence-calibrated technology, based on peer-reviewed validation, is specifically engineered to satisfy the explicit requirements of these bodies, thereby de-risking HTA submissions and ensuring methodological rigour.

I. The Foundational Requirement: Quantifiable Trust and Accuracy

The core technical failure of generic LLMs lies in their inability to provide reliable Uncertainty Quantification (UQ). They often report high confidence even when their answers are incorrect, making human oversight ineffective.²Compliance with HTA standards demands that AI tools not just perform, but prove their reliability.

Loon addresses this through a validated, confidence-calibrated methodology, published in Value in Health (2025). The system provides a quantitative confidence score for every screening decision, which is then used to structure the Human-in-the-Loop (HITL) workflow.

Peer-Reviewed Compliance Metrics

The validation confirmed that by routing only low-confidence outputs for human review (approximately $\le 5.8\%$ of citations), the overall methodological quality of the SLR reached a superior level, satisfying HTA requirements for rigour.

II. Regulatory Compliance: Meeting HTA Mandates

The Loon system is architecturally designed to provide the specific evidence required by major regulatory and scientific organizations to validate the use of AI in evidence synthesis.

2.1. NICE & CDA-AMC: Human Augmentation and Auditability

Both the NICE Position Statement ³ and the CDA-AMC Guidelines ⁴ require that AI must "augment human involvement, not replace it" and that users must report the risks (e.g., bias) and the steps taken to address them.

2.2. ISPOR ELEVATE-AI LLMs Framework: Mitigating Hallucination Risk

The ISPOR ELEVATE-AI framework addresses the core technical flaws of LLMs in HEOR, specifically the challenges of hallucination, data inaccuracy, and the need for rigorous reporting.⁶

III. The Business Case: De-Risking Submissions and Optimizing Resources

For purchasing Directors and VPs, the decision to adopt this technology hinges on three factors: compliance assurance, cost efficiency, and time savings.

1. Compliance Assurance: By adopting a peer-reviewed, confidence-calibrated solution, the HEOR department gains quantifiable evidence (99.0% accuracy, 99.9% NPV) to defend against regulatory challenges regarding AI usage. This directly protects the integrity of market access dossiers.

2. Resource Optimization: The validated ability to focus expert review only on the lowest-confidence decisions achieves unparalleled efficiency. This optimization frees up Methodologists and clinicians from low-value screening, reallocating their time to critical tasks like economic modeling and interpretation.⁸

3. Time-to-Market Acceleration: The reduction of a complex systematic review's workload from over 6,000 person-hours to approximately 85 hours provides an immediate and material advantage in compressing evidence generation timelines, which is critical for supporting accelerated submissions and achieving faster time-to-market.⁸

IV. Conclusion: A De-Risked and Fast-tracked Path to Market Access

The strategic advantage of adopting Loon's confidence-calibrated AI is the removal of the methodological and regulatory risk associated with generic LLMs or unvalidated, uncalibrated tools. The high-stakes environment of HTA demands validation and the highest scientific rigour. The peer-reviewed metrics confirm that Loon's technology provides the quantitative confidence necessary to satisfy the global regulatory standards for quality, transparency, and human oversight.

Loon's pioneering confidence-calibrated methodology combined with peer-reviewed validation provides the necessary technical and scientific foundation for full compliance with the requirements of NICE, CDA-AMC, and ISPOR, providing the biopharmaceutical industry with the first commercially validated technological pathway that moved AI from an exploratory tool to a verified, core component of the evidence generation function.

References

1. Janoudi, G., et al. Validating an Agentic Artificial Intelligence Abstract Screener across 8 Reviews Showed 99% Recall, Calibrated Confidence Scores, and a Sub-6% Human Check, Lifting Precision to 90%. Value in Health, 2025. (Source for 99.0% accuracy/sensitivity, sub-6% human review threshold, and core efficiency data).

2. Azam F., et al. Evaluating the Confidence Levels of Large Language Models in Answering Medical Questions: A Multi-Specialty Analysis. Journal of Medical Internet Research, 2025. (Addresses the paradox of worse-performing LLMs exhibiting higher confidence).

3. National Institute for Health and Care Excellence (NICE). Position Statement: Use of AI in Evidence Generation.November 2024.

4. Canada’s Drug Agency (CDA-AMC). Position Statement on the Use of AI Methods in Health Technology Assessment. April 2025.

5. Janoudi, G., et al. Evaluating Loon Lens Pro, an AI-Driven Tool for Full-Text Screening in Systematic Reviews: A Validation Study. medRxiv, 2025. (Describes the zero-shot/unsupervised methodology for bias mitigation).

6. ISPOR Working Group on Generative AI. The ELEVATE-AI LLMs Framework: An Evaluation Framework for Use of Large Language Models in HEOR. ISPOR Working Group Report, 2025.

7. ISPOR Working Group on Generative AI. ELEVATE-GenAI: Reporting Guidelines for the Use of Large Language Models in Health Economics and Outcomes Research. Value in Health, 2025.

8. Al-Najjar, A., et al. The Evolving Role of AI in Systematic Literature Reviews: From Automation to Augmentation. Journal of Medical Systems, 2024. (Addresses the assistive role of AI and enabling researchers to focus on complex tasks/optimization).