Detection & evals
H2O EvalGPT
H2O EvalGPT (represented within the H2O.ai ecosystem as the evaluation and governance core of Enterprise h2oGPTe) is an enterprise-grade tool for content detection, evaluation, and verification.
What is H2O EvalGPT?
H2O EvalGPT (represented within the H2O.ai ecosystem as the evaluation and governance core of Enterprise h2oGPTe) is an enterprise-grade tool for content detection, evaluation, and verification.As part of the H2O AI Super Agent orchestration layer, it provides citation-backed conversational Agentic RAG, Document AI, and multi-model support, ensuring every answer is traceable to source document passages.
The platform connects to diverse model hosts including OpenAI, Anthropic, Bedrock, Azure OpenAI, Databricks, and self-hosted vLLM.It features intelligent routing and cost controls at the user, team, and model levels.Its core strength lies in unifying predictive, generative, and agentic AI capabilities within a single governed framework, offering one audit trail and one bill.
Through the built-in Eval Studio, users can evaluate the performance of RAG systems and agents, logging every prompt, retrieval, tool call, and response.Whether processing PDFs, scans, or complex tables, the system leverages Mississippi vision-language models for OCR and layout understanding, transforming unstructured data into queryable fields.
Key features
- Citation-BackedPerform multi-step retrieval and reasoning across contracts, policies, tickets, and wikis, with answers citing specific source documents and passages to eliminate hallucinations.
- Advanced DocumenIngest PDFs, scans, faxes, and images using Mississippi vision-language models for OCR and layout understanding, extracting tables and fields to make them queryable.
- Multi-Model InteConnect to any LLM from any host (including OpenAI, Anthropic, Bedrock, etc.) and intelligently route requests based on cost and quality requirements.
- Eval Studio EvalComprehensively evaluate RAG retrieval quality and agent execution results, logging full audit trails to ensure compliance and transparency.
- H2O AI Super AgeIncludes nine agent types (such as Deep Research, Data Science, Code Generation) that plan and execute complex workflows from natural language instructions.
- Granular Cost CoProvides detailed cost attribution and controls per user, team, and model, helping enterprises manage budgets while scaling GenAI applications.
How to use it
- 1.Ingest Data Sources: Upload or connect document sources, allowing Document AI to automatically parse scans, tables, and forms.
- 2.Configure Models: Select desired LLM providers or self-hosted models and set cost limits for different collections.
- 3.Ask Questions: Query via the chat interface to receive answers with source citations, verifying information accuracy.
- 4.Build Agents: Turn workflows into agents, connect tools via MCP servers, and add necessary approval gates.
- 5.Evaluate & Govern: Use Eval Studio to assess RAG and agent performance, reviewing audit logs to optimize system behavior.
Where it fits
- Legal & ComplianRapidly search through large volumes of contracts and policy documents to get precisely cited answers, assisting legal teams with due diligence.
- Enterprise KnowlIntegrate internal wikis, tickets, and technical documentation to provide employees with a unified intelligent Q&A assistant, reducing time spent searching for information.
- Financial DocumeAutomatically extract key data fields from invoices, statements, and faxes, enabling structured querying and analysis of unstructured financial data.
Pricing and free quota
Pricing and any free quota follow the official site.
Who it is for
- Enterprise IT & Security Teams
- Legal & Compliance Experts
- Data Scientists & AI Engineers
- Knowledge Management Leaders
FAQ
Which models does H2O EvalGPT support?
It supports connecting to OpenAI, Anthropic, Bedrock, Azure OpenAI, Databricks, as well as self-hosted vLLM and Cascade-tuned models.
How is answer authenticity ensured?
The system uses Agentic RAG technology, where all answers include citations to source documents and specific passages, supporting multi-step retrieval for verification.
Is on-premises deployment supported?
Yes, it supports on-premises, air-gapped, private cloud, or public cloud deployment to meet various data sovereignty requirements.