Skip to content

Validate AI-generated data

AI agents are very good at producing plausible structured output. Plausible is not the same as valid.

Validibot gives an agentic system a checkpoint between generation and action. You define the contract once as a reusable workflow, deploy Validibot inside your own environment, and require generated artifacts to pass before they are stored, published, simulated, or sent to another system.

The useful mental model is simple:

Let the model propose. Let an independent workflow verify.

This is an antidote to hallucination where correctness can be expressed as a schema, rule, graph constraint, or executable model. It is not a general truth detector. A passing run proves that the configured checks passed; it cannot prove properties the workflow did not test.

Why validation belongs after generation

A prompt can ask for valid JSON, correct units, known identifiers, or sensible cross-field values. The model can still return a confident-looking result that breaks one of those requirements.

Prompt instructions are guidance to a probabilistic generator. A Validibot workflow is an executable contract. It produces the same result for the same input, workflow version, and validator versions, and it reports defects in a shape software can act on.

That separation matters. If the same model both generates an artifact and judges whether it is correct, it can repeat or rationalise its original error. Use deterministic validators first, then add AI review only for criteria that cannot be encoded another way.

Build validation in layers

The best workflow starts with cheap, exact checks and moves toward richer or more expensive ones.

Layer Validibot validator Useful checks on agent output
Structure JSON Schema or XML Schema Required fields, types, enums, formats, nesting, and document shape
Business rules Basic validator with CEL assertions Cross-field relationships, ranges, units, identifiers, and policy invariants
Tabular quality Tabular validator CSV column types, uniqueness, missing values, regex patterns, and per-row rules
Semantic consistency SHACL validator RDF classes, properties, cardinality, and graph relationships
Behaviour EnergyPlus, FMU, or a custom advanced validator Whether a generated technical model actually runs and produces acceptable outputs
Subjective review AI validator Natural-language criteria that cannot yet be expressed deterministically

You can chain these validators in one workflow. For example, a building-design agent might first validate an epJSON document against a schema, then enforce project-specific constraints with CEL, and finally run EnergyPlus to check the model's behaviour. A syntactically convincing model that cannot simulate never passes the gate.

Use a bounded repair loop

A robust agent integration follows this sequence:

  1. Generate a candidate artifact.
  2. Submit it to the workflow chosen by the operator.
  3. Wait for the run to finish.
  4. Read each finding's severity, message, and data location.
  5. Repair the reported defects without silently changing unrelated content.
  6. Submit the repaired artifact as a new run.
  7. Stop after a small, fixed number of attempts and escalate unresolved findings to a person.
  8. Release the artifact only when the required workflow passes.

Bounded retries prevent an agent from burning time or compute on a contract it cannot satisfy. Keep the failed runs: they explain what changed and why the final artifact was accepted.

Connect over the REST API

Every self-hosted deployment exposes a versioned REST API when the API feature is enabled. If your instance is at https://validibot.example, its current OpenAPI document is available from:

https://validibot.example/api/v1/schema/?format=json

Treat that deployment-local document as authoritative. It reflects the exact version the operator installed and lists the supported request and response schemas.

The common launch pattern is:

POST /api/v1/orgs/{org_slug}/workflows/{workflow_slug}/runs/

A synchronous workflow can return a completed result immediately. An asynchronous workflow returns an accepted response with a run URL to poll. See Sending data to the API for raw-body, JSON-envelope, and multipart examples, and Authentication for bearer tokens.

Use a dedicated automation identity with the minimum role it needs. An agent that only launches workflows normally needs Executor access; one that only reads results can use Viewer access.

Connect over MCP

Self-hosted Pro deployments can enable Validibot's MCP server. MCP gives compatible agents a smaller, task-oriented surface than the full REST API:

  • discover workflows the authenticated user may access;
  • inspect a workflow before sending a file;
  • submit a candidate for validation;
  • check or wait for the result.

The operator supplies the MCP URL, typically on a separate origin such as https://mcp.validibot.example/mcp. Workflows must explicitly allow agent access, and the MCP server preserves the authenticated user's existing permissions.

See AI Agent Integration (MCP) for connection and authentication details.

Keep the evidence with the output

A useful validation result records more than pass or fail. It should remain possible to answer:

  • Which workflow and version ran?
  • Which validators and rules were applied?
  • Which input was checked?
  • What findings were produced?
  • When did the run finish?

Validibot keeps runs and structured findings so the consuming system can show why an artifact was rejected or accepted. Commercial editions can add signed credentials when a result needs to be verified outside the deployment.

Keep generated data inside your environment

Self-hosted Validibot runs on infrastructure you control. Product telemetry is off by default, and validation data does not need to leave your environment. That makes it suitable for research files, engineering models, customer data, and other artifacts that should not be sent to a public validation service.

If you enable the AI validator, its configured model provider may receive the content that step evaluates. Use deterministic validators when data must stay entirely local, or configure a model endpoint that meets your data-handling requirements.

Design rules that age well

  • Version schemas and workflows alongside the system that consumes the data.
  • Make findings specific enough for an agent to repair: include a path, stable rule identifier, expected condition, and actual value where safe.
  • Prefer allowlists and explicit enums for identifiers an agent must not invent.
  • Check relationships, not just individual fields. Many plausible errors are internally inconsistent rather than syntactically invalid.
  • Validate units and ranges before launching expensive simulations.
  • Record the workflow version with the accepted artifact.
  • Never turn a passing result into a broader claim than the workflow supports.

Next steps

Spotted a problem on this page? Report it or suggest an edit