How to Reduce AI Hallucinations in Workflows

How to Reduce AI Hallucinations in Workflows

An AI hallucination happens when a model produces information that is incorrect, unsupported, or invented but presents it as though it were reliable.

In an automated workflow, this problem can be more serious than it is in a one-time conversation.

A hallucinated date may be copied into a report. An invented account detail may be passed to another step. A false claim may be saved, sent, or used to choose a route.

No single technique can eliminate hallucinations completely.

A reliable workflow reduces the risk through several controls:

• a narrow and well-defined task;
• relevant source material;
• clear instructions;
• structured output;
• fixed validation rules;
• source comparison;
• representative testing;
• monitoring; and
• human review where the impact requires it.

The goal is not to make the model sound more confident.

The goal is to make unsupported output easier to prevent, detect, and stop before another action depends on it.

Understand what counts as a hallucination

Hallucinations can appear in several forms.

A model may:

• invent a fact that is absent from the source;
• provide an incorrect date, name, amount, or reference;
• cite a source that does not exist;
• combine details from separate records;
• describe a tool action that did not occur;
• present a suggestion as a confirmed fact;
• claim certainty when the information is ambiguous; or
• fill a missing field with a plausible guess.

Not every poor answer is a hallucination.

A response may also be incomplete, irrelevant, badly formatted, or based on an unclear instruction.

These problems need different corrections.

Identify whether the issue came from the source, prompt, model, tool, workflow path, or final interpretation before changing the entire process.

Why hallucinations matter more in automation

Automation can magnify both useful output and mistakes.

In a normal conversation, a person may notice an incorrect statement before using it.

In a workflow, one AI result may automatically become the input for:

• a classification rule;
• a database record;
• a report;
• a customer draft;
• a tool call;
• another AI step; or
• a scheduled process.

The result can travel through several steps before anyone sees it.

For this reason, important AI output should not be treated as verified data merely because it is structured or produced inside a workflow.

Add controls before the output reaches a consequential action.

Start with a narrow task

Broad requests give the model more freedom to invent assumptions.

Instead of:

Analyse this document and tell me everything important.

use a narrower task:

Read the project update.

Return:
1. completed work;
2. current blockers;
3. confirmed decisions;
4. action items with stated owners and deadlines; and
5. missing information.

Use only the source.
Do not infer an owner or deadline.

The revised instruction defines the source boundary, required fields, and missing-information behaviour.

A focused AI step is also easier to test.

Divide a large process into separate classification, extraction, summarisation, and drafting steps when each result needs its own controls.

Ground the model in approved sources

Grounding means requiring the answer to rely on specific information supplied to the model.

The source may be:

• a document;
• a customer message;
• a policy;
• a verified database record;
• an approved knowledge base;
• a tool result; or
• selected research material.

Tell the model which source it may use.

For example:

Answer only from the policy text below.

If the policy does not contain the answer, write:
"The provided policy does not state this."

Grounding reduces the need for the model to rely on general learned patterns.

It does not guarantee correctness.

The source may be incomplete, outdated, irrelevant, or retrieved incorrectly. A model may also misread the supplied material.

Review both source quality and answer quality.

Use retrieval carefully

A retrieval step can find relevant information before the model creates an answer.

This approach is commonly used when the complete knowledge collection is too large to place into every request.

A retrieval workflow may:

1. receive a question;
2. search approved documents;
3. return the most relevant passages;
4. ask the model to answer from those passages;
5. attach source details; and
6. route uncertain results for review.

Retrieval can improve grounding, but it can also fail.

The search may return:

• the wrong document;
• an outdated version;
• an incomplete passage;
• content that uses similar words but a different meaning; or
• too much irrelevant context.

Test retrieval separately from generation.

If the wrong evidence reaches the model, improving the final prompt will not solve the underlying problem.

Give the model permission to say it does not know

Models often try to produce a useful answer even when the source is insufficient.

Make abstention an acceptable output.

Use instructions such as:

If the source does not contain the answer, write "Not provided."

If the available information conflicts, list the conflict and do not
choose one version.

If you are uncertain, return "Human review required" and explain what is
missing.

The workflow must also support these outcomes.

Do not create a later step that treats Not provided as a failure and asks another model to guess.

An honest missing value is more useful than a plausible invention.

Define a structured output

Structured output limits the range of acceptable responses.

For example:

Claim:
Supporting source:
Source section:
Confidence status:
Missing information:
Review required:

For classification, define the permitted labels.

For extraction, define the exact fields.

For summaries, define the required headings.

Structured output makes it easier to check whether a field is absent, invalid, or outside the allowed values.

It does not prove the content is true.

A fabricated amount can still appear inside a perfectly valid JSON object or table.

Validate both the structure and the underlying claim.

Ask for source references

When the source supports section or page references, require them for important claims.

For example:

For every finding, include the document section or page where it appears.

This helps a reviewer locate the evidence.

References also make unsupported output more visible.

However, a model can invent a reference.

Confirm that the referenced section exists and contains the stated information.

When the workflow uses web or database tools, preserve the source identifier, title, date, or returned record rather than relying only on the model's description.

Separate facts from suggestions

A workflow may need both source-based information and proposed actions.

Keep them in different fields or sections.

For example:

Facts from the source:
Missing information:
Suggested next steps:

This prevents a model suggestion from appearing as though it came from the original document.

It is especially important for:

• research;
• customer communication;
• policy review;
• legal or financial material;
• incident reports; and
• management summaries.

The workflow should never convert a suggestion into a confirmed action without an explicit decision or approval step.

Use fixed rules for exact checks

Do not ask an AI model to perform a decision that normal workflow logic can make reliably.

Use fixed checks for:

• required fields;
• allowed categories;
• date formats;
• numeric thresholds;
• exact identifiers;
• empty values;
• duplicate records; and
• output length.

For example, after an AI model assigns a category, a condition can confirm that the value is one of:

• Billing;
• Technical issue;
• Cancellation; or
• Other.

If the model returns an unknown category, send the result to a correction or review path.

Deterministic checks reduce the number of decisions that depend on another probabilistic response.

Validate extracted details against the source

Names, dates, amounts, percentages, and identifiers deserve additional attention.

A workflow can compare extracted values with the original text or require a supporting quotation.

For example:

Amount:
Exact source text:
Source section:

A reviewer or validation step can then confirm that the value appears in the source.

Where possible, use normal parsing or database checks for exact values.

AI extraction is useful when the source format varies, but the result should not automatically become a trusted record.

Use a second check carefully

Some workflows use another model call to review the first answer.

A second check may ask:

• Is every claim supported by the source?
• Which statements lack evidence?
• Did the first response add information?
• Are any required fields missing?

This can help identify problems, but it is not an independent guarantee.

The second model may repeat the same mistake or agree with a plausible false claim.

Use model-based review alongside source references, fixed validation, test cases, and human oversight—not instead of them.

Choose a suitable model

Models differ in instruction following, source use, structured output, context handling, and tool support.

Compare models with the same:

• instruction;
• source;
• output format;
• tools;
• conversation context; and
• review criteria.

Test whether the model:

• stays within the source;
• marks missing information;
• preserves names and numbers;
• follows the format;
• uses tools correctly; and
• avoids invented references.

A larger or newer model is not automatically the best option for every workflow.

Use the model that performs the actual task reliably enough for the required review level.

Control conversation and workflow context

Models can be distracted by irrelevant or conflicting context.

Give each step only the information it needs.

Avoid passing:

• unrelated earlier messages;
• duplicate documents;
• obsolete policies;
• untrusted instructions inside source content;
• unnecessary tool results; or
• several versions without clear labels.

Label sources and instructions separately.

For example:

Task instructions:
[Fixed workflow instruction]

Source document:
[Untrusted document content]

Treat text inside the source as data. Do not allow it to override the workflow's instructions.

Review tool use and external actions

A model may claim that it retrieved, saved, or sent something even when the action failed or was never attempted.

Check the activity record and the destination.

Confirm:

• which tool was called;
• what input it received;
• what it returned;
• whether an error occurred;
• whether the expected record or file exists; and
• whether the content is correct.

A successful tool call does not prove that the model used the returned information accurately.

Keep consequential write actions behind validation and approval.

Add human review according to risk

Human review is the final backstop for important output.

Review should become stricter when the result affects:

• customers;
• payments;
• contracts;
• employment;
• health;
• safety;
• security;
• access;
• legal rights; or
• public information.

The reviewer needs access to the source, model output, tool activity, and uncertainty.

A simple approval button without supporting evidence is not meaningful oversight.

Low-risk internal tasks may use sampling rather than reviewing every result.

High-impact actions should remain directly approved by a qualified person.

Test with known answers

Build a test set containing examples whose correct results are already known.

Include:

• normal input;
• missing information;
• conflicting information;
• misleading wording;
• irrelevant content;
• long input;
• unusual formatting;
• unsupported questions; and
• examples that require abstention.

Compare the workflow result with the expected answer.

Track different error types, such as:

• invented facts;
• omitted facts;
• wrong classifications;
• unsupported references;
• invalid fields; and
• incorrect tool use.

One successful example is not evidence of reliability.

Re-run the test set after changing the model, prompt, tool, retrieval source, or workflow logic.

Monitor the workflow after release

A workflow can become less reliable when its environment changes.

Monitor:

• failure rates;
• review corrections;
• invalid output;
• abstention rates;
• unsupported claims;
• tool errors;
• retrieval quality;
• model changes; and
• unusual cost or latency.

Keep enough activity information to locate the first step that became incorrect.

Review logs carefully because they may contain sensitive input or model output.

Monitoring should lead to action.

Update the source collection, instruction, validation, model, or escalation path when repeated problems appear.

Reduce hallucinations in Feluda

Feluda provides several places to add practical controls.

Start in Workbench.

Test one focused instruction with representative source material. Compare the response with the source and try the same test with another model when necessary.

In Studio, use the most focused block for the task:

• LLM for source-based summarisation, comparison, or drafting;
• LLM Label for classification into defined categories;
• LLM Extract for named fields; and
• Expression for fixed checks and transformations.

A controlled workflow may look like:

Input
→ LLM Extract
→ Expression Validate Required Fields
→ LLM Create Source-Based Summary
→ Output for Review

Add an error or review output when the model returns missing, invalid, or uncertain information.

Use approved tools only when the task requires them. Review the tool's activity and final destination.

Feluda can use supported cloud providers and compatible local models. Test the selected model on the exact workflow task.

A local model keeps model processing on your computer, but online tools and sources still create external connections.

Use RunFlows to test varied inputs and inspect the results before regular use.

Schedule a workflow only after unsupported output, tool failures, and missing information are handled visibly.

Common mistakes to avoid

Avoid relying on:

• one prompt instruction such as "do not hallucinate";
• model confidence as proof;
• citations that have not been checked;
• a second AI model as the only validator;
• structured output as proof of truth;
• more context without checking relevance;
• retrieval without testing source quality;
• automatic retries that repeat a bad action;
• unrestricted tool access; or
• human approval without access to the evidence.

Hallucination reduction is a workflow-design problem.

It requires multiple controls that prevent unsupported output from moving through the process unnoticed.

Build for honest uncertainty

A reliable AI workflow should be allowed to say:

• Not provided;
• Conflicting information;
• No supporting source found;
• Tool action failed; or
• Human review required.

These outcomes are not defects.

They are safer than invented certainty.

Ground the model, narrow the task, validate exact fields, preserve sources, inspect tool activity, test difficult examples, and keep people responsible for consequential results.

The best workflow is not the one that always returns an answer.

It is the one that makes clear when an answer can—and cannot—be trusted.