Use MCP Tools with Local AI Models

Use MCP Tools with Local AI Models

Feluda can use local AI models together with MCP tools.

A local model runs on your own computer through a compatible provider such as Ollama or LM Studio.

An MCP tool gives that model an approved way to retrieve information or perform a supported action.

This combination can be useful for:

• processing sensitive text locally;
• searching local files or internal sources;
• building private workflows;
• reducing dependence on cloud models;
• using connected business tools; and
• running repeatable tasks from your own device.

The most important point is that the model location and tool location are separate.

A local model does not make a remote MCP tool local.

Understand the complete task path

A local-model tool task may follow this path:

User Input
→ Local AI Model
→ MCP Tool
→ Connected Source
→ Tool Result
→ Local AI Model
→ Final Answer

The model may process its instructions on your computer.

The MCP tool may still send information to:

• a local service;
• another computer on your network;
• an internal organisational server; or
• a remote online service.

Review the complete path before using sensitive information.

What stays local

Model processing can remain on your device when:

• Feluda uses a local provider;
• the selected model runs through Ollama, LM Studio, or another compatible local endpoint;
• the source is local;
• the MCP server is local;
• the tool uses only local information; and
• the result remains on the device.

In that setup, an internet connection may not be needed.

What may leave the device

Information may leave your computer when the workflow uses:

• a remote MCP server;
• an online search tool;
• a cloud storage service;
• an external business system;
• a remote database;
• a messaging service;
• a cloud destination; or
• another connected online service.

The local model does not prevent that external connection.

Review every tool separately.

Local models and local MCP servers

A local model can use an MCP server running on the same computer.

This may support tools for:

• reading approved local files;
• searching a local knowledge source;
• using a local application;
• accessing a local database;
• writing to a local folder; or
• interacting with another on-device service.

Confirm that the complete tool path remains local.

Some local tools may still contact an external service behind the scenes.

Local models and remote MCP servers

A local model can also use a remote MCP server.

In that case:

• model inference happens locally;
• the tool request is sent to the remote server;
• the remote service processes the request;
• the result returns to Feluda; and
• the local model interprets the result.

Describe the workflow as partly local, not fully local.

Before you begin

Make sure that:

• Feluda is installed;
• a local model runner is installed;
• a model has been downloaded;
• the local model server is running;
• Feluda can see the model;
• the MCP server is connected;
• the required MCP tool appears;
• you understand where that tool runs; and
• you know whether it reads or writes information.

Test with non-sensitive sample data first.

Connect a local AI provider

Open AI Providers from the Feluda sidebar.

Add or open your local provider.

Common options include:

• Ollama;
• LM Studio; or
• another compatible local endpoint.

Confirm the base URL.

Standard local setups often use:

• Ollama: http://localhost:11434
• LM Studio: http://localhost:1234

Use the endpoint shown by your local model application when it differs.

Start the local model server

The model runner must be available while Feluda uses it.

For Ollama, confirm that the service is running.

For LM Studio, open its server area and start the local server.

Then return to Feluda and fetch or refresh the available models.

Choose the local model

Select a model suitable for the task and your computer.

Consider:

• available memory;
• model size;
• speed;
• instruction following;
• tool-use ability;
• structured output;
• language needs; and
• expected input length.

A larger model may provide better results but use more memory and take longer.

Confirm tool-use support

Not every local model uses tools equally well.

Test whether the model can:

• recognise when the tool is needed;
• select the correct tool;
• provide required fields;
• understand returned data;
• follow read-only limits;
• stop after success; and
• report tool errors clearly.

Use Workbench for the first test.

Use MCP tools in Workbench

To test a local model with an MCP tool:

1. open Workbench;
2. select the local provider;
3. select the local model;
4. open Tools;
5. enable one MCP tool;
6. enter a clear instruction;
7. send the message; and
8. review the Activity drawer.

Keep the first test small.

Use one tool during early testing

Enable only the tool needed for the task.

This makes it easier to understand:

• whether the model recognised the tool;
• what input it prepared;
• what result came back;
• whether a warning appeared;
• whether the model repeated the call; and
• whether the answer matched the tool result.

Add more tools only after the first tool works reliably.

Write a clear instruction

Local models often benefit from direct and structured instructions.

For example:

Use only the enabled Internal Knowledge Search tool.

Search for the current onboarding process.

Return:
1. the main steps;
2. the source details returned by the tool;
3. missing information; and
4. uncertainty.

Use only the tool result.
Do not guess.
Do not create or change anything.

This instruction defines the tool, output, and limits.

Review Workbench Activity

After the answer, open Activity.

Confirm:

• the expected tool was called;
• the input was accurate;
• the tool result was complete;
• warnings were reviewed;
• errors were understood;
• no unrelated tool was called; and
• the final answer matched the returned data.

A local model can still misread a correct tool result.

Compare the raw result with the answer

Check exact:

• names;
• dates;
• amounts;
• identifiers;
• statuses;
• source details;
• missing values; and
• warnings.

Smaller local models may omit fields or add unsupported information.

Use a structured answer format and compare both layers.

Use read-only tools first

A read-only tool is the safest starting point.

It may:

• search;
• retrieve;
• inspect;
• list;
• compare; or
• read information.

Confirm that the tool does not create or change anything.

Use a read-only account when the connected service supports it.

Use write tools carefully

A write tool may:

• create a task;
• update a record;
• save a file;
• send a message;
• create a note;
• change a status; or
• remove information.

Test these tools only after the local model handles read tools reliably.

Use a draft-first process and human approval for important actions.

Use a draft-first write process

Ask the local model to prepare the action without performing it.

For example:

Prepare the proposed record update.

Show:
* the record identifier;
* the current values;
* the proposed values;
* the source; and
* the reason for the change.

Do not submit the update yet.

Review the proposal before authorising the write tool.

Confirm the destination

Before approving a write action, verify:

• the MCP server;
• the connected service;
• the account;
• the environment;
• the record;
• the file path;
• the recipient;
• the content; and
• whether the action can be reversed.

A local model does not make an incorrect destination safe.

Protect credentials

Local model providers normally do not need an external model key.

MCP servers may still require:

• an access token;
• an API key;
• an account;
• a protected header; or
• another credential.

Store private values only in Feluda's intended protected settings.

Never place them in:

• Workbench prompts;
• Studio instructions;
• Input blocks;
• Journal entries;
• source documents; or
• ordinary notes.

Limit tool input

Even with a local model, the tool may receive information outside the model process.

Before the tool call, ask:

• Does the tool need the full document?
• Can one section be used?
• Can personal details be removed?
• Can a reference number replace a name?
• Can a shorter search query work?
• Can sample data be used?

Send only what the tool needs.

Protect tool results

An MCP tool may return sensitive information.

Before displaying, saving, or sharing the result, confirm:

• which fields are necessary;
• who should see them;
• whether personal information can be removed;
• whether the Journal is an appropriate destination;
• whether the result should remain local; and
• whether organisational rules apply.

Local model processing does not remove the need to protect returned data.

Use MCP tools in Studio

After a successful Workbench test, add the local model and tool to a Studio workflow.

A simple structure is:

Input
→ Local LLM with MCP Tool
→ Output

Use the same:

• local provider;
• model;
• MCP tool;
• instruction; and
• sample input

that worked in Workbench.

Separate retrieval from analysis

When the workflow becomes more complex, use separate steps.

For example:

Input
→ Retrieve Data with MCP Tool
→ Emit Raw Result
→ Analyse with Local Model
→ Output

This helps you identify whether a problem came from the tool or the model.

Use Emit blocks

Add an Emit block during testing.

It can show:

• raw tool output;
• extracted fields;
• classification;
• a proposed write action; or
• another intermediate value.

This visibility is especially helpful when a smaller model changes or omits information.

Handle empty results

A tool can complete successfully and return no match.

The local model should say:

No matching information was returned.

It should not invent a result.

Add a clear no-match path in the workflow when needed.

Handle tool errors

The MCP call may fail because:

• the MCP server is unavailable;
• authentication expired;
• the network is unavailable;
• a local service stopped;
• required input is missing;
• permission is denied;
• the source is unavailable; or
• the request timed out.

Add an error path that explains what needs review.

Handle model errors

The local model may fail because:

• the model server stopped;
• the wrong endpoint is configured;
• the model is not loaded;
• memory is insufficient;
• the model does not support the required tool use;
• the prompt is too complex;
• the input is too long; or
• another local workflow uses the available resources.

Test the model separately from the MCP server.

Test each layer separately

Use this order:

1. Test the local model without tools.
2. Test the MCP server connection.
3. Test one read-only MCP tool.
4. Review the tool result.
5. Test the local model's interpretation.
6. Test the complete workflow.
7. Test any write action separately.
8. Test scheduled use last.

This makes failures easier to locate.

Understand offline use

A local model can work without an internet connection.

A complete MCP workflow can work offline only when:

• the model is local;
• the MCP server is local;
• the source is local;
• authentication does not require an online service;
• the destination is local; and
• every required tool works without the internet.

A remote MCP tool makes the workflow internet-dependent.

Test offline behaviour

When offline use matters, test it intentionally.

Confirm that:

• the local model remains available;
• the local MCP server remains available;
• the source can be reached;
• tool calls complete;
• no remote dependency appears;
• errors are visible; and
• the result remains available.

Do not assume a workflow is offline-capable because its model is local.

Review hardware use

Local models use your computer's memory and processing power.

MCP tools may add their own local workload.

Watch for:

• slow responses;
• model-loading failures;
• high memory use;
• delayed tool calls;
• application instability;
• timeouts; and
• competing workflows.

Use a smaller model or reduce simultaneous runs when needed.

Avoid overlapping local workloads

Several local-model workflows may compete for the same resources.

This is especially important when they:

• use large models;
• process long documents;
• generate images;
• call local file tools;
• use a local database; or
• run on a schedule.

Test concurrent runs before relying on them.

Choose a suitable local model

Use a model that matches the task and hardware.

A smaller model may be suitable for:

• extraction;
• simple classification;
• short summaries;
• structured field handling; or
• narrow tool tasks.

A larger model may be better for:

• complex reasoning;
• ambiguous requests;
• multi-step tool use;
• long source material; or
• detailed comparisons.

Test the complete task rather than choosing by model size alone.

Improve local-model tool use

When the model struggles:

• shorten the prompt;
• use one tool;
• name the tool directly;
• request a fixed output structure;
• reduce source length;
• separate retrieval from analysis;
• add Emit blocks;
• use exact missing-value rules; or
• choose a more capable local model.

Simple workflows are easier for smaller models to follow.

Compare local models

To compare models fairly:

1. use the same MCP tool;
2. use the same tool input;
3. use the same instruction;
4. start a new conversation for each model;
5. review each Activity log;
6. compare tool selection;
7. compare returned-value handling; and
8. compare unsupported claims.

Choose the model that performs the entire task reliably.

Mix local and cloud models carefully

One workflow can use both local and cloud models.

For example:

Local Model
→ MCP Retrieval
→ Local Redaction
→ Cloud Model
→ Final Output

Review exactly what is passed from the local step to the cloud step.

Remove sensitive details before cloud processing when required.

Do not assume earlier local processing keeps later cloud input private.

Use local models for redaction carefully

A local model may help remove sensitive details before another step.

Test whether it consistently removes:

• names;
• addresses;
• account numbers;
• personal identifiers;
• private notes; and
• confidential fields.

Automated redaction should be reviewed before sensitive content is sent elsewhere.

Schedule local-model MCP workflows carefully

A scheduled workflow needs every local service to be available at run time.

Confirm that:

• the computer is on;
• Feluda is open;
• the local model server is running;
• the model is installed;
• the MCP server is running;
• local files or databases are available;
• remote tools have network access when needed;
• enough memory is available; and
• overlapping runs are controlled.

Test the schedule with non-sensitive input first.

Review scheduled results

After a scheduled run, check:

• whether the model loaded;
• whether the MCP server was available;
• which tool was called;
• what input it received;
• what result returned;
• whether the local model interpreted it correctly;
• whether a write action completed;
• whether a duplicate was created; and
• whether a reviewer checked the result.

Pause the schedule after repeated failures.

If the local model does not call the tool

Check whether:

• the tool is enabled;
• the instruction names it;
• the model supports tool use;
• the prompt is too vague;
• too many tools are enabled;
• the conversation contains conflicting context; or
• a more capable model is needed.

Use a new conversation and one direct task.

If the local model calls the wrong tool

Disable unrelated tools.

Use a direct instruction:

Use only the enabled Internal Knowledge Search tool.
Do not call any create, update, delete, messaging, or file-writing tool.

Review Activity to confirm the change.

If the local model misreads the result

Compare the raw tool output with the answer.

Check:

• exact fields;
• dates;
• amounts;
• identifiers;
• statuses;
• warnings;
• missing values; and
• source references.

Ask for a fixed field-by-field output.

If the model repeats tool calls

Stop the task when repeated calls may create unwanted actions.

Repeated calls may be caused by:

• unclear instructions;
• empty results;
• tool errors;
• the model not recognising success;
• several similar tools; or
• a workflow loop.

Simplify the task and use one tool.

If the workflow is slow

Check:

• model size;
• available memory;
• source length;
• number of tool calls;
• local server performance;
• external network delays;
• other active workflows; and
• output length.

Reduce one demand at a time and repeat the same test.

If the workflow fails offline

Identify the hidden remote dependency.

It may be:

• the MCP server;
• the connected source;
• authentication;
• a web search tool;
• a cloud destination;
• a cloud model in a later step; or
• another external service.

Replace or remove that dependency only when an approved local alternative exists.

A practical local MCP routine

Use this process:

1. Configure the local provider.
2. Start the local model server.
3. Confirm the model appears in Feluda.
4. Identify whether the MCP server is local or remote.
5. Review the complete data path.
6. Enable one read-only tool.
7. Test with non-sensitive input.
8. Review Workbench Activity.
9. Compare the raw result with the answer.
10. Add the task to Studio.
11. Use Emit blocks during testing.
12. Add no-result and error paths.
13. Test write actions separately.
14. Confirm external destinations.
15. Test offline and scheduled use separately.

Local AI models can keep model processing on your own computer.

MCP tools extend what those models can do.

Reviewing both locations helps you understand which parts of the task remain local and which parts connect elsewhere.