Use a Local AI Model

Use a Local AI Model

A local AI model runs on your own computer instead of on a cloud provider's servers.

Feluda can connect to compatible local model applications such as Ollama and LM Studio. Once connected, you can use the model in Workbench and in your workflows.

Local AI can be useful when you:

• prefer to process information on your own computer;
• want to work without an internet connection;
• do not want to depend on a cloud AI account;
• want to compare different downloadable models; or
• need an AI model for repeated local tasks.

You do not need an API key for a normal local setup.

What you need

Before connecting a local model to Feluda, you need:

• the Feluda desktop application;
• a compatible local model application;
• at least one downloaded AI model; and
• enough available memory for the selected model.

Two common local model applications are:

You only need one of these applications to begin.

How local AI works with Feluda

The local model application runs the model on your computer.

Feluda connects to that application and makes the model available in areas such as Workbench and Studio.

A simple local setup follows this path:

1. install Ollama or LM Studio;
2. download a model;
3. start the local model service;
4. connect it through AI Providers in Feluda;
5. load the available models; and
6. test one in Workbench.

The local model application must remain running while Feluda is using it.

Choose between Ollama and LM Studio

Both applications can make local models available to Feluda.

Choose Ollama when you want a simple model runner and are comfortable following short command-based instructions.

Choose LM Studio when you prefer a visual application for browsing, downloading, loading, and starting models.

The AI model you choose matters more than the application used to run it. Models differ in quality, speed, supported features, and computer requirements.

Option 1: Set up Ollama

Install Ollama

Visit the official Ollama website and download the version for your operating system.

Complete the installation and open Ollama when required.

Download a model

Ollama lets you download models by name.

Follow the model instructions provided by Ollama. For example, its model library shows the command needed for each available model.

If you are new to local AI, begin with a smaller general-purpose model. A smaller model is more likely to run comfortably on an everyday computer.

Wait for the download to complete before continuing.

Make sure Ollama is running

Ollama normally starts its local service automatically.

Its standard local address is:

http://localhost:11434

Keep this address unless you intentionally changed Ollama's setup.

Connect Ollama to Feluda

In Feluda:

1. open AI Providers;
2. choose Ollama;
3. confirm the local address;
4. select the option to load or fetch available models;
5. choose one of the models installed through Ollama; and
6. save the provider.

If no models appear, confirm that Ollama is running and that at least one model has finished downloading.

Option 2: Set up LM Studio

Install LM Studio

Visit the official LM Studio website and download the version for your operating system.

Complete the installation and open the application.

Download and load a model

Use LM Studio to search for a model.

Before downloading, review the model size and the memory guidance shown by LM Studio. Start with a smaller model if you are unsure what your computer can run.

After the download completes, load the model.

Start the local server

Open the server area in LM Studio and start its local server.

Its standard local address is commonly:

http://localhost:1234

Use the address shown in LM Studio if it differs from this example.

Keep LM Studio and its local server running while Feluda uses the model.

Connect LM Studio to Feluda

In Feluda:

1. open AI Providers;
2. choose the matching local provider;
3. enter the local address shown by LM Studio;
4. load or fetch the available models;
5. select the model currently available through LM Studio; and
6. save the provider.

If Feluda cannot find the model, return to LM Studio and confirm that the model is loaded and the local server is active.

Choose your first local model

Local models come in different sizes.

Smaller models usually:

• use less memory;
• start more quickly;
• respond faster on ordinary computers; and
• work well for simple summaries, classification, and drafting.

Larger models may produce stronger results for some tasks, but they require more memory and may respond slowly without suitable hardware.

For your first test, choose a small or medium general-purpose instruction model.

Avoid downloading several large models before you know what your computer can run comfortably.

Computer requirements

Feluda itself does not determine how demanding a local model will be. The selected model creates most of the memory and processing requirements.

As a simple guide:

• a computer with limited memory should begin with a small model;
• a computer with 8 GB of available memory may handle some lightweight models;
• 16 GB or more gives you a wider choice; and
• larger models may require substantially more memory or a suitable graphics processor.

These are general guidelines. Actual performance depends on the model, model format, operating system, and other applications running at the same time.

Check the model page in Ollama or LM Studio before downloading it.

Test the model in Workbench

After saving the local provider:

1. open Workbench;
2. select the local provider;
3. select the downloaded model;
4. enter a short test message; and
5. review the response.

Try:

Summarise the following sentence in five words:

Feluda helps people build and reuse AI workflows.

If the model responds, your local connection is working.

Use a local model in Studio

A local model can also be selected in an AI step inside Studio.

When configuring the relevant workflow step:

1. select your local provider;
2. select the model;
3. add the instruction for the task;
4. save the workflow; and
5. test it with a simple example.

Keep the local model application running whenever you run that workflow.

If the model is unavailable, the workflow step that depends on it cannot complete.

Work offline

After Feluda, the local model application, and the model itself have been downloaded, you can use the model without a cloud AI provider.

Work can remain offline when every part of the task is local.

A task may still use the internet when it includes:

• a web search;
• an online tool;
• an external data source;
• a cloud model in another workflow step; or
• a Gene that connects to an outside service.

Review the complete task before assuming that every step remains on your computer.

Local models and privacy

A local model can process prompts and documents on your own computer.

This gives you more control over where model processing takes place, but you should still use sensitive information carefully.

Before using confidential material:

• confirm that the selected provider is local;
• review any enabled tools;
• check whether the workflow includes an online step;
• remove information that is not needed; and
• review the final result before using it.

Also protect your computer, user account, files, and backups. Local processing does not replace normal device security.

Compare local models

No single local model is best for every task.

You can compare models in Workbench by using the same instruction and sample information with each one.

Compare:

• whether the model follows your instructions;
• whether it includes the important information;
• how quickly it responds;
• how much memory it uses; and
• whether it supports the features required by your task.

Use sample content during comparison so that each model receives the same input.

Combine local and cloud models

Feluda can use both local and cloud providers.

You may choose a local model for one task and a cloud model for another. A workflow may also use different providers in different steps.

For example, you could use a local model to organise private source material, then use a cloud model only after unnecessary personal details have been removed.

Review the information passed into every step. Data sent to a cloud model or external tool leaves the local model environment and is handled by the selected service.

If Feluda cannot find your local model

Check the following:

• the local model application is open;
• its server is running;
• the model has finished downloading;
• the model is loaded when LM Studio requires it;
• the local address and port are correct;
• Feluda and the local application are running on the same computer; and
• your firewall is not blocking the local connection.

Then return to AI Providers and load the model list again.

If the model is slow

Local response speed depends on your computer and model choice.

Try these steps:

• close other memory-heavy applications;
• use a smaller model;
• shorten very long input;
• restart the local model application;
• allow the first response more time while the model loads; and
• compare the task with another local model.

The first response may take longer because the model needs to be loaded into memory.

If the model stops responding

Confirm that Ollama or LM Studio is still running.

If it is running but no longer responds:

1. stop the current request;
2. restart the local model service;
3. reload the model if needed;
4. return to Feluda;
5. confirm the provider and model; and
6. send a short test message.

Choose a smaller model if the problem happens repeatedly when your computer runs low on available memory.

Start with one simple task

Your local AI setup is complete when the model appears in Feluda and answers a message in Workbench.

Begin with a small task such as:

• summarising a short note;
• classifying a message;
• extracting a few details;
• rewriting a paragraph; or
• preparing a simple outline.

Once the model works reliably, you can use it in a workflow, compare it with other models, or combine it with approved tools.