How to Choose a GaaS Provider
Choosing an Agentic as a Service provider requires more than comparing model names or product demonstrations.
A GaaS provider may operate agents that:
- access internal information;
- choose tools;
- move across several systems;
- prepare decisions;
- create or update records;
- run on a schedule; and
- retain task state.
The provider therefore becomes part of the organisation's operating process.
A strong selection process should evaluate the complete service:
- the task it is designed to perform;
- the agent and workflow design;
- the models and tools involved;
- data handling;
- permissions;
- monitoring;
- evaluation;
- human oversight;
- reliability;
- support;
- commercial terms; and
- the ability to leave or migrate later.
The best provider is not the one with the most autonomous demonstration.
It is the one that can produce a useful, controlled, measurable outcome for a clearly defined use case.
Start with the problem, not the vendor
Before reviewing providers, define the task.
Record:
- the current process;
- the desired outcome;
- the people involved;
- the systems involved;
- the sources used;
- the actions performed;
- the current time and cost;
- common errors;
- review requirements; and
- the consequences of failure.
This prevents the buying process from becoming a search for an impressive agent without a practical role.
A useful requirement may be:
Prepare a weekly supplier-risk report from approved records and create
follow-up drafts for manager review.
A weak requirement is:
We need an autonomous agent for procurement.
Confirm that GaaS is the right service model
Some problems need GaaS.
Others need:
- a normal SaaS feature;
- one AI model call;
- a deterministic workflow;
- a search tool;
- an internal agent;
- a custom application; or
- a manual process improvement.
GaaS is most suitable when:
- the goal is clear;
- several steps are required;
- some variation exists;
- approved tools can be defined;
- the service will be used repeatedly;
- exceptions can be identified;
- the outcome can be measured; and
- the organisation wants a provider to manage part of the agentic stack.
Do not buy a managed agentic service when a simpler and more predictable solution solves the problem.
Define non-negotiable requirements
Separate essential requirements from optional features.
Non-negotiable requirements may include:
- a specific deployment region;
- local or private model support;
- named data-processing locations;
- approved model providers;
- no training on customer data;
- read-only operation;
- human approval before writes;
- exportable activity records;
- support for MCP;
- specific business integrations;
- deletion and retention controls;
- availability commitments; or
- a defined exit process.
This makes early provider screening faster.
Evaluate task fit
Ask the provider to describe the exact tasks its service handles well.
A credible answer should include:
- supported input;
- expected output;
- required systems;
- normal exceptions;
- unsupported cases;
- completion criteria;
- review requirements; and
- measured performance.
Be cautious when a provider claims one general agent can reliably perform any business task.
Focused services are often easier to evaluate, govern, and improve.
Ask for a real workflow demonstration
A polished conversational demonstration does not prove operational capability.
Ask the provider to show the full workflow for a representative task.
The demonstration should include:
- task intake;
- context retrieval;
- tool selection;
- an incomplete-input case;
- a no-result case;
- an approval step;
- an error;
- a completed action;
- activity evidence; and
- final status.
Use your own safe sample data when possible.
A provider should be able to demonstrate failure handling, not only the ideal path.
Understand what the provider actually manages
The term managed service can hide important differences.
Ask whether the provider manages:
- agent instructions;
- models;
- model routing;
- tool integrations;
- workflow logic;
- task state;
- memory;
- credentials;
- monitoring;
- evaluation;
- scaling;
- incidents;
- updates; and
- customer support.
Also ask which responsibilities remain with the customer.
A platform that provides an agent builder may still require the customer to design, deploy, test, and monitor everything.
That can be useful, but it is different from a fully managed GaaS offering.
Review the agent architecture
The provider should explain the service architecture in understandable terms.
Ask:
- Is one agent used or several?
- Which decisions are adaptive?
- Which steps are fixed?
- Which model performs each role?
- How is task state stored?
- How are agent loops limited?
- How are errors routed?
- When is a human involved?
- How is the final result validated?
Complexity should have a clear purpose.
A multi-agent design is not automatically better than one controlled agent inside a structured workflow.
Review model choices
Model evaluation should go beyond brand names.
Ask:
- Which models are used?
- Can the provider change models without notice?
- Are different models used for different tasks?
- Can a customer restrict providers?
- Are local or private models supported?
- How are model changes tested?
- How is model performance monitored?
- What happens when a model is unavailable?
- Is customer data used to train any model?
The strongest model is not always the best operational choice.
Cost, latency, privacy, tool use, and task accuracy also matter.
Review tools and integrations
Create an inventory of every tool the agent may use.
For each tool, identify:
- purpose;
- provider;
- account;
- source;
- destination;
- read or write behaviour;
- required permissions;
- input;
- output;
- retry behaviour; and
- failure handling.
Ask whether the provider supports:
- APIs;
- webhooks;
- databases;
- file systems;
- local tools;
- cloud applications;
- MCP servers;
- custom integrations; and
- reusable capability packages.
Integration breadth matters less than controlled access to the systems you actually need.
Confirm permission controls
A provider should support least-privilege access.
Look for controls that can limit:
- accounts;
- projects;
- workspaces;
- record types;
- tools;
- read access;
- write access;
- deletion;
- URLs;
- IP addresses;
- file paths;
- ports;
- destinations; and
- execution time.
Ask whether read and write credentials can be separated.
Avoid providers that require broad administrator access for routine tasks without a clear technical reason.
Review credential protection
Ask how credentials are:
- entered;
- encrypted;
- stored;
- scoped;
- accessed by tools;
- hidden from models;
- rotated;
- revoked;
- recovered; and
- removed at contract end.
The provider should not require users to place raw secrets inside prompts, workflows, notes, or ordinary files.
Determine whether credentials belong to the customer, provider, or a shared service account.
Ownership must remain clear.
Map the complete data path
Ask the provider to show where information travels.
Include:
- user interface;
- agent runtime;
- model provider;
- tool server;
- memory or state storage;
- logs;
- monitoring;
- connected applications;
- support systems; and
- final destinations.
For each location, ask:
- what data is processed;
- where it is processed;
- how long it is retained;
- who can access it;
- whether it is encrypted;
- whether subcontractors are involved;
- how deletion works; and
- whether data crosses regions.
Do not accept “secure by design” as a substitute for a clear data-flow explanation.
Review privacy and data-use terms
Confirm:
- whether customer data trains models;
- whether prompts are retained;
- whether tool output is retained;
- whether support staff can view content;
- whether memory persists between tasks;
- whether users can correct stored information;
- whether customers control retention;
- how backups are deleted; and
- what happens after termination.
The contract should match the technical design.
Evaluate observability
A GaaS customer should be able to inspect important activity.
Useful evidence includes:
- task identifier;
- triggering user or system;
- agent version;
- workflow version;
- model;
- selected tools;
- tool input;
- tool output;
- source;
- destination;
- approvals;
- warnings;
- errors;
- retries;
- timestamps;
- final status; and
- confirmed external actions.
Ask whether records are searchable, exportable, and retained for an appropriate period.
A final answer without execution evidence is not sufficient for important tasks.
Evaluate human-control features
The provider should support human control where impact requires it.
Look for:
- draft-first operation;
- approval queues;
- exact action previews;
- destination confirmation;
- rejection;
- correction;
- pause controls;
- task cancellation;
- escalation;
- manual takeover; and
- reviewer identity records.
Approval should bind to the exact action displayed.
If the destination or content changes, the service should request approval again.
Review guardrails and stopping conditions
Ask how the service limits:
- tool calls;
- retries;
- runtime;
- cost;
- data volume;
- repeated errors;
- source expansion;
- destination changes;
- agent delegation; and
- unsupported actions.
Confirm what causes the agent to:
- stop;
- ask for clarification;
- route to a person;
- return a partial result; or
- fail safely.
A provider should not treat continuous action as a sign of intelligence.
Review prompt-injection protections
Agents may retrieve malicious or misleading instructions from documents, websites, and messages.
Ask how the provider:
- separates instructions from retrieved content;
- treats external content as untrusted;
- limits tool access;
- validates destinations;
- detects suspicious instructions;
- tests adversarial cases;
- blocks data exfiltration; and
- handles compromised sources.
Prompt injection cannot be solved by one warning in the system prompt.
It requires layered controls.
Review retry and duplicate prevention
Ask the provider to demonstrate what happens when a write tool times out.
A reliable service should:
- record the request identifier;
- inspect the destination;
- determine whether the first action completed;
- avoid an immediate duplicate;
- limit retries;
- show partial status; and
- route uncertainty for review.
This is especially important for tasks that create records, files, messages, transactions, or notifications.
Evaluate result validation
Ask what happens after the agent produces an answer or action.
Validation may check:
- required fields;
- source references;
- dates;
- amounts;
- identifiers;
- output structure;
- allowed values;
- destination;
- duplication;
- policy rules; and
- task completion.
Important factual or operational values should use deterministic checks where possible.
The provider should distinguish generated confidence from verified evidence.
Request evaluation results
Ask for evidence from representative task evaluations.
Useful measures include:
- task completion rate;
- correct tool selection;
- source accuracy;
- field accuracy;
- validation pass rate;
- exception rate;
- human correction rate;
- duplicate-action rate;
- false-completion rate;
- runtime;
- cost per accepted task; and
- recovery after failure.
Ask how the test set was created and whether it includes difficult cases.
A general model benchmark does not prove that the service can perform your workflow.
Test exceptional cases
Your proof of concept should include:
- missing input;
- no matching result;
- conflicting sources;
- invalid data;
- permission denial;
- unavailable tool;
- model outage;
- slow response;
- timed-out write;
- repeated request;
- changed result format;
- denied approval;
- prompt injection; and
- wrong-destination prevention.
A provider that discourages failure testing is not ready for important operational use.
Review reliability
Ask for information about:
- availability;
- maintenance;
- regional resilience;
- queues;
- task recovery;
- backups;
- disaster recovery;
- model-provider outages;
- tool outages;
- data restoration; and
- service-status communication.
Determine whether an interrupted task:
- resumes;
- restarts;
- fails;
- waits;
- duplicates earlier work; or
- requires human review.
Review incident response
Confirm:
- how incidents are detected;
- who is notified;
- how customers report issues;
- how writes are stopped;
- whether agents can be disabled quickly;
- how credentials are revoked;
- how affected activity is identified;
- how evidence is preserved;
- how recovery is tested; and
- how post-incident findings are shared.
Ask for a sample incident process.
Evaluate support
Agentic services affect business processes, so support quality matters.
Review:
- support hours;
- response targets;
- escalation paths;
- named contacts;
- technical expertise;
- onboarding support;
- workflow support;
- incident support;
- model and tool troubleshooting;
- customer training; and
- documentation.
Determine which support level is included and which services require an additional agreement.
Review change management
The provider may update:
- models;
- prompts;
- agents;
- workflow logic;
- tools;
- routing;
- memory;
- validation;
- user interfaces; and
- safety controls.
Ask:
- Which changes require notice?
- Can customers test updates first?
- Are versions visible?
- Can a release be rolled back?
- How are integrations protected from breaking changes?
- Are evaluation results available after material updates?
Behavioural changes need the same attention as normal software releases.
Evaluate portability
A credible provider should explain what can be exported.
Review portability for:
- customer data;
- prompts;
- agent instructions;
- workflows;
- tool mappings;
- configuration;
- memory;
- activity records;
- evaluation results;
- approval history; and
- generated output.
Ask whether exports use documented formats.
Portability reduces operational risk and improves negotiating power.
Review termination and offboarding
Before signing, understand how the service ends.
Confirm:
- how agents are disabled;
- how schedules stop;
- how connections are removed;
- how credentials are revoked;
- how customer data is exported;
- how stored data is deleted;
- how backups are handled;
- how deletion is confirmed;
- how support continues during transition; and
- whether additional exit charges apply.
Offboarding should not be designed only after the relationship ends.
Understand the pricing model
GaaS pricing may include:
- subscriptions;
- users;
- managed agents;
- tasks;
- runs;
- model usage;
- tool calls;
- processing time;
- storage;
- integrations;
- monitoring;
- support; or
- accepted outcomes.
Ask how the provider treats:
- failed runs;
- retries;
- partial completion;
- human review;
- duplicate actions;
- long-running tasks;
- premium models;
- external service charges; and
- test environments.
Estimate the cost of a completed, accepted task.
Do not compare only the advertised price per model call.
Review contractual responsibility
The agreement should clarify:
- service scope;
- customer responsibilities;
- provider responsibilities;
- availability;
- support;
- data use;
- security;
- subcontractors;
- incident notification;
- intellectual property;
- audit rights;
- liability;
- retention;
- deletion;
- portability; and
- termination.
High-impact deployments require appropriate legal, security, privacy, and procurement review.
Evaluate the provider organisation
Review whether the provider can support an ongoing service.
Consider:
- leadership;
- technical expertise;
- financial stability;
- customer references;
- support capacity;
- security programme;
- incident history;
- product roadmap;
- dependency on one model provider;
- documentation quality; and
- willingness to provide evidence.
A small provider can still be suitable.
The evaluation should match the importance of the task.
Ask for customer references
Useful reference questions include:
- Which task do you use the service for?
- How long did deployment take?
- What needed more work than expected?
- How often do people correct results?
- How transparent are failures?
- How responsive is support?
- Have updates changed behaviour?
- Can you export your records?
- What does the service cost at real volume?
- Would you choose the provider again?
References should involve a similar use case or operating environment.
Run a controlled proof of value
A proof of value should be narrower than a full deployment.
Use:
- one defined task;
- representative test cases;
- non-sensitive or approved data;
- limited tools;
- read-only access where possible;
- a safe destination;
- named reviewers;
- baseline measures;
- clear success thresholds; and
- an end date.
Do not allow a pilot to become an ungoverned production system.
Measure the proof of value
Compare the GaaS service with the current process.
Measure:
- completion quality;
- preparation time;
- elapsed time;
- missed cases;
- human corrections;
- source accuracy;
- exception volume;
- tool failures;
- duplicate actions;
- user satisfaction;
- cost per accepted result; and
- support effort.
Record negative results as carefully as positive ones.
Use a weighted provider scorecard
A scorecard can reduce decisions based on demonstrations alone.
Example categories:
| Category | Suggested emphasis |
|---|---|
| Task fit and outcome quality | High |
| Security and privacy | High |
| Permissions and human control | High |
| Observability and auditability | High |
| Reliability and recovery | High |
| Integration fit | Medium to high |
| Portability | Medium to high |
| Support and change management | Medium |
| Pricing and total cost | Medium |
| User experience | Medium |
Adjust the weighting to the risk and importance of the task.
A high score in presentation quality should not compensate for weak access controls.
Red flags to watch for
Be cautious when a provider:
- cannot define the supported task;
- promises near-perfect autonomy;
- refuses to show failures;
- requires broad administrator access;
- cannot name its model or infrastructure providers;
- cannot explain the data path;
- hides tool activity;
- does not support export;
- has no clear incident process;
- treats all errors with automatic retries;
- cannot separate test and production;
- changes models without evaluation;
- has unclear credential ownership;
- cannot explain deletion; or
- measures success only by model usage.
How Feluda can support provider evaluation
Feluda can help organisations test agentic capabilities in a controlled environment.
Workbench can support interactive tool tests.
Studio can represent the workflow, approvals, branches, and error paths.
MCP servers and Genes can expose focused capabilities.
RunFlows can help users inspect tool activity, intermediate results, warnings, and errors.
Schedule Manager can test recurring execution.
Permissions can constrain URLs, IP addresses, file paths, and ports.
Local model support can help compare cloud, local, and hybrid processing options.
These capabilities make it easier to evaluate a provider or service design against a known workflow rather than relying only on a sales demonstration.
Final provider-selection checklist
Before choosing a provider, confirm that:
- the use case is clearly defined;
- GaaS is the appropriate service model;
- supported and unsupported tasks are documented;
- provider responsibilities are clear;
- the architecture is understandable;
- models and tools are known;
- permissions are narrow;
- credentials are protected;
- the data path is documented;
- privacy terms match the design;
- activity is visible and exportable;
- human control is practical;
- retries and duplicate writes are controlled;
- evaluation evidence is relevant;
- exceptional cases have been tested;
- reliability and recovery are understood;
- support and incident response are adequate;
- material updates are controlled;
- pricing reflects real task volume;
- data and workflows are portable;
- termination is practical; and
- the proof of value meets agreed thresholds.
The practical conclusion
Choosing a GaaS provider is an operating decision, not only a software purchase.
The provider may become responsible for agents that use data, tools, and business systems on the customer's behalf.
Selection should therefore focus on evidence:
- Can the service complete the defined task?
- Can the organisation limit what it can access?
- Can reviewers see what it did?
- Can important actions be stopped or approved?
- Can failures be understood and recovered?
- Can value be measured?
- Can the customer leave without losing control of its data and processes?
A provider that answers these questions clearly is more valuable than one that offers the broadest autonomy with the least visibility.