PRACTICAL GUIDE
Self-hosted and on-premise AI: how to choose a private deployment architecture
Self-hosted AI is not simply installing a model inside the company. You need to decide where data lives, which components run internally and who controls identity, networking, tools, logs and updates.
· IA Empleado
A self-hosted or on-premise AI architecture aims to increase control over data, connectivity and operations, but it does not require every component to run on the same server. A business can host agents and data in its own infrastructure, use a managed private model, combine external services under egress rules or fully isolate selected workflows. The right choice depends on data sensitivity, regulatory requirements, integration with internal systems, latency, operational capacity and total cost. This guide separates those decisions so that “private AI” is not treated as one single architecture.
01
1. Define data requirements
Classify information, residency, retention and involved systems before selecting technology. Not all data requires the same level of isolation. Before automation, document the current state, the people involved, the data they use and the exceptions that occur. This baseline prevents improvements caused by unrelated changes from being attributed to AI and makes before-and-after comparison more consistent. It also identifies tasks that can be solved with simple deterministic rules without adding unnecessary complexity.
Implementation should separate interpretation, validation and execution. AI can help understand context or prepare a proposal, while deterministic rules, permissions and approvals control higher-impact actions. For this area we recommend measuring data categories and restrictions by category, recording human corrections and reviewing regularly whether automation continues to produce net savings and stable quality.
02
2. Choose the deployment pattern
Compare on-premise, private cloud and dedicated environments by control, cost, connectivity and customer operational capability. Before automation, document the current state, the people involved, the data they use and the exceptions that occur. This baseline prevents improvements caused by unrelated changes from being attributed to AI and makes before-and-after comparison more consistent. It also identifies tasks that can be solved with simple deterministic rules without adding unnecessary complexity.
Implementation should separate interpretation, validation and execution. AI can help understand context or prepare a proposal, while deterministic rules, permissions and approvals control higher-impact actions. For this area we recommend measuring cost, isolation and operational effort, recording human corrections and reviewing regularly whether automation continues to produce net savings and stable quality.
03
3. Design identities
Separate users, services and agents. Integrate SSO where needed and avoid shared credentials that weaken auditability. Before automation, document the current state, the people involved, the data they use and the exceptions that occur. This baseline prevents improvements caused by unrelated changes from being attributed to AI and makes before-and-after comparison more consistent. It also identifies tasks that can be solved with simple deterministic rules without adding unnecessary complexity.
Implementation should separate interpretation, validation and execution. AI can help understand context or prepare a proposal, while deterministic rules, permissions and approvals control higher-impact actions. For this area we recommend measuring unique identities and privileged access, recording human corrections and reviewing regularly whether automation continues to produce net savings and stable quality.
04
4. Apply least privilege
Define specific operations per tool. An agent that reads stock does not need permission to modify bank details. Before automation, document the current state, the people involved, the data they use and the exceptions that occur. This baseline prevents improvements caused by unrelated changes from being attributed to AI and makes before-and-after comparison more consistent. It also identifies tasks that can be solved with simple deterministic rules without adding unnecessary complexity.
Implementation should separate interpretation, validation and execution. AI can help understand context or prepare a proposal, while deterministic rules, permissions and approvals control higher-impact actions. For this area we recommend measuring granted permissions versus permissions actually used, recording human corrections and reviewing regularly whether automation continues to produce net savings and stable quality.
05
5. Segment the network
Place components according to sensitivity and limit lateral traffic. Connectors to internal systems can run close to sources without exposing them publicly. Before automation, document the current state, the people involved, the data they use and the exceptions that occur. This baseline prevents improvements caused by unrelated changes from being attributed to AI and makes before-and-after comparison more consistent. It also identifies tasks that can be solved with simple deterministic rules without adding unnecessary complexity.
Implementation should separate interpretation, validation and execution. AI can help understand context or prepare a proposal, while deterministic rules, permissions and approvals control higher-impact actions. For this area we recommend measuring segments, rules and permitted connections, recording human corrections and reviewing regularly whether automation continues to produce net savings and stable quality.
06
6. Control Internet egress
If the agent needs external models or APIs, use explicit destinations, proxies and egress policies. Do not grant unrestricted Internet access for convenience. Before automation, document the current state, the people involved, the data they use and the exceptions that occur. This baseline prevents improvements caused by unrelated changes from being attributed to AI and makes before-and-after comparison more consistent. It also identifies tasks that can be solved with simple deterministic rules without adding unnecessary complexity.
Implementation should separate interpretation, validation and execution. AI can help understand context or prepare a proposal, while deterministic rules, permissions and approvals control higher-impact actions. For this area we recommend measuring external destinations and egress volume, recording human corrections and reviewing regularly whether automation continues to produce net savings and stable quality.
07
7. Manage secrets
Centralise keys, tokens and certificates in a manager with rotation and access control. Avoid secrets in shared long-lived variables or repositories. Before automation, document the current state, the people involved, the data they use and the exceptions that occur. This baseline prevents improvements caused by unrelated changes from being attributed to AI and makes before-and-after comparison more consistent. It also identifies tasks that can be solved with simple deterministic rules without adding unnecessary complexity.
Implementation should separate interpretation, validation and execution. AI can help understand context or prepare a proposal, while deterministic rules, permissions and approvals control higher-impact actions. For this area we recommend measuring secret age and completed rotations, recording human corrections and reviewing regularly whether automation continues to produce net savings and stable quality.
08
8. Encrypt data
Apply encryption in transit and at rest according to risk. Define who manages keys and what happens during recovery. Before automation, document the current state, the people involved, the data they use and the exceptions that occur. This baseline prevents improvements caused by unrelated changes from being attributed to AI and makes before-and-after comparison more consistent. It also identifies tasks that can be solved with simple deterministic rules without adding unnecessary complexity.
Implementation should separate interpretation, validation and execution. AI can help understand context or prepare a proposal, while deterministic rules, permissions and approvals control higher-impact actions. For this area we recommend measuring encryption coverage and recovery tests, recording human corrections and reviewing regularly whether automation continues to produce net savings and stable quality.
09
9. Minimise context sent to models
Send only the fields required for the task. Redaction, pseudonymisation or local processing can reduce external data exposure. Before automation, document the current state, the people involved, the data they use and the exceptions that occur. This baseline prevents improvements caused by unrelated changes from being attributed to AI and makes before-and-after comparison more consistent. It also identifies tasks that can be solved with simple deterministic rules without adding unnecessary complexity.
Implementation should separate interpretation, validation and execution. AI can help understand context or prepare a proposal, while deterministic rules, permissions and approvals control higher-impact actions. For this area we recommend measuring data per request and removed fields, recording human corrections and reviewing regularly whether automation continues to produce net savings and stable quality.
10
10. Design audit logs
Record who initiated a task, which tools were used, which approval existed and what outcome occurred. Avoid logging secrets or unnecessary content. Before automation, document the current state, the people involved, the data they use and the exceptions that occur. This baseline prevents improvements caused by unrelated changes from being attributed to AI and makes before-and-after comparison more consistent. It also identifies tasks that can be solved with simple deterministic rules without adding unnecessary complexity.
Implementation should separate interpretation, validation and execution. AI can help understand context or prepare a proposal, while deterministic rules, permissions and approvals control higher-impact actions. For this area we recommend measuring auditable events and log quality, recording human corrections and reviewing regularly whether automation continues to produce net savings and stable quality.
11
11. Configure observability
Metrics, traces and alerts should show latency, errors, usage and exceptions. A private environment without observability merely moves the problem. Before automation, document the current state, the people involved, the data they use and the exceptions that occur. This baseline prevents improvements caused by unrelated changes from being attributed to AI and makes before-and-after comparison more consistent. It also identifies tasks that can be solved with simple deterministic rules without adding unnecessary complexity.
Implementation should separate interpretation, validation and execution. AI can help understand context or prepare a proposal, while deterministic rules, permissions and approvals control higher-impact actions. For this area we recommend measuring useful alerts, availability and latency, recording human corrections and reviewing regularly whether automation continues to produce net savings and stable quality.
12
12. Design backups
Determine which components need backup and at what frequency. Not all temporary data must persist, but critical configuration and state should. Before automation, document the current state, the people involved, the data they use and the exceptions that occur. This baseline prevents improvements caused by unrelated changes from being attributed to AI and makes before-and-after comparison more consistent. It also identifies tasks that can be solved with simple deterministic rules without adding unnecessary complexity.
Implementation should separate interpretation, validation and execution. AI can help understand context or prepare a proposal, while deterministic rules, permissions and approvals control higher-impact actions. For this area we recommend measuring backup success and asset coverage, recording human corrections and reviewing regularly whether automation continues to produce net savings and stable quality.
13
13. Test restoration
An untested backup is only a hypothesis. Run periodic restores and measure how long the service takes to return to operation. Before automation, document the current state, the people involved, the data they use and the exceptions that occur. This baseline prevents improvements caused by unrelated changes from being attributed to AI and makes before-and-after comparison more consistent. It also identifies tasks that can be solved with simple deterministic rules without adding unnecessary complexity.
Implementation should separate interpretation, validation and execution. AI can help understand context or prepare a proposal, while deterministic rules, permissions and approvals control higher-impact actions. For this area we recommend measuring RTO, RPO and successful restore percentage, recording human corrections and reviewing regularly whether automation continues to produce net savings and stable quality.
14
14. Govern updates
Version models, prompts, images and connectors. Test changes in a controlled environment and preserve rollback capability. Before automation, document the current state, the people involved, the data they use and the exceptions that occur. This baseline prevents improvements caused by unrelated changes from being attributed to AI and makes before-and-after comparison more consistent. It also identifies tasks that can be solved with simple deterministic rules without adding unnecessary complexity.
Implementation should separate interpretation, validation and execution. AI can help understand context or prepare a proposal, while deterministic rules, permissions and approvals control higher-impact actions. For this area we recommend measuring failed changes and rollback time, recording human corrections and reviewing regularly whether automation continues to produce net savings and stable quality.
15
15. Define responsibilities
Document who maintains hosts, application, secrets, backups, connectors and models. Boundaries between customer and provider should be explicit. Before automation, document the current state, the people involved, the data they use and the exceptions that occur. This baseline prevents improvements caused by unrelated changes from being attributed to AI and makes before-and-after comparison more consistent. It also identifies tasks that can be solved with simple deterministic rules without adding unnecessary complexity.
Implementation should separate interpretation, validation and execution. AI can help understand context or prepare a proposal, while deterministic rules, permissions and approvals control higher-impact actions. For this area we recommend measuring ownerless incidents and escalation time, recording human corrections and reviewing regularly whether automation continues to produce net savings and stable quality.
16
16. Plan capacity
CPU, memory, storage, queues and API limits should be sized for normal volume and peaks. Privacy does not replace performance planning. Before automation, document the current state, the people involved, the data they use and the exceptions that occur. This baseline prevents improvements caused by unrelated changes from being attributed to AI and makes before-and-after comparison more consistent. It also identifies tasks that can be solved with simple deterministic rules without adding unnecessary complexity.
Implementation should separate interpretation, validation and execution. AI can help understand context or prepare a proposal, while deterministic rules, permissions and approvals control higher-impact actions. For this area we recommend measuring utilisation, saturation and queues, recording human corrections and reviewing regularly whether automation continues to produce net savings and stable quality.
17
17. Review architecture periodically
Requirements and providers change. Periodic review confirms whether the isolation level remains appropriate and whether architecture can be simplified without increasing risk. Before automation, document the current state, the people involved, the data they use and the exceptions that occur. This baseline prevents improvements caused by unrelated changes from being attributed to AI and makes before-and-after comparison more consistent. It also identifies tasks that can be solved with simple deterministic rules without adding unnecessary complexity.
Implementation should separate interpretation, validation and execution. AI can help understand context or prepare a proposal, while deterministic rules, permissions and approvals control higher-impact actions. For this area we recommend measuring findings, technical debt and operating cost, recording human corrections and reviewing regularly whether automation continues to produce net savings and stable quality.
TAKEAWAYS
Key ideas
Private does not simply mean installing a server.
Classify data before choosing architecture.
Separate identities and apply least privilege.
Control networking and Internet egress.
Centralise secrets and encryption.
Minimise context sent to external models.
Record decisions without duplicating sensitive data.
Test backups and restoration.
Version and govern updates.
Define operational responsibilities clearly.
GO DEEPER
Private AI for business: deploy AI agents and AI Employees with control over data and infrastructure.
Private AI lets businesses run agents and automations in dedicated, self-hosted or on-premise infrastructure when they need stronger control over where data flows, how systems authenticate and who operates the platform. The goal is not to isolate AI by default, but to choose an architecture proportionate to privacy, integration, availability and governance requirements. A robust deployment separates model, data, tools and permissions and defines networking, identity, secrets, observability, backups, updates and operational ownership from the start.
APPLY IT