Private AI is an operating architecture in which the data path, model access, permissions, logging, and retention are deliberately controlled. It does not automatically mean running a local model. Some workflows are well served by a managed API with suitable contracts and data controls; others require Swiss hosting, local inference, on-premise operation, or a hybrid combination.
What does private AI mean in practice?
A private AI architecture answers five questions clearly:
- Which data leaves each system?
- Which provider and model processes it?
- Where are inputs, outputs, and logs stored?
- Who may access models, sources, and tools?
- How can data be deleted, exported, and audited?
The term therefore describes control over the full processing path, not only the location of a server.
Which deployment models are available?
Managed model API
A managed API provides fast access to capable models while reducing infrastructure work. It is suitable when contract terms, data processing, retention, region, and security controls fit the use case.
Do not check only whether a provider excludes customer data from training. Logging, abuse monitoring, support access, subprocessors, transfers, deletion periods, and the region actually used are also relevant.
Swiss cloud or managed private environment
A Swiss hosting or cloud environment can simplify data-location requirements and contractual accountability. It does not automatically guarantee full data sovereignty. Operator access, backups, monitoring, model sourcing, and external interfaces remain part of the architecture review.
This model often fits organisations that want local accountability without operating all inference hardware themselves.
Local inference
With local inference, the model runs on controlled hardware in an owned or dedicated environment. This reduces external model calls and may suit confidential documents, low-latency workflows, or offline operation.
The organisation takes on more responsibility for model operations, patches, capacity, evaluation, access controls, and monitoring. A locally operated model is not automatically secure, accurate, or less expensive.
On-premise operation
On-premise means operating infrastructure inside the organisation's controlled environment. It may be necessary for strict network, contract, or data requirements. It is usually the most demanding option for operations, scaling, and model updates.
On-premise deployment should follow from a specific risk model, not from a general assumption that internal always means safer.
Hybrid AI architecture
Hybrid architectures route tasks according to sensitivity and capability needs. Confidential extraction may run locally, while non-sensitive summarisation or complex general tasks use a managed API. A gateway enforces rules for data classes, models, providers, and fallbacks.
For many organisations, hybrid is the most practical option because not every workflow needs the same protection level.
How should an organisation choose the right architecture?
1. Classify data and risk
Do not begin with a model comparison. Classify data by confidentiality, regulatory relevance, trade-secret value, personal-data content, and the impact of an error. Document which categories permit external processing and which do not.
2. Review the workflow, not only the chatbot
A workflow includes more than a prompt and answer. Sources, retrieval, tool calls, logs, memory, human approvals, and export paths may transfer data to additional systems. Map the complete data flow from input through retention.
3. Measure the quality requirement
Local and open models may work very well for extraction, classification, and domain-specific tasks. Complex reasoning, coding, or multimodal work may require other models. Test real tasks with a stable evaluation set instead of inferring suitability from benchmarks.
4. Calculate the full operating cost
Do not compare API prices with hardware cost alone. Include engineering, operations, utilisation, redundancy, monitoring, updates, evaluation, and support. Local infrastructure can be economical at stable high usage but create unnecessary fixed cost for small or variable workloads.
5. Define exit and portability
A controlled architecture documents models, prompts, evaluation sets, data formats, and interfaces so that a provider change remains possible. Portability does not mean every model produces identical results. It means the switching cost is visible and testable.
Which controls are required regardless of hosting?
Every private AI deployment needs:
- role-based access to sources and tools,
- separate development and production environments,
- logging of important access and actions,
- defined retention and deletion processes,
- evaluation before model or prompt changes,
- limits on agent steps, tools, and cost,
- documented human approval for critical outcomes.
Hosting is only one control layer. A server in Switzerland does not prevent incorrect permissions or uncontrolled agent actions.
Is Swiss hosting always required?
No. The correct answer depends on the data, contracts, risk model, interfaces, and internal requirements. Swiss hosting may be useful or required, but it does not replace a complete privacy and security assessment. Conversely, an international cloud is not automatically unsuitable when the data path, contracts, and controls satisfy the requirements.
Organisations should involve their privacy and legal advisers for specific legal requirements. The technical architecture must then implement those requirements in a verifiable way.
Sources and further guidance
- Swiss Federal Data Protection and Information Commissioner: https://www.edoeb.admin.ch/
- Swiss Federal Act on Data Protection: https://www.fedlex.admin.ch/eli/cc/2022/491/en
- NIST AI Risk Management Framework: https://www.nist.gov/itl/ai-risk-management-framework
- OWASP Top 10 for Large Language Model Applications: https://owasp.org/www-project-top-10-for-large-language-model-applications/
Next step
For one real workflow, create a data-flow map covering sources, models, providers, logs, storage locations, and responsibilities. Then compare two or three deployment models on quality, risk, cost, and portability. This turns private AI into a verifiable architecture decision rather than a vague hosting promise.