For: Healthcare administrators, boutique law firm partners, and security architects.

Why Medical AI Cannot Live in the Cloud

Artificial Intelligence has reached an inflection point. The ability for an LLM to read a 5,000-page medical history and output a precise chronological summary in ten seconds is nothing short of revolutionary.

But for high-stakes professionals—doctors, medical researchers, and lawyers—there is a massive, often unspoken problem: You cannot upload Patient Health Information (PHI) or Legal Intellectual Property to a cloud provider without assuming catastrophic liability.

When you paste a patient’s chart into ChatGPT or an equivalent cloud AI, you are transmitting sensitive data outside of your secure network. Even with “enterprise” agreements promising zero data training, the risk of a third-party breach or a misconfigured cloud bucket is too high for true compliance.

The solution isn’t to abstain from AI. The solution is to bring the AI entirely in-house.

This post outlines the architecture for the Sentinel AI Workstation: a fully localized, air-gapped inference engine that runs state-of-the-art LLMs natively, secured by an enterprise-grade Identity and Access Management (IAM) perimeter.

The Local Inference Paradox

Running an LLM locally (using engines like Ollama to run models like Llama-3 or MedAlpaca) solves the data sovereignty problem perfectly. The data never leaves your physical office.

However, installing an LLM on a machine creates a new security nightmare. Who has access to it? If an intern sits down at the terminal, can they query the AI to summarize a VIP patient’s file? If a nurse logs in, can they see the chat history of a senior partner?

Anyone can install a local AI. Managing who is allowed to talk to it, and what they are allowed to see, requires Zero-Trust Architecture.

Building the Zero-Trust Perimeter

To construct a truly secure local AI workstation, we decouple the AI inference engine from the security layer. We wrap the AI in a fortress composed of three best-in-class open-source tools: APISIX, Keycloak, and Open Policy Agent (OPA).

1. The Gateway Layer (Apache APISIX)

APISIX acts as the sole entry point to the AI. The local LLM and its user interface (Open WebUI) are bound strictly to internal network ports that cannot be accessed directly. Every single request must pass through APISIX. APISIX also provides critical rate-limiting; if multiple doctors query the AI simultaneously, APISIX manages the queue to ensure the local GPU does not overheat or crash.

2. The Identity Layer (Keycloak)

Before APISIX allows a user to even see the chat interface, it redirects them to Keycloak. Keycloak acts as the central Identity Provider. It enforces Single Sign-On (SSO) and, critically, Multi-Factor Authentication (MFA). For the highest security environments, Keycloak can be configured to require physical hardware keys (like a YubiKey) before granting access. Once authenticated, Keycloak issues a cryptographically signed JSON Web Token (JWT) containing the user’s specific roles.

3. The Policy Layer (Open Policy Agent)

This is where fine-grained authorization happens. Just because a user is logged in doesn’t mean they can do whatever they want. On every request, APISIX asks OPA: “Is Nurse Jane allowed to access the Oncology Agent?” OPA evaluates its declarative Rego policies in milliseconds, ensuring that users only interact with the agents and chat histories they are strictly authorized to see.

The Architecture

Here is what the flow looks like:

sequenceDiagram
    participant User as Doctor/Researcher
    participant APISIX as APISIX Gateway
    participant Keycloak as Keycloak IdP
    participant OPA as Open Policy Agent
    participant UI as Open WebUI
    participant LLM as Local Ollama (MedAlpaca)

    User->>APISIX: GET /chat
    APISIX->>Keycloak: Redirect to Login (OIDC)
    User->>Keycloak: Authenticate (Password + YubiKey)
    Keycloak-->>APISIX: Issue signed JWT w/ Roles
    APISIX->>OPA: Allow request? (Pass JWT)
    OPA-->>APISIX: Decision: True (Role = Specialist)
    APISIX->>UI: Proxy request to Dashboard
    User->>UI: Prompt: "Summarize patient chart..."
    UI->>LLM: Internal API Call
    LLM-->>UI: Generates secure, local response
    UI-->>User: Displays response

The “Human-in-the-Loop” Mandate

Even with bulletproof infrastructure, it must be stated: AI is an accelerator, not a decision-maker. These systems are designed to draft documents, synthesize vast amounts of text, and surface insights. The final clinical or legal decision must always rest with the licensed professional. The architecture is designed to protect the data, but the human must protect the outcome.

Conclusion & Next Steps

If you are a boutique law firm, a private medical practice, or a research team dealing with highly sensitive Intellectual Property, you do not have to choose between cutting-edge AI capabilities and data security.

You can have a specialized, intelligent agent sitting securely in your office, running on hardware you own, protected by the same identity architecture used by Fortune 500 banks.

---

Do you need this for your practice? I build and ship these zero-trust secure appliances (The Sentinel AI Workstation) as turnkey solutions for high-stakes professionals. Hand me your access requirements, and I will ship you a fully configured hardware unit ready to plug into your network.

Contact me to schedule a demo of the Sentinel architecture.

---

This architecture leverages the APISIX + Keycloak patterns developed at AuthGeek. To see more of our Zero-Trust gateway configurations, visit demo.authgeek.com.