​
Overview

The Aporia architecture is split into two planes to avoid sensitive data exposure and simplify maintenance.
  • The control plane lives in Aporia’s cloud and serves the policy configuration, along with the UI and metadata.
  • The data plane can be deployed in your cloud environment, runs the policies themselves and provides an OpenAI-compatible endpoint.

​
Architecture

Built on a robust Kubernetes architecture, the data plane is designed to expand horizontally, adapting to the volume and demands of your LLM applications. The data plane lives in your cloud provider account, and it’s a fully stateless application where all configuration is retrieved from the control plane. Any LLM prompt & response is processed in-memory only, unless users opt to storing them in an Postgres database in the customer’s cloud. Users can either use the OpenAI proxy or call the detection API directly. The data plane generates non-sensitive metadata that is pushed to the control plane (e.g. toxicity score, hallucination score).

​
Data plane modes

The data plane supports 2 modes:
  • Azure OpenAI mode - In this basic mode, all policies run using Azure OpenAI. While in this mode you can run the data plane without any GPUs, this mode does not support policy fine-tuning, and the accuracy/latency of the policies will be lower.
  • Full mode - In this mode, we’ll run our fine-tuned small language models (SLMs) on your infrastructure. This achieves our state-of-the-art accuracy + latency but requires access to GPUs.
The following architecture image describes the full mode:

​
Dependencies

  • Kubernetes (e.g. Amazon EKS)
  • Postgres (e.g. Amazon RDS)
  • RabbitMQ (e.g. Amazon MQ)

​
Security

​
Networking

All communication to the Aporia is done via a single port based on HTTPS. You can choose your own internal domain for Aporia, provide your own TLS certificates, and put Aporia behind your existing API gateway. Communication is encrypted with industry standard security protocols such as TLS 1.3. By default, Aporia will configure networking for you, but you can also control data plane networking with customer-managed VPC or VNet. Aporia does not change or modify any of your security and governance policies. Local firewalls complement security groups and subnet firewall policies to block unexpected inbound connections.

​
Application

The data plane runs in your cloud provider account in a Kubernetes cluster. Aporia supports AWS, Google Cloud and Azure. Aporia automatically runs the latest hardened base images, which are typically updated every 2-4 weeks. All containers run in unprivileged mode as non-root users. Every release is scanned for vulnerabilities, including container OS, third-party libraries, as well as static and dynamic code scanning. Aporia code is peer reviewed by developers with security training. Significant design documents go through comprehensive security reviews. Issues are tracked against the timeline shown in this table. Aporia’s founding team come from the elite cybersecurity Unit 8200 of the Israeli Defense Forces.