Security Layers for Enterprise LLM-Based Applications

In the era of Large language model (LLM) availability and their uses, enterprise has to be careful to design their applications, considering external threat and attacks. We have been discussing some of the best practices to enable security for LLM-based applications at the Enterprise level. There are a bunch of tools in the market; however, our article focuses more on the method instead of Tools, as we have proposed the following layers of security, which include traditional methods and new, evolving methods. More details are given below:

API Gateway:

User-initiated traffic passes through the API gateway. We have to implement authorization and authentication. Additionally, we can leverage the API gateway for the following services:

1. Auth & RBAC
2. Rate Limiting
3. WAF & Bot Protection
4. TLS + GW level Logging

AI Gateway:

The next component is the AI Gateway, which can scan both incoming and outgoing traffic. It can perform the following things:
1. PII/PHI Protection
2. Prompt Security
3. Output Guardrails
4. Audit & Compliance

Application or Code level:

For a genAI application, we can build an additional framework, which can add functionality to sanitize incoming text. This new table will contain the metadata for sensitive information, prompts, jailbreak text, and other elements used for validation. Text will be sensitized before sending to LLM. Similarly, outgoing text will also be sanitized at the application level.

Database:
The database is an important component; we can't expose enterprise data to everyone. Here comes role based, ABAC, column/ row level security, other than encrypting data.

Summary:

leveraging API/AI Gateway, code level data sanitization and enabling data security and governance principal, we can make safe and better AI application.