​
Overview

The DOS Policy prevents system degradation or shutdown caused by a flood of requests from a single user or IP address. It helps protect LLM services from being overwhelmed by excessive traffic. This policy monitors and limits the number of requests a user can make in a one-minute window. Once the limit is exceeded, the user is blocked from making further requests until the following minute.

​
User Configuration

  • Threshold Range: 0 - 1,000 requests per minute.
  • Default: 100 requests per minute.
Once the threshold is reached, any further requests from the user will be blocked until the start of the next minute.

​
User ID Integration

To ensure this policy functions correctly, the user should provide a unique User ID to activate the policy. Without the User ID, the policy will not function. The User ID parameter should be passed in the request body as user:.

​
Security Standards

  1. OWASP LLM Top 10 Mapping: LLM04 - Model Denial of Service.
  2. NIST Mapping: Denial of Service Attacks.
  3. MITRE ATLAS Mapping: AML.T0029 - Denial of ML Service.