Build your own custom policy by writing a prompt.
target
and a modality
for your policy.
target
is either prompt
or response
, and determines if the policy should run on prompts or responses, respectively. Note that if any of the extractions in the evaluation instructions or system prompt run on the response, then the policy target must also be response
modality
is either legit
or violate
, and determines how the response from the LLM (which is always TRUE
or FALSE
) will be interpreted. In legit
modality, a TRUE
response means the message is legitimate and there are no issues, while a FALSE
response means there is an issue with the checked message. In violate
modality, the opposite is true.{question}
, {context}
and {answer}
extractions are supported.
Extractions in the evaluation instructions should be used as though they were regular words (unlike advanced mode, in which extractions are replaced by the extracted content at runtime).
TRUE
or FALSE
.{extraction}
tag will be replaced with the actual content extracted from the message that is being checked.temperature
and top_p
for the LLM.
{extraction_descriptor}
, where extraction_descriptor
can be any extraction that is configured for your projects (e.g. {question}
, {answer}
).
If you want the text to contain the string {extraction_descriptor}
without being treated as an extraction, you can escape it as follows: {{extraction_descriptor}}