Skip to main content

Moderating chat

The basic setup​

Create one client when your server starts, and check each message before you send or show it:

import { SafetyClient, isSafe } from '@chidakashi-ai/safety';

const client = new SafetyClient({ apiKey: process.env.SAFETY_API_KEY! });

const result = await client.checkText(text, 'chat');
if (!isSafe(result)) return res.status(422).json({ error: 'message_blocked' });
await deliver(text);

Decide what happens when a check fails​

A check throws if the service can't be reached, times out, hits the project's rate limit, or your organization is out of credit. Decide in advance whether to deliver the message anyway (fail open) or hold it back (fail closed):

try {
  const result = await client.checkText(text, 'chat');
  return isSafe(result);
} catch (err) {
  logger.error(err.message);
  return FAIL_OPEN;   // true or false: your choice
}

The SDK doesn't retry. See Errors.

Keep it fast​

  • Set a short timeout. The default is 120 seconds, which is too long for a chat message:

    const client = new SafetyClient({ ...creds, timeoutMs: 3000 });

  • Leave showReason off unless you need the reason. Checks are faster without it.

Checking an AI model's prompt and reply​

if (!isSafe(await client.checkText(prompt, 'assistant-input'))) return refuse();
const answer = await callModel(prompt);
if (!isSafe(await client.checkText(answer, 'assistant-output'))) return refuse();
return answer;