Mod Assist App
Content moderation is vital for maintaining a safe, inclusive digital workspace. It serves to uphold user safety by filtering harmful content like spam and hate speech, preserving brand reputation as a secure platform, and complying with legal regulations on content removal. Manual moderation is increasingly impractical due to the volume of conversations and the psychological risks to human moderators.
The Mod Assist app automates this process, offering an efficient, scalable solution for content moderation. It enhances digital workspace security by automating key moderation tasks. It detects harmful content based on predefined rules, flags and reports these messages for moderator review, and enriches reports with contextual details like sender information, room location, and timestamps. This streamlines the moderation process and provides comprehensive data for informed decision-making.
Install Mod Assist App
The app will only be available to workspaces running version 6.3.0 and higher.
To install the Mod Assist app,
Go to Administration > Apps > Marketplace.
Search for the Mod Assist app and click on it.
Click Install and accept the needed permissions to install.
Get an OpenAI API Key
The Mod Assist app requires API keys to authenticate your OpenAI account on Rocket.Chat and make requests.
Your OpenAI account must have active billing with credits. For a more detailed guide on billing, please refer to OpenAI's prepaid billing.
Log into your OpenAI account and click on your profile icon at the top-right corner of the screen.
From the profile dropdown, select View API Keys.
If you don't have any API keys, click + Create new secret key button to create a new API key.
Copy the API key to your clipboard. You'll need it for configuring the app in your workspace.
Configuring the Mod Assist App
To configure the Mod Assist App after installing it,
On the Mod Assist App Info screen, navigate to Settings.
Update the required fields:
AI Moderated Rooms: Select the rooms where the moderation app should automatically screen for inappropriate content.
Exclude Roles: Select and whitelist trusted roles whose messages can bypass moderation. It does not support custom roles.
Moderate accounts created in the past (number in days): This setting enables the app to moderate messages from accounts created within a specific number of days. For example, if you set a 30-day period, the app will moderate the message if the sender's account was created within the last 30 days. Messages from accounts created prior to this timeframe will bypass moderation.
OpenAI API Key: Paste the Open AI API key you copied earlier.
Moderation Categories: Select the types of inappropriate content you want the app to moderate. Natural language processing techniques are used to detect all types of inappropriate content such as profanity, toxicity, etc.
Toxicity: Any message that is harsh, disrespectful, negative, malicious, harmful, or hateful is toxic in nature.
Profanity: Any message where the language is bad, abusive, foul, vulgar, or obscene is deemed as profanity type.
Financial Risk: Scam messages sent by fraudsters and imposters tricking users into divulging sensitive personal information or financial details that can lead to identity theft, financial loss, or unauthorized access.
Flirtation: Messages that are indecent, intimate, unwanted, and inappropriate can be deemed as a flirtatious content type.
Click Save Changes.
The API key used when configuring the app is applied to the entire workspace. Users in your workspace don't need to get their own API keys.
Now that you've successfully installed and configured your Mod Assist app, proceed to explore Using the Mod Assist App.
Last updated