On today's episode of the Modcast Podcast, we move on from discussing moderating in teams to discuss issues that can stem from moderation; how to address spam. Joining us are our guests Joe and Ayu, and, as always, our hosts Sean and Kairi.
We have, today, the pleasure of being joined by Joe and Ayu. Joe is a computer science student from the UK who has been using Discord for 5 years. He is the owner of the Python Discord, moderates the r/Python Subreddit and is the engineering lead for the Modcast Podcast.
Alongside Joe, we have Ayu with us. Ayu develops and maintains the anti-spam bot Beemo. She is an essential developer and administrator for Riot Games’ Discords and own a community server focused on Riot Games’ games. She is also a member of the Discord Moderator Program.
Spam can have different definitions in different communities and on different platforms. To get us all on the same page, Kairi asked our guests to define spam. Ayu says that spam is unsolicited messaging, specifically, a lot of unsolicited messaging. This can be a lot of unsolicited messaging from one account, or individual unsolicited messages from a large group of accounts working together. Joe goes on to say that there are various types of spam; targeted spam, where community members are trying to annoy you, and non-targeted spam, where bots are used to annoy multiple groups or make money in some way.
Sean asks our guests if they respond to different types of spam in different ways, using the examples of a bot raid or 5 users joining to ask server members to participate in an online poll. Joe’s server has handed out over 1800 infractions with spam being the reason with punishments ranging from warnings all the way up to permanent bans. He goes further to say that with bot raids, other actions may need to be taken, such as shutting down a channel, whereas with user raids, Discords built in purge ban feature may be sufficient. Ayu says that while bot raids have the potential to be more harmful, they are also much easier to stopgate. Since the bots behaviors are automatic, it’s possible to set up automatic countermeasures to prevent them. Those automatic countermeasures can also be set up for user spam, for example to automute if a user is sending messages too quickly. Sean asks if the definition and understanding of spam changes from platform to platform, using Twitch, Discord and Reddit as examples. In larger streams, Twitch chat can move incredibly fast and be largely unreadable, while even 700,000 member Discord servers can and should have a readable chat. On Reddit it's completely different because it's more of a message board than a real time communication platform. The worst thing that Joe has encountered on the r/Python Reddit, which has seven times the amount of users on the Discord, was a user sending the same malicious script in post comments. He says that that is the type of spam that the platform attracts, with multi-user spam being something that likely exists but isn't something he's come across. Single user spam is easily caught by the automated anti-spam tools that the subreddit employs. Ayu’s only experience with moderation outside of Discord has been with Minecraft, which attracts similar types of spam with the exception of not having DM based bot spam. Kairi chimes in to say that in Twitch chat, spam is often allowed and encouraged. Users will spam emojis or copypastas and moderators will join in. On Twitch this is fun behavior that the person broadcasting enjoys, whereas the same behavior on other platforms would be a situation that would require the immediate attention of a moderator.
Twitch is just its own different animal entirely when it comes to spam. - Kairi
Sean responds that on Twitch, particularly in large streams, the chat isn't really there for users to have a conversation with each other. The chat serves more to help get across a general vibe of how people are feeling about what the broadcaster is doing. The behavior people bring into busy Twitch chats wouldn't be permissible anywhere else online because the chat features on other platforms serve different purposes.
Sean poses the question of what motivates users to spam. Some users are personally motivated to disrupt the chat but that only seems to be a small contingent of spammers. Ayu responds that a lot of spam is motivated by profit; particularly in the world of DM bot raids. The people behind these DM bot raids are either paid by someone to offer it as a service, or are paid per click on a link that they are sending out. This type of spam isn’t targeted at your community specifically, it is targeted at whatever servers will be the most profitable to target since the infrastructure required costs money to build and operate. Both Joe and Ayu say that the spam she sees most frequently day to day is retributive and a result of disgruntled users, whether they are users who have been members of the server or people who dislike Python or Riot Games and want to disrupt the communities. Kairi asks what our guests’ experiences with users who are just spammy has been like. These users may not have malicious intentions, but are still disrupting the community by not really contributing to the conversation, asking the same questions repeatedly, and similar behavior. Joe responds that he’s introduced tools like slowmode into his server to dissuade users from splitting messages onto newlines unnecessarily. Joe also mentions that he has unintentionally increased the amount of spam in his community by introducing more requirements to gain permission to join voice channels in his server. Users are required to send a certain number of messages before being able to join, and that has led users to hop from channel to channel with low effort/engagement messages to quickly gain access.
I wonder if people have made GPT-3 spam user bots to get like Mee6 levels or something. -Ayu
Sean builds off of Ayu’s musings to say that leveling bots in general incentivize participation without having very specific metrics on what participation entails. While they drive up the number of messages sent in a community, they don’t necessarily drive up the quality of the conversation. Ayu uses a leveling bot in their servers and has built in or considered some internal tools to prevent abuse and prevent encouraging low quality participation. Some tools that can be used to identify spammy contributions would be to check the word count of the messages, a check to verify that the message content is actual words, and a check to see if the messages are repetitive. Having options to reset a user's XP, or prevent a specific user from gaining any at all can also be useful to more heavily dissuade XP farming behaviors. Sean notes that there used to be bots that had a global XP system, where you could see someone's total XP from all of the servers that they were in that used that bot. This XP has no real world value and isn’t exchangeable for anything, it was a purely vanity metric that was only visible in servers that used those bots. Sean says that some users set up private servers with the bot and ran scripts on their accounts to increase their global stats, some to such a degree that they ended up being flagged by Discords infrastructure team because their messages were spamming Discord. People will go to incredible lengths for vanity metrics.
Automating Spam Detection
Most raids and spam are automated, and thus a great way to catch spam is playing them at their own game; with automation! Any server with concrete moderation behind it will have at least some spam catching automated, so Kairi here asks our guests how they automate their spam moderation.
Over at the Python Discord, Joe shares with us that they use a Bespoke moderation bot, with its own set of filters. Starting off, he explains the most basic filter, the burst filter. This filter is triggered based on how many messages a user sends in a short amount of time. Other things it looks for are duplicate messages, and mentions, alongside another filter, shared burst, which is triggered by a certain amount of messages in a channel as a whole as opposed to a certain user in a period of time.
I think, as I’ve mentioned, not many of these spambots are creative in their content, it’s very easy to pick up. Obviously, if a message has an embed in it, then you know it’s a self bot. If a message has the same content as the previous five messages, you know it’s a spam bot.
Once these filters are triggered, there isn’t an autoban but rather a manual review process by a moderator, alongside a ten minute mute. Of the 2000 times the filter was triggered, only half of those were a ban, with one fifth of those (one tenth of the overall) being temporary bans.
Kairi here brought up an interesting question about when a ban is permanent as opposed to temporary, which Joe tied back to a previous point about raids versus just a low quality user, the latter of which results in a temporary ban.
Ayu, on the other hand, specializes in DM anti-spam as compared to server anti-spam. This focuses on raids that are typically more automated in which accounts join en masse and DM users with server invites or scams. This also works out for some non-DM spams as Ayu tells us.
By the way, it doesn’t have to be DMs, like, most automated raids have the same creation pattern, because, y’know, they’re buying them from some farm to get past the captchas that Discord has set up.
These filters are implemented in their bot called Beemo, which stops most low effort raids, some middle effort, but not high effort, specifically targeted raids, as said by Ayu. Paid raids are typically paid for with the expectation of at least breaking even, so Ayu finds that “if you set the bar high enough, you actually will generally stop raids.” Unlike Joe’s bot, Beemo bans right away as Ayu is confident that the bot will only catch userbots, and bans are IP bans (and bots can share the same IP). Being the main developer for Riot servers, Ayu experiences many raids against Riot’s Discord servers for their games, most of which are stopped by doing the bare minimum as it discourages the raiders, and makes them want to move to other servers without raid prevention. As Sean points out, it’s an “outrunning the bear” game.
You don’t have to build a wall that actually stops them, per se, you just have to build a wall that is high enough that it annoys them enough to go somewhere else.
This applies not just for server raids, but also Discord’s anti-spam. It’s impossible to prevent people from creating new accounts, but it’s possible to make it annoying enough to want them to go somewhere else. These types of people, Sean comments, will typically come back a couple months down the line to see what has changed in “breaking in” with their spam accounts.
Previously, Joe brought up a couple of the things he looks at when classifying spammers, leading Sean to prompt Ayu on the types of things he looks at with Beemo. Manually moderators will normally take a peak at join lists for patterns in join date, creation date, name, and profile pictures among other things when dealing with raids. Just like manual moderators, Beemo will look at these and take them into account, one of the key ones being creation date. Since spammers will most likely buy these accounts, or mass create accounts, creation dates will all be very similar. Joins will also likely be fast as it’s automated and thus the program will want speed, however, this is not always the case. Creation date here is more consistent, but creating an algorithm to compare both and judge them is the best for the most thorough inspection.
Automating Spam Moderation
While this was briefly touched upon earlier, Sean takes our guests into a deeper dive on what they do and why when spam is caught. As Joe already gave a brief answer, Ayu went first, starting off with talking about manual spam and manually actioning it. Here, there isn’t much time constraint, and thus moderators can look at the situation from a number of angles and act upon it as such. On the complete opposite end of the spectrum, there’s automated DM ads. This has much stricter time constraints, as “every second that you leave that user on your server, they could be DMing other users.” Even Discord verification levels can be bypassed as Ayu understands, resulting in their mindset to only ban, as well as the aforementioned IP bans by Discord. There are exceptions, such as when Ayu’s testing a new method, in which case they kick and send a message to the user being kicked to minimize the potential for harm against real users.
Kairi here shares her experience of seeing bots that detect raids and mutes the users, as well as giving moderators options of what they want to do; ban all the users, kick all the users, do nothing.
Ayu finds this flawed for two reasons, the first of which is that giving a role (the muted role) bypasses verification which allows bots to DM users, and the second of which being that these bots are typically easy to catch with an algorithm, to the point that it’s not worth getting a second opinion.
Joe, on the other hand, agrees that for raids instabanning works perfectly but in the general case the Python spam filter gets tripped by many manual users, thus banning those would just lead to many appeals. DM raids, Joe believes, depend on the communities as the Python Discord rarely attracts any DM scams, or at least, they haven’t come across any in their modmail or other reports. A suggested reason by Kairi, and backed up by Joe and Sean, is that these scams aren’t targeted towards communities full of programmers, as they are not likely to fall for it at a high enough rate to be worth it.
In regards to users caught in raid filters, Sean asked our guests how they deal with appeals. Joe, for the Python Discord, has an email for appeals which the administration team has access to. As it is an email, it generally gives higher quality appeals but they are looking into getting a more standardized format, as many users fail to give their user information to the email.
Contrasting this, Ayu does not do appeals. This is, in part, due to the differences in DM raids which Ayu’s servers attract versus the raids Joe’s server attracts. For most in-server spam cases, Ayu doesn’t ban, although there are, of course, exceptions such as racial slur spam. There are also no messages sent to banned bots as that would require the ban to be delayed, resulting in the bot existing on the server for a much higher period of time due to Discord rate limits. Joe shared that he does the same for automated raid bans, as they are typically not users who will actually appeal. Moreover, Sean shared that Discord does the same, where user appeals are processed and possibly accepted, but bot accounts are given heavy doubt.
There are cases where they at least attempt to test the systems, where they will write in from their bot spam account … and they’re like “I’m a human being, what did you do!?” and we’re like, y’know, not clear that you’re a human being, based on a whole bunch of information that we can see, including you sending messages faster than human beings send messages.
The vast majority, Sean says, are throwaway accounts and thus do not appeal, as seen in Discord’s transparency report.
When Automation Fails
No one is perfect. Humans aren’t, and of course, bots aren’t either. Because of this, sometimes bots will not detect the spam, or the raids. Kairi here asks our guests both about situations where their bots fail, and what they do to deal with it manually.
Joe started out first, sharing that their bot is open source, and thus really targeted raids will read the anti-spam code and code their bots around it. In response, and with the Python Discord being a guild full of programmers, the staff team will have to write an anti-spam module against the raid on the fly, but it does annoy Joe a fair bit.
Over on Ayu’s side, what brings them the most pain is slow raids. The slower raids are, the more likely you are to accidentally ban a legit user, as users have a higher chance of having a similar creation or join time. Another thing Beemo cannot stop is an example Sean gave of a spammer drip feeding bots, one a day for a month. For the former, Beemo is coded such that the longer it takes someone to join the server, the lower the creation date difference is allowed to be before they’re detected and banned by Beemo. At that point, Kairi suggested that dealing with them is now more down to in-server activity or reports from users, which Ayu confirmed. These raids, however, are high cost and thus are not frequent.
When manually handling spam, the most common thing Ayu sees is chaining the same emote together and just general shitpostiness throughout the chat, by different users, where a moderator just needs to come and tell people to stop or face a mute. As pointed out by Kairi, this is another platform difference as Twitch chat typically encourages this behavior, whereas Discord servers discourage it. Of course, the amount of people in Twitch chats versus Discord servers, and the more live interaction encourages this but it is not the case for all streamers, as some streamers like our previous guest Devin Nash will have long messages in the chat for viewer interaction, which are sometimes banned under the pretense of copypasta in other Twitch chats.
Our final mini-topic before turning to Q&A is a question from Sean posed to both our guests, asking how often are their spam rules tweaked. For the Python Discord, Joe observes the filters every time a raid hits the chat as that means the filter has lacked something, which can then be written into the anti-raid module. With Beemo, Ayu will “very proactively” change the filter if she's informed of a raid slipping by, as she believes that there is no perfect filter. In the previously mentioned case of bots slowly seeping in, the effort to defeat that can be more effort than it’s worth. Over at Discord’s side, Sean shared that even the ToS team has experienced sleeper bots of about 50,000 bots created from December 2018, meaning they’ve been sleeping for about two years. After the second or third time they saw them, however, they managed to get them all and nuke them all. Joe also mentioned, with regards to filter changes, that when the demographic changes, such as being featured on Discovery, new types of bots are brought in (for example, by people streaming raiding Discord servers).
As always, if you've got a question suggestion for guests, or a topic suggestion, hop over to our Discord and send it our way in #quest-q-n-a, or #topical!
webtax#9393: Should spam / shitposts have a place in a community. Is a shitpost / spam channel a good idea?
These can take form in different ways, like counting channels as Kairi brings up, all of which in the end are spam. Ayu answered first, adding that if it’s bringing no harm and purely enhancing the community, without adding channel clutter, why not. Sean combatted this with the idea of meme channels and edgy humor channels, which some people enjoy, also lower the quality bar of the server. These channels, Ayu mentions, are quite different from counting channels as they are much harder to moderate.
Joe jumped in after a question from Kairi regarding general channels for spamming anything. Shitpost and memes channels are commonly requested in the Python Discord, all of which were shut down.
In general, it does harm the reputation of the server, and you kind of get leaks. We recently hosted a Python 2020 Core Developers Sprint, and I think it would harm our reputation if we had these 30 Python Core Developers come to the server and be greeted with a meme channel where people are spouting all sorts of gibberish.
Kairi brings up however, that for different servers it can fit in, such as servers for Twitch streamers where spam chats would be more accepted. Jumping back in, Joe also drops that shitposting channels can harm user onboarding. In the end, it was generally agreed that it depends on the type of server- “what is your server truly about”, as Sean says.
binchlord#0998 If you allow certain types of spam (ie. announcement reaction spam, lots of people saying happy birthday, excited overuse of a new channel or new server feature [maybe stickers for example]) where should you draw the line and how do you explain to members why some things are allowed and some aren't?
Ayu slides in to say that she is not a fan of allowing reaction spam and would explain that allowing custom reactions will inevitably lead to spam but also open the doors for other types of abuse and the ability to add offensive or otherwise inappropriate reactions to a very prominent message. Joe starts off by responding about Happy Birthday messages; in a server like Python that has on topic and off topic channels, as long as the happy birthday spam is in the off topic channels, it doesn’t negatively impact the server. It doesn’t really matter if the off topic channel is out of commission for an hour because it isn’t a core part of the server. He goes on to say however, that when stickers are launched to everyone, they will likely require a new anti-spam rule on his bot to rate limit their use. Sean says from a platform perspective, that when a feature like stickers comes out, people will use the feature significantly more at release than they will normally just because it’s new and shiny and exciting. It can be hard to balance from a moderation perspective because it can be disruptive but in a few weeks the use will level off. Adding his two cents, Sean says that things like the examples binchlord listed have an element of positive engagement and are typically celebratory in some way. These actions may meet the dictionary definition of spam, but aren’t unpleasant or unsolicited and that’s a good way to differentiate.
When I’m raided by bots I usually clear out all of the messages they spam or ban them before they have the chance. Do you report these raids to the platform and if so how do you go about it?
Kairi notes that on Twitch, when messages are deleted, they are still visible to the moderators and broadcaster, making them easy to report. On Discord however, once you delete a message, it is gone. Joe mentions that as you delete messages on Discord, there is a toggle asking you if you would also like to report it to the Trust and Safety team. He also notes that Discord has tools to check if accounts seem to have been made for spam by checking creation dates, join dates, etc. so the account ID’s involved in the raid can still be reported. He doesn’t report raids from a handful of disgruntled users, but will report large bot raids to the platform. Ayu mentions that often when they report DM bot raids to Discord, they will get a response that Discord has already flagged the group of accounts, so they do not report everything they come across. They also mention that they think it is important to delete spammed messages even if the chat has moved on and the accounts have been banned because users may see the content and think that it was or is permissible. Joe does the same and uses bot features from his moderation bot to clear out any spam very quickly.
As always, we hope you enjoyed this episode and it gave you insight on how to combat spam in your communities. Thank you to Ayu and Joe for coming on as guests, and Sean and Kairi for hosting as always.
You can find us over at Twitter and Discord to submit a question or interact with the team and other listeners and moderator, and you can listen to the full episode on Spotify, Anchor, Google Podcasts, Apple Podcasts, YouTube, Twitch, or Pocket Casts.
Sean Li, Host (@seansli)
Kairi, Host (@kairixio)
Panley, Director (@panley01)
Mike, Producer (@michaelrswenson)
Brandon, Audio Editor (@_MetalChain)
Angel, Producer (@AngelRosePetals)
Joe Banks, Engineering lead (@JoeBanksDev)
Dan Humphreys, Social lead (@dan_pixelflow)
Delite, Social media manager (@MaybeDelited)
Drew, Social media manager (@SoTotallyDrew1)
Ahmed, Content writer (@DropheartYT)
Chad, Content writer