Context, Context, Context: the problem with language moderation
When it comes to moderation, one of the biggest issues faced is context. Many things can be said in many different ways- some inappropriate or harmful, and others not. I’m sure we have all at one point said to a family member or friend “I’m gonna kill you”- whether its when your little sister steals your favourite dress or your best friend drunk texts your ex for you- but most (hopefully) don’t mean it and simply say it as a joke.. This is just one of the reasons why getting language moderation for text and speech mastered by AI is so difficult, as objectively this phrase has a quite obvious harmful meaning.
The Internet and particularly social media is a great tool for education. It allows both short and lengthy explanations of current events, whilst still catering to different age ranges and intellects. At DragonflAI, we often use our social media, especially Instagram, to not only spread the message about what we do to companies, but to also educate about the issues that we are trying to tackle through our technology. Like us, many organisations, charities and businesses use social media for educational purposes as well as selling and spreading awareness of themselves- however these educational posts will often include keywords that can trigger the platform’s moderation software, reporting it as non appropriate or harmful to the channel’s users. For example, here at DragonflAI we talk a lot about issues relating to child sexual abuse material (CSAM) and other very sensitive subject areas, without using names and descriptions for the types of abuse that we are trying to reduce, it would be very hard for us to get our point across in a coherent and understandable way, therefore we have to use these triggering keywords- but purely for educational purposes.
Recently, with the developments occurring in Afghanistan, many social media users, campaigners and activists have been using their platforms as a way to educate and spread knowledge about this very serious and current world issue. With that, comes the need to use wording which, again, may be triggering to technology used by these multi-media platforms- reporting it as unsuitable and removing it from their app. This results in educational sources, which could make real differences in the level of education and spread of information, being edited or removed to make social media more PG rated. However, sometimes in order to get the point across, like us, users do have to use very to the point and real language otherwise editing may take away the shock factor which unfortunately is sometimes needed in order to make change.
Like we said before, context really is everything. On dating apps for example you may be talking about your sexual orientation using terms which are valid to describe, however others may use those terms in a derogatory or insulting way when talking to someone else. Dating apps shouldn’t ban the words that describe the LGBTQ+ community, as they should be able to express and define themselves however they personally feel comfortable. But this does mean that individuals unfortunately have the capacity to contextualise some of these words in hateful and derogatory manners. While we certainly don’t mean to imply that AI moderation being trained on keywords and phrases is useless (it most certainly is not), this example is simply to highlight that it is not a complete solution yet. Racial or religious abuse is another prime example, and in some cases can be easier to train, but what happens if song lyrics are being quoted, or a speech made decades ago?
Many problems have arisen on social media platforms relating to content that was reported by their systems and removed due to the belief that it was of a harmful or offensive nature. Examples include human rights activists and campaigners’ pieces being taken down as the platform viewed it as terrorist content, along with those who shared their stories of racial abuse during the rise of the Black Lives Matter campaign, simply made to educate and highlight the seriousness of the problems of racism in the world today. These accounts including vile language yes, but they had all been said to individuals in the real world, so what was the difference with them using that same language and accounting of events in order to raise awareness for this issue, and like we have said previously, sometimes to make a strong point and exercise change a level of shock is necessary.
DragonflAI are ultimately trying to stop CSAM, and reduce the amount of human moderation needed for moderating nudity and explicit images, which have proven to have real and very serious mental health issues for moderators viewing this type of content. Disemvoweling is just one tool being used to try and cut down on the amount of trolling comments that are needed to be moderated- this involves software that removes the vowels from examples of hate speech, trolling or bullying and harassment online, making sure that the comment made is pretty much unreadable. A popular blog called the Boing Boing used this technique and said that it enabled the “emotional sting” to be removed from their comments section, however they also did that state that this technique required a lot of human effort and that the lack of one fully automated system in place to deal with this, that they underestimated the amount of human dedication needed. The use of this type of technology is largely related to Teresa Nielsen Hayden who first did this manually in her own blog, however since 2009 this technique has disappeared in the world of moderation, and instead more advanced technology has come into place to flag up these types of issues.
Although at the time tools like disemvoweling were efficient in removing the shock factor of harmful comments, they were not time efficient, therefore paving the way for new modes of technology to come forth to try and tackle the now more prevalent problem than ever of trolling and hate speech online. In terms of the educational content that is flagged on social media sites, we can only hope that our work and companies like us can shape the the future our AI to judge context and allow important issues to be discussed on a global level, allowing for more information and knowledge to be shared and spread in order to evoke real change.
