Years ago, i suggested a algorithmic approach to moderation to a Open Source project i contribute too. They ultimately went another way (classic moderation), but the idea was pretty neat.
You basically create a feudal vouching system, were highly engaged community members, vouch for others, who again vouch for others.
If people in this "Dharma-tree" accumulate problematic behaviour points, the structure at first, bubbles better behaved members to the top. If the bad behaviour continues, by single members or sub-groups, the higher echelon of members will cut that branch loose, or loose their own social standing.
Reapplying members, would have to apply at the lowest ranks (distributing the work) and those would risk dharma loss if they vouched for sb unworthy in the trial out phase.
We never solved the bad tree problem though. What if there exists a whole tree of bad apples vouching for another? You can not rely on the other feudal "houses" indicating this correctly, due to ingame rivalry.
I was part of solving a content moderation problem for a tech company forum once.
The most troublesome users were often the most prolific posters. The people who had a lot of free time on their hands to post every single day were often the most disgruntled, seizing every issue as a chance to stir up more controversy.
It was tough enough to reel in difficult users when they had no power. Giving extra power to people who posted the most would have only made the problem worse, not better.
The most valuable content came from users who didn't post that often, but posted valuable and well-received content occasionally. I'm not sure they would have much interest in moderating content, though, because they would rather produce content than deal with troublesome content from other people.
Content moderation is a pain. The trolls always have infinitely more free time than you do.
I used to moderate a few message boards, and I fully agree.
I think empowering the "power users" like this inevitably leads to Stack Overflow-style communities, where arrogant responses are the norm, the taste of a few regulates the many, and the culture of the community ossifies because new contributors are not welcomed.
The factor missing with a SO style "meritocracy" is that it needs to be hard to rise up, but very easy to fall down from misconduct: in other words, a built in accountability mechanism to ensure they don't abuse their power and become arrogant over it. Not necessarily ban worthy offenses (lest, they simply be banned), but ones that make a less welcome community.
I'm lost on how to do this within an internet community, however. You don't want burners being made to report over a disagreement in opinion and impact them. Or worse, power users tearing each other down in retaliation. I don't think "election periods" would work well either because few people really care to be a moderator to begin with. I can only see it being done by having professional moderators and having a manager review any potential user complaints. Which I'm sure has other problems.
The fact that there's very little overlap between those with the inclination to contribute and those with value to contribute, is fundamental to virtually all communication channels / fora. Unless the channel is exceedingly aggressively managed, the former will outweigh the latter.
An early form of this appeared as an observation on Hal Varian's page at UC Berkeley (he's an economist, was with the School of Information Management Science, and now works for Google).
On HN it's interesting to look at what kind of riff-raff appear high on the leaderboard, vs. revered names in the field who may have posted only a few times.
Neither truth nor quality are popularity (or volume) contests.
You can also test this, similarly to how Stackoverflow does it: send people a post that you know is bad (or good) and check that they flag it. If they don't, let them know that they are doing it wrong, lock them out of moderation, or silently ignore their voting and use it as a signal of voting rings.
yes, let computers do the repeatable work and humans do the original thinking.
i still haven't seen a moderation system better than slashdot, which community-sourced its moderation/meta-moderation semi-randomly. though it still had issues with gaming and spam, it seems like a good base to build from. and yet we ended up with twitter, facebook, reddit, yelp, etc. that optimize for (ad) views, not quality.
slashdot had good bones, but what it was missing was a pool of known good moderators. Without them, you end up with a death spiral as bad actors upvote and meta moderate each other and drive good mods away.
Yes, I'm convinced at some point we're going to figure out an algorithm to solve content moderation with some version of crowdsourcing like this based on reputation, though I'd prefer a system based on building up trustworthiness through one's actions (consistently flagging similarly to already-trustworthy people).
But the challenge is still the same one you describe -- what do you do with competing "groups" or subcommunities that flag radically different things. What do you do when both supporters of each side of a civil war in a country consider the other side's social media posts to be misinformation and flags them? Or even just in a polarized political climate?
I still think (hope) there would have to be some kind of behavioral signal that could be used to handle this -- such as identifying users who are "broadly" trustworthy across a range of topics/contexts and rely primarily on their judgments, while identifying "rings" or communities that are internally consistent but not broadly representative, and so discount that "false trustworthiness".
But that means a quite sophisticated algorithm able to identify these rings/clusters and the probability that a given piece of content belongs to one, and I'm not aware of any algorithm anyone's come up with for that yet. (There are sites like HN which successfully detect small voting rings, but that's a far simpler task.)
In the Freenet Project we’re solving content moderation with propagating trust — and visibility slowly increasing with social interaction, which is free for honest users (who just want to communicate) but has a cost for spammers and disrupters.
The main tool is to drop the idea of a global definition of trustworthiness: You have seed IDs to find people who supply CAPTCHAs, but otherwise all trust is computed locally.
I wonder if you could try to address this by limiting who can flag a given post.
Even just doing it very naively and choosing, say, a fifth of your users for each post and only giving them the option to flag it might help significantly. It would probably make it more difficult to motivate the members of these problematic groups to actually coordinate if the average expected result was the inability to participate.
And you could do it in more sophisticated ways too, and form flag-capable subsets of users for each post based on estimates about their similarity, as well as any other metrics you come up with - such as selecting more "trustworthy" users more often. This would help gather a range of dissimilar opinions. If lots of dissimilar users are flagging some content, that seems like it should be a strong signal.
Presumably instead of global moderation you could have pluggable meta moderation where you pick the moderators so you can have fun stuff like Moderator A whom you follow banned Bob therefore you can't see his posts or his comments on your posts but I don't follow A I follow B who believes Bob is a paragon of righteousness and so I see Bobs words everywhere and we all in effect have an even more fragmented view of the world than we have today with conversations that are widely divergent even within the same social media group or thread.
I think we need to critically evaluate what we call echo chambers. The continent, country, state, city, street you live in all exhibit patterns of echo chambers. In a sense, our planet itself is an echo chamber. Every human network is an echo chamber that boosts signals to varying degrees. A lot of times, this is a good thing! Like when people come together to help each other. The real problem is when the network itself is designed to boost certain signals (e.g. outrage, controversy) over others to a point where our society breaks down. Many of today's centralized networks profit greatly from misinformation, anger, and other negative signals. IMO that is the problem we need to tackle.
Which has a single front page which shows the same headlines to everyone where people who disagree can all see each others posts and we can disagree with each other so long as we can avoid being jerks to one another.
At worst you lose imaginary internet points if you say something that the group doesn't agree with.
It is something of a collective filter bubble, and there is a pervasive criticism from those who largely don't participate here of typically HN behaviours. I agree with a fair bit of that. There are certainly topics HN doesn't seem to be able to reasonably discuss. (This seems to be a frustration of the mods as well. They're certainly aware of the issue.)
My own (in my view at least) largely contrarian voice seems to have been reasonably well tolerated here, though.
This is easy to solve if you don't have a "public timeline", e.g. if you only see posts that have been vouched by people you follow. Like using Twitter but without topics, hashtags, and search: the only content you see has been either authored by someone you directly follow, or retweeted by someone you directly follow.
If you keep seeing content you like (through retweets), you can follow that person directly to get more. If you see content you dislike, you can unfollow the person who brought it into your timeline (by retweeting it).
Of course this would work a bit better if there was a way for accounts to categorize posts they author or retweet. You might follow me for tech-related content but not care much about my French politics content, which I would be happy to categorize as I post/retweet but have no way to do on current Twitter.
I think lousy annoying manual moderation in smaller communities is hard to beat. Human beings have flaws but we have hundreds of thousands of years of adaptation to making small social circles work that might not work AS well in groups of hundreds or low thousands but they can be made to work acceptably.
When you say highly engaged community members I hear people who have no life who derive self importance via imaginary internet points and social position not by doing things but by running their mouths. While it claims to encourage community it discourages it by potentially punishing association. It would make people afraid of being associated with ideas others consider bad which sounds great if you imagine communities run by people that are largely or mostly good and intelligent when in fact people are largely bad selfish and stupid.
It would be ruthlessly gamed by individuals whose status would be based on their efforts to stir up drama that sounds fantastic when its directed at people like Epstein or Harvey Weinstein less so when you realize that this would be effective at silencing people regardless of guilt because people would need to as you say cut the branch loose.
I have literally never heard a worse system of meta moderation proposed.
Every system that relies on crowdsourcing and/or reputation to solve such problems is doomed to fail. Remember when manual curation and recommendation of products/places/content was supposed to be fully replaced by online ratings & reviews?
> was supposed to be fully replaced by online ratings & reviews?
I mean, it has though.
When I want to buy something new, I find Amazon reviews to be far more helpful than anything else that has ever existed. Obviously you can't only look at ratings or only read the first review, but it's pretty easy to find the signal amid the noise.
Similarly, TripAdvisor has given me far better recommendations of sights to see while traveling when compared to Lonely Planet. Yelp is eons better for finding great restaurants than Zagat ever was. And so on.
I don't understand how you think these systems are "doomed to fail" when they already exist, are used by hundreds of millions of people, and are better than what they replaced?
Rather than use high engagement as a basis for vouching, arbitrarily selected communities, perhaps of about 50--150 active participants (posting or moderating) might be better. Think Mastodon, but in order to be federated, good posts must be "voted off the island", in the good sense.
I've been applying a somewhat similar notion to collecting and managing reading material and suggestions, what I call BOTI, or Best of the Interval. It's a round-robin style system (think "43 Folders" from GTD), where I compile a list of references over a period of time (monthly to annually seems to be most appropriate for me, though hour / day / week / month / year / decade / ... could be applied). At the end of an interval, some limited number of items is carried forward.
This is one way of addressing the "firehose of content" nature of information, recognising that in any given time period, you only have so much personal bandwidth to dedicate.
With the federated model, content federating is itself subject to assessment, and is effectively re-vetted. Note that different communities might favour different content: cooking, kittens, Kabbalah, canoodling, cypherpunk, core meltdowns, classic cars, concerthalls. Vetted / re-vetted streams themselves might be of interest.
Both community- and time-based elements of this could get interesting.
Just as there is no stopping “crime” there is no stopping bad content.
Besides - these are evolutionary games being played between grazers (content consumers) and predators (alarmingly large group).
As long as there is signal in a forum, there will come to be a method to subvert it.
Honestly the question I would ask people is how do you measure bad behavior on a forum.
Any technical idea, such as your tree, is doomed to eventual obsolescence. The question is how long it would take, and how effective it would be, and how you would measure it.
Also, if there is a severe penalty for vouching for bad people, but not much gain for vouching for someone, will you end up with no one wanting to vouch for anyone else?
Bad behaviour of this nature tends strongly to operate at scale.
Bad actors form massive communities, and quickly. You can look at the rise of various networks, such as T_D on Reddit, or the rapid growth (and often fast decline) of other such forums elsewhere. I find it very difficult to believe that such growth is organic.
(Commercial and brand accounts / pages / platforms often follow similar trends, and for similar reasons. Advertising and propaganda are one and the same.)
Another reason to vouch for someone is that you trust their judgement and want them to have more power in the system to hide content that you personally don't want to see.
It's true that this will lead to echo chambers, but by looking at vouching relationships rather than the contents of posts, it should be easier to detect the echo chambers and give people the opportunity to expand their horizons.
In a system that tends to reward closing your horizons with a sense of safety and belonging the trend wont be towards expanding horizons. Don't build systems that don't work like you want them to in practice because in theory people could use them better.
Our existing systems already generate echo chambers (due to the emergent properties of algorithms and/or the choices people make when they consume media), and that's in the best case where your horizons aren't just set by the political leanings of the employees of the companies that run the systems that provide most of the content.
As long as people could choose different communities that had moderation policies which suited their members, it would allow those who wanted to hear other perspectives to do so, and might make people aware of what sort of echo chamber they are in.
I like this, a lot. Well, I don't like that it sounds like it will prevent outsiders from participating, those who have no-one who will vouch for them, and it does sound like it would encourage a mono-culture of thought. But I like the idea of socializing the costs of bad behavior. Indeed, those socialized costs would extend to the real world. I'm intrigued and perturbed at same time.
The lazy solution to rivalry getting out of control is bicameralism. Make tree-based governance where most of the action is, but design another chamber that can veto it without the same rivalries involved.
This is much like how the private torrenting trackers are doing, but no very number of points. So maybe is existing some precedence for some system in this like.
You basically create a feudal vouching system, were highly engaged community members, vouch for others, who again vouch for others. If people in this "Dharma-tree" accumulate problematic behaviour points, the structure at first, bubbles better behaved members to the top. If the bad behaviour continues, by single members or sub-groups, the higher echelon of members will cut that branch loose, or loose their own social standing.
Reapplying members, would have to apply at the lowest ranks (distributing the work) and those would risk dharma loss if they vouched for sb unworthy in the trial out phase.
We never solved the bad tree problem though. What if there exists a whole tree of bad apples vouching for another? You can not rely on the other feudal "houses" indicating this correctly, due to ingame rivalry.