[Novalug] [OT] 'Comment moderators' -- Mailing list or web-based?

[Gregory Maxwell]

On Tue, Jul 7, 2009 at 11:05 AM, John Warren<jpwarren00 at gmail.com> wrote:
> I'm fully aware of Slash and CAPTCHA's.  Like I said our software handles
> pre-filtering (CAPTCHA and, text analysis).  This is an estimated half
> million comments a day throughput after filtering for bots and easily
> screened vulgarity.
>
> Community filtering is not an option, as the community specifically is the
> problem.  This site has to absolutely remain as neutral as possible (not a
> problem at Slashdot).
>
> Think of this as an extremely public site, for a massive group of people,
> that diservedly and undiservedly draws trememndous ire and distrust both
> outside and inside of the operation of the site.  Peak traffic is measured
> in tens of thousands of hits per second, and user registration for
> commenting is not currently an option on the table.
>
> It's a tough problem, hence I was wondering if anyone had any experience
> with services that claim to help in this area.  It's possible we are
> breaking new ground and nothing out there right now that can help.
>
> How would you deal with it?

Smarter computers! http://stupidfilter.org/main/

;)

Automatic shaming http://xkcd.com/481/


More seriously, the sort of moderation system mentioned up-thread is a
pretty piss poor solution. Yes, it eliminates a lot of garbage, but so
does virtually any other approach. The slashdot solution ends up just
being technology mediated group-think.   Go read slashdot at -1 and
amongst all the crap that any other approach will eliminate you'll
find a multitude of level headed views that weren't modded up because
they went unseen as they were provided by outsiders. (It's also not
uncommon to find the subject of an article commenting and not get
moderated up).


Really the presented problem sounds like this:
(1) people are posting rubbish
(2) machine filtering is doing all that machine filtering can do
(3) the scale and politics of the matter inhibit the possibility of
using a small set of trained people to filter it is probably just not
viable.

To me it sounds like you're saying "The river is washing over my
fields and destroying my crops, but I simply can't muster the energy
to boil away the river,  can I hire someone to boil this river for
me?".

I think the solution to this problem will come not from trying to
filter the bad comments, but instead trying to create a culture which
minimizes the bad comments. Once that is achieved you can simply let
the good vastly outnumber the bad.

For example, make sure the site is structure in such a way that people
are exposed to many good examples before getting an opportunity to
comment on their own.  Make sure that it's clear what the purposes of
the site is, and what kind of input will be productive and what won't.

Any way that you can encourage people to put a little more time and
effort into their comments should pay off with increased comment
quality.

Perhaps you've reached the limits of automated /filtering/ but
filtering isn't the only thing that machine classification can be used
for... What would be the effect of using something like stupidfilter
to trigger occasional automated replies from fake people like "Your
argument isn't especially clear to me, can you break it down for me?"
:)


One example to look at is the English Wikipedia.  It's operating at
roughly the the scale you're describing. (Mean traffic levels of 65k
hits/sec; hundreds of thousands of edits per day; no registration
required).  Wikipedia has its share of problems, accuracy, etc.  but
none of them can really be described as comment moderation problems.
This is accomplished with very little resembling actual moderation.
Obviously the primary articles benefit from the 'wiki'-nature: That
users can remove and refine the article text but even the discussion
spaces where modifying other peoples comments is strongly discouraged,
 people overwhelmingly conduct themselves in a professional and
clearly spoken matter on topics where people spend most of their time
hurtling personal attacks at each other:

Some random examples:
http://en.wikipedia.org/wiki/Talk:Washington_Nationals#The_Definitive_answer_to_the_Expos.2FNationals_controversy
http://en.wikipedia.org/wiki/Talk:Abortion/Archive_22#New_Study_Needs_to_Be_Cited
http://en.wikipedia.org/wiki/Talk:George_W._Bush#Calling_Waterboarding_Torture_is_factual


Cheers—