Will and I are discussing, hopefully we'll find a solution that works.
In the mean time, I noticed the first two pieces of spam today went undetected (third was one properly classified). I've modified the script to report as an older version of internet explorer (apparently browser version is a good indication of if you're a spammer) and changed the posting IP address to a United States proxy. I guess that this is really the only way I can tune performance. I've written an email to askimet asking about neutral user-agents and ipaddresses, we'll see what they say.
@MAGZine Is there any way to keep the flag system for abusive behavior, or will that render your idea useless?
I realize that you all need to be sure before you implement anything, to prevent account deletion that isn't warranted. Can you or the powers that be implement something in the meantime to help slow the bleeding? A measure that's not perfect, but a way you can tweak it to see what would be the best from experience?
I can do post-deleting only. That would probably be the half measure. That, or we manually review the logs to make sure no users real users would be deleted and continue manual spam cleanup until (or if) we're sure we want to flip the switch.
I've finished a very early-stage implementation ("that was easy"), and started running it on my virtual machine. We'll see what crops up overnight, I guess, while I wait for more community/Will input.
@andy.fancher: Usually, when we delete spam users, we mark them for deletion (which removes all posts of the user's and prevents further commenting).
Maybe a good trial would be to simply delete the post instead of the user and see how that plays out. Of course, there is a chance that user accounts are deleted.
e: another option would be to drive the detection through the flags system. Rather than having a human sit on the other side of the flags system, we could simply submit all entries for spam detection. However, then that takes away from the alternative uses of the flag system (e.g. abusive behavior)
@MAGZine: It sounds like that will help deal with the spam problem or at least mitigate it somewhat to allow people to come to understand that there is more on the forum than spam (the other day i had to go through 4 pages to find actual content!) but i don't think it addresses the underlying issue of the forums lack of usability and features, the tested forums are like something from 1995, theres no FB or Twitter integration, no inbuilt search, it just seems really clunky.
I really don't want to come across as being down on the whole thing, honestly there is a great community here it just needs the tools to allow it to grow and to express itself, personally I'm not a huge fan of reddits i much prefer forum based communities, or failing that FB Groups.
Im assuming a large part of the problem is the current forum software is integrated into the tested website meaning we use the same accounts for comments on posts and managing subscriptions? I used to run a large XenForo forum (50,000 members, 16 million posts) But we never tried integration and just focused on running the forums as a stand alone entity but i know lots of other sites which allow integration, maybe something to look at if/when you look to give the site a facelift?
In the meantime however i full heartedly agree that anything to limit the amount of spam is worthwhile
AFAIK, the same thing to assume is that there is zero engineering budget for the forums. In that sense, any sort of forum implementation (while possible) is out of the question. I've personally integrated an external corporate identity system with Invision Power Board, but I'm not on the Tested Payroll and don't have access, resources, etc to undertake such an upgrade.
As Will said, it's what we have, or (probably) nothing at all.
@MantaBase: @StephenWestervelt: @Andy3E: @gekitsu: @will: @Nscafe: I'm just picking some people on the last page that were active in this discussion to solicit feelings and musings.
I've been meaning to get this project up and running for the last 6 months but I visited the forums today and was disheartened by the state of it all, so it sort of lit the spark in me once again.
Basically, I'm in the process of writing a scraper that will watch the board index for new posts. When someone makes one, it'll collect the relevant details, submit it to Akismet, and then ban the user on my behalf if the result comes back as positive (for spam).
I wanted to implement this through machine learning, but Akismet has way more data than I do, and has engineers working on it. And, as much as I'd like to work on a pet project, I don't have enough time to dedicate to anything more than essentially gluing APIs together.
There is no confidence ratio on these requests. Either the comment is spam, or it's not, so we risk incurring false positives that I can't easily undo. However, I feel like this is the best ready-made spam solution, and I've nearly implemented it. We could run it on a trial basis, where I just have it write to a file every user that it would ban, and compare after the week is up (though there would be a lot) before turning it on.
I might not be a maker in the traditional sense, but I am a software maker! Thoughts on this solution? I honestly feel this is the best shot that we have at restoring the forums to their former glory.
I will not turn anything on at this time.
EDIT: I, at this time, can also not make any commitments to when we can try this, as my semester is near wrapping up and I might not have enough time to dedicate to finishing this up. Simply collecting sentiment on this endeavour so I know how to proceed.
EDIT 2: If we go this route, we should purchase a monthly commercial Akismet license. It's $5/mo. I'm not sure if they'd ever catch on as to what I'm doing, but in the interest of ethics and moral decisions...
@will: @scotrail: Fedex is just as bad. Everytime I order something from then there's some tiny amount of tax + $10 to cover their "advancement".
Usually you can avoid these fees if you have an account with the company, and input your account number before shipping. Said account is usually tied with a credit card no, which they'll charge instead of "advancing" you the money.
i cleaned up just tonight 1 account with 256+ posts, and one with 128+ posts and another with 50+ posts.
Thar be spam.
@General Desire said:
Maybe it's the South taking revenge for our hate?
Revenge is a plate best served over and over and over again.
p.s. lifeforce t-boost.
wait, guys, does tested have a spam problem?
The meta-spam is real.
Nor did I ever suggest that the South should not be made fun of
Actually, it seems to me that's exactly what you're doing. Read the title of the thread. Also read the last bit of your post. You have a clear call to action: do not make fun of the south. Just saying.
My point is that it would be great if folks made a conscious effort to make fun of the South. My gripe, minor as it is (and I have acknowledged from the beginning that it is minor - not everything has to be 0% or 100%), is that the "hatin'" on the South is not a conscious attempt to poke fun or criticize but an unconscious assumption that the best way to express human ignorance is via the Southerner. Have a dumb thing to say? Say it like a redneck. Want to pretend like you can't understand tech? Say it like a redneck. I will confess that Will gives a general impression of regretting that he was raised in the South, and perhaps that has colored some of what I’ve noticed.
Personally, I usually opt for an old person stereotype if I want to imply that I don't understand how a piece of tech works.Anyhow, I odn't really get that impression from Will, and it's not for use to judge one way or another how he feels. To my knowledge, he sitll has family/goes home for holidays, so it can't be that bad ;-)
Let me put it another way - you make a blonde joke, "ha ha." Now, instead, you create a "dumb women" meter that you use to rank how advanced a tech product is by how hard it would be for "dumb women" to figure out how to use it. (You can substitute "dumb women" with your choice of stereotype). Can you see the difference? Suddenly, it's more inflammatory than funny and it doesn't belong in civil discourse.
Fairly sure you have a category mistake here, as you're confusing sexism for stereotypes. Careful about conflating the two. Also, there is a whole other discussion on tested about this, so lets not open that jar of worms here. I'd also say there's a difference between creating a one-off joke and consistently marginalizing a given denomination. Telling a blonde joke is ok (though some may take issue), but creating an environment that demonizes or marginalizes a certain demographic, at least in my books, is not. And no, I don't feel like that's what Will & co are doing.
I was just making an appeal for civility. I think most folks can recognize the fact that our current society is rotten. Well, it won't start getting better until we actually try to make it better. There was a time when folks wouldn't use foul language in public, not because of "censorship" but because it was not a civil thing to do. People did not WANT to live in a society that was so vulgar it used curse words as common descriptors. Nowadays I can't even listen to the Tested podcasts without headphones because I've got kids running around the house. And what was the end result? The dilution of words, not the betterment of society (is there, really, a difference between “I have a bad headache” and “I’ve got a fucking headache”? Not anymore).
It doesn’t have to be all one way or the other – I’m not asking for eggshell walking or pro-Southerner videos – just a recognition that we truly aren’t the most ignorant bastards that have ever walked the earth (with the exception of West Virginians, naturally).
Yeah... there was a big discussion in the MaximumPC days about language on the podcast, and ultimately this is where we ended up. I doubt that it'll ever return to uncensored. As for the last bit, I can promise you that everyone is fully aware that the South are not the most ignorant bastards that have ever walked the Earth. Promise.
I'm Canadian. I have stereotypes of my own that constantly get used/abused. I laugh about them, because why not.
@General Desire said:
I do think lately more real people are finding the forums as well, which is a good thing. The prostitute-bots come in waves, but generally the forums are clean around 6PM GMT, so someone is doing some nice active clean-up. Can't wait for a decent behind-the systems thing to come in effect though, but yes, they're on it.
yep, that's about when I wake up most days :D
AFAIK, there's a bug where you can't edit a post that you just made. You need to first refresh the page, then you can edit.
In my experience, anyhow.
@OrigamiMarie: I've been trying to clean up the forums once in the evening before I go to bed (10pm PST) and once after I wake up (7AM PST). This is the majority of when the spam falls. Unfortunately, I can't clean spam when I'm sleeping (or I would).
Fighting spam is a solved problem. However, the engineers behind Tested do not have enough time to dedicate to investing in spam protection or a bevy of other features. It sucks, I know.
@mfh647: @Nscafe: the correct answer is editing functionality was removed from new users because spammers.