Request for Feedback: Spam, possible solutions, and a bit about you

Created by will on Jan. 15, 2015, 12:02 p.m.
  • Sticky
  • When the Tested admin/moderator teams take the weekend off and the forums get 6+ pages of spam, it's really noticeable.

  • Spam in the thread about spam. It has become self aware.

  • @Falcon: Well done.

    And, at this point, Web Czars, why not just try SOMETHING. Because doing nothing has predictably resulted in not much.

  • I guess they aren't going to try anything? It's time to consider this forum dead and move on.

  • @will: Maybe code an auto delete system where a post that receives 20 user thumbs down it gets automatically deleted. The account of the poster gets automatically locked. This will leave the administration to discretion of the users democracy. Even if it isn't spam, if a post gets 20 thumbs down from users it should be deleted for the sake of maintaining content quality.

  • How about just blocking free users from posting links until account has been verified by a mod. This is what happens on almost all other sites.

  • Is there any way that you could set it up so that a user can only post one post per hour or something? Or a five post limit for noew registrations. Along with only one registration from any IP address. That should discourage a lot of it.

  • @jccarlton: It doesn't. We have the limits and they either wait, or have figured out a way around them. The IP bans are useless as well, as they use proxies on either infected PCs in the states or just use TOR

  • @TheConanRider: Most of the spam we're getting now doesn't even have links. People cray.

  • @will: The forum again is filled. Most do have links. Those who don't, have phone numbers which aren't feasible to isolate automatically.

    Doing ANYTHING will help. Talking about it doesn't further the goals of eliminating the problem. At some point, you have to try something.

    Other forums don't have these issues and have the same, or less hands-on moderation. This is a system issue, not abnormal/extreme spamming attacks.

  • @bottleworks: I literally spend two hours a day cleaning it out. They're constantly making some small changes to help auto-detect, but the big change to automatically remove stuff that's flagged is held up waiting on another team. There is literally nothing else I can do.

    I'm not ok adding moderators with the amount of spam we have right now because the tools we have for mods suck ass too. It's not right to ask people to volunteer their time to do this kind of work.

  • I think there are plenty of people who would be willing to spend a bit of time clearing out the forums, even if the tools do duck ass. Adding a few moderators would spread the load to many people, and not require a single person to spend two hours a day doing it.

    I know I would be more than willing to help keep the forums clear if it frees your time up to create more content, and do other things for the site. I am sure there are plenty of members who would be willing to do the same.

    If people are willing to do the job, even if it sucks, why not allow them to? If they stop because it sucks, just remove their permissions and no harm done.

  • Wow today was a bad one, 18 pages to the last purge. Wonder if tested is on a list somewhere...

  • A couple ideas based off today's spam. They may not all be unique ideas, but they are what popped into my head. I know most of them will take some programming but seem reasonable:

    Do not allow new accounts to make new topic for X period of time, at least 24 hours maybe a week. I believe this is fairly common, and would not be a barrier to entry. Comments would still be allowed, just no new threads, so new users could still respond to topic and content. Not fool proof, but would raise the bar for spammers and may convince them to go elsewhere.

    Do not allow new accounts (< 1 month or so) to make more than 1 new post for X period of time. Probably minimum an hour, maybe a day. Again not a barrier to entry, new users should have no problem contributing content this way. Again, not fool proof, but likely would raise the bar higher than most spammers would like.

    You could even let it be some sort of backoff rate. One hour min between posts, then another hour and it resets. If you post again within the second hour, its two more hours before posting is allowed, and two more for reset, etc.

    As others have said, moderator approval for first new thread, or even first few threads, would not really make it harder for new users. Just a minor inconvenience of having to wait. Captcha for first month of new threads would also work.

    On the higher end of things, I think there would be some way to look at the actual content of the post / title and if someone new has posted similar titles within some time period they will get marked as spam. This would require some NLP probably, but most of the spam has 2-3 of the same words in all of the titles. Once a pattern is seen, then the user could be marked as spam, and remove the posts, or something along those lines.

    Don't have to be perfect with any of the methods above, just want to increase the effort level for the spammers and convince them to go elsewhere.

  • I'm not trying to place the blame on anyone, but it seems like a site where Adam and Jamie are part of the logo and post tons of content would have the resources to make sure that spam wasn't an issue.

    I would love to go into the forums and see discussions about recent videos and builds and maybe users could post what they've been making. But every time I go in there it's just flooded with spam. Giantbomb doesn't have this problem so maybe you guys could look into what they're doing.

    I love the site and all the content that is posted but the forum experience is not working.

  • I think limiting new users ability to post large quantities until they have gone through a sort of approval is something you should look into.

    I have no idea how many new users Tested gets per day, so i dont know how viable something like that is.

    but i definitely think that everyone first posts should have to be waived by a moderator before they go public.

    Some people register to make a new thread, limiting new users ability to create threads is a bummer.

    I think a similar system of moderators having to approve the thread before it goes live is the way to go.

    Doesn't really matter if it takes 3 days to get your post approved, honestly.

  • @will

    I once fought something like this with a relatively simple solution, which is pretty much a manual CAPTCHA

    "In Principle" is much easier to approve than to ban

    1. Lock out the forums to white listed people.

    2. Create a section +Thread for requesting entry into the forums. Moderator can approve those who follow the rules for application on that thread. (Like a New member application thread)

    Example Rules: To formally request your entry into "Forum" please state your intent/favorite topic and copy the Code: AWDK36 into your post. (or pose a question: IE. What is Norms last name?)

    Note: State of intent is only a means to check for humanity, and copying the code will help filter out automated replyspam unless the botters figure it out. (to which you just change the question)

    Select volunteer mods can reduce the workload to make sure it doesn't get backdoored.

    Robots are meant to attack a specific countermeasure that exists in another forum and/or in popular use. This method has more human involvement, but the amount of legitimate users coming in is much smaller than the bots that swarm in. So its easier to approve vs banning spammers which number in the thousands.

    This will avoid a paywall, but add a little time to get in the forum which I'm sure people will be patient for.

  • @Foe: The spam is being generated by people and not automated services, at least so we're being told.

  • Are there really people that sit around and post a bunch of spam into random forums? And they get paid to do this?!?

    I think we just need to have some moderators to approve a users first few posts and then they're good to post after that. Heck, make me a mod and I'll start doing it right now. It'll be something nice to do while I drink my morning coffee.

  • @Nscafe:

    That's interesting to know, but you know you can't just simply leave it alone without a few more mods. Fans are more willing to volunteer their time to clean up a mess that you yourselves may not have time for. To defeat the motivation of the spammers, human or not, you need to make it not worth their time by utilizing help from the community that can clean the forums with a very quick response time while you can concentrate with back-end solutions rather than waste hours cleaning the forums.

    Get a volunteer mod for every timezone is my recommendation, and get as many temp mods (with limited powers) that you can in proportion to the amount of spammers.

    From experience, The system that I used (as I posted above) was last resort, and was completely effective and discouraged bad behavior because it was easy to spot weird usernames or stuff was copy pasted or no effort was made, and It took 24 hours each time for them to at least attempt again after getting banned. Even If they did sneak a few users or two you can simply ban them when they do show up, and it would be too many hoops for them to continue this on a daily basis.

    Although using this method would force spammers to begin PrivateMsg spamming other users, so be sure to shut that off for new users until they are approved.

  • @Foe: The forum solution Tested uses apparently doesn't allow for grades of mod-tools, or effective anti-spam measurements at all, and they don't want to make two dozen people fully tooled-up mods as it is more trouble than the forums are worth at the moment.

    Its one of those situations where the forum doesn't have much real activity due to spam, and cleaning up the spam isnt enough of a priority because there isn't enough forum activity.

  • @kim_a: So what you're saying is, we're the Windows Phone of forums?

  • Or we can temporarily migrate to a separate forum platform until can finalize a set of backend countermeasures, and to hookup spambot databases which identifies blacklisted IP adresses from multiple honeypot databases.

    I think phpBB forums automatically hookup to databases like these.

    has a bit to counter human spam

  • Well, after a 40 page spam thing on the weekend, here is my new thought - if it matters (my old post got deleted when I got accidentally flagged and it (they) never got restored.

    Put limits on non-premium members. I don't want to be an elitist, but it seems like the only way to clean things up.

    Maybe non-premium (NP) new threads to 1-2 a day. Limit NP posts to 4-6 an hour. Limit user registration to 2-3 users per email and 6-9 per IP. Use Captcha (yes, I know they are humans, but anything that makes it take more time for a spammer makes them want to move on). For new user registration, put a 10 minute delay on the email.

    Also, see if the forum/cms can red flag anything outside of ASCII +ASCII extended for moderator approval. The availability of filters to red flag certain "words" (and number combinations) for moderator approval. Also, red flag new threads from NP that have URL's (but leave regular posts open)

    I doubt heuristic filters are available, but if so consider Capital ratios, and punctuation ratios. As well, IP bursts (Google and JSTOR use this on us at work, and it is - far as I can tell - pretty effective at reducing certain bad behaviors).

    Lastly, give (very) well trusted premium members some (very) limited moderating capability.

    It's too bad that what appears to be a very limited set of spammers has decided to take up residence, but it's really really bad on weekends now.

    At the end of the day, you can only manage spam - not eliminate it. Making it take time and effort for a person who wants to post gobs of things (spam) while making it easy for a legit user to still post - is the balance. The whole point of being a spammer is that it is supposed to be "easy" to make money because it is easy to post. If it is hard to post, they will move on.

    Just IMO.

  • If there is a filter i notice that 50% of spammy threads have 91 in the title

  • Is it possible to add a block and hide feature so we could block the spambots and hide the posts? That would make the forum usable until a mod could remove them.
  • I have to believe that somebody hates this site for some reason. I think that for the time being, as many have said that you will have to put up a sticky that says that the forums are for premium subscribers only because of the spam issues. and create posts that mock the spammers.

  • As a poor non premium member, I would hate to see the forms go premium only.

  • For a temporary solution I wonder if they could turn off the creation of new threads, but leave the ability to post to existing threads. Seeing as the spam seems to be limited to new threads it might eliminate the clutter.

  • Make them premium, even for a temporary time frame. I was all for an open forum, its better for the sites health in large, but this is freaking ridiculous. We are looking at a daily 30+ pages of spam,

    Just lock this shit down until the higher ups get off their butts and provide the sire with what it needs to control this problem.It is super depressing to come to the site to post a thread and you just say screw it because it will be buried in 40 pages of Spam....

    Help us Tested. .... we are drowning in black magic adds.

    Stephen Westervelt