Request for Feedback: Spam, possible solutions, and a bit about you

Created by will on Jan. 15, 2015, 12:02 p.m.
  • Sticky
  • Woo! Another spam comment in the spam thread! Gold star.

  • Just a small contribution:

    Maybe put in a time limit for non-members?

  • Maybe a penny or whatever covers costs donation level for forum write access? That might be easiest. Also just posting time limits seem easy too.

  • Does anybody have any experience with computer telephone systems? All the ads have phone numbers. Set up a computer with an auto dialer and an automated system waits until somebody picks up and says f you, stop spamming the tested forums, hangs up and does it again until he phone lines are saturated and essentially useless. They spam us we spam them. They're the ones with money to lose from this.

  • Scratch that idea. I checked the country code. 098 is Iran Tested does not want to be making thousands of calls to Iran I imagine the NSA would want to know why. Is there somebody in Iran that hates Tested?

  • @Will

    It seems that you guys have indicated that you are doing something about these forums, or gave up on this entirely. Can we get any updates?

  • Moderators and such @Rallier

    The Off-Topic tab's almost complete first page is spam.

    Update its everywhere now.



  • @will: Personally I'm not a huge fan of the forum software here, it really lacks the usability of something like Xenforo or hell even vBulliten from back in the day, they also have a much much larger community who have developed pretty good search engines and spam filters, just my $0.02

  • i’m with @Andy3E on this. the forum software does not have good usability and is low on features – there isn’t even a forum search – and it obviously isn’t up to even providing basic spam protection without a disproportionate amount of work.

    honestly, i don’t even see what advantages this software here brings to the table. i get the appeal of programming my own stuff, and i occasionally do so when it’s for my own enjoyment, where money and performance requirements aren’t anywhere near the top of the list. but once something has to work under real world conditions and there are already established solutions that see widespread successful deployment, reinventing the wheel instead of taking one from the shelf, and maybe hack it to do exactly what you want, can not pay off.

    for a long term workable solution, i don’t see an alternative to setting up one of the established forum systems. if it is important enough, i am preeeetty sure it is possible to make the site comment account signup process also place the relevant user data entries in the forum user database, creating a parallel forum account. this platform here will ultimately never be not in the way of getting things done with as little friction as possible.

  • @will: :( sad to hear that.

    whenever i head to the forums and it’s not buried in 30+ pages of spam, i keep thinking of how good it could be if the awkward software wasn’t in the way so much. i admit i might be overly sensitive in that regard, though. ;)

  • (lurk mode: off)

    Having been a denizen of forums online since the internet was monochrome, know and heed the following:

    You won't fix the spam problem without a lot of money.

    S/N will continue to get worse as forum population goes up.

    Only two things make forums better in my experience:

    Forum/CMS software is immaterial. If it gets popular it's going to get compromised.

    1) using real, verified names.

    2) a small ($1 would be nice) Paypal entrance fee.

  • This is my first time, viewing the forums(and posting) although I have posted a few times under content comments. Came to the site after listening to the Podcast for a year(Around 2012), never thought about you having a site I would be interested in. Now I have finally decided to visit the forums, to see if anything here is relevant to me...I guess I'll be grandfathered in.

    So its been at least 3 years or 4/5, between listening to the podcast checking out the content and now checking out the forums. I currently participate in so many forums that its hard to fit in another. Your content is very compelling. Also you folks (All of you on the podcasts) have the rare gift of gab. Its my favorite to listen to as I drive over 100K miles a year. If I decide to become a Premo it will be because of your great podcasts! If you told me I could only listen to your podcasts as a premium member...I would pay right up..Don't know if its a helpful post or not.

  • @gekitsu: If we had massive traffic, I'd agree with you. This is a very minimal forum software. For the volume of posts and discussion we get here, more advanced software would be overkill. Of course, it could be a chicken or egg situation, but my general feeling is that most of the conversation that used to happen on messageboards has become centralized and lives on social networks and places like reddit now.

  • @will: you’re probably right about this. as sad as it is – i really miss how common this insular forum camaraderie was.

  • If you are going to kill the forums then you have lost a wee bit

    But if you lock the forums down so no more new members without a e-mail request then you haven't lost as much

    I don't use the forums much but would be sad to see them go as i avoid stuff like bookcase like the plague

  • @will: I think if the forums had more functionality and usability they would see more traffic, and I also think the community here could really benefit from a good forum set-up, something like a cross between the rpf and avforums, there is great content produced here and have a more centralised place to allow comments and discussion to grow and expand would add a lot I think.

    although as with any community it wouldn't happen overnight, but it certainly has the staying power driven by the great podcasts and content.

  • I check the forums two or three times a day even with the spam. I would be truly disappointed if they disappeared.

  • @MantaBase: @StephenWestervelt: @Andy3E: @gekitsu: @will: @Nscafe: I'm just picking some people on the last page that were active in this discussion to solicit feelings and musings.

    I've been meaning to get this project up and running for the last 6 months but I visited the forums today and was disheartened by the state of it all, so it sort of lit the spark in me once again.

    Basically, I'm in the process of writing a scraper that will watch the board index for new posts. When someone makes one, it'll collect the relevant details, submit it to Akismet, and then ban the user on my behalf if the result comes back as positive (for spam).

    I wanted to implement this through machine learning, but Akismet has way more data than I do, and has engineers working on it. And, as much as I'd like to work on a pet project, I don't have enough time to dedicate to anything more than essentially gluing APIs together.

    There is no confidence ratio on these requests. Either the comment is spam, or it's not, so we risk incurring false positives that I can't easily undo. However, I feel like this is the best ready-made spam solution, and I've nearly implemented it. We could run it on a trial basis, where I just have it write to a file every user that it would ban, and compare after the week is up (though there would be a lot) before turning it on.

    I might not be a maker in the traditional sense, but I am a software maker! Thoughts on this solution? I honestly feel this is the best shot that we have at restoring the forums to their former glory.

    I will not turn anything on at this time.

    EDIT: I, at this time, can also not make any commitments to when we can try this, as my semester is near wrapping up and I might not have enough time to dedicate to finishing this up. Simply collecting sentiment on this endeavour so I know how to proceed.

    EDIT 2: If we go this route, we should purchase a monthly commercial Akismet license. It's $5/mo. I'm not sure if they'd ever catch on as to what I'm doing, but in the interest of ethics and moral decisions...

  • @MAGZine As a new user, I wouldn't have a problem with what you propose at all. I love the forums here, more-so than anything on reddit or another social network platform. It is elegant in its simplicity. Subjects on the left, topics in the middle, easy to navigate and there is a good mix of what you want to see and nuggets of interesting I've found searching for other things.

    I am willing to accept that if I start a topic, if it is deemed spam by your program, then it will be deleted. The responsibility to make a thread topic meet the requirements falls squarely on our shoulders as users. I love this forum, and the community is one of the best I've experienced in my time surfing the web. I'd hate to see it go because a group of people constantly drown it in a deluge of skin care and shaman curses.

    Being a software maker is awesome. You may not have physical representations of projects like more traditional makers do, but this kind of things takes serious knowledge and an ability to apply it. You have some serious chops if you can manage all that. I wish I could do something like that to directly help all the people who visit this site.

  • @andy.fancher: Usually, when we delete spam users, we mark them for deletion (which removes all posts of the user's and prevents further commenting).

    Maybe a good trial would be to simply delete the post instead of the user and see how that plays out. Of course, there is a chance that user accounts are deleted.

    e: another option would be to drive the detection through the flags system. Rather than having a human sit on the other side of the flags system, we could simply submit all entries for spam detection. However, then that takes away from the alternative uses of the flag system (e.g. abusive behavior)

  • @MAGZine Is there any way to keep the flag system for abusive behavior, or will that render your idea useless?

    I realize that you all need to be sure before you implement anything, to prevent account deletion that isn't warranted. Can you or the powers that be implement something in the meantime to help slow the bleeding? A measure that's not perfect, but a way you can tweak it to see what would be the best from experience?

  • @andy.fancher said:

    @MAGZine Is there any way to keep the flag system for abusive behavior, or will that render your idea useless?

    I realize that you all need to be sure before you implement anything, to prevent account deletion that isn't warranted. Can you or the powers that be implement something in the meantime to help slow the bleeding? A measure that's not perfect, but a way you can tweak it to see what would be the best from experience?

    I can do post-deleting only. That would probably be the half measure. That, or we manually review the logs to make sure no users real users would be deleted and continue manual spam cleanup until (or if) we're sure we want to flip the switch.

    I've finished a very early-stage implementation ("that was easy"), and started running it on my virtual machine. We'll see what crops up overnight, I guess, while I wait for more community/Will input.

  • @MAGZine: Check your PMs ;)

  • @MAGZine@will Whatever you guys end up implementing, we appreciate it immensely. This is a community I'm proud to be part of, and that starts from the top and filters down to us.

  • @MAGZine: So, you want feedback....

    First, it's really good to see any forward movement on this.

    Second, I think a trial period is certainly in order if you go the proposed route. I basically never got my posts back (as far as I know) from the last time I was accidentally banned (Thanks Will! lol). Not that I was seriously attached to them, but I was in the middle of a thread or two when it happened. So I would want to know (if I were in your place) in advance exactly how many false positive it's looking like.

    Third, I am sorry that the system doesn't allow other options - since I think much of the spam could be curbed/managed by the suggestions some have made (including Will's comments). Restrictions on new users, proxy banning, etc....

    And lastly, looking at the TOS for that service, it looks like we (well, Tested) would be legit even if it had to be the $50 month one which could be covered by premium memberships (12 of them).

    Anyways, that's my feedback. I hope it is useful somehow. You certainly know more than I about the underpinnings of this forum operation.

  • Will and I are discussing, hopefully we'll find a solution that works.

    In the mean time, I noticed the first two pieces of spam today went undetected (third was one properly classified). I've modified the script to report as an older version of internet explorer (apparently browser version is a good indication of if you're a spammer) and changed the posting IP address to a United States proxy. I guess that this is really the only way I can tune performance. I've written an email to askimet asking about neutral user-agents and ipaddresses, we'll see what they say.

  • @MAGZine Is there any way to flag a post without going into the actual thread? Or possibly flagging a user who is a spammer? I try to flag some of the spam I see each night, but it is so time consuming to do it that way.

  • FYI, we pushed the first of several changes earlier this week. We have more stuff coming, including rules for member accounts and flagging, a more active flagging implementation, and eventually integration with anti-spam services. The problem with using Aksimet isn't the cost of the service, it's the cost of developing the integration.

  • @will Thank you for your continued diligence to try and corral this problem. I love a spam free forum asa much as the next person, but it is refreshing when the higher ups like you are active in their own forums.

  • I am quite happy to spend a few minutes a day banning folk that want to make my p*nis bigger

    As to quote scotty

    You cannae break the laws of physics jim