UPDATED: 12/09/2004 15:46 JST From now on, please check the central Spam Karma page to get the latest updates and news on this plugin.
Yet another techy update for my fellow bloggers using WordPress.
Now that it’s reached version 1.4 and that most (all?) major bugs have been ironed out, I feel it’s time to introduce the latest member in the ever-expanding WordPress plugin family…
Spam Karma is a mean critter that truly enjoys killing
In fact it is so mean that we had to keep it in a special military-grade containment unit on this server.
Genetically engineered in the dark recess of our Secret Spam Research Labs and trained through months of reflex conditioning and shock therapy, this thing, once unleashed on your comments, will only let go of its death grip after the last spam has been shredded to pieces.
We haven’t fed it for a week now, and it could smell spam miles away in its sleep.
But while a fierce and merciless spam killer, this plugin is also a perfect companion for your kids and friend’s comments. Only the unmistakable foul stench of spam will trigger its ire… while questionable, yet potentially legit, comments will always be given a chance to clear themselves before being irremediably disposed of.
If you are using WP Plugin Mgr, install is as easy as a click on the “Check Updates” button and a click on the “One-Click Install”… Yep, that’s all.
For those still stuck in the last century, a manual install archive is available here. Please, please, RTFM: it’s short, sweet and contains essential details.
Once installed, make sure you check at least once the Option screen (in wp-admin, click on Options >> Spam Karma).
I strongly recommend you check for updates (if you are using WPPM it will do it automatically for you) at least once a week so as to make sure you benefit from the latest bug fixes I might make.
Spam Karma v. 1.4 is now compatible with WordPress 1.2: however due to the lack of certain functions in WP 1.2 Plugin API, some of the features are missing (Option Page integration etc). It is fully enabled for use with any fairly recent release of WP alpha 1.3.
Cool, but How does it work?
Layman’s Explanation
Spam Karma works by running every new comment through a battery of filters and checks. Each of which increase or decrease the comment’s ‘Karma’ value. Depending on the final score, the comment is either:
- Approved
- Discarded silently as spam (no email is sent to you, unless you specifically require it, but a digest is sent to you every X spams deleted).
- Placed in Moderation mode. With the possibility for the commenter to auto-moderate his own comment by proving he’s not a spammer (by filling a Captcha or checking a confirmation email).
This whole process insures (by order of priority):
- No deleted false positive (bad bad bad).
- Extremely few moderated false positives (annoying): uses Captcha and email auto-moderation to keep these at a minimum.
- No published spam.
- very little spam held in moderation (must be destroyed directly: really annoying to have to moderate it).
Further more, Spam Karma works in an intelligent way to automatically update its filtering database and grow stronger with each spam it catches…
In short: blocks spam with no unnecessary annoyance, for you or your visitors. The way it should be.
The Detailed Explanation
For our more tech oriented friends, here are a few more insights on the rather complex process used by Spam Karma to decide what’s spam and what’s not. Each of the following filter is given a weight varying on many factors, ranking from user-controlled values (e.g.: after how many days is a post “old”?) to the credibility that can be given to a test (e.g.: a missing header is less important than a blacklisted IP).
Mostly, Spam Karma looks at the following things:
- If the poster is logged in the current blog, and what his user level is (e.g. automatically approve Admin posts).
- Presence of HTML entities (e.g. {, ʚ etc).
- Presence of a HTTP_VIA header.
- Proper use of the posting form (hash value must be present).
- Time taken to fill the comment (e.g.: if it’s less than a few seconds, most likely spam).
- Posting granularity. First time posters posting many comments at once vs. old-timers (with comments previously approved by the admin).
- Previous diagnostic from WP’s built in comment check (set on the ‘Discussion’ panel).
- IP and regex match for URLs contained inside the comment (small weight only for non-URL text matching a URL regex).
- Realtime Blacklist (RBL) Server check for IP and URLs.
- Comment’s age (e.g. penalize comments on very old post).
In addition to these filters, Spam Karma uses different treatments and backup checks to insure it becomes better at stopping further spam and that it never deletes mistakenly a legit comment:
- Ambiguous comments (that can neither be deleted or approved) are given a second check: commenter is asked to solve a Captcha or use the email auto-moderation (an email containing a hash to unlock the comment is sent to the commenter’s email address). If confirmed, the comment’s Karma is bumped up and the comment is either published or held for further review, if not confirmed within a certain period, its Karma is lowered and it is either deleted or kept into moderation (if it was sufficiently high to begin with).
- When a comment is struck as spam, its IP and URL(s) are harvested and submitted to the Admin for inclusion in the blacklist. In the meantime, they are used as “auto-added” values, with a lesser weight than permanent blacklist entries.
- When destroying a spam comment, it checks for recently posted comments that match similar values and retroactively moderate them (e.g.: a spammer could manage to slip X numbers of spams onto a blog, but upon reaching a certain suspicious threshold, all the comments would get retroactively moderated, then deleted).
- Spam Karma uses a central DB to retrieve IP and URL updates. By default, it will query the DB automatically every 2 days (can be disabled). Central DB can be configured. Each install of Spam Karma can work as a sort of P2P relay in the update process (both fetching updates and publishing its own updated list for others to grab).
Thanks and Acknowledgement
Many, many people have contributed, knowingly or not, to this plugin, with their ideas, code, help, testing, advice and support… I ended up rewriting most of the code I took from these plugins, but it nonetheless gave me a solid base to start with quickly. Thanks guys.
- Happy dad Owen Winkler, whose OSA plugin served as a base to write Spam Karma
- Mark Jaquith, for some code and ideas from his Snowball Effect and Captcha plugin.
- Jay Allen, the author of MT-Blacklist
- Laughing Lizard who ported MT-Blacklist to WP and whose 3 Strikes plugin was problably the first to implement the “karma” concept…
- Mookitty, whose efficient Spaminator plugin (working on a very similar principle), and the discussion we had around it, got me thinking…
- James Off, whose RBL server is used by default by this plugin.
- chrys, Steph aka bunnywabbit, maru, rgh, masquerade, neuro and everybody else on #wordpress, for their patient help debugging this plugin…
If you encounter any error or misclassification of comments (false positive, undetected spam), please contact me and preferably include the whole comment content, such as it appears in the admin screen (with the Spam Karma debug values).
Any comment or suggestion always welcome…
One Plugin to block them all
One Plugin to keep track of them
One Plugin to moderate them all and in the queue delete them.
I installed both the plugin manager and the spam karma for the first time today. I’m getting an error when I get to the settings section… i’ve put a screenshot at http://deanmckenzie.org/images/spamkarmass.jpg Not sure what to do at this point.
Kickass.
Hi. Thanks for such an awesome plugin. Good stuff!
For Dean (and those who have similar DB troubles with the Blacklist creation step)
This is likely due to an incompatibility with a previously installed plugin that would have already created a different Blacklist table.
To fix it, simply download the latest version (currently 1.5) and, if you are upgrading from a previous non-working version, click on “Reset Blacklist” and “Update Blacklist” to create and fill the Blacklist table correctly again…
This should do it, if it doesn’t, please contact me.
Awesome plug-in. Thx very much! I’ve been getting about 100 daily spams in my comments and this caught the three that posted within seconds of installing it. Keep up the great work!
Regards,
Kory
Awesome plug-in. Thx very much! I’ve been getting about 100 daily spams in my comments and this caught the three that posted within seconds of installing it. Should I remove the filter words/phrases from WP’s default area? Keep up the great work!
Regards,
Kory
Removing entries from your WP built-in “spam words” list is not mandatory… but I’d recommend doing it at least for any word/expression that is not absolutely guaranteed to be spam.
Supposedly, Spam Karma should be able to intercept and treat on its own every comment including the ones that WP has singled out this way… You can use the “Use Options set on the Discussions Page” option to make sure SK takes in account what WP found.
However, SK should work fine without it. and if that list contains a lot of entries, it might become a performance issue; better erase it then (replace it by regex entries in SK if you want).
Great plugin, thanks a lot! 🙂 Only issue is that on WP 1.2, the digest is seemingly not working – I am getting an email after each deleted spam, even though the option is set for 10. It seems that when I click the link to go back to setup, most of the options are blank, as if the plugin has “forgotten” the settings. Thanks for any assistance you may be able to provide.
-Carsten
I get an error everytime I activate this plugin – and I beleive update the blacklist:
Database error: [The used table type doesn’t support FULLTEXT indexes]
ALTER TABLE wp_comments ADD FULLTEXT KEY spamk_fulltext (comment_content)
I’ve tried to manualy query the database to no avail. As well it is catching regualr comments (in fact ALL comments) as spam. I’m running 1.3 alpha4 with Ryan Boren’s Kubrick for 1.3 – any ideas?
Hi. The install of Spam-Karma was easy, so I decided to test it out to see it in action. Yet, when I submitted the form all I get is this:
————————-
Comment Verification
Your comment could not be posted immediately as it triggered some of the anti-spam filters that run on this blog. Please complete the form below in order to get it posted directly.
If you are not able to see the image due to a browser issue or a handicap, please use the e-mail confirmation link at the bottom of this page.
If you do not confirm this posting within a certain period (by either solving the test below or using the mail auto-moderation), your comment will likely be destroyed.
————–
I don’t see a captcha or a form that allows people to edit their comment. I tested the comment form out in Safari, Internet Explorer, and Mozilla. Any info on this would be much appreciated. Thanks.
P.S., I’m currently using WordPress 1.3 alpha 5.
Regarding the Captcha not displaying: if you proceed with a manual install, it is essential that you follow the install instruction and chmod the “captcha_temp” directory to 777 (make it writable by the server). This step is not necessary if you use WPPM.
There shouldn’t be any form to allow people to edit their comment on this moderation page, but, if you have this option enabled, a link to send a confirmation email should appear at the bottom of this page. If it does not, make sure you entered a valid e-mail address.
Lastly, depending on what your comment contained, it might have justified or no, to be moderated… if you want, you can go check in the moderation panel and the comment should contain debug data explaining why the comment was moderated… if you send them all to me, I can check if this is a bug or expected behaviour…
Got an error with a comment:
Comment still went through. Some bad value must be getting put into the regex.
This is a great pluging! I keep getting hammered by annoying spambots all the time and Spam Karma stops them all. I *DO* have a problem though. The settings keep getting RESET automatically. If I for example change the digest settings to 20 it will be set to 1 again. Very annoying.. :/ Any ideas of what to do?
I’ve installed Spam Karma on a WP1.2 install, manually. Is it supposed to continue to allow emails to be sent when a comment needs to be moderated. We had it sending an email when it deleted one. So we’d get two emails, one saying that I had a comment to be approved and another saying that Spam Karma ate the comment. The only reason I had SK sending a message for each time it ate one was to see that it was working without waiting for a digest. Is SK supposed to prevent wp from sending a “awaiting moderation” message?
The digest doesn’t seem to be working either, I have it set to send a digets for every 20 deleted. The Current digest window says that it’s deleted 798 spam messages, and 0 of 20 in the digest in progress.
Quite a few problems with this plugin so far, running on WP 1.2. The first and major problem is that it’s not sending me notifications or digests or anything resembling responses. It also seems to be nuking every single comment; I’ve been trying to post anonymous comments but it just sends all posts straight to hell. Handy for stopping all the comment spam, but at the cost of all further comments. Not good.
Also, it seems to have quite a few problems saving my preferences; more than half the time when I go back to the config page (plugins/spam-karma.php?spamk_setup) all the config options requiring numbers are blank, the digest reads “Digest in progress ( of 10)” with no digest entries, even though it IS keeping track of all the spam deleted.
Is this just a WP 1.2 thing? I really don’t want to have to go back through and hack all my WP pages back again after an upgrade to 1.2.1, but if that’ll fix the problem I’ll do it.
There’s a bug with the $time_taken zero divide fix. If you use it like it is, the time taken is always the lesser of the actual time taken or 0.01 seconds. So it’s always 0.01 seconds. Usually, the comment is blocked because 0.01 seconds is less than the minimum required delay time setting.
Move the min() to the place where the zero divide would take place, a few lines down.
Thanks a lot Owen, nasty little bug indeed… That’s what happen when you do hasty bug fixing before 10am.
Anyway, if you happened to have downloaded/upgraded SK in the past 10 hours, please upgrade immediately.
If you have the kind of problems Jeff is experiencing: Make sure you are using WP 1.2.1 (SK should work with 1.2 too, but honestly, I am not supporting it since 1.2.1 is fully compatible and fixes many important bugs) and most importantly, if you enable the OSA check: you must use a comment form identical to the one shipped with WP (no changing the form name): reverting the comment form file to its original fixed Jeff’s issues and will probably fix yours too.
Great plugin! I have a question about the digest. Mine says
Deleted Spam: 4
Digest in Progress: (1 out of 3)
I don’t understand what digest in progress means. Am I supposed to do something to get to the other 2 items?
Thanks for any help.
Tom
Whoa, this plugin is awesome. It’s exactly what you’ve described it as, although I seem to have a problem. I have WordPress set to email me when comments are posted. It no longer does that. And it DOES email me every time a spam comment is deleted, one of those “please approve” emails, even though it’s set not to do that at all, no digest or anything. I think it’s 1.7 alpha; the one that the wp plugin manager has set for one click install. Is there a solution? If not I won’t complain because the plugin is truly awesome… I just respond to most comments by email.
Hmm, can’t get it to work :/
Activated or not, I can’t load the page :
Fatal error: Cannot redeclare url_shorten() (previously declared in /home/ozh/wp-admin/admin-functions.php:3) in /home/ozh/wp-admin/admin-functions.php on line 3
(using 1.3)
Any clue ?
Some thoughts:
* I’m not receiving moderation notices from WordPress any more, even though I had three messages go to moderation today.
* The submenu for SK 1.7a2 appears but doesn’t link to the right place in WP 1.3a5.
* It would be very nice if I could omit comments from the digest that don’t include the OSA code. No code, no post, period – I don’t even care what you’re hawking.
* Digest comments could include the posting time/date in their info.
wow. this plugin rocks. I moved from WPBlacklist because I couldn’t control the email delivery (I was getting all the spam comment email notices as well. one day it was about 50 an hour as my site was getting hammered). This also gave me the excuse of installing wp 1.3a5, and frankly, I couldn’t be happier about that either. Keep up the awesome work!
I just used plug-in manager to install spam karma. WP is 1.2.1
I can’t seem to configure it. If I click the link for 1.2 I get
If I try the other link to configure I just get a 404. So I’m not sure what I’ve done wrong, but if it is possible for any help, I would be most grateful.
I’ve successfully installed Sk on WP1.2.1 and I’m no longer receiving emails when comments are approved. This saddens me.
What can I do about this?
Important Update
To all those who reported bugs or issues above that I haven’t fixed by email yet: please download the newest Update (1.8), it includes many fixes and solves all the issues reported to me so far (in particular with WP 1.2).
Upgrade is strongly recommended to everybody, regardless…
WP 1.2 users, please note: the upgrade process will reset your settings and erase your digest, so please check your option page before and after the update.
Hello everybody.
I have tried installing karma 1.8 om wp 1.2.1. Im getting this error message
Updating Blacklist Options
Populating WP-Blacklist Table from: ‘http://www.unknowngenius.com/blog/blacklist/’
ERROR: Could not download from this Blacklist URL.http://www.unknowngenius.com/blog/blacklist/
Populated WP-Blacklist table: imported 0 values, skipped 0 duplicates.
Error: did not import anything…
Warning: fread(): Length parameter must be greater than 0. in httpd.www/wordpress/wp-includes/streams.php on line 107
Project-Id-Version: WordPress 1.2 Localization-dk POT-Creation-Date: 2004-05-22 18:59-0500 PO-Revision-Date: 2004-08-11 10:59-0000 Last-Translator: René Clausen Nielsen Language-Team: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit
Can anybody help me
Arvid Normann