Dave's Blog » WordPress

WP-plugins.net domain name for sale!

Dave — Thu, 18 Dec 2008 21:38:29 +0000

Update: Sold.

In the last installment of our Boring Geek Updates™ series for the year and as part of my general strategy of complete WordPress disengagement, I finally decided to do something with that long-neglected pet project of mine: WP-plugins.net

Namely: sell it.

As you can tell from the ‘beta’ sign and the outdated reference to an ongoing update process, to be completed ‘any time now’, this site has long shifted out of my personal field of interests. Don’t really have the motivation, definitely don’t have the time.

But, maybe you do!

If so, now is the time to make me an offer.

In a nutshell:

For whatever price you pay, you get: full control of wp-plugins.net (I’ll have it transfered to your registrar), all the code that sits on it and currently powers both frontend and backend, plus the entire plugin database.
What you won’t get: hosting (you gotta have your own server), full ownership of the code (some of it is already GPL’ed, so I can’t do anything about it, but you can still use it however you like) and I will most likely remove all plugin authors’ email information from the database, unless you can make a compelling case for me not to.
I would like it if: you have plans for the site that involve a certain continuity with the existing service. Whether you keep the entire thing as is and just maintain it or entirely overhaul it with new and exciting features, I’d just like to be sure this domain doesn’t become a cybersquat in a month or two. Basically, if you can prove some loose level of involvement with the WP development community (really: anything will do… You are not interviewing for a job here), I’ll be much more likely to take your offer into consideration.
I have frankly no idea how much this domain/site should go for. ~~Which is why I’m asking you to send me an offer, by email~~ (update: domain sold, please do not contact me about it), with whatever amount you think fair (along with a two-line plan of what you intend to do with it, see above). For reference, just note that I have routinely gotten (and declined) offers for advertising deals running about $1000/year… Which pretty much sets a starting price (but then again, make a convincing case if you can’t afford that much and maybe the spirit of christmas will overcome me and I’ll let it go for less than that for a good cause).

Update 12/19:

Credits: whatever you do with the domain (as said previously: hopefully something for the common good), I would appreciate credit (name + link anywhere on the site will do). Then again, if you have a strong case not to, just take it to me and we’ll see: this is a “would like”, not a “must”… Either way, please let me know.
Deadline: I’ll accept email offers until Monday 8pm GMT. At which time I might contact a few of you for additional information or clarifications on your plan, then let everybody know shortly afterward.

Anyway, all interested parties please send your short and honest offers to the following address: wpplugins4sale@gmail.com.

Thank you and happy eggnog-drinking binge!

Post originally published on: Dave's Blog (please leave your comments over there)

WP-plugins.net domain name for sale!

Spam Karma goes GPL

Dave — Sun, 13 Jul 2008 18:29:04 +0000

Geek news warning: sane people and anybody for whom such acronyms as PHP or GPL merely evocate some brand new drugs the kids might be into these days: you are probably better off skipping this one.

I’ll try to keep it short.

Spam Karma 2 is now released as GPL v.2. This essentially means you can do anything you want to it, except claim you made it (copyright and attribution notice must remain there). You should also note that any attempt at deriving some ill-deserved profit from it through harebrained web marketing schemes will earn you both my long-standing scorn and a nut-shriveling decrease to your actual karma.

I suppose another angle to that post’s title could be:
Officially discontinuing Spam Karma’s development: so long and thanks for all the fish
as this is what this truly is about.

But, such a title would be slightly misleading (and no doubt heavily quoted out of context): Indeed, I am hereby officially announcing that I will no longer support, maintain or further develop Spam Karma (beside some very occasional, very limited poking, until the transition to a self-maintained project is completed). However, thanks to the magic of free software, all the unsung heroes of the Open Source world will soon rise to take over and bring you a stronger, better, more closely supported version of Spam Karma!

Okay, what’s more likely to happen is that nobody will really bother taking over, except perhaps a handful well-intentioned but utterly clueless beginner coders who will quickly find themselves overwhelmed by the task and next be seen running away screaming at the top of their lungs. Hey, I’m not blaming anybody: I wouldn’t waste my time on a non-paying, open-source community project either…

But on the off-chance that you would (and trust me it won’t do anything to help you get laid either), I have set up a Google Code repository, which could become the jumping point to some magnificent community-based development effort (or not). If you are interested in participating in any way, contact me (mail or contact form) with a *brief* description of who you are, what you can do and what you wanna do. I don’t need a resume (I am not hiring), just a very quick idea of what level of responsibility you’d be willing to take on the project. I’ll put in the first couple people that seem to know what they are doing (and do not sound like they’ll be selling everything to Russian mafia-owned spam sites) as administrators of the project, and hopefully from there on, things will work by themselves…

If you think you’d like to tackle any aspect of SK2 development (including possibly porting it to other platforms), here is your chance. Speak now or go back to more fruitful and life-rewarding endeavours forever.

Oh, and as for the “reasons”, well, here they are:

1) Life.
Much as I love the challenge and excitement of coding an anti-spam filter and thinking up new tricks to defeat parasitic life-forms of the web, I just don’t have the time anymore. And to be honest, if I did have the time, I probably would have other challenging, exciting new projects I’d rather tackle. I’m fickle like that.

2) WordPress
I will really try to keep that one short, because I could probably write a novel of that. And it wouldn’t be a very interesting read.
In a word: WordPress kinda sucks nowadays. Its retarded upgrade rate makes it nearly impossible to keep up, in turn making it a constant security threat on my servers. And each time I finally cave in and install one of those “mandatory security upgrade”, it also installs 600 Ko of other theme compatibility-breaking fluffy crap that I never asked for in the first place. Usually setting the ground for the next cycle of security-exploit-rushed-upgrade. To sum up, it’s become incredibly bloated and tedious to support. Replacing it on my own servers is very high on my list of things to do (which means somewhat in the first 1000 items).

Having no interest for WordPress anymore, I have thus very little interest for WordPress-related development.

As for WP coming bundled with its own anti-spam plugin, I could also go on for hours on that. The fact that a community-based open-source project is used to distribute a commercially licensed piece of software doesn’t make me particularly happy. But frankly I haven’t cared and still don’t care enough to even raise a stint. At any rate I know lots of people (me included, obviously), aren’t convinced by the way Akismet works and are happier doing the filtering on our own servers, so there is definitely room for SK2-like plugins out there.

Anyway, thanks everybody for your support all these years and let’s gather a round of applause for our brand new Spam Karma GPL Edition!

Update: in addition to the Google Code-hosted project, there is now a dev mailing list set up on Google Groups, go check it out and feel free to sign up if you are interested in SK2′s future development!

Post originally published on: Dave's Blog (please leave your comments over there)

Spam Karma goes GPL

Big in Estonia

Dave — Wed, 05 Sep 2007 17:12:34 +0000

Because Big in Japan is so 1995, I am proud to announce that I am now also officially Big in Estonia. Secondary to my being featured in this month’s issue of arvutimaailm, a computer magazine so big in Estonia, it doesn’t even bother putting its content on the internets.

The article is signed by Elver Loho and features, as far as I can tell, some collected ramblings of mine on miscellaneous items of computer security and spam, as well as a scarily huge photograph, that must by now adorn the wall of every single Russian mafia hitman east of the Volga. My command of Estonian being unfortunately very low, I can only assume the article bears no reference to my habit of drinking a bowl of fresh kitten blood every night before sleep. I will try and post the PDF here, but need to make sure it’s ok first (and also get the final version, as I only have a working layout).

Which reminds me there’s been lately a couple articles, newspaper columns and books bearing mention of my name and/or one of my miscellaneous ongoing evil plans. Unfortunately, I have been pretty bad at collecting clippings for my mom’s trophy wall and haven’t really kept count, but I still wanted to recommend Maria Langer’s book, since she was so kind as to send me a personal copy (long, very long ago now… sorry Maria…).

Post originally published on: Dave's Blog (please leave your comments over there)

Big in Estonia

Followup on WordPress Security Issue

Dave — Thu, 27 Jul 2006 10:25:54 +0000

Time for some Q&A here…

Below are a few of the most oft-heard questions/statements about my previous panic-level-3 announcement regarding a serious security issue in WP and how to easily fix it temporarily (one checkbox to untick)… Along with answers:

“Is this a joke/hoax?”

Is the date the 1st of April?

“How critical is it?”

Critical enough.
Not all WordPress users are at risk, but I don’t suppose that would be of much comfort to you if you belonged to the 30% hackable installs.

“Will you tell me more about this exploit? I swear I won’t tell anybody else! This will stay between you, me and the World Wide Web…”

Frankly, if you do not understand why I won’t even be giving the slightest hint of what the exact problem is before enough people have applied the temporary fix (disabling ‘user can register’, under Options >> General) and an upgrade has been released, you are the last person I should be telling to about this.

“Are you a WordPress official? a WordPress developer? Anybody with a title I can trust?”

Absolutely not. In fact, less than ever.

“Why don’t you leave that up to the Big Guys Who Know What’s Best For You®™ and go back to getting smashed on gin somewhere under a Parisian bridge then?”

Take your pick:

Because over the past year of distant involvement in the WP community, I have come to question and, well, often outright disagree with the way the Big Guys Who Know What’s Best For You®™ handled similar problems in the past.

Because, all modesty set aside, I am not sure how their strategy for handling such problems (which I have seen in action in the past) has proven better at containing disaster than the one I adopted here.

Or perhaps simply because, as some Big Guys Who Know What’s Best For You®™ have implied in one helpful bit of Shoot-the-messenger communication, I am an attention-craving moron with nothing better to do with his time than scare his fellow WordPress users into unchecking one single option in their admin screen.

“Why do you relish so much in fear-mongering, FUD and…
OMFG…
What is this?
Snakes!!!! There are motherfuckin’ SNAKES on WordPress!!!
Ahhhh! do something!!!”

Right.
Why in the world would anyone want to make their security announcements sound too important?
All apologies for that bit of hysteria. Let me go back and add a few soothing pastel tones and some muzak to the original post.
Let’s rephrase too:

Hello, there’s a teeny itty bitty problem with your WordPress install, feel free to do something about it, but don’t worry too much otherwise, it’s all good man.

There. Better?

Now, seriously, to those who have been equating my behaviour with shouting “bomb” in a crowded airport: get. a. fucking. grip.
This is the internet, not an airport. Computer mice stampede rarely kill anybody. And I’m sure even the most cheetos-infused computer nerd’s heart muscle can take the news without prematurely collapsing.

Making it sound serious is a) the only way to spread the news b) perfectly justified given the fact that, you know, this is pretty serious.

Truthfully, I don’t think there is much harm done in upping the amount of caution people display when using server software. Such exploits are common, and not solely a WP problem (all server applications have had similar issues at one point or another). Lack of awareness in the general public, if anything, is the danger. Sorry if that doesn’t really bode well with some people’s marketing pink-cloud vision of the world.

Just consider yourself lucky I didn’t use:

(directorial credit: Leftjustified)

“You mean you didn’t just write that thing in a fit of aimless panic to scare other people out?”

There again, despite what some well-meaning people seem to have charitably been hanging on my head, I have given a bit of thought to the problem before deciding the best course of action was to invite people to disable an option in their admin tools. And for those who really care, here is how I see things:

An exploit in WP was brought to my attention. It was neither the first one, nor probably the last one. Exploits are an unavoidable byproduct of major projects, they are not a fatality: you fix them and move on.
My very first reaction with this, was telling Geoff Eby, who discovered it, to contact WP devs and take it to them in private.
As would sometimes happen, it seems, the answer wasn’t really overwhelming, and it appeared some devs (suspicion confirmed ever since) even considered this to be of the “Not WordPress’, Somebody Else’s Problem” category. A position I reckon everybody has recanted ever since, but that threatened to be the official position then.
Now, I am sure a fix will eventually be released. It will be the fourth upgrade this year. And I bet you the whole “major security issue” thing would not exactly be put in large blinking letters on top of the official download site. Ya know, you don’t want people to panic and start thinking there may be a problem or anything.
I have absolutely no stats to confirm that, but I can also bet you there are still thousands of installs of WordPress that haven’t even installed the previous security upgrades. “Too bad for them”, I hear you say? Well, sure, it’s not very smart but you can’t blame them for eventually tiring of following announcements and refusing to sort out themselves between minor bug-fixing releases and major security-plugging ones (a difference that’s pretty much inexistent in all WP official channels).
As a result, I get word every day of one WP install or another that’s been hacked, usually using an exploit supposedly fixed months ago (just have a look at some of the comments in the previous post).
I would be amazed, alarmist tone notwithstanding, if more than 50% of WP installs throughout the world were upgraded to a safe version by the end of the Summer.
On the other hand, the second the patch is released, every malicious coder with two bits of brain will know exactly how to exploit older versions and will go to work.
The fix I invited everybody to apply (namely: uncheck the ‘allow user registration’ option in WP’s General Options panel) is both simple and does not reveal the exact nature of the exploit. Not only do I trust it to be more widely applied than a full-on upgrade by the less computer-proficient users, but it will ensure, by the time the upgrade is there and the exploit publicized, that much fewer vulnerable installs remain.
So, think of it as a two-step process where we can afford to raise awareness and offer a simple temporary fix without risking to help script kiddies too much, then release the usual half-ignored patch and not have to watch the entire WP world crumble as dozens of unprotected blogs get defaced by 12-year old pimply teenagers.

Or, you could just follow “protocol”, discuss the exploit at length in private developer mailing lists, safely out of sight from the general public (but well read by possibly every last malicious coder in the world) and release yet another semi-silent security upgrade while hoping good people hear about the news before the bad ones do…

But feel free to ignore this ranting of a madman: after all, perhaps I was just bored and figured this would be the best way to have some fun, without incurring serious cavity damage in an airport security room for mentioning snakes while boarding.

For anything else, feel free to contact your nearest WordPress Official for further word on what to do. I’ve done my bit.

Update: WordPress upgrade 2.0.4 should now patch this bug. If your version of WordPress is equal to or higher than 2.0.4, feel free to ignore the warning above. If not, then you should/MUST upgrade (more details in the comments).

Post originally published on: Dave's Blog (please leave your comments over there)

Followup on WordPress Security Issue

Critical Announcement affecting ALL WordPress users

Dave — Wed, 26 Jul 2006 15:10:40 +0000

If you are running WordPress as your blogging platform and if you have been trusting enough to leave User registration enabled for guests, DISABLE IT IMMEDIATELY (in wp-admin >> options: make sure “Anyone can register” is not checked).

Additionally, delete or disable ANY guest account already created by people you are not sure about.

Leaving it open and letting people sign-up for guest accounts on your WordPress blog could lead to incredibly nasty stuff happening if anybody so desired. And trust me I am not exaggerating this. So don’t wait a second to disable this option and please relay the message.

WordPress dev team has been notified a while back and I dare hope they will soon start acting on it, if only by relaying a similar announcement through the official channel (as well as, of course, releasing a proper patch).

Sorry for the shrill hysterical tone, but this is a big deal. However, disable that one option and you are fine, no need to panic further

[cheers go to Geoff Eby for discovering and bringing this insane security exploit to my attention]

Update: a small follow-up addressing comments and concerns I have received ever since this last warning, is posted here. Feel free to ignore completely unless you really care about inner WordPress politics (yawn).

Update 2: WordPress upgrade 2.0.4 should now patch this bug. If your version of WordPress is equal to or higher than 2.0.4, feel free to ignore the warning above. If not, then you should/MUST upgrade (more details in the comments).

Post originally published on: Dave's Blog (please leave your comments over there)

Critical Announcement affecting ALL WordPress users

Better leave at the top of your game…

Dave — Fri, 14 Apr 2006 17:07:55 +0000

Wherein the author enumerates much meaningless data and uses them as springboard for some slightly more topical meandering…

1,059 WordPress plugins currently sit on wp-plugins.net. Not bad for a project that was half-shunned by the official WP pubah(s) from the very beginning. Kinda getting worried by the amount of bandwidth this is eating off my quota right now (read: somewhere in the 200% vicinity). But we’ll cross that bridge when it starts falling.
11,232 SK2 downloads for the year 2006 so far. There again, not bad for a plugin that doesn’t happen to be the one packaged by default in WordPress 2.0.
968 comments (mostly Trackbacks and Pingbacks, as I closed comments on this page a while back) on SK2′s homepage. Can’t help but notice an uncannily high percentage of posts from Germany. Is SK2 like, the David Hasselhoff of anti-spam plugins?

As you can tell, despite being on cruise-control mode, the Deliverables Department of UnknownGenius Corp. is doing nicely. As for where it’s heading, I suppose I may use the occasion to offer a quick update:

The short answer is that it is going nowhere.

The longer answer is that, ultimately, I will be phasing out all WordPress development (and most web coding, actually) from my activities.

For those who care about the Why, I will try to provide some elements without delving too deep into the multiple layers of frustration and unrelated motives for my general disinterest toward WordPress at the moment:

Recently, Skippy summed-up quite nicely the building frustration in the WP community. Mine has been growing along a similar path and, after voicing my concerns a few times last year, I eventually decided to save me the time and energy and silently started shifting out most of my WP-related activities. Like Skippy, I take issues with the way WP development is “managed”, I also have serious concerns about the increased melting of public open-source code with semi-private interests, as reflected by the blatantly commercial marketing strategies of recent releases. Don’t get me wrong: I am not one of those open-source zealots who considers it a crime for software authors to make a dime off their work, I just do not like the way it’s done here.

On a wider scale, I guess I am getting tired of the whole “Web 2.0″ micro-bubble, the underwhelmingly banal concepts it rests on and the mix of greed and naive enthusiasm that propels it. I was there the first time around, and believe me: little else of durable value was invented during that era beside the Skyy & Red Bull cocktail. Oh, I’m sure a few people will manage to make some cash this time too, and I certainly wish them all the success they deserve. But I see little reason contributing my time graciously to help selling Automattic to Yahoo or some other acquisition-hungry greying Bay Area corporation, which is what developing for WordPress increasingly felt like, as of late.

Please do not panic (and do not listen to the well-meaning people that might be inclined to tell you otherwise): I am not dropping Spam Karma nor wp-plugins.net any time soon. I will keep doing as I have over the past few months: maintain and improve as mandatory, without any sort of long term planning (in the positive or the negative) as long as things remain stable. If push comes to shove and I really have to make a decision, I will at the very least ensure that the legacy of these projects is maintained, and you will long be blogging on neuro-quantic interfaces before you have to worry about alternative ways to ward comment spam off your blog. If anything, I have no intention to stop my blogging, and WP remains, at the moment, the best option for my needs, so you can find reassurance in my own necessities.

As for other tentative projects of the past (WPPM, minor WP plugins etc.), I am saddened to say that the chances of resuscitation are inching closer to zero with each passing day. I have toyed with miscellaneous new ideas (including starting the new blogging platform of my dreams from scratch), but decided that, in the end, this wasn’t the direction I wanted to take with my time. Sure, I have to refrain an impulse to start coding, each time I pass the now-defunct one-click install FAQ, but really, it’s all for the best. Of course, if some generous benefactor shows up with $3,000 in cash and asks to see the most kick-ass WPPM 2 s/he’s ever seen, I won’t be difficult to convince (I’m a fairly venal ilk of genius these days), barring that unlikely event, we shall say new coding adventures will be kept for brighter days and a very distant hypothetical future.

Post originally published on: Dave's Blog (please leave your comments over there)

Better leave at the top of your game…

Spam Karma donations: You Rock!

Dave — Sun, 05 Feb 2006 18:15:44 +0000

Less than a week ago, we were launching our major campaign: “A bullet for every Spammer, a lifetime supply of quality gin for dr Dave“.

We called for your generosity in helping make this dream come true (especially the drinkable part).

The results are in.

You, people… how shall I put it… You people… well… you truly and utterly rock. All of you. You are the best users a comment-spam plugin developer could ever want. And I’m not saying that because I am currently typing this atop an air mattress floating on an olympic-sized pool of gin, a feeding tube taped to the corner of my mouth. Although it might be helping.

I know I said there were no hard milestones in funding and I am usually loathe to boast of donation figures, but I feel I must make an exception and mention here that, for the past week alone, your donations have totalled a little over 800 US dollars, shared between dozens of people, spanning from $2 to $50, all equally humbling as a developer.

Do you realize how many days of uninterrupted drug binge this money represents? In fact, you may never hear from me again.

I have compiled a list of all donators (past, present and hopefully future): if you have donated at any point, please go look for your name and ensure it’s there, the way you want it (I wasn’t able to dig URLs for some people despite my best detective work). For those interested in paying for additional exposure, a small extra fee will get you a laser-engraving of your name and website on a prominent part of my liver.

Also: I will be contacting shortly all those who offered their time to help with SK2 documentation and support.

As for Spam Karma’s release schedule, here is a tentative roadmap:

SK2.2 Beta is now available for download. Among other things, it fixes tons of miscellaneous compatibility issues with exotic server installs, WP versions and browsers. It also incorporates some small tune-ups and improvements that should help deal with recent spambot advances (although nothing radical yet). Note that it is still a beta: only install if you are comfortable with a certain risk factor and make sure you religiously upgrade to final as soon as it’s out.
If testing of the current beta doesn’t reveal flesh-eating tendencies, a Spam Karma 2.2 Final should be out by next week-end.
By then, I will be running a second donation drive in order to fund my trip to Borneo, where I shall be attempting to capture enough monkeys to ship along with every future version of SK3 (all part of the plan to use P2P blacklisting).
Failing that, a 2.3 version of SK2 should happen sometime in March that brings some amount of Naive Bayes filtering to the mix.
And then there is this large iron padded door over there, behind which our research staff is hard at work on classified anti-spam weaponry. Unfortunately I cannot let you in at the moment.

In the meantime, keep the suggestions coming and let me know of any bug you spot in either betas or final versions of Spam Karma.

Now if you’ll excuse me, I need to collect my check and go snort a pound of cocaine off a hooker’s cleavage.

Post originally published on: Dave's Blog (please leave your comments over there)

Spam Karma donations: You Rock!

The State of Spam [Karma]

Dave — Mon, 30 Jan 2006 04:12:48 +0000

First blog update on Spam Karma, WordPress development and Spam in many months, and a crucial one at that. Being notoriously verbose to the point of irrelevance, yet with lots to say today, I have tried to provide a telegraphic sum-up below, feel free to skip and go straight to the parts you may care about (hint for the busy ones: the plot thickens mostly around part 6 and 7).

And now for the details:

1. How well is SK2 stopping spam currently?

If you’ve been using SK2 for a while until now, you know it’s working pretty damn well. Over the past year, on the different blogs I manage (some of which receive a steady stream of both legit and spam comments, TBs and PBs): over 99% of spam was caught and under 0.1% false positive (pretty much zero, actually).

The only spam comments that made it through, were usually spams posted manually: that is, where a human would browse to the site, maybe even read the post and post a topical comment looking nearly like ham, save for a blatantly “commercial” site linked in the URL field. These were nearly impossible to stop, as SK2 works 90% on detecting spambots and relies only moderately on blacklisting (which helps to keep its false-positive rates extremely low).

These “manual” spams, though, never were much of an issue, as the essence of spam is automation, without which it loses all its appeal: Assuming it takes a few seconds for an admin to manually moderate spam, and given the numbers of bloggers vs. spammers, anything under hundreds of spams per seconds, is just not worth a spammer’s time.

Also one important thing to understand is that SK2 learns and improves: Flagging the spams it let through, helps stopping the next ones. It is fairly normal for a fresh install to let a few spams through at the beginning, but flagging them and thus allowing SK2 to build its blacklists and pattern lists, should immediately improve the catching rate dramatically.

2. Then why have I seen so much spam going through lately?

Unfortunately, as some of you might have noticed, SK2′s performances as seen from the outside, seem to have dropped suddenly over the past few days. While the bulk of the spam still remains at the door, a meaningful percentage now manages to fly right through SK2′s basic filters. And given the numbers involved, even 1% of all spam attempts is a lot to deal with. There again: SK2′s blacklists learn, and conscientiously flagging each uncaught spam should help keep things under control, but this is still a major quality drop from SK2′s usual performance.

The reason for this sudden burst, is a new breed of spam, or more likely, of spambots. It is confirmed now that some spammers have gotten hold of much more efficient spamming tools. Ones that bypass some of SK2′s strongest filters without trouble.

Also of note is the fact that Trackbacks and Pingbacks are absolutely unaffected by this issue (although a small unrelated bug was fixed in the latter SK2.1 releases and you may want to upgrade again from the site: more on this later).

3. How does this new spambot generation work?

This is a very difficult question, since it involves lots of guessing and detective work. Pretty much like in a war, we do not have access to the enemy’s weapons designs. A very uneven war, actually, since the enemy does have access to ours.

There are ways, though, to gather information about what spambots do, and try guessing how they do it.

[long and uselessly detailed technical droning: you probably want to skip that if you aren't an anti-spam plugin developer yourself:]

First of all, these spams do not present most of the idiotic traits of their lower colleagues: they do not try cramming hundreds of URLs or inserting hundreds of easily spotted junk keywords in the comment content. Instead, they use only the dedicated name and homepage fields to sneak in spam URL and keywords. The comment content is often perfectly innocuous, sometimes even topical (by copying parts of another comment or a trackbacking post). All in all, these spams could easily be missed by a human moderator who wouldn’t look carefully at the contact name and URL.

When dissected in the http server logs, the spam looks strikingly human-generated: queries for all the files (pictures, css, favicon and javascript included), sometimes a valid referrer URL is provided, links are followed (e.g. from the frontpage to a specific post), the user-agent, of course is valid and claims to be a regular browser. Timestamps generated by a single spamming IP even seem to point to a typically human erratic way of browsing. Most importantly, the spam bypasses SK2′s Javascript filter, which indicates an ability to parse javascript.

However, looking closer at timestamps and a host of other small details, I am fairly certain these aren’t posted by a human, but are indeed a new breed of spambots. There are many ways I can think of, to make such a spambot with javascript-parsing ability and other “mimicking” skills… In fact, I’m just surprised it hadn’t been done before. But this new development is also worrying, as it seems to indicate that spammers have finally gotten hold of real coders to do the job: whereas previous spambots could have been the work of any random script-kiddies with half a brain and a vague knowledge of scripting, these seem a bit more thought out in their design and their implementation. This is particularly worrying as I do not know of any anti-spam system currently that I, or a somewhat similarly skilled coder (that is: not that incredibly skilled) couldn’t force through eventually.

So far, the overal dumbness of spambot programmers gave anti-spam plugins a very easy edge. Things will change if real coders start taking an interest in this no-doubt very lucrative market and starts churning out efficient spambots program to the spam monkeys. And do not doubt a second there aren’t or won’t be such black hat developers in this market (the same way there are in other domains of internet spam)… Even if Mark Pilgrim was slightly off the mark in his apocalyptic sum-up of the situation, he was certainly right on one point: there is huge money involved, certainly enough to pay the hourly services of a decent professional coder… perhaps even [cue ominous strings on the soundtrack] a coder already involved in the blogging community.

No, not me (unless I’ve been sleepcoding again).

4. Will any other anti-spam tool fare better than SK2 with this particular spam (or spam in general)?

First off, SK2 is hardly out of the game: even as it is, and with a few tweakings, it can easily be brought back to a satisfying, if not perfect, level of protection. Not to mention a possible harder, faster and better successor to SK2 (more on that later).

As for the rest.

You’ll have to believe me when I say I truly wished for a better offer in anti-spam tools. Far from seeing it as some sort of “competition” (to what? a product I am neither selling nor making any revenue off?), I consider diversity in spam-fighting tools the most efficient way to fight spam. The same way bio-diversity is your guarantee against viruses and germs, presenting a wide array of defense tools to spammers means they can less easily focus their attention on one in particular and try to break it.

What we really do not need, however, is yet another blissfully ignorant moron releasing some stupid 5-line, 3-year outdated, kiddie trick that will not fool a single spammer and waste hours of users’ time. Unfortunately there are a lot of these. So let me go through a quick roundup of what worked, works, and never worked, I’ll skip the details for today, so you’ll have to take my word when I say that:

Captchas: work. Despite the ultra-theoretical “captcha breaking” scheme urban legend, spammers aren’t about to break a captcha on your blog. The big downside of Captchas, is that they are extremely user-unfriendly, intrusive and most of all: hurt accessibility (how do blind users do?).
Pretty much any other plugins won’t work. Blacklists, “spam words”, stupid script renaming tricks and all: all pretty useless taken one by one. Some used to work years ago, all have been successfully broken by spammers. Some are even dangerous by the number of false positives they yield. Just save your time and skip them. Javascript payloads also likely won’t be working (I’d love to hear from anybody currently using such a type of plugin, but I’m pretty sure of this one).
Bad Behavior will not stop these specific spammers. For the simple reason that BB is not designed to filter spam. It is only meant to stop the 70% stupid bots that do stupid things. Unfortunately bots are getting smarter, and the ones you wanna worry about are in the top percent of these 30%, thus far out of reach of BB.
Akismet works. Roughly with the same result rates as SK2. Possibly a slightly higher catching rate, but also a higher false positive rate (which is a big no-no, in my opinion, but that’s up to you). Other concerns generally thrown around include privacy, reliability and terms of use (it is free, but you are entirely dependent on a third party server). My personal issue is that I am doubtful of the long-term resilience of a monolithic DB such as Akismet’s when confronted to both Denial of Service attempts and data poisoning. There is some breathing room until spammers turn their unbridled attention to these weaknesses, but the fact Akismet is now bundled with WP will only accelerates things.

As you can tell, there is scant little out there, only a few plugins that all fare somewhat on a par with SK2, all with their pros and cons. Most important of all, there is currently nothing I wouldn’t feel confident breaking through, was I to start in the business of spamming tomorrow…

Just wire the amount to my swiss account.

I kid.

6. Is there really nothing you can do?

Of course there is.

I have a very fertile imagination, and still a couple tricks to throw in the way of the spamming monkeys, spanning from small bits of tweaking all the way to major, insane and quite possibly break-through concept ideas. Very few in the middle actually. Problem being of course that the more potentially efficient tools would also tend to be the more time-consuming, hazardous ones.

Let me try to sum up the whole state of Spamdom such as I see it, with a tedious numerical analogy:

Say spam-protection goes from 1 to 100, where 1 is “sitting duck”, and 100 is “so protected that Houdini himself wouldn’t get a spam through”. Now let’s say most anti-spam plugins tend to hit somewhere in the 1-10 range, with a few, such as Akismet or SK2, hitting something like a 20 (perhaps also rising a bit as time and improvements went).
Simultaneously spamming techniques have also been adapting and improving, and it’s fair to say they are now approaching a 20, and steadily rising. Essentially, spammers are lazy (or pragmatic, depends on how you see it) and their target is to be just above the anti-spam barrier, not much higher.
Now, among the anti-spam tricks left in reserve, I’d say I got a few small ones that should without too much effort bump SK2 a few points up (with compounded effect, something like a 25), which is nice, but certainly won’t buy more than a few weeks/months.

Since they are also by far the easiest ones to implement, I am already working on them.

There are two other separate projects I’ve been toying, testing and prototyping with: a first one involving a somewhat novel approach to Naive Bayes filtering (definitely not on comment content), which would be a definite +10 on our SpamScale, and another, considerably more complex and difficult to explain in details, that could be crudely summed up as a P2P Blacklisting system.

That last idea I have been thinking through for a looong time now. I have some confidence that it may hold the key to the End of Blog Spam as We Know It… A definite +50 on our scale…

Of course, these last two ones, are also the ones that will take serious time investments before even figuring if I can do something with them… Which takes us to the one and only question you all care about:

7. Why aren’t you busy working on the next anti-spam solution before this spam thing becomes out of control?

Well, because as I said above, it is a lot of work. Work that would add to the top of the already heavy SK2-related workload I deal with daily. Don’t get me wrong, as I’ve stated previously: I love developing, I love developing SK2 and most of the time I love hearing from you (even if sometimes I get irrepressible urges to ram online manuals down some throats). But being a fully human carbon-based entity with little photosynthesis abilities, I happen to need food near-daily…

Also due to recent life changes, I am now a tad busier (being a full-time student) and much poorer (being a full-time student) than before. Hence the regrettable need I am in, to privilege works that either feed me or keep my university peers and professors content.

Can you tell where this is getting? No? OK:

To make it short, I am launching a Fund Drive…

The idea is simple: if you use SK2, if you like it, if you’d like to see more of it in the future, if you’d like this future to be sooner than never, if you’d like to help fund the crack habit of a starving student who also happens to dedicate way too much of his free time to eradicating spam, if you think this is worth a few cents, hell even a few dollars, if you can afford to spend this money without robbing your kid or your cat of their next birthday present… Consider donating:

There are currently a few thousands of you actively using SK2 (yep, crazy huh?)… I figure if we weed out the cheapos and those who honestly can’t afford it, plus those who consider their small use of SK2 not worth a monetary contribution (hey, I don’t pay for all my shareware… I’m nobody to throw you the first stone), that might still leave a few dozens of you? If each one contributes a few bucks, that should be enough for me to justify spending a few weeks working on SK3 rather than flipping burgers to pay for booze (and occasionally food and rent).

Non-monetary donations of any sorts are all gladly accepted: food specialties from where you live (especially if it’s distilled and drinkable, but the solid kind is cool too), postcards and anything else that won’t cause a police raid to my place at 6 in the morning… Note that due to recent health regulations, I can no longer accept your first-born child in payment for services, but thanks for offering.

If, like me, you are a starving student who cannot afford to divert any of your drug money to pay for my costly addictions, then consider donating some time. There will be need for it: mostly in doc writing (FAQs, user guide, maybe even a support forum at some point since the whole 2-hours emailing a day is becoming a bit tedious). Just put your name in and my people will get in touch with your people when the time arises.

If making a donation, please provide a nickname (if you don’t want your full name to be used) and your blog’s address, as I will probably make a donation page to list all those (if any) who donated.

8. Would you seriously stop developing SK if you don’t get money?

Of course not.

But it is unfortunately true that I will have to lower my involvement with anti-spam dev in favour of more, err, survival-oriented activities. Obviously, I’d much rather be paid for something I love doing (like squashing spam and spammers) than any random job… But it isn’t much of a choice.

I guess I should set some sort of imaginary milestones in terms of funding and how far/fast it would take me on the SK3 development trail, but I’d rather not look like a complete moron when all but a fraction of it will have trickled in at the end of the month… So I’ll just give you my word that I’ll do my best with what I get, and probably with what I don’t get either…

No matter what happens, I will be releasing SK2.2 (with minor tweaks and bug-fixing) at some point… Hopefully within a week… The two bigger components will honestly depend on how much interest they raise and the time I can afford to spend on them (we are talking at least month-long projects)…

Oh, and let me remind you that donations are not, I repeat: not, mandatory in any way whatsoever.
This is not a change in licensing: SK2 is and will remain free for all non-commercial use and redistribution (note that you can still use SK2 on a commercial blog, the only restriction is on packaging and distributing or otherwise selling SK2 for profit: in which case I ask that you contact me for permission first).

I also wanted to take the occasion to thank very sincerely all those who have already donated money, time or simply kind words through email: you have made my day on many occasions, and helped making it worth it so far.

Thanks a lot and do not hesitate to spread the word!

Post originally published on: Dave's Blog (please leave your comments over there)

The State of Spam [Karma]

WP-plugins.net Changes

Dave — Sun, 24 Jul 2005 16:53:10 +0000

So it’s 1am on a Sunday night, not much more has happened in term of exciting stuff ever since my latest week-end update (no more earthquake, no tsunami, no godzilla…), I am busy debugging code written by a maniac who apparently thinks that picking random combinations of three to six letters is a perfectly acceptable naming convention for all functions and variables in a multi-thousand lines program (and yea, I’m aware that this last bit means absolutely nothing to a rough 90% of my beloved readers: please color me equally stumped, albeit not for the same reasons)… It’s time for…

Techie Update of the Month

I have been putting in a bit of long-overdue work into wp-plugins.net, the ultimate WP plugins repository.

Namely:

No more ridiculously long loading time of the index page. No annoying clicky-clicky browsing either: thanks to the magic of AJAX (some fancy JavaScript stuff), plugin descriptions are loaded dynamically, without refreshing the entire page each time.
I finally made the permalinks google-friendly (well, human-friendly too, actually), as well as added permalinks per plugin author. From now on, a plugin can be accessed directly by using:
http://wp-plugins.net/plugin/your-plugin-short-name/
(e.g. http://wp-plugins.net/plugin/SK2/)
and an author’s complete collection, using:
http://wp-plugins.net/author/your-account-login/
(e.g. http://wp-plugins.net/author/drdave/).
Old permalinks using IDs still work.
Finally added Google Adsense contextual ads.

Before you start pointing out that Adsense isn’t exactly an “improvement” to the site, let me explain:

I am no huge fan of net advertising in general, not even when used to support a “community service”. If anything, I’d rather see such services supported through donations, as I don’t really buy into the general idea that covering everything in advertisement in the name of free service, is an acceptable trade-off for the loss of a mercantile-free landscape (whether a real physical one, with grass and billboards, or its online equivalent, with html and click-under banners). But that might be the idealistic tree-hugging in me talking.

Anyway, while very targeted projects like Spam Karma have been getting a small but steady flow of generous donations, a site like wp-plugins and its thousands of daily users has yet to generate so much as one spontaneous thank-you email. Note that I am not complaining here: I see the logic perfectly well, considering that, unlike a specific plugin which may do a lot, for few people, this site does a little, for many people. I wouldn’t personally think of making a donation as a casual user of such a site.

Hence: advertisement it is.

So far (from this week-end’s reports), it seems worth the unnecessary clogging of screen estate, in that it should pay entirely for my hosting bills at the end of the month, maybe even a Sapphire & Tonic and a bowl of ramen in Omoide Yokochou… which, when you think of it, is about what I would be paid for my hours in the fields, if I was an indentured servant and the year was 1705.

Which brings us to why this should be taken as an improvement:

Being an extremely venal ilk of Genius, I was starting to consider putting a brake on wp-plugins.net development. While closing the site was out of the question, spending any more time developing and enhancing it, became increasingly hard to justify, in light of my current schedule and alimentary necessities. In that regard, placing ads was/is an experiment, and the fact it seems able to sustain it own cost, means I shall finally be spending some time working on miscellaneous improvements in the near future. Which ought to make us all happier.

In conclusion, feel free to go have fun with wp-plugins.net, hurry up to add your WP plugin if that’s not already done and don’t hesitate to click through the ads if you see something you like (happily ignore them if not). Also don’t hesitate to contact me (through the form or in the comments) for bugs or feature requests…

Post originally published on: Dave's Blog (please leave your comments over there)

WP-plugins.net Changes

WordPress 1.5.1 and Spam Karma 2.0

Dave — Tue, 10 May 2005 09:24:31 +0000

WordPress 1.5.1 is now officially released. If you are a WordPress user, you really ought to upgrade. This version fixes many of the bugs and shortcomings that were introduced with the botched release of WP 1.5. It takes seconds to upgrade from 1.5 (just overwrite everything in your blog directory except the wp-content directory). Shout out to all WordPress developers and contributors… Great job guys…

With this, I am glad to announce the official release of Spam Karma 2‘s first public beta.

In fact, it is pretty much final-grade quality and could probably do without the “beta” label… I’m just a big fan of the greek alphabet.

Many of the lingering issues with the last alpha have been fixed, a few missing features have been added (it now supports curl for those whose host doesn’t allow url_fopen). Check out the dedicated page for details.

Now go and spread the word! there are still far too many clunky SK1.x in the wild out there…

Also, feel free to contribute to the newly-opened official Wiki page for SK2: your help is much appreciated!

Update: Ahem, it would appear I spoke a bit fast. There is a rather nasty bug in this update that can bork your RSS feed. I do recommend updating nonetheless (and follow instructions in the link above to fix the bug).

Regarding that bug, let me add a little word. Honestly, I considered skipping this rant (and I waited quite a bit to ensure it really was what I was thinking about), but I am just to aggravated to keep my mouth shut. Consider this my final word on that branch of topic, though:

Obviously, and as usual, this blatant bug could probably have been avoided with a bit of real-world deployment strategy and testing before release, but I won’t even bother restating my opinion on the matter… What bears mentioning, though, is the fact that I did report it personally, more than 2 weeks before the release date.

The bug is at least somewhat directly connected to this change. I noticed it, shortly after the code was changed in SVN, emailed Matt about it, didn’t really get much of a response (apart from an invitation to basically go and fix it for him) and considered it none of my business any more (I had way enough things to take care of at the time).

It didn’t even cross my mind, though, that this code, after being clearly reported to Matt as broken, could make it seamlessly and without apparent revision into what was supposed to be a bug-fixing release.

Quite frankly, I feel rather insulted, if anything…

Post originally published on: Dave's Blog (please leave your comments over there)

WordPress 1.5.1 and Spam Karma 2.0

Angry Emails

Dave — Sun, 17 Apr 2005 03:48:41 +0000

I recently explained my decision to start packaging and distributing my own version of WordPress in order to make my life simpler, as to the distribution of Spam Karma 2.

Simultaneously, I had to deal with an outbreak of issues (or more exactly vocalizations on these issues: it’s unlikely the issues have been any more common than before. They were just given more attention on these pages) with Spam Karma 1.

I have taken the time to write a very detailed response to cover all aspects of the False Positives problem, and provide anybody coming to me with a comment denied by Spam Karma, with the Why?, How? and What Now? of their problem. The bottom line being that, while I’m sincerely sorry about these, I am not responsible in the slightest for the use people make of this program (especially considering how easy it is to screw up the install when you hold yourself too smart to read the docs) and even less so now that there is an upgrade available.

But I care, and think I have proven it in the past, to whoever bothered contacting me without resorting to death threats immediately.

There is, however, one way to make me not care about your problem… It goes a little bit like this:

Act 1 (Exposition):

From: ***
To: drdave@infamousgenius.com
Date: April 16, 2005 8:59:07 PM JST
Received: from some AOL customer by ***.mx.aol.com [...]

I resent being identified as and accused of being a “nasty comment” spammer. For YOUR information and edification I have never ever, sent or been involved with the sending of ANY spam. To find that I have been blocked/banned from a site as a result of YOUR program is an affront I won’t stand for. I would refer you to the remarks posted on your BLOG whatever for details of what I originally wroted that was intercepted by your program, and deleted. I fully expect that YOU contact the site administrator of the site I am apparently now banned from and straighten out the problem that YOU and YOUR software are directly responsible for. In the event I do NOT hear from YOU regarding this situation, rest assured I WILL file complaints with the FCC for interception and diversion of lawful communications be it direct or indeirect as a consequence of the code which you wrote and distributed with and for express intent and purpose of intercepting and diverting communications. I am truly sorry there may be issues with SPAM, but that in and of itself does NOT empower YOU or anybody else to interfere with lawful communications, period.
Signature removed for the protection of the criminally stupid

Act 2:

From: drdave@infamousgenius.com
To: ***
Date: April 16, 2005 9:04:04 PM JST

http://unknowngenius.com/blog/wordpress/spam-karma/false-positives/

Signature removed for the protection of my own mailbox

Act 3:

From: *** (yea: still the same, in case you have difficulties matching his inimitable style)
To: drdave@infamousgenius.com
Date: April 17, 2005 4:17:05 AM JST

http://unknowngenius.com/blog/wordpress/spam-karma/false-positives/

???

Nothing there justifies, or remediates the fact your program, or version thereof, has unjustifiably filtered out and mislabeled a proper communication, and further mis-identified me as engaging in unlawful conduct. Causing my name and information to be listed and blocked from a site for no reason.

You wrote the program that filtered my message, that intercepted and deleted such communication, and quite obnoxiously in scope broadly misjudged me to be a malcontent and wrongdoer.

You authored the accusation casting negative aspersions of my being a nasty comment spammer, which others see and believe to be accurate because a computer program said so, hence must be true regardless of fact.

I expect you to contact the administrator of the following site and clarify to them that, I am NOT a spammer and, did NOT post, or in this case attempt to post, a nasty comment or spam. That NO evidence involving my attempted posting supports the false-positive finding and accusation against me as set forth by your program.

[some random blog URL removed]

It is not my place to inform users of your program of false-positive discrimination issues, nor my place to remotely mention or suggest altering detection settings as described in your response page. You may disclaim any liability to users of your program for any consequential damage, however, I am not a user, and you cannot go around making and publishing false accusations of unlawful conduct, orcausing to be made and published as your documented program has clearly done. You can’t simply disclaim liability for harm or dismiss collateral damage to third parties as “innocent bystander(s)” caught up in some crusade.

As a consequence of your actions, direct or indirect, (i.e. the program you wrote and distributed). I have been falsely accused of a malicious and illegal activity (spamming) and unjustly harmed (denied access to a public forum). To whatever extent direct, commission with intent, or indirect, by neglect, error, ommision, or simply inadequate logic [more likely] the net result being the same, significant precedence abounds, it’s wrong. As such I don’t think it unreasonable to expect you to clear up what your program caused.

Signature removed for the protection of the increasingly criminally stupid

Act 4 (Denouement):

From: drdave@infamousgenius.com
To: ***
Date: April 17, 2005 12:50:51 PM JST

Dear non-spammer,

Not only do you seem to be lacking proper reading and cognitive abilities, otherwise you would have realised that this lengthy response I took the time to write (while I probably could have been doing much more enjoyable things, such as sleeping), did in fact answer your questions at length, but you are also highly annoying, if only by your constant use of pseudo-legalese verbiage in these highly redundant communications.

I will draw your attention to the fact that, unlike your completely unreadable emails, I paid particular attention to formulating my own answer in a somewhat easy to grasp, if not easy to read, form.

While I am certainly prone to much understanding toward whatever tone is employed by people contacting me on the spur of the moment, regarding these matters, my tolerance quickly fades away when it becomes obvious I am not dealing with some reasonable user, justifiably irritated by what he just experienced, yet able to comprehend the matter at hand if explained calmly, but with some perpetually angry and bitter individual, whose entire life seems devoted to bending the world to his liking, through angry communications and the use of big words.

Your tone and aforementioned use of ridiculously out-of-place paralegal terms, lead me to believe you are one of two things:
1) A lawyer or closely associated profession. a US-residing one at that. and quite likely of the ambulance-chaser persuasion too.
2) Just one of these thousands random morons inhabiting the US, who dream of suing their way through life and have successfully thwarted any attempt at creativity that has ever come in the past two decades and didn’t have 200 lawyers to back it up.

The fact that you seem to think you have something of a legal case here, would imply an absolute ignorance of the most basic tenets of the legal system, and therefore: 2). But I’m gonna give you benefit of the doubt and say that, after all, you might be a very stupid lawyer, or more likely, under the delusion that lots of redundant wording and liberal use of UPPER CASE is gonna impress me into sending you a settlement check before I go and personally contact each and every user of my software and remind them that, exactly as the documentation states, this software might actually stop a commenter that, despite their obvious lack of manners and potentially sub-average IQ, is not, in any way attempting to spam.

Well guess what? I am not very impressed. nor scared.

So please feel free to get in touch with my lawyer, but please be sure to bring an interpreter, because he only speaks Japanese. and he hates Americans too.

Just because you seem to be into that kind of legal grassfucking, allow me to remind you that we are talking about a software that is distributed with the explicit mention of “AS IS” and “WITH NO IMPLICATION OF LIABILITY WHATSOEVER” (see, I can use all caps too).

As to why your comment was stopped, since you might still be curious about that one (and probably didn’t bother to read through the link above, where you might have found an answer to that question):

Why I’d love to say there’s some sort of design in there to detect and thwart any attempt at commenting based on the IQ of the commenter. It’d be both exaggerating my skills as a developer and insulting the numerous other courteous and reasonable people whose comment happened to be stopped by mistake on any other occasion… Therefore I’ll give you the quick and entirely accurate answer: you are an AOL user (and while I can see how this might sound like a confirmation of the initial IQ-based filter theory, let me assure you there is nothing but the simplest of technology involved here, and this particular aspect is fully documented in SK’s manual).

To conclude this waste of 20 perfectly fine minutes of a beautiful sunny Sunday (well, at least in my corner of the world. Judging by your attitude, I suspect you might not be experiencing the same warm Spring weather, wherever you are hailing from), I will hereby kindly inform you that a properly anonymized version of this exchange will be gracing the pages of my own website, as you are reading this. There again if you feel like legally objecting to this, feel free to contact my lawyer: he is definitely worth getting acquainted with (ask him to tell you these great stories of how, as a child during the war, he used to be sent on vacation to Hiroshima island with his sister)…

Otherwise, please accept my sincerest apologies regarding your misfortune and receive my complete assurance that I will do absolutely nothing about it.

Oh yea, and do get a life.

Karmically Yours,
Signature removed for the protection of my own mailbox

Or, to quote one of my most eminent role-model in life: This conversation can serve no purpose any more. Goodbye.

Update: Because who doesn’t like a cliché movie-ending twist where the supposedly dead-and-buried evil guy springs back to life for a last ~~fright~~ laugh, you can check out the epilogue in the comments….

Post originally published on: Dave's Blog (please leave your comments over there)

Angry Emails

WordPress Upgrade available for Download

Dave — Fri, 15 Apr 2005 12:01:33 +0000

As I just lengthily explained, I have very little love for corporate hierarchies and all the folklore that goes with it. Ironically, the ostensibly “free” world of major Open-Source development never fails to irk me in the exact same way…

I have my theories about that, and they mostly have to do with the fact that Open-Source is full of the very same people, who, for one reason or another, might not have made it at the top of the corporate ladder in the traditional world, but are fiercely decided to enjoy the exact same privilege in their own version thereof. Less money, therefore a void to fill, usually by inflating the ego until it touches on all sides. These are all theories… But I do not care to expand on them right this second, if only to add very quickly that Open-Source Software is also filled with an overwhelming majority of selfless, dedicated, bright people, who, in the end, bring the balance way into the positive.

Yet, I do have my beef with many OSS practices in general, and practitioners in particular, but because I have yet to find the recipe for immortality (and just in case you are looking for it too, I can tell you so far that neither baths in young virgin’s blood, nor daily consumption of half a gallon of gin, work), I do my best to spend as little time as humanly possible on such crap. Sure I yammer. a lot. but I am also quite good at packing my marbles and moving to another side of the playground without sulking too much, whenever I really can’t take my little playmates’ bullshit any more…

In short, we, at Dr Dave Logs Inc., do our best to toe that delicate line between diva whims and lonesome-cowboy-fading-in-the-sunset attitude… something that’s not always easy (especially with these damn stiletto heels on). Mostly, that means staying on the edge of the soccer field, practice against the wall, address a few invectives at the lousier moves every once in a while, and go fill in for the team when there’s an emmergency. Nothing to be particularly proud of, nothing to be ashamed of either. We quite like it that way, and do not intend on stopping any time soon (neither the speaking of ourselves in the plural form, for that matter).

This is where the author stops, goes back and count number of preceding paragraphs. 1… 2… 3… and… 4 !… paragraphs of disclaimer…
I think we are good to go now!

When I went to bed yesterday, I was pissed. Very tired (physically, emotionally and mentally), and very pissed. I thought better than posting something then, and anyway, my eyelids firmly refused to stay open, which is kind of a problem to check what you are typing. But I was dead set on opening the shit sprinklers to full power today, as soon as I would have a second.

And then…

You know what’s ironic with my decision to stop my daily bottle of Tequila in the morning for a month or two?

You could expect this to make me somewhat cranky: what with the really boring immobility of my surrounding, the apparent emptiness, now that I only see one of each object, the awfully bland taste of my morning Orange Juice, when it’s not mixed with Skyy vodka, all that… Despite all that: I feel completely euphoric these days…

So euphoric, in fact, that I will not, as initially intended, post a ten-page rant on my current issues with the way WordPress development is done.

A rant that would have continued exactly where I left it a few months ago, when I went vocal with my concerns that lead devs were basically wiping themselves with the hands of plugin developers. Contending that, while all projects need leadership, the idea of community development is also for dialog to happen, not for one to decide and all to follow.

Hell, had I felt particularly mean, I might have viciously pointed how a little bit of humility sounded like a good idea at the moment. I would have pointed out why it is fucking rude beyond belief to blatantly ignore insistant, yet polite, requests for more disclosure, on all matters… And not talking about detailed bookkeeping either: very silly basic things, like giving some sort of a roadmap and timeframe that isn’t written on the back of somebody’s personal agenda.

All sort of mean things really…

And then I would have followed by the announcement that I was hitching a ride out of the WP game. Perhaps not a fork (god, oh no… the day I betrays my faithful chopsticks for a fork, feel free to stab me with either one), but at least a wise retreat to my own grounds, where I no longer have to depend on anybody’s whims but my own (and it may be a lot already, but I tend to have more patience with those).

That was more or less the plan…

Instead, this morning, I just brought my laptop out in Tokyo’s long awaited shiny Spring weather, sat in my garden, and took 30 seconds to write a blindingly stupid little script.

All that script does, is download all code from the SVN server sitting there, repackage it a bit (roughly: removing a few useless files), and upload it here. OK, it does a few other things, but I’ll only be able to tell you about this part in a day or two when I’m done with that part.

Why did I do this?

Because I was fucking tired of having a fairly decent plugin sitting on my server, while everybody kept on downloading an obsolete version that hadn’t been worked on for many months now, thus ensuring a continuous flow of insults to my mailbox. All that because the software it was meant to work with, 6 full months ago, still wasn’t able to release one simple, correctly planned, working version. And sure, “it will be out any day now”, I know that. But the fact that “nobody” deigned giving me a simple answer as to when exactly, meanwhile giving me to think, ever so faintly, that one possible reason to delay a release could be to basically pull the rug from under SK’s feet (I know: this is an unlikely one, but the mere thought is quite infuriating).

As it turns out, in the game of rude behaviours and unilateral decisions, I am not so bad either. Sure, it’s kinda childish, but certainly not gratuitous:

So, please, go ahead, enjoy and upgrade both WP and SK right now…

More to come soon.

Post originally published on: Dave's Blog (please leave your comments over there)

WordPress Upgrade available for Download

Frustrated Users and New Developments

Dave — Fri, 18 Feb 2005 19:41:25 +0000

This entry was originally gonna be a comment posted on Dave’s Chalkboard in response to this post. But then I realized it had taken the size of a novella, and furthermore, most of its content is probably relevant to other people too. So here goes another entry about Spam Karma…
Sorry, I know this is getting tedious, I’m tired of talking about it too… I promise this is the last time you hear about it until I finally get off my ass and release SK 2.0

First of all, believe me I am the first one sorry to hear that some people are being consistently singled out by SK: it is far from a perfect tool, especially in light of recent changes brought by WP 1.5 (regarding trackbacks for example, WP code was changed in a way that runs all trackbacks through comment filters, not a great decision imho, since it has the result of breaking lots of backward compatibility with filters that were only intended to work on comments)…

In your case, I suspect your IP might have ended up on some major Realtime Blacklist (RBL) servers such as Spamhaus: these lists are stuffed with false positives and are not under my control and I actually advise people to turn that filter off unless spam really keeps coming through.
Another very likely culprit is the use of a proxy server that mask (or changes) your IP. This is a definite comment killer… Since it’s the signature move of spambots trying to spoof IPs. Not using this criterion would make spam-filtering nigh impossible… I’ve been looking into ways to detect friendly proxies and force them to use the same IP and not cache the page, without success so far…
Setting SK on “lenient”, as somebody pointed out, is probably a good idea too…
If you want to contact me directly by email and try posting a test comment on my blog, I’ll be able to tell you exactly why it’s not working (in fact, anybody could tell you, since they’ll receive your comments in their SK digest, along with detailed headers).

Anyway, trust me, I’m the first one unhappy with SK’s imperfection, especially given its high rate of adoption these days: there’s nothing more frustrating than seeing bugs you don’t have time to fix, being downloaded by hundreds of people…

I WILL resume development, and I believe I can bring SK much closer to a 0% false positive score, which was the initial goal (yea, we kinda drifted, somewhere along the escalating arm race, when it became so annoying to deal with spam, that I really had to crank the filters up).

In response to Adam, I am actually both studying and trying to make a living full time, which, along with attempts at preserving some kind of social life, leaves little extra time for side projects…
However, SK is absolutely open-source (MIT license) and anybody is free to take the ball and run with it (with proper credits etc, of course). As for a more coordinated effort: I have occasionally been getting help and snippets from people, but the size of SK (it’s totalling around 3000 lines of PHP right now), makes it a non-trivial coding project and requires some level of involvement. Plus its code has mutated into something rather horrible, over successive versions…
So I haven’t really found anybody willing to put that much effort yet.
That being said, and even though I have officially pulled the brake on support and development for now, I usually make an effort to integrate any snippets, diffs or bug fixes sent to me…

Regarding Referrer Karma: I probably shouldn’t open my mouth again, but I will, and I’ll say that the potential for user frustration and overall false positive banning is much, much, less than with SK.
First of all, RK does not ban users: worst case scenario, they are simply asked to click on a redirection link (that rids their http query of its potentially spammish referrer). And this will only happen when two important conditions are not met:

Their referrer is not on any of the default whitelist or hasn’t been whitelisted by a previous successful attempt.
The referrer URL is reachable, but doesn’t contain their URL.

At the moment, the only major source of false positives is webmail servers, since it’s impossible for RK to check these. But this is why there is an extensive whitelist, and I am trying to slowly add all major email and search engines. In the meantime, once again, the worst that can happen is that people have to click on a link to see your site (and you can easily whitelist the referrer in your settings once you spot them). I might work on better auto-whitelisting in the future. For now that’s all there is.

I have also added some (fully optional) integration features to make it use SK’s IP blacklist.

Basically, the concept being to stop lying there and taking it while thinking of England…
Even when spam comment do not make it to the blog, their relentless attempts eats up heaps of Bandwidth and CPU (especially, I suspect, with SK’s heavy filtering process in the middle). RK’s new version simply blocks them at the door, before any serious computing starts… and there again, does it intelligently, since users can easily unblock themselves and see your site by merely clicking a link. Check out Referrer Karma‘s page for the latest details on this feature.

Post originally published on: Dave's Blog (please leave your comments over there)

Frustrated Users and New Developments

Almost Famous

Dave — Wed, 16 Feb 2005 20:26:20 +0000

You know, despite my best efforts, I have never managed to reach celebrity status in Japan… and when I see the kind of humiliation my friend the “talento” has to go through, every time he is featured on TV (full story for another day), I am quite happy with that.

However, I can now die a happy man, since a small part of my work has finally been recognized for what it’s worth, and featured in a Fijian Newspaper. Yea, you read correctly: I am a star in Fiji.

Well, kinda.

You can read the whole story on Nacken‘s blog. Basically, Nacken’s got into a polemic over some censorship issue on Fijian TV, which brought him the visit on his blog of a well-meaning advocate of public decency. That dear lady, quite naturally tried to leave an answer to Nacken’s plea for open-mindedness and mature attitude toward adult themes on TV.

And that’s when things take a comical turn:

You see, it would appear Fiji has a strong propension toward triggering Spam Karma‘s filters, I think this is mostly due to the proliferation of proxy servers in order to reduce bandwidth costs (proxy servers can sometimes mask users’ IPs, which doesn’t bode well with Spam Karma at all, since it is a way too common mark of nasty spambots too). Or maybe that lady did something stupid that looked suspiciously like spam to SK.

Anyway, you guessed it, the poor lady’s comment was swiftly killed and she was presented by that now infamous line, which, I am sure, must have infuriated more than one innocent would-be commenter throughout the world:

Sorry but the total effect of your comment’s actions and conduct during the successive phases of its existence, did not justify giving it a second chance in this world of pain and misery…

In other words: it got trashed.
Bad Karma, man, bad karma…

Of course, that lady – bless her simple soul – naturally assumed this was Nacken’s real time answer to her comment, and was shocked at such a blatant censorship from the man who had risen against her own plea for censorship. Apparently, Nacken’s explanation over the phone did little to lift that confusion, because the next day, the paper printed this letter…

Look! They quoted me!

All right, quite obviously we are not getting new Spam Karma users from this article, but this is still totally awesome. I might have Nacken mail me a few copies of this article to frame and send to all the relatives that told me I was stupid when I dropped out of school to focus entirely on that “get rich and famous through cockroach breeding and plugin writing” scheme. Now who’s laughing, huh?

Ironically, I had just released this afternoon a small upgrade for Spam Karma with a much-needed toned down deny message (see change logs for details). After all, spambots won’t read the message, and I don’t need to be an household object of hatred for the next 10 generations of frustrated bloggers, so a little politeness and explanations cannot hurt.

In other SK news, a quick Google search indicates that there are about 164.000 pages mentioning the string “spam karma” and 85,000 pages featuring SK’s standard “Advice to spammers” message. I have only one thing to say: Wow. Thanks.

I also guess that means I should get off my ass and start working on a kick-ass version 2.0. And I will. One day…

To be honest, I am quite ashamed by the current level of result of SK 1.x… It needs some serious reworking, and the addition of a few filters I have been pondering for the past few weeks. Unfortunately, gainful employment still has the priority over fun spam-kicking projects, so it will have to wait.

Post originally published on: Dave's Blog (please leave your comments over there)

Almost Famous

Spam Karma Zen

Dave — Mon, 07 Feb 2005 05:15:59 +0000

Found this awesomely dorky time-waster through Cosmic Buddha and here is what it’s got to say about Spam Karma:

This new pizzle is destined ta become tha permanent news repository fo’ all th’n Spam Karma
[...]
Spizzay Karma in tha dogg pound.
[...]
Spam Karma cuz this is how we do it.
[...]
Spizzam Karma . Ya fuck with us, we gots to fuck you up.
[...]
You can leave a response, or trackback fizzy yo own site fo gettin yo pimp on.
[...]
S-P-to-tha-izzam Karma has gizzle nuts . Relax, cus I’m bout to take my respect!
Chosen excerpts from search results

Now I ask all the spammer beeatch out there, you sure you wanna fuck with my homies?

Post originally published on: Dave's Blog (please leave your comments over there)

Spam Karma Zen

Sick of this Crap

Dave — Thu, 13 Jan 2005 22:14:20 +0000

Yea, me too.

As I mentioned many times before, recent development and support on both Spam Karma and WPPM have taken a serious toll on a schedule that certainly didn’t need the extra excitement.

On an average, I receive over a dozen emails/comments a day regarding SK or WP-related support. A good 90% of which are usually RTFM-related and not in any way due to a bug in SK. Lately, I have spent upward of two hours, every single day, dealing with plugin development issues (mostly SK). Very often to come to the conclusion that the bug I’m going after has been introduced by some changes in WP’s code, user hacks, exotic server configurations or any of the hundred parameters I have little control over.

And this, of course, for the mere glory of it all. Because it is doubtful I will ever make a buck off it (and that’s really not the goal), nor is this type of development ever likely to impress anybody reading my resume (the kind of people who employ me usually, ignore until the very meaning of the word ‘blog’).

But this is quite alright.

The many thank-you notes, sincere props, pitches in the tip jar, as well as the personal benefit from using these tools on my own blog, definitely go a long way toward making it worth my time. And I am certainly not gonna start complaining because a project of mine gets some amount of popularity. User adoption is indeed the greatest form of appreciation for one’s work.

Why am I putting Spam Karma’s development on hold, then?

Well, because on top of being something I barely have time to do, developing SK has recently become one of the most excruciatingly irritating experience of my whole life as a developer.

Hang that on two reasons: 1) blogs 2) WordPress.

“Blogs”: because developing a blog-related tool means that, whenever a bug or incompatibility comes up, instead of doing the normal user thing and contacting the developer with some data regarding occurrences of the bug, most bloggers simply decide it’s infinitely more productive to bitch about it on their blog, addressing a readership that most likely can’t do anything about it and doesn’t care. This would only be half-irritating if not for the fact that the problem usually turns out to be coming from the user’s failure to read the docs or his less-than-standard setup.

Note that I even went to the length of explicitly mentioning this problem in Spam Karma’s doc. But then again, the people who jump on their blog as soon as they encounter an issue before posting the slightest comment or email are also the ones who usually skip on the doc-reading part.

Second and principal reason why developing SK is becoming more of a drag than I am willing to handle: WordPress.
WordPress is a wonderful tool, it is probably the best publishing platform available out there at the moment. Essentially thanks to the huge community that revolves around it: developing plug-ins, patching the code and guiding new users. Literally hundreds of people putting their time to the service of the community.

However, I have a serious beef with the way its development has been going lately: chiefly, I am getting sick and tired of discovering massive changes in ostensibly alpha-phase code, every other morning. Changes that are neither discussed nor announced on any of the main community channels. Announcing major alterations to the code architecture is not only simple courtesy toward the people working with it, it is also bloody common-sense, if you hope to keep them interested in contributing.

This goes along with the overal flakiness of the release scheme (there again: absolutely devoid of any communication) and the fact that, rather than fixing fundamental flaws, current development seems to focus essentially on adding trendy features, overlapping existing ones (please just do not ask me one more time why there is a wp-plugins.net and a wp-plugins.org) or making sweeping, untested and half-efficient last-minute changes to the code.

Well, the development leads are, after all, perfectly entitled to conduct development the way they see fit. It is their utmost right.

But it’s also my right to be pissed and tired of swimming against the flow: I am just not that interested in dealing with this particular brand of ego-tripping any more.

And this is why, until further notice, you can consider development on Spam Karma frozen.

I will try to release the last bug-fixing version I have ready, but this is nowhere near my list of top-priorities for the day. It is also unlikely to fix whatever has been broken by the latest release of WP alpha 1.5: consider SK officially incompatible with 1.5 alpha.

This is definitely not to say that I consider SK in its current incarnation to be an efficient and sufficient answer to blog spam: truth is, it is barely keeping afloat of the latest spambots (which is already better than most, but probably won’t last). I know very exactly what is needed and what could be done to bring it back up to a satisfying level of efficience: I might get back on it some day when I have both time and motivation.

I am not stopping all spam-related development. I actually plan to keep working on that project I have been alluding to for a while. If only because, at its essence, it is intended to be entirely blog-independent. Therefore avoiding me the kind of WP-related frustration mentioned above. We’ll see in the future about adapting it for WordPress or possibly even integrating it with an hypothetical 2.0 release of SK…

In the meantime, for all of you happy with the current release of SK, it should keep doing a decent job for as long as you stay away from CVS code. After that, I believe there should be many an alternative available.

Now if you will excuse me, I have a long awaiting date with a bottle of Bombay Saphire…

Note: I also want to use the occasion to introduce the awesome Documentation for Spam Karma kindly put together by Johnathan Abad. Definitely go check it out: it’s full of useful stuff for every level of users.

Post originally published on: Dave's Blog (please leave your comments over there)

Sick of this Crap

Spam Armament Race

Dave — Wed, 05 Jan 2005 23:48:17 +0000

From my childhood readings I remember this particular chapter off Jules Verne‘s visionary masterpiece From the Earth to the Moon.

The first volume would go over the fabrication of a humongous cannon being built to send a few adventurers on the Moon (yea, this is 19th century science-fiction all right, but not as far off as one would think) and included an historical background of both the main character: Impey Barbicane, founder of shots during the Civil war, and his personal nemesis-turned-ally: Captain Nichols, founder of armored plates. That one chapter described extensively the armament race that opposed the two men through their new inventions on the battlefield.

Every time the cannon grew bigger, the armor became thicker, and vice-versa.

Anyway, spam strongly reminds me of that. Just when everybody started enjoying a deceivingly quiet reprieve in the Spam Wars, the filthy baboons come back and hit again, harder, and nastier. This time using a different angle.

Nothing surprising, if you ask me, although I was kinda hoping that TrackBacks would have long been made a bit tougher by default. There goes my hopeless wish for more long-term prevention, less short-term patching…

Well, the latest Spam Karma update does little for the long-term, but I think we can all agree the situation is bad enough that anything short of unplugging that ethernet cable will have to do for now. Spam Karma 1.15 should quench the torrent of spammy mud that is currently pouring over WordPress blogs. Make sure you read all the details in the Change Logs regarding the activation of the new TrackBack filter.

However, do not expect miracles: First, TB spam works much differently from Comment spam. By the time a TB makes it to Spam Karma, a lot of useful information are no longer available, while, on the other hand, filters relying on the presence of a human (such as Captcha checks) are not possible either, since TBs are meant to be automated. The solution, imho, would be for WP to ditch TrackBacks more or less completely in favour of extended PingBack support (harder to spam). In the meantime, it would be nice to have more control over each feature separately in the Options.

As I previously mentioned, I have great projects for the future. Maybe a bit too great, since I still haven’t gotten any closer to finishing to put my notes in a coherent, readable format, let alone implement a proof of concept. Truth is, for as much as I enjoy helping the WP community and squashing spams left and right, the popularity gained by Spam Karma has brought it to a level where supporting each request and working on new improvements to keep up with nasty Spambots is fast becoming a full time occupation. At a time where my current combination of more-or-less gainful employment, university stuff and girlfriend damage-control is hardly calling for more.

Don’t worry: I am not giving up on Spam Fighting or Spam Karma quite yet… Only asking everybody to be patient with their problems: I will work on every outstanding bug and issue that I can put my hands on, while at the same time trying to move forward with development of an efficient and coherent Spam fighting strategy. By all means keep emailing me about possible bug, but please read carefully comments and manual before to ensure your problem hasn’t been solved by somebody else already.

We shall prevail.

Post originally published on: Dave's Blog (please leave your comments over there)

Spam Armament Race

Tip Jar

Dave — Wed, 08 Dec 2004 17:52:34 +0000

Update: Please note that this was originally written for Spam Karma v.1 and therefore some elements (licensing especially) no longer apply to SK2. Same spirit overall though.

OK, so before I go any further, let me make things as clear as I can:

Spam Karma is and will always be absolutely free. Free as in beer, free as in love.

So free, actually, that you could practically take it, change twenty lines and start selling it as yours. Except your own karma would probably shrivel as a result.

Anyway, the gist of it is that you are under no obligation, legal or moral, to pay anything for its use. In fact, you are entirely welcome and encouraged to use it for free. Though a supportive e-mail or comment is always nice too.

Now that we got that part out of the way:

Lately I have been submerged by people inquiring on how they could repay me for all the great work I have done on Spam Karma and all that.

Ok, so maybe not exactly “submerged”, but I got two emails and one comment in this direction today, which is more people offering to get me things than during the whole three months of my stint in a Tokyo strip club.

Not one to disappoint, and because bandwidth doesn’t come that cheap these days, I figured there should be no harm in helping all the people willing to part with a dollar or two, by providing this Paypal link:

An Amazon Wishlist probably would have been more adapted, but I really do not have the time to compile a list right now, plus we all now I’d sell these books for crack anyway.

Let me add that, while you can be sure that this money will be put to good use (buying a gallon-sized bottle of vodka or a replacement hip for my disabled grandma, whichever I can afford first), it won’t actually get you anything beside my eternal gratitude: neither the slightest guarantee that I’ll keep developing Spam Karma, nor any kind of preferential treatment with your bugs. Both development and support will be utterly unaffected by this and will keep depending solely on: 1) level of anger of my deserted entourage 2) time I can afford to devote to support without making it a fourth full-time day job.

I am not saying that if the donations reached insanely high proportions, I would not drop most other projects to concentrate solely on this one… But I’d more likely just blow the money on a plane ticket to the nearest beach paradise in the southern hemisphere never to be heard again.

Anyway, feel free to send your spare dollars, yens, pesos or traveller checks if you really really enjoyed using Spam Karma, but do not feel like you have to (and my grandma doesn’t need this hip prosthesis so badly anyway). I also accept first newborns, but I need a release form with them.

Oh, and don’t forget to go add your comments to Spam Karma 2.0 Call for Requests.

Post originally published on: Dave's Blog (please leave your comments over there)

Tip Jar

Spam Karma 2.0 Feature Requests

Dave — Wed, 08 Dec 2004 17:35:05 +0000

First thing: there is now a static page entirely dedicated to Spam Karma. Among other things, it will always contain the current version number as well as links to other relevant piece of information.

Now that we pretty much got Spam Karma 1.x nice and stable, it’s time to get ready for 2.0!

Below is what I have more or less already planned for it, please feel free to add your own wishes, desires and suggestions in the comments.

Improvements:

Automatically clean-up captcha graphics (yea, long overdue).
Move string and regex matching to SQL (gotta check if that one is really worth it, performance-wise).
Log time and date for each deleted spam.
Improve Digest presentation (colors, formatting, perhaps some css trickery to hide details by default).

Features:

Provide “restore deleted comment” feature.
Send realtime blacklist updates back to a central server.
Implement some sort of P2P update protocol for the blacklist (huh, that one might be for 3.0)
Run check for open-proxy servers on suspicious IPs.
Option to appear on the central Spam Karma High Score website.
Flash implementation of Space Invaders where you can destroy little spams falling from the sky (just kidding… or am I?).

Filters:

Majorly kick spammer’s ass (ban IP etc) if comment is on a non-existing entry ID (should have been done long ago, but I didn’t realize WP 1.2.1 still didn’t fix that exploit).
Parse suspected Spam URL target page and look for spam words.
Check user-agent (with a new category in the blacklist table).
Parse href title and content and check for blacklist matches.

Ok, what else?

Post originally published on: Dave's Blog (please leave your comments over there)

Spam Karma 2.0 Feature Requests

Introducing Spam Karma

Dave — Fri, 19 Nov 2004 05:06:02 +0000

UPDATED: 12/09/2004 15:46 JST From now on, please check the central Spam Karma page to get the latest updates and news on this plugin.

Yet another techy update for my fellow bloggers using WordPress.

Now that it’s reached version 1.4 and that most (all?) major bugs have been ironed out, I feel it’s time to introduce the latest member in the ever-expanding WordPress plugin family…

Spam Karma is a mean critter that truly enjoys killing

In fact it is so mean that we had to keep it in a special military-grade containment unit on this server.

Genetically engineered in the dark recess of our Secret Spam Research Labs and trained through months of reflex conditioning and shock therapy, this thing, once unleashed on your comments, will only let go of its death grip after the last spam has been shredded to pieces.

We haven’t fed it for a week now, and it could smell spam miles away in its sleep.

But while a fierce and merciless spam killer, this plugin is also a perfect companion for your kids and friend’s comments. Only the unmistakable foul stench of spam will trigger its ire… while questionable, yet potentially legit, comments will always be given a chance to clear themselves before being irremediably disposed of.

If you are using WP Plugin Mgr, install is as easy as a click on the “Check Updates” button and a click on the “One-Click Install”… Yep, that’s all.
For those still stuck in the last century, a manual install archive is available here. Please, please, RTFM: it’s short, sweet and contains essential details.

Once installed, make sure you check at least once the Option screen (in wp-admin, click on Options >> Spam Karma).

I strongly recommend you check for updates (if you are using WPPM it will do it automatically for you) at least once a week so as to make sure you benefit from the latest bug fixes I might make.

Spam Karma v. 1.4 is now compatible with WordPress 1.2: however due to the lack of certain functions in WP 1.2 Plugin API, some of the features are missing (Option Page integration etc). It is fully enabled for use with any fairly recent release of WP alpha 1.3.

Cool, but How does it work?

Layman’s Explanation

Spam Karma works by running every new comment through a battery of filters and checks. Each of which increase or decrease the comment’s ‘Karma’ value. Depending on the final score, the comment is either:

Approved
Discarded silently as spam (no email is sent to you, unless you specifically require it, but a digest is sent to you every X spams deleted).
Placed in Moderation mode. With the possibility for the commenter to auto-moderate his own comment by proving he’s not a spammer (by filling a Captcha or checking a confirmation email).

This whole process insures (by order of priority):

No deleted false positive (bad bad bad).
Extremely few moderated false positives (annoying): uses Captcha and email auto-moderation to keep these at a minimum.
No published spam.
very little spam held in moderation (must be destroyed directly: really annoying to have to moderate it).

Further more, Spam Karma works in an intelligent way to automatically update its filtering database and grow stronger with each spam it catches…

In short: blocks spam with no unnecessary annoyance, for you or your visitors. The way it should be.

The Detailed Explanation

For our more tech oriented friends, here are a few more insights on the rather complex process used by Spam Karma to decide what’s spam and what’s not. Each of the following filter is given a weight varying on many factors, ranking from user-controlled values (e.g.: after how many days is a post “old”?) to the credibility that can be given to a test (e.g.: a missing header is less important than a blacklisted IP).

Mostly, Spam Karma looks at the following things:

If the poster is logged in the current blog, and what his user level is (e.g. automatically approve Admin posts).
Presence of HTML entities (e.g. {, ʚ etc).
Presence of a HTTP_VIA header.
Proper use of the posting form (hash value must be present).
Time taken to fill the comment (e.g.: if it’s less than a few seconds, most likely spam).
Posting granularity. First time posters posting many comments at once vs. old-timers (with comments previously approved by the admin).
Previous diagnostic from WP’s built in comment check (set on the ‘Discussion’ panel).
IP and regex match for URLs contained inside the comment (small weight only for non-URL text matching a URL regex).
Realtime Blacklist (RBL) Server check for IP and URLs.
Comment’s age (e.g. penalize comments on very old post).

In addition to these filters, Spam Karma uses different treatments and backup checks to insure it becomes better at stopping further spam and that it never deletes mistakenly a legit comment:

Ambiguous comments (that can neither be deleted or approved) are given a second check: commenter is asked to solve a Captcha or use the email auto-moderation (an email containing a hash to unlock the comment is sent to the commenter’s email address). If confirmed, the comment’s Karma is bumped up and the comment is either published or held for further review, if not confirmed within a certain period, its Karma is lowered and it is either deleted or kept into moderation (if it was sufficiently high to begin with).
When a comment is struck as spam, its IP and URL(s) are harvested and submitted to the Admin for inclusion in the blacklist. In the meantime, they are used as “auto-added” values, with a lesser weight than permanent blacklist entries.
When destroying a spam comment, it checks for recently posted comments that match similar values and retroactively moderate them (e.g.: a spammer could manage to slip X numbers of spams onto a blog, but upon reaching a certain suspicious threshold, all the comments would get retroactively moderated, then deleted).
Spam Karma uses a central DB to retrieve IP and URL updates. By default, it will query the DB automatically every 2 days (can be disabled). Central DB can be configured. Each install of Spam Karma can work as a sort of P2P relay in the update process (both fetching updates and publishing its own updated list for others to grab).

Thanks and Acknowledgement
Many, many people have contributed, knowingly or not, to this plugin, with their ideas, code, help, testing, advice and support… I ended up rewriting most of the code I took from these plugins, but it nonetheless gave me a solid base to start with quickly. Thanks guys.

Happy dad Owen Winkler, whose OSA plugin served as a base to write Spam Karma
Mark Jaquith, for some code and ideas from his Snowball Effect and Captcha plugin.
Jay Allen, the author of MT-Blacklist
Laughing Lizard who ported MT-Blacklist to WP and whose 3 Strikes plugin was problably the first to implement the “karma” concept…
Mookitty, whose efficient Spaminator plugin (working on a very similar principle), and the discussion we had around it, got me thinking…
James Off, whose RBL server is used by default by this plugin.
chrys, Steph aka bunnywabbit, maru, rgh, masquerade, neuro and everybody else on #wordpress, for their patient help debugging this plugin…

If you encounter any error or misclassification of comments (false positive, undetected spam), please contact me and preferably include the whole comment content, such as it appears in the admin screen (with the Spam Karma debug values).

Any comment or suggestion always welcome…

Post originally published on: Dave's Blog (please leave your comments over there)

Introducing Spam Karma

Plugin Database Update (v. 1.2)

Dave — Tue, 19 Oct 2004 22:06:26 +0000

Update:: latest WPPM news are here (doesn’t affect wp-plugins.net)

A new version of WP Plugins Manager with many, many new features…

Website Database

A “filter” function (advanced search to come soon).
More plugin meta-data and miscellaneous cosmetic changes (One-click installable plugins appear in green now).

A brand new RSS 2 Feed with every updates and new versions, straight from the DB!

But the most important improvements are not visible in the frontend, check out ~~wp-plugin-mgr~~ (temporarily taken offline, see note a the top) for the full experience (if you already installed it, just overwrite the old file with this one)

Now keeps track of all installed plugins (including those not in the DB).
“Filter” function.
Changelogs available for plugins that have been updated.
Lots of cosmetic changes and more metadata for each plugin.
One-Click Upgrade when a new version is available.
More and more plugins available through One-Click install (new options made available to developers make it easier for them).
One-Click install support plugin auto-configuration (no need to edit these source files manually anymore).

Read the FAQ for more details.

WP Users: help the DB grow, contact the developers of your favorite plugins and ask them about adding them into the DB (it only takes a few minutes), and recommend they take the extra time to provide a One-Click archive!

A lot fo new stuff for Plugin Developers in the Developer Backend tools:

More fields available, miscellaneous bug fixes.
Added an “Updates” table (not entirely completed yet): currently allows you to provide a changelog when releasing new versions of your plugins.
Improved “One-Click Install” options: now lets you easily define a set of preferences that will be asked to the user at install time (no need for him to edit the files). Check out the detailed explanations in the Dev FAQ.

Note to all the WP Plugin Developers: Packaging your plugin to be One-Click installable is really easy, barely requires anything in most case, please take the extra time to do it, as it will ensure many more users are able to try your plugin.

Many more to come (feel free to post suggestions).

Post originally published on: Dave's Blog (please leave your comments over there)

Plugin Database Update (v. 1.2)

WordPress Plugins Manager v. 1.0

Dave — Tue, 12 Oct 2004 16:25:29 +0000

Last Update: The WordPress Plugin manager is no longer available for download and support has been discontinued. More info here.
10/20/04 Update: Check out the new version…

If you are a WordPress user, go check out the shiny new WordPress Plugin Database!

If you have written plugins for WordPress, go add them already!

If you do not use WordPress, then, well… you should.

But I’ll still talk to you if you don’t…

The whole thing will be migrated soon enough over to its own domain (wp-plugins.org or something like that, but I need to hear from a certain radioactive caribou first). In the meantime, it will work all the same.

This little thing is meant to solve the atrocious headache and unfathomable mess that are WP plugins. Also hopefully help promote these many invaluable piece of code that usually end up ignored by the WP public at large, for lack of a decent central repository.

In addition, ~~wp-plugin-mgr~~ (temporarily offline) will keep a list of installed plugins, notify you of new plugins and upgrades, let you install/remove them in one click and bring you fame, success and happiness (you need the commercial version for that last feature. contact me for a quote).

To install, simply ~~download the file~~ [link removed, see above], drop it in your WP root folder and browse to it (e.g., if your blog is at http://www.yourdomain.com/blog, point your browser to http://www.yourdomain.com/blog/wp-plugin-mgr.php).

For more info, see the Users FAQ and the Developers FAQ.

Feel free to address suggestions, bug reports, mad praises and death threats in the comments below (please, please RTFM twice before).

Post originally published on: Dave's Blog (please leave your comments over there)

WordPress Plugins Manager v. 1.0

WP-keitai-mail Plugin v. 1.0

Dave — Sun, 22 Aug 2004 19:40:10 +0000

Finally got around to writing some kind of doc for my mail posting script for WordPress. And I can therefore finally release it for the world to use and enjoy.

Please welcome a new member in the WP plugin & hack family: wp-keitaimail v. 1.0b!

Actually, it is not technically a plugin, nor a hack, rather a standalone script that interfaces with WP to let you post entries to your blog via Email.
The name ‘keitai’ simply means ‘cell phone’ in Japanese… This script might as well have been called wp-cellphone-mail, I just liked the sound of ‘keitai’.

You can see a working demo on the right-hand side of this blog where it powers the “keitai log”. I have been using it for nearly a month now and it should be really stable albeit a tad complicated to set up (see below).

A quick roundabouts of the current features:

works as a script executed on email reception (unlike WP’s built-in feature), which means you do not have to run a cron job and your posts appear instantaneously on your blog.
handles jpeg pictures, will automatically create a thumbnail if they are too big and insert all necessary code in your post.
can be run on a separate machine from your WP install: it uses xmlrpc to post the entry and can be set to use ftp to upload the picture files.
can handle as many blogs and authors as you want. For each email account you can customize any aspect of the posts (image formatting, posting category etc). For example, you can easily set up the script to let 5 authors post to your blog with each one under a different category.

Now for the bad news:
This script is not trivial to set up. I have tried to include very detailed instructions that should supposedly be enough for even non technical-minded people. But hooking up the script to the email address can be somewhat complicated, and more importantly, it heavily depends on each hosting solution… which makes it quite hard to document.
Overall, you should be fine if you have ONE of the following:

A “mail forward” feature in your server control panel (especially if the control panel in question uses the very common ‘cpanel’ solution) that lets you forward mails to a script (it usually does).

a file entitled .procmailrc in your server root (meaning your mail server uses procmail).

an Exim recipe file. However, you will need to know how to edit it (or look it up online, it’s not that hard), since I haven’t had time to write instructions for Exim.

If you think you have one of these, check out the full instructions in keitai-prefs.php for details on hooking the script and set it to use your blog. I tried to be as clear as possible, but if you run into any problems using them, please let me know. You do not have to be a programmer or know anything about PHP to use this script, however, it is probably a good idea to hold on for now if you are not comfortable editing a settings file manually and tweaking around your server settings.

Hopefully, next release (date very much unknown) will make a lot of this easier.

In the meantime, if you wanna use this wonderful little toy, just download the archive here. Unzip it, edit the keitai-prefs.php file, following the detailed instructions it contains and upload the whole lot to your server. The script doesn’t need to be inside your web root (somewhere inside the ‘www’ or ‘public_html’ folders) since it is not accessed through the web, actually, it is preferable if it is not.

Among the features I plan to add for an hypothetical 1.2 version are:

Simpler interface to edit the script preferences
POP3 support (if you want to use it like WP’s built in script).
Password protection for posting.
Access to post settings through email commands (e.g. you could choose the category to post it to).
Support for more media types (movies, other graphic formats).
Support for GPS coordinates.
Full kanji support. Although this seems to be problem with the xmlrpc lib, the script has troubles with emails containing kanjis as of now. YMMV though.

as well as:

Support for EXIF data
Better cleaning of content: remove unnecessary new lines etc. (allow textile parsing?)

All that depending on the level of interest and the time I will be able to spend on it (not much for now). If you’d like to see other features or are having issue getting the script to work correctly, please let me know.

Enjoy!

Post originally published on: Dave's Blog (please leave your comments over there)

WP-keitai-mail Plugin v. 1.0

Introducing: Dr Dave’s Logs with Multilingual Goodness

Dave — Fri, 13 Aug 2004 19:00:13 +0000

Just installed the awesome language picker plugin for WordPress.

At long last, Dr Dave’s Logs is joining the truly multilingual blogging crowds !

Do not expect real full-on translations for any of my posts. I have neither the time nor the slightest motivation to post redundant content. However, I will probably post from time to time a full entry or an extended discussion in the language relevant to certain items of a particular country/culture, without fear of boring my beloved ignorant American English-speaking-only readership.

Japanese is a special case though: half for my own practice, half to spare my poor Japanese friends the pain and suffering of reading my already convoluted English through Babelfish, I’ll try as much as possible to post a small sum-up of every entries somewhat relevant to my life in Japan and/or my friends here. If you are a student of the Japanese language trying to improve your practice, I strongly sugest you stay the hell away from these, as they certainly won’t do you any good. I do not assume any responsibility for the permanent damage to your practice you might incur by exposing yourself to my crappy grammar and overall appalling level of written Japanese.

If you are really bored with yourself and have a better master of the language than me (basically: if you speak any Japanese), feel free to mock my errors and even possibly point them out to me. I yearn to learn.

Note: if you are reading this blog through RSS, you won’t be able to see alternate language content or links. You need to use the website version. I’ll work on fixing this later.

Post originally published on: Dave's Blog (please leave your comments over there)

Introducing: Dr Dave’s Logs with Multilingual Goodness

Helping WordPress Deal with Exotic Text Encoding

Dave — Fri, 13 Aug 2004 15:59:51 +0000

In my ongoing quest to bring flawless multi-language (multi-encoding to be more accurate) support to WordPress, I just had a blindingly simple, yet highly efficient, idea for an improvement.

If your blog is accustomed to receiving trackbacks or comments containing non-standard characters (accents, kanjis etc.), then you have probably noticed that a fair share end up getting mangled in the process. WP is not really at fault here, since this is caused by some browsers’ failure to respect the encoding set in a page when sending form content (e.g. submitting a comment). No need to tell you which poor excuse for a browser so shamelessly ignore proper web standards. This is of little comfort anyway, since in the end, all that matters is that WordPress is getting toh-mah-toh when it is expecting toh-may-toh, and pretty much ends up displaying poh-tah-to to everybody else.

The fix, as I was saying is ridiculously easy. And to the best of my knowledge it won’t break anything in your current WP install. Worst that could happen is that it won’t fix your problem, but it won’t break your blog.

OK. Let’s start with the How To (gory details afterward):

If you have an .htaccess file in your blog folder, open it.
If you do not have one (or if you don’t have the faintest idea what it is: which means you probably do not have one). You need to create a new file called .htaccess (mind the period at the beginning of the filename) and upload it to your blog root folder.
Append the following lines to your existing or newly created .htaccess file:
php_value mbstring.internal_encoding UTF-8 php_flag mbstring.encoding_translation on
You are done!

As an optional extra step, you can also edit the wp-settings.php file in your WordPress install and inserts the following two lines at the very end, just before the closing “?>” tag:

if (function_exists("mb_internal_encoding")) @mb_http_output(get_bloginfo('charset'));

This optional step is only somewhat necessary if your blog uses an encoding different from the standard UTF-8 encoding recommended by WordPress and sane people the world over. If once again you have no idea what this is all about, you can safely assume your blog is using UTF-8 (as it should be) and skip this step. If you are not using UTF-8, then: 1) you probably should not be complaining about any encoding issues you brought onto yourself by refusing to use UTF-8 in the first place 2) the htaccess hack won’t work without adding the two lines above and might still not work afterward (though I think it should).

What this all does [aka the boring stuff]:

The first line of the htaccess hack insures that, if multi-byte support is available (namely the mbstring PHP module), it gets used to automagically translate any input to whatever you ask it to use. That’s damn convenient and done everywhere in the code once you enable this, meaning that even the most hidden functions benefit from the change and will always deal with the right encoding (if they perform encoding translations of their own, they shouldn’t be affected negatively). The second line set the encoding to be used internally (and therefore translated to, whenever performing automatic translation) as UTF-8.

The optional second part of the hack asks PHP to translate its output from UTF-8 to whatever the blog encoding setting might be, thus ensuring your blog will appear consistent with what the blog meta-tags announce. Since that content has already been translated and stored as UTF-8, this is superfluous for people who have their blog set to use UTF-8. But it is probably a good thing to have in order to ensure flawless code compatibility (i.e. it won’t break if you decide to change the settings).

The reason why we cannot directly set the internal encoding to whatever the blog needs to output is twofold:

It can’t be done in the htaccess file, since we need to access WP’s settings through PHP to know what encoding to use.
Most importantly, not all encodings are fit to be used by PHP as an internal encoding (while pretty much any encoding can be used for output), one more good reason to use UTF-8 for everything.

Unfortunately, it is not possible to circumvent the htaccess step and call ini_set("mbstring.encoding_translation", "on") from wp-settings.php as we do for mb_http_output(), since this setting is PHP_INI_PERDIR level.

Before you ask: this hack won’t break your install, even if you do not have the PHP mbstring module installed: htaccess directives will simply be ignored and the function_exists() check will prevent the wp-settings.php hack from being executed. Therefore nothing will be changed for people who do not have mbstring enabled and the overhead is absolutely inconsequential.

I do not think there is any viable way to support encoding conversions without an install of PHP that has the mbstring module enabled. If your server does not offer it and you really want to be able to support “non-English” characters, you should consider switching hosting companies. All the US hosting solutions I have used to this day (that’s many) had mbstring enabled (though not necessarily with the correct settings, but that’s what this hack will rectify), so really it’s not that hard to find.

I guess it would be great to see this implemented in a form or another in the main code base, although I do realize some people might not be enthused over the idea of shipping WordPress with a preset .htaccess file.

Any thought or proposed improvement/correction on the matter greatly appreciated.

Post originally published on: Dave's Blog (please leave your comments over there)

Helping WordPress Deal with Exotic Text Encoding

Dave's Blog » WordPress

WP-plugins.net domain name for sale!

Spam Karma goes GPL

Big in Estonia

Followup on WordPress Security Issue

“Is this a joke/hoax?”

“How critical is it?”

“Will you tell me more about this exploit? I swear I won’t tell anybody else! This will stay between you, me and the World Wide Web…”

“Are you a WordPress official? a WordPress developer? Anybody with a title I can trust?”

“Why don’t you leave that up to the Big Guys Who Know What’s Best For You®™ and go back to getting smashed on gin somewhere under a Parisian bridge then?”

“Why do you relish so much in fear-mongering, FUD and… OMFG… What is this? Snakes!!!! There are motherfuckin’ SNAKES on WordPress!!! Ahhhh! do something!!!”

“You mean you didn’t just write that thing in a fit of aimless panic to scare other people out?”

Critical Announcement affecting ALL WordPress users

Better leave at the top of your game…

Spam Karma donations: You Rock!

The State of Spam [Karma]

1. How well is SK2 stopping spam currently?

2. What’s wrong in the peaceful Kingdom of SpamKarmia then?

3. How evil?

4. Won’t anybody show up and save the day?

5. Is there really nothing you can do?

6. Then why aren’t you busy doing it, you lazy bastard

7. You wouldn’t leave us to die here, would you?

1. How well is SK2 stopping spam currently?

2. Then why have I seen so much spam going through lately?

3. How does this new spambot generation work?

4. Will any other anti-spam tool fare better than SK2 with this particular spam (or spam in general)?

6. Is there really nothing you can do?

7. Why aren’t you busy working on the next anti-spam solution before this spam thing becomes out of control?

8. Would you seriously stop developing SK if you don’t get money?

WP-plugins.net Changes

Techie Update of the Month

WordPress 1.5.1 and Spam Karma 2.0

Angry Emails

WordPress Upgrade available for Download

Frustrated Users and New Developments

Almost Famous

Spam Karma Zen

Sick of this Crap

Spam Armament Race

Tip Jar

Spam Karma 2.0 Feature Requests

Introducing Spam Karma

Plugin Database Update (v. 1.2)

WordPress Plugins Manager v. 1.0

WP-keitai-mail Plugin v. 1.0

Introducing: Dr Dave’s Logs with Multilingual Goodness

Helping WordPress Deal with Exotic Text Encoding

“Why do you relish so much in fear-mongering, FUD and…
OMFG…
What is this?
Snakes!!!! There are motherfuckin’ SNAKES on WordPress!!!
Ahhhh! do something!!!”