Sucks To Be You

I run a lot of different pieces of software to try and figure out what’s going on with my website. Pair.com gave me the choice of Webalizer or Analog. Plus I’ve got Sitemeter and ExtremeTracking javascripts loading at the bottom of the pages. Not to mention my Refer page. Maybe it’s overkill but this site is still in the “building” stage. How do most people find my site? Who are my return visitors? What pages are the most popular? That kind of thing.

So when I was checking out my stats for the first two days of August I was annoyed to find forum avatar leechers—people who had directly linked to the images on my site for display on another site instead of downloading them and providing them on their own. It’s just rude. They are eating my bandwidth. Not enough to cause me overage trouble but it’s still shows a complete disregard for me. Basically every ISP on the planet gives out a bit of webspace with your account. Not to mention the tons of free places. How hard is it to save that image to the desktop and upload it to your own server?

Plus it totally blows away my legitimate tracking. Especially when people use them as forum avatars on popular sites. (And when they post alot.) One girl in Australia who was linking to a picture of Barbarella from my old Right Turn Clyde site managed to rack up 2200 hits on that picture in 2 days! And some guy in Italy had 1000 hits on a picture of The Donnas from that old site of mine. Of course those rocket to the top of all the lists in my tracking knocking out stuff that I really want to know about.

So I decided to get even using Apache’s nifty little mod_rewrite:

I added this to my .htaccess file:

RewriteEngine On

RewriteCond %{HTTP_REFERER} ^http://www.vogue.com.au/forum/.* [NC]
RewriteRule .*barbarella.jpg$ /images/i_steal_bandwidth.gif

RewriteCond %{HTTP_REFERER} ^http://www.toronews.net/forum/.* [NC]
RewriteRule .*donnas5.jpg$ /images/i_steal_bandwidth.gif

So now Ms. Australia looks like this:

Barbarella Steals Bandwidth

And this guy is so bad-ass I almost crapped myself:

Toro Steals Bandwidth

You can find lots of solutions on websites to block all image requests that don’t come from your own site, but I use my site to serve images and files for lots of different things—my own forum avatars on numerous sites, example screenshots for software that I beta test, etc. It would be a real pain to have to continually go in and add new sites for access. So I set this up to just block those two. Of course there are others leeching from me. But those were the worst.

And of course this solution doesn’t solve my tracking problem. But revenge can be sweet. I’ll block them totally in a week or two:

RewriteEngine On

RewriteCond %{HTTP_REFERER} ^http://www.vogue.com.au/forum/.* [NC]
RewriteRule .*(jpg|JPG|gif|GIF)$ - [F]

RewriteCond %{HTTP_REFERER} ^http://www.toronews.net/forum/.* [NC]
RewriteRule .*(jpg|JPG|gif|GIF)$ - [F]

A Bit About Gallery Photo Moblogging

It turns out that the email to Gallery gateway that I set up was uglier than I thought. I would strongly recommend not using these scripts to post photos from your camera phone to a webpage. I thought it was just that my implementation of procmail was bad because I didn’t really know what I was doing. Well it was bad, but the scripts aren’t very good either. At least the mimesave.pl one isn’t. I’m still learning Perl so I only have a bit of an idea of what’s going on, but it definitely does a few bad things. I would stay away from it unless you’re willing to rewrite some sections. The parsing of the mime attachments is the real culprit. It decodes the mime attachment and reads the raw data into a new file and generates a random name for it with a .gif extension instead of looking at the actual name of the attached file.

This is a much better way to do it. You’ll need to learn a bit about procmail—though you have to do it with those bad scripts above anyway. Instead of the flakey mimesave.pl script, you use the nice Unix C-program ripMIME. Pair.com, my hosting service, does an amazing job of having many, many programs pre-installed on all their servers. If you look at the list of Perl modules they have installed, it’s staggering. Or at least I think it is. Maybe those are standard on most web servers. I don’t know. Anyway, they already had ripMIME installed so I didn’t have to compile it myself.

Infinite Ink has an amazing starting guide to procmail. I’m certainly no expert now but it taught me enough to set up a couple of proper recipes on a special Gallery gateway email address I created to send pictures from my phone and have them posted to my Gallery webpage. This article by Zig, and this one by Phil, have some nice specifics about dealing with the Qmail mail transfer agent on pair.com and interfacing it with procmail. There’s also a ton of information here at PairUsers—not only about email but pretty much every aspect of anything you might want to do with your website.

This is a simplified version of the ~/.procmailrc file that I setup to handle the email to Gallery gateway:

SHELL=/bin/sh
PMDIR=$HOME/procmail
LOGFILE=$PMDIR/pmlog

# prevent qmail (the program that is calling procmail
# as a filter) from delivering the original mail. We'll
# handle all delivery from here, thankyouverymuch.
EXITCODE=99

# This is for support of the virtual mailboxes on pair.com
# The $MYACCOUNT variable is sent to Procmail when it is envoked from Qmail.
# You need to change "my_account" and "mydomain.com" to your own.
MYDIR=/usr/boxes/my_account/mydomain.com/$MYACCOUNT^/.imap


#### End Variables section; Begin Processing section ####  

# This is setup for virtual mailboxes on pair.com
# You need to change "myaccount-mydomain:com-mygatewayemail@mydomain.com"
# and "phoneemailaddress" and "http://gallery" and "uname" and
# "pwd" and "album". Also change path to "galleryadd.pl".
:0:email2gallery.lock
* ^Delivered-To:.*myaccount-mydomain:com-mygatewayemail@mydomain.com
* ^Content-Type:.*multipart/mixed
* ^From:.*phoneemailaddress
| ripmime -i - -d $HOME/tempdir/ --syslog_on --no_nameless &&
  ./galleryadd.pl -l http://gallery -u uname -p pwd -a album $HOME/tempdir/* &&
  rm $HOME/tempdir/*

# If email comes in for mygatewayemail and it doesn't match any of the
# rules above, trash it.
:0
* ^Delivered-To:.*myaccount-mydomain:com-mygatewayemail@mydomain.com
/dev/null

As it exists right now, it will only really work for pair.com accounts if you swap out the placeholders I mention in the comments for your own information. Unless there are other web space providers who set up their mail like Pair does. It seems a little unique to me. But the basic idea is there for people at other places. I strongly suggest you go through Infinte Ink’s quick start guide to learn how it works and then compare my recipe to the one that jmullen provided in the link above to see how I had to adapt it to my particular situation.

For extra security, a password system should be implemented as well. But as long as you keep your gateway email address private, this should keep out most spam. You could even add in a check for other headers that might show in email sent from your phone. There are probably a few other bits that would make it a little more unique and a little less possible to spam.

The Best Of Both Worlds

No, this is not another Star Trek post. Nor a reference to the choice of Dave or Sammy in the new Van Halen greatest hits album. (We’re all desparately trying to forget the fact that for a brief time there was a third.)

This is actually heralding the newly announced Movable Type 3.1 coming to a weblog near you on August 31. Many have critiqued Six Apart, particularly with the recent change in licensing, for continuing to provide weblog software that natively only supports static pages, while many other weblog and CMS systems are PHP based.

No longer. (Wait for it.) Movable Type 3.1 will offer the best of both worlds where publishers can decide on a per-template basis whether to offer a dynamic or static page. (We have titular line.) This is a huge step forward. There are distict advantages to both methods. Static pages offer a much lower server load while dynamic pages can generate information on the fly, instantly incorporating new content.

Other improvements include post scheduling, a subcategory management system, and new hooks for plug-in developers. Not to mention all the winning plug-ins from the recent contest.

I personally am anxious for this new software. It should certainly add whole new levels to what you can accomplish with your weblog.

Trackback Spam Sucks

I have been getting hit pretty hardcore with the Trackback Spam today. It seems that every time a take a look at my email there’s another 50 messages from MT saying that I have new trackbacks. And they’re not real trackbacks of people writing about something I wrote—they’re links to sex sites. So every couple of hours I’ve been going in and clearing them out. It’s not terribly difficult under MT3, but it is annoying. I sure hope that Blacklist 2.0 is released soon. (Hell, I’d even go for a beta! hint hint.)

For those who have been experiencing the annoying Trackback Spam under MT2.6.x, Jay has some easy instructions for getting rid of it with Blacklist 1.6.4.

Gooooooaaaaal!

Jay Allen creator of the amazing Movable Type plug-in, Blacklist, that helps greatly reduce the amount of comment and trackback spam on your website, just won the Grand Prize in the MT Plug-In Developer’s Contest.

Congratulations, Jay! It couldn’t happen to a more deserving guy.

Now maybe with the extra speed on that new G5 he’ll be getting, he can hurry up get the 2.0 version out to us plebians. 😉

Quickly Adding Smilies To Movable Type

When I did some upgrades to my website a week ago, I added in support for smilies. I used Lisa’s excellent tutorial to set them up on my site. I’d suggest you do the same. I can offer one extra bit to make things go a little faster.

Download one of Jason’s Basic Sets of 17 smilies to use on your site. Create a directory /images/smilies in the root level of your weblog website and upload the 17 GIFs there. Then use the two blocks of code I’ve provided below instead of Lisa’s code and you’ll get Jason’s Basic Set working without any extra typing. Pretty easy.

Use this code instead of the code in Step 3 of Lisa’s Tutorial, and use this code instead of the code in Lisa’s Step 7.

Of course you might be the kind of person who needs “special” smilies, in which case you’re on your own. But if you want to get the basic 17 going quickly, this should help.

Update 6:30am — I had to remove the code from the post and insert links to text files with the code. With smilies installed on my site, it was substituting the smiley graphics themselves for part of the code! Too smart for my own good sometimes—or maybe too dumb—still trying to figure out which.

Getting Ems And Ens To Work In Smarty Pants

This one’s painfully obvious when you think about it. Unfortunately I had get the author to explain it to me.

The default behavior of Daring Fireball’s excellent plug-in Smarty Pants is called mode “1”. In this mode you can get an Em-dash ( — ) by typing two dashes next to each other ( -- ). You cannot get an En-dash at all. You need to run Smarty Pants in mode “2” to have both active. Then you get an Em-dash ( — ) with three dashes next to each other ( --- ) and an En-dash ( — ) with two ( -- ). Here’s the extra little (non-) tricky bit that I wasn’t picking up on:

If you run Markdown and Smarty Pants together by using the pull-down menu in your Movable Type entry page, then you don’t need to make any changes to your templates but Smarty Pants only works in mode “1” the default behavior. If you want to run it in mode “2”, you need to add smarty_pants="2" to any MT tag where you want it to kick in and you need to set the pull-down menu to Markdown only. It’s the “and” part that I missed.

Now it’s all running great for me. I just have to go back and update my entries. For those of you who don’t know what the whole deal is with the Em-dash and En-dash, this is a great resource.

Thanks John for helping me out with this.

Using Another Weblog For Link Lists

In the course of the most recent redesign of my website, I setup the various link lists on my site—Your Blogs, Favorite Sites, etc.—as separate weblogs that are fed into my main weblog. Anyone who has gone in and updated the templates in Movable Type knows what a pain it can be. By using a separate weblog for link lists with one entry per link, it is very easy to make changes to those lists.

The proper way to do this uses PHP. If you can’t run PHP on your server, you’ll need to modify this process slightly. Plus it won’t be quite as easy to keep things up to date. You’ll need to rebuild your site every time you want new links to show up.

I got the idea for my link list from two places, vova’s “Manage Your Link List as a Blog” thread in the Movable Type support forums, and Mike James’ “Sideblog ‘Interesting Links’ setup HOWTO”.

These are both rather old topics and very likely someone has already come up with the same ideas I’m pushing pixels about right here, but just in case, I’m writing this. My version of the link list uses great ideas from both places with a little extra twist of my own. Here’s how I did it:

  1. Make an includes directory on your website. This process will create one or more files that will be included into your webpages. For the sake of being organized, I created this folder to keep these files separate. I called my folder “inc” and put it in the root level of my website with the permissions set to 755.
  2. Make a new weblog. In my case, I called it “MFTI Links”. Set the “Local Site Path” to the new includes directory you created, and the “Site URL” to your website’s URL, plus the includes directory. You can ignore the two Archive settings. But you should set the correct Timezone. Save your changes.
  3. In the Preferences screen of your Weblog Config, use the following settings: Default Text Formatting to “Convert Line Breaks”, Default Post Status to “Publish”, Number of Words in Excerpt to “0”, Preferred Archive Type to “No Archives”. Make sure all boxes in Publicity and Comment Configuration are unchecked. Set Allow Comments Default to “None”. Save your changes.
  4. In the Archive Files screen of your Weblog Config, delete all archives. Save your changes.
  5. Delete all your templates. Go to the “Templates” screen. Delete all Index and Archive-Related templates.
  6. Create categories for each of the different types of link lists you want. I created a category called “Blog” for my “Your Blogs” list and one called “Favorite” for my “Favorite Sites” list. Make as many as you want.
  7. Back in the Templates screen, make a new Index template for each category you just created. vova’s idea was to create one template and include all categories in it. This means when you insert your link list, everything is together. Sure it’ll be sorted with headings, but everything winds up together. So in my case you’d see “Your Blogs” and the list and then immediately below it, “Favorite Sites” and the list. If you create individual templates for each category, you can put them where ever you want. My “Your Blogs” list is in my sidebar while “Favorite Sites” is on my About page. As an example, my “Your Blogs” template is called “Blog Links” and the Output File is “bloglinks.php”.
  8. Use this code for the templates:
    
    <h2>Your Blogs</h2>
    <ul>
    <MTEntries category="Blog" sort_by="title" sort_order="ascend">
    <li><a href="<$MTEntryBody convert_breaks="0"$>" 
    title="<$MTEntryMore remove_html='1' $>"><$MTEntryTitle$></a></li>
    </MTEntries>
    </ul>
    

    Obviously you should change this to whatever is best for you. Probably the two most important things would be to put whatever title you want to the list between the <h2> and </h2> tags, and to change category="Blog" to whatever your category is.

  9. Make some entries into your new weblog. This is the format:Title = Name of the webpage (i.e. Monsters from the Id)
    Primary Category = The category mentioned above (i.e. Favorite)
    Entry Body = The URL of the webpage (i.e. http://www.idmonsters.com/)
    Extended Entry = The title that shows up when you hold your cursor over the link (i.e. Jon Michaels’ Weblog)
    Excerpt = An optional number or letter for sortingThis last one is where my method really differs from the other ideas people had. In the code listed in step 8 above you’ll notice sort_by="title" and sort_order="ascend". This will display the list in alphabetically my the name of the webpage from A to Z. Changing the sort order to "descend" would list the links from Z to A. If you removed the code sort_by="title" entirely, the list would be sorted by entry date. The final option is to use the code sort_by="excerpt". This would allow you to order the list in anyway you wish. Use numbers or letters to rank importance. Want your friend’s site to be the first on the list? Put a “1” or an “a” in the Excerpt field.
  10. Add the PHP include to your main weblog’s templates. Use this code:
    <?php include('<$MTBlogSitePath$>inc/bloglinks.php');?>

    Change the “inc” to whatever directory you created in step 1. Change “bloglinks.php” to whatever you called the Output File in step 7. Then put the code in your sidebar or where ever you want the link list displayed. Make sure to include it in all necessary templates.

  11. If your site uses PHP files (with the .php extension), you’re all set to go. Rebuild all the pages and you’ll be up and running. If you have HTML files (with a .html extension) you’ll need to make a change to your .htaccess file. (Or create one if you don’t have it.) It should be sitting in the root level of your website. Open it in a text editor (or create it), and add these two lines:
    DirectoryIndex index.html index.htm index.php
    AddType application/x-httpd-php .html .htm 

    This will let you process PHP code in your HTML files.

If you can’t run PHP at all on your site, you’ll need to rework this whole idea. Probably using the <$MTInclude$> tag. And as I mentioned, you’ll have to rebuild your site every time you want new links to show up.

More Site Changes

I said it would be back, and it is. I’ve brought back my Monsters from the Id title graphic. In that previous post I said I wouldn’t bring it back until “all the bells and whistles” were in place. Well, as anyone who does their own website probably knows, all the bells and whistles will never be in place. But I did do some major upgrades.

First you will notice that the “My Links” section is gone and replace with “Navigate” at the top of my Sidebar. “My Links” has now moved into the brand-spankin’ new About page that you can access from said “Navigate” section.

I’ve also redesigned how my Archives work. I took the references to the monthly and category archives out of my Sidebar and put everything under a master Archives page. I’ve changed my monthly archives and category archives to only show the 40 word excerpt of each post. You can click the link for the full post. The main reason for this is that pages were getting too big. I’ve only been running this site a few months. But 50+ posts in my “Musings” category where I tend to get a little long-winded, made a huge page to load. What would it be like in a year? So all the data is still there, it’s just not duplicated on many different pages. Those you coming in looking for a specific item that you searched for on Google or another engine, can simply type in what you’re looking for into my search box and you should get the same thing.

I also added in the Referrers page that I talked about in my previous post. Less then 24 hours after putting it in, I had to make some major changes to it. For one thing I wasn’t stopping the bot spam that I first got. Vibehosting.com is now on my nemesis list. I had to implement both the BotWhack and IPWhack from this page. Even though I talked about how cool it was that it could track all pages at my website, I decided to shutdown referral tracking on “The Donnas — Rock ‘n’ Roll Machines” and “Right Turn Clyde”. Neither of those sites have been updated in about 4 years, but since they’ve been around that long, there are links to them from everywhere. They get much more traffic than this little weblog. I might setup a separate referrer tracker for them at some point but for now, I’m really just interested in Monsters from the Id.

I setup a very cool link list system using a second weblog and PHP. It’s based on a couple different methods that other’s have used with a few little changes of my own. I’ll write about the specifics at some point so others can make use of it. The long and short of this is basically no different for the end user. It just makes things much easier for me to add and delete links in my “Your Blogs” list and my “Favorite Sites” list. Plus I new and slightly more dynamically exciting list should be showing up in the not too distant future.

And what would any site be without smilies. I had to do. I’ve mentioned before—I can’t remember if it was in a comment on this site or someone else’s—that I try not to use them too much. I try to write in a clear manner so that my tone in inferred through my words. But obviously that’s not always possible. So the occasional smiley might be called for. Plus I turned them on for Comments. That’s more of a “stop in and have fun” area. People should be able to make cute faces. This too I did slightly different from the way that’s mentioned on other sites. I’ll post something on that as well.

But wait! There’s more! I enabled Markdown and Smarty Pants in the comments. That’s right folks! Now you too can benefit from the easy markup that is Markdown.

So enjoy all the new fun. I do have a few other tricks floating around in the back of my mind that might just pop up here one day.

Where Did You Come From?

It figures. I don’t know whether it was Murphy or the previously mentioned “narrative convention” but the very instant that I installed Dean Allen’s excellent Refer scripts to keep track of referrals to the various pages of my website, I was referral porn spammed by a passing bot. I have now made some changes to the scripts to reduce referral spam. Hopefully that helps.

Oh well. For those of you who might be interested there are some links to what is probably some hot stuff in my new referral page. At some point later today, I’ll update my various MT templates to include a link to this.

There are a couple of very cool things about these scripts. The first is that they are not a plug-in to Movable Type. This means that they’ll keep track of all pages on your website—even ones not served by MT.

The second cool thing is that by making a simple change to your .htaccess file (which the installer explains) you don’t have to insert any new lines into your webpages. Apache and PHP will take care of everything for you. Nearly instant on.

The third cool thing is that it only uses one table in a MySQL database so you can easily configure it to add the table into your existing MT database. (My ISP only allows me a limited number of MySQL databases. I wasn’t sure I wanted to use up one of those just for referrers.) Just make sure you don’t use one of the existing MT table names.

As you can probably infer from the text above, Refer requires PHP and MySQL to run. If your ISP doesn’t allow those things, you’re out of luck. Sorry.