Perfect Pitch

November 3rd, 2009

Web developer Jeremy Keith writes in his blog about DMCA takedowns, and what appears to be Google waffling on their “don’t be evil” creed:

I was reminded that I have an account over at Google Webmaster Tools set up for three of my sites: adactio.com, huffduffer.com and thesession.org. I logged in today for the first time in ages and started poking around.

I noticed that I had some unread messages. Who knew that Google Webmaster Tools has a messaging system? I guess all software really does evolve until it can send email.

One of the messages had the subject line Blocked URLs:

For legal reasons, we’ve excluded from our search results content located at or under the following URL/directory:

http://www.thesession.org/discussions/display/21250

This content has been removed from all Google search results.

Cause: Somone has filed a DMCA complaint against your site.

What now?

I visited the URL and found a fairly tame discussion about Perfect Pitch. Here’s the only part of the discussion that references an external resource in a non-flattering light:

I think that is referring to www.PerfectPitch.com. I’m not saying anything about such commercially-oriented courses because I don’t know them, but I think we’d all be wise to bear in mind the general comments voiced in the first two posts on this thread.

That single reference to a third-party site is, apparently, enough to trigger a DMCA complaint.

So let’s get this straight. In a discussion about perfect pitch, someone mentions the website perfectpitch.com. They don’t repost any materials from the site. They don’t even link to the site. They don’t really say anything particularly disparaging. But it all takes is for the owner of perfectpitch.com to abuse the Digitial Millenium Copyright Act with a spurious complaint and just like that, Google removes the discussion from its search index.

Jeremy goes on to urge anyone with a publicly-viewable blog, Twitter, or whatnot to link to, repost, or otherwise disseminate his article. I couldn’t agree more: while the DMCA does uphold the rights of copyright holders, abuses like this shouldn’t be tolerated. In fact, I’m willing to bet a small sum that I receive an unjustified DMCA complaint from PerfectPitch.com for merely mentioning them in this repost. We shall see.

Eliminating “Page has expired” warnings

November 2nd, 2009

Every programmer working with PHP session cookies has gotten “Page has expired” warnings from time to time. The most common cause is simple: a user, when a session cookie is set, clicks on the Back button. Because the pages are not being cached, the page has, in essence, expired.

As Chris Shiflett points out in his blog post (which you should read after this), the fix is simple. PHP, by default, sends a Cache-Control: nocache header. To override it, set this header in your php.ini file or with a call to ini_set():

1
session.cache_limiter = private

This will cause the server to send a Cache-Control: private header instead. Restart your Web server, and pages will be cached for three hours by default. As always, test that out and see if it causes problems.

How-to: Change your server’s IP address

October 25th, 2009

The Problem

It happens to the best of us. I administer an Internet-facing server for a client, who has native IPv4 connectivity, and IPv6 connectivity via 6to4. All was going well, until one day, the owner needed more IPv4 addresses. So we contacted their ISP (Comcast) and upgraded our /30 block (one IPv4 address) to a /29 block. That’s all and well, except:

  1. The new /29 subnet does not contain the /30 subnet.
  2. Comcast could not give a transition period. As soon as they pushed the config file to the gateway, it would reboot, instantly deactivating the old /30 and activating the new /29.

The Ideal Solution

Needless to say, this is a bit of a problem. In a perfect world, Comcast would activate the /29 alongside the /30, keep both up for a few days, and then shut down the old /30. That would give us enough seamless transition time, so our clients and customers would see no interruption of service.

The Accepted Solution

This solution is partly drawn from experience, and partly from a “Gee, in retrospect, I should have done that” position. For the rest of this post, I will use the following conventions:

192.0.2.1/30
The old IPv4 address and subnet, in CIDR notation.
192.0.2.100/29
The new IPv4 subnet.
2001:db8:0:1::1
The old IPv6 address. (Yes, I know this isn’t in the 6to4 subnet, 2002::/16. I’m opting to use the documentation prefix instead.)
2001:db8:0:1::100
The new IPv6 address.
example.com
The server’s DNS name.

The following post also assumes that you are running your DNS on BIND 9. If you don’t, then the specific DNS changes neededare left as an exercise for the reader; fortunately, this isn’t anything complicated.

1. Notify your users.

Firstly, if your site has any sort of announcement system, like an RSS feed, or a login page, put up a notice warning users of downtime. At the very least, inform them that there will be downtime, and have them plan to not use your site immediately after the IP address changeover. This may seem like a no-brainer, but even the most hardcore of nerds working for a company shouldn’t forget about customer service.

Because our site is tech-oriented, I also included directions on how to reach our site if it refuses to load (that is, having the user change their hosts file). A bit overkill, but I like to cover all bases.

2. Decrease your zone’s TTL.

Secondly, what you want to do is update your DNS records. Two things need to be addressed. The first is to change your zone’s TTL down to a smaller value. Change this to the maximum amount of downtime that is acceptable. They keep their DNS TTL at 86,400 seconds (one day). For this, I’d recommend changing it to 10,800 seconds (3 hours). Change your SOA record accordingly:

1
2
3
4
5
6
7
8
$TTL 10800  ;change this to 10,800 seconds (three hours)
www.example.com. IN SOA authoritative.dnsserver.com. youremail.example.com. (
        2009102501      ;Serial — increment this, as always
        3h              ;Refresh — change this to "3h" (10,800 seconds)
        1h              ;Retry
        4w              ;Expire
        3h              ;Negative caching TTL
 )

Lower TTL values will cause your authoritative nameserver to get a great deal more traffic. If you outsource your DNS hosting like this client does, that may anger your DNS host.

3. Add your new IP addresses to your zone

The second thing to do is update your zone files to include both the old and new addresses. If you have both IPv4 and IPv6 connectivity, then this means that your zone file will have four IP addresses. Bear in mind the following facts about DNS when a host has more than one IP address:

  • Generally, DNS servers will shuffle the records, and return the IP addresses in a different order every time.
  • Generally, a client will pick a random IP address out of the list; A and AAAA records don’t have a priority value like MX and SRV records do.
  • Almost always, a client will try another IP address if the first fails.

Thus, with two IP addresses in the zone file, your clients will connect quickly about 50% of the time during the DNS transition. This is a very simple change:

1
2
3
4
www.example.com. IN A 192.0.2.1
www.example.com. IN A 192.0.2.100
www.example.com. IN AAAA 2001:db8:0:1::1
www.example.com. IN AAAA 2001:db8:0:1::100

Save your zone file, sign it (if you use DNSSEC), and do an

1
rndc reload

to reload the zone and notify your authoritative nameserver. Remember your old TTL value? Wait at least that much time before proceeding with the IP changeover, especially if you have international users.

4. Edit your server’s IP configuration.

You’re going to have a little downtime, and a period of confused users. But keep thinking ahead. Now, log into your server (preferably over a physical console) and update its IP configuration. On Gentoo Linux, this consisted of:

  • Editing the /etc/conf.d/net file, to include the new IP information:
    1
    2
    3
    4
    5
    6
    7
    config_eth0=(
        "192.0.2.1/30"
        "192.0.2.100/29"
        "2001:db8:0:1::1/64"
        "2001:db8:0:100::1/64"
        # Also include your other IP addresses here.
    )
  • Because this server is also an IPv6 router for the local subnet, radvd is announcing the IPv6 prefix to the private IPv4 subnet. Adding a prefix to the /etc/radvd.conf file:
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    prefix 2001:db8:0:1::1/64 {
        AdvValidLifetime 86400;
        AdvPreferredLifetime 86400;
        AdvOnLink on;
        AdvAutonomous on;
        AdvRouterAddr on;
    };
    prefix 2001:db8:0:100::1/64 {
        AdvValidLifetime 86400;
        AdvPreferredLifetime 86400;
        AdvOnLink on;
        AdvAutonomous on;
        AdvRouterAddr on;
    };

    Save and reload radvd.

  • Checking the DHCP server settings. DHCP may not be affected, but DHCPv6 will definitely be affected by the changing subnets.
  • If you have the fixed IP address recorded anywhere else, make sure you add the new IP addresses! (I can’t help you on this one, but they did have the IP address specified in: (1) Postfix’s /etc/postfix/main.cf file, with the smtp_bind_address and smtp_bind_address6 directives, (2) switch configurations, and (3) IP address filters in Apache for their Intranet applications.

Reboot the server if necessary. If you can, reboot it anyway — even you, Linux users.

5. Change your IP address.

The big moment has come. Contact your ISP and have them perform the changeover. This resulted in about a minute of downtime as their cable modem received the new settings and rebooted. As soon as you can afterwards:

  • Remove the old IP addresses from the zone file. Increment the serial, reset the TTL, save, sign, and rndc reload.
  • Remove the old IPv6 subnet from /etc/radvd.conf.
  • Reboot some workstations and check basic connectivity to and from the server and the IPv4 and IPv6 Internet. Then go offsite and try to reach the server with the DNS name and new IP addresses.
  • Be prepared to answer phone calls and emails from clients who can’t reach the site.

I hope that this helps anyone else in this predicament. I still don’t know why Comcast couldn’t offer a transition period, especially to a business customer. Regardless, the changeover for this client went very smoothly, and there were no issues reported. Everything is now running just fine, and they’re making use of their new — and limited — IPv4 address space.

An interim fix for GPGMail on Snow Leopard

October 12th, 2009

GPGMail logo, a padlock on top of a stack of envelopes, with

GPGMail has been an invaluable tool for users of both GPG and Apple’s Mail application. For those unfamiliar with GPG (and its commercial equivalent, PGP), it uses a set of keys — one public, one secret — to encrypt and/or attach digital signatures to email. (Read the Wikipedia article for more information, as it’s not entirely relevant here.)

Unfortunately, the marriage of GPG to Mail has been a bittersweet relationship. Apple doesn’t publish the internal API’s of Mail, and has not provided much of a plug-in architecture for it. Therefore, GPGMail had to be developed by using undocumented functions, which Apple had no obligation to maintain between major releases of Mac OS X. Like all other major upgrades, Mac users upgrading to Snow Leopard were left to pick up the pieces. To add to the pain, the sole programmer behind this no longer has time to maintain the software.

While the software is getting a long-overdue rewrite from new hands, two enterprising users found a solution, which I’m reposting to spread the word.

  1. Download and install GPGMail and MacGPG.
  2. Find Mail in the Applications folder. From the File menu, choose Get Info.
  3. Check Run in 32-bit mode. (64-bit support will have to wait until a new version of GPGMail is formally released.)
  4. Now, go to ~/Library/Mail/Bundles; right-click on GPGMail.mailbundle and choose Show Package Contents.
  5. Add the following lines to Info.plist, depending on which version of Mac OS X that you’re running:
    • Mac OS X 10.6:

      1
      2
      3
      4
      5
      <key>SupportedPluginCompatibilityUUIDs</key>
      <array>
          <string>B3F3FC72-315D-4323-BE85-7AB76090224D</string>
          <string>225E0A48-2CDB-44A6-8D99-A9BB8AF6BA04</string>
      </array>
    • Mac OS X 10.6.1:
      1
      2
      3
      4
      5
      <key>SupportedPluginCompatibilityUUIDs</key>
      <array>
          <string>2610F061-32C6-4C6B-B90A-7A3102F9B9C8</string>
          <string>99BB3782-6C16-4C6F-B910-25ED1C1CB38B</string>
      </array>
    • Mac OS X 10.6.2 (beta):
      1
      2
      3
      4
      5
      <key>SupportedPluginCompatibilityUUIDs</key>
      <array>
          <string>0CB5F2A0-A173-4809-86E3-9317261F1745</string>
          <string>2F0CF6F9-35BA-4812-9CB2-155C0FDB9B0F</string>
      </array>
  6. Save the file, and then open Mail.

Everything should be working fine now! Many thanks to detlefschmitt and pretemsteinmetz from the SourceForge.net forums for this solution. If you found this helpful, please thank the original posters, and then urge Apple to do any of the following:

  1. Provide native support for GnuPG in Mail, as they do S/MIME.
  2. Publish a Mail plugin API, to make development of GPGMail easier.
  3. Assign one or more members of the Mail development team to the GPGMail project.

Update, October 12, 2009 at 2:45 pm: As expected, GPGMail loads, but still has major issues on Snow Leopard, related to the user interface and encryption/decryption. I’ve filed a couple of bugs on the tracker; if you spot some, you should, too.

dnssec-signzone and “File not found” error

August 11th, 2009

DNSSEC is a very good thing, in my opinion. For almost twenty years, there has been at least a proof of concept about attacks against DNS. Kaminsky’s DNS flaw last year only strengthened the push to implement cryptographic extensions, and now, ICANN says that com. and . (the root zone) will be signed by year’s end, which means that DNSSEC can be properly implemented.

Generating the keys and signing a bunch of zones, however, quickly becomes arduous. For one domain, it’s not bad. When you administer two external domains, four reverse IPv4 and IPv6 zones, and an internal zone or two, it quickly becomes a hassle. Like any Linux user who spends some time at the command-line, I wrote a rudimentary shell script to automate this process. The latest version is at the end of this post, but read on. I ran into a problem, and not finding anything on Google, I decided to document it here.

Now, I like a nice clean directory listing. When you use SSH as your primary means of getting things done on the server, it’s a requisite. I wrote my script to rename the generated keys, and place them all in the keys folder before signing the zone ($1 being the zone name):

1
2
3
4
/bin/mv $ZSKNAME.key keys/$1.zsk.key
/bin/mv $ZSKNAME.private keys/$1.zsk.key.private
/usr/sbin/dnssec-signzone -t -N increment -H 10 -k $1.ksk.key \
    -e +7776000 -o $1. $1.zone /var/bind/pri/$1.zsk.key

All seemed well until I ran it:

# ./signzone.sh example.com
Will sign example.com.zone
Generating new key-signing keypair.
Generating zone-signing keypair.
Signing zone.
dnssec-signzone: cannot load dnskey keys/example.com.zsk.key: file not found
Done.

It complained about the file not existing, but it definitely did:

# stat example.com.zsk.key
  File: `example.com.zsk.key'
  Size: 395          Blocks: 8          IO Block: 4096   regular file
Device: 6802h/26626d   Inode: 17989721    Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2009-08-11 11:20:12.000000000 -0400
Modify: 2009-08-11 11:20:12.000000000 -0400
Change: 2009-08-11 11:20:12.000000000 -0400

It took me a few hours of troubleshooting, modifying the script, wondering about NSEC3 records, and even rebuilding BIND 9.6.1-p1 from source. Finally, I installed and used strace on my script, which revealed the problems:

  1. BIND, especially dnssec-signzone, expect the public keys to end in .key and the private keys to end in .private. Changing their extensions from .key.private to .private fixed it.
  2. dnssec-signzone expects, as an argument, the name of the keypair, minus the extension. With the correctly-named keypair of example.com.ksk.key and example.com.ksk.private, dnssec-signzone should be run with the -k example.com.ksk argument. Note the lack of extensions.

After fixing that, dnssec-signzone stopped complaining, and I was able to sign all of my zones without a hitch. Restarted BIND, and it’s sending out notifies as I type this.

Now, for those of you who came here for the finished, bug-free script, here you go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
#!/bin/sh
# signzone.sh, Copyright © 2009 Colin Cogle.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program.  If not, see &lt;http ://www.gnu.org/licenses/&gt;.
#
# Usage:  signzone.sh &lt;name of zone to sign&gt;
# Example:  signzone.sh example.com
# Example:  signzone.sh 8.b.d.0.1.0.0.2.ip6.arpa

if [ -f $1.zone ];
then
    echo "Will sign $1.zone"
else
    echo "$1.zone does not exist.";
    exit;
fi

#
# Stash our keys in the keys subdirectory, just to keep things clean.
#
if [ -f keys/$1.ksk.key ];
then
    echo "Using pre-generated key-signing keypair."
else
    echo "Generating new key-signing keypair."
    KSKNAME=$(/usr/bin/dnssec-keygen -f KSK -e -a NSEC3RSASHA1 -b 2048 -n ZONE $1)
    /bin/mv $KSKNAME.key keys/$1.ksk.key
    /bin/mv $KSKNAME.private keys/$1.ksk.private
fi

if [ -f keys/$1.zsk.key ];
then
    echo "Using pre-generated zone-signing keypair."
else
    echo "Generating zone-signing keypair."
    ZSKNAME=$(/usr/bin/dnssec-keygen -a NSEC3RSASHA1 -b 2048 -n ZONE $1)
    /bin/mv $ZSKNAME.key keys/$1.zsk.key
    /bin/mv $ZSKNAME.private keys/$1.zsk.private
fi

echo "Signing zone."
/usr/sbin/dnssec-signzone -a -g -t -N increment -H 10 -3 caf3 -d keys -k keys/$1.ksk -e +7776000 -o $1. $1.zone keys/$1.zsk

echo "Done."

Do note that this Bash script is a rough draft, and represents only about ten minutes of coding and referencing man pages. I’ve licensed it under the AGPL if anyone wants to make a copy for themselves.

Naming Software

August 7th, 2009

Naming software has got to be one of the most difficult steps of the software development lifecycle.

This winter, I quickly and single-handedly developed a Web-based help desk software. Clients log in to submit trouble reports, and they can track and edit them as things progress. It also allows them to track their assets. It makes great use of PHP, HTML 5, CSS 3.0, AJAX, geolocation, Google Gears, and more; it’s a balanced mix of tried-and-true and bleeding-edge technology.

The world’s only (known) installation is on my company’s Web server. However, since then, it’s grown by leaps and bounds, but still running on the same spaghetti PHP code; even though it was developed in a rush, if my old college professors could see what I did, they’d revoke my degree.

So, what I’d like to do is clean it up, document it, make it portable, and release it as a free and open-source project. Naturally, the company would sell support contracts, like Red Hat does with Linux — there’s the moneymaking portion of it, enough to please my boss.

However, I just can’t think of a good name. The best name that I came up with is Mercury, eponymous after the messenger of the Greek gods. Trouble is, while “Project Mercury” rolls off the tongue with aplomb, and what I’m calling the development process, it’s not the greatest of names for a final product.

New and returning readers, it’s discussion time. What’s in a name? How do you name software? Have any ideas?