Wednesday, 19 January 2011

PSA: Hard disk passwords

"You can't remember your password?"

Like Simon the BOFH, I have many passwords to remember, and occasionally my brain can't conjure the right one at the right time. Thus when, for the first time in several months, I rebooted Traal (and yes, despite my reiplophobia, I do sometimes reboot - this time was to replace a noisy CPU fan) and realised that I couldn't bring him up again. Some time ago, I put a password on the hard disk (ATA password), and the computer won't touch the hard disk without it being entered. After half an hour of trying to remember the password, followed by an hour of searching my other systems for where I might have written down a hint as to what my password was, I turned to password recovery services.

There are several services that, for a fee, will low-level format your drive and wipe out both password and data. This was not what I had in mind, as it is basically a way to get a cheap(er) hard drive; yes, it's cheaper than replacing the drive, but I would prefer to keep all my data, thanks! Forums posts and blog articles consistently stated that it's impossible to remove an ATA password without wiping your data, but one site promised to do just that.

A-FF Repair Station boasts that it can remove passwords from a large variety of hard drives. Generally it seems best to add the drive as a secondary in a desktop; with SATA drives, this is quite easy, as desktop and laptop SATA is the same connection (with PATA/IDE you need a 44-to-40 adapter). The free download of the Repair Station will examine the drives attached to the system, identify them, and tell you which one(s) have passwords; it will not, however, remove the password until you pay for it. It scanned my drive and told me that it ought to be able to remove the password, so I got out the credit card and prepared to pay for my inability to remember unusual passwords.

The purchase was smooth and troublefree, and I then told Repair Station to have another look at my drive. Unfortunately it then - and only then - told me that it was unable to depasswordify this drive, which was a great disappointment. It did not, however, charge my account for this failed recovery, and left the credit there; and later on, after I'd figured out the password on my own, I was able to get a refund to my credit card of the full amount. Quoting from Daniel Clay of Atola support:
Upon investigating session logs at our end, I see that the problem is related to the specific firmware revision of your hard drive. There are some rare firmware revisions of WD drives that Repair Station fails to unlock, while initially misdiagnosing as "unlockable". This is actually quite rare (well less than 1% of all sessions). Our simple refund policy was designed exactly to cover these.

While 1% may sound quite high (eg 1% downtime average for a server is appalling, 1% misdelivered mail is a lot), what this really means is that they have a free service that can, with an excellent degree of reliability, predict whether or not the drive is unlockable. And as their refund policy covers the rest, this is fairly safe.

So, conclusions.

1) Popular opinion, that it's absolutely impossible to remove an ATA password, is not necessarily right. I can't say for sure that A-FF Repair Station will be able to work as promised, as I haven't seen it in action, but it certainly appears to be valid for a large variety of drives. (I'm not prepared to password a drive and pay for its removal JUST to verify the program, though. I'll let someone else do that.)

2) It's still worth remembering your password. The removal service costs $50 (less if you buy in bulk) and you have to have a spare computer to plug it in to, and only certain drives are supported.

3) If you do have an issue with A-FF Data Recovery, their support people will handle it. I had a full refund back on my credit card within three days, and considering that the first "day" started at 11pm, that's decent turnaround time.

Ultimately, of course, no encryption on a removable device can ever be perfect. It would be possible, albeit slow, to just brute-force the password, and I think there are services that offer this. But you need special hardware or a REALLY patient person, so it's not practical. However, A-FF can remove passwords in fixed time, making ATA passwords largely bypassable; if your data is too sensitive to be released, use software encryption using industry-standard cryptography (maybe in addition to the hard disk password). Nothing's perfect, and ultimately, the only way to be sure is... to make sure your laptop isn't stolen. But you knew that already. :)

Friday, 14 January 2011

A secure ticketing system for Melbourne

Or: MyPKI.

Actually, this is sufficiently general that it could probably apply to any city, but I have Melbourne in mind as I write this. Though it has its faults, it IS still a great city to live in!

A public transport ticketing system has to cope with a large number of possibilities. It has to handle:

  • Regular commuters, taking regular services every day, and knowing they're going to do so for the foreseeable future;
  • Periodic travellers who might use the train a couple of times a month, and want it convenient when they do;
  • Short-term visitors, using PT for a fixed period before departing the city;
  • One-off trips with no prospect of using the system again any time soon.


  • Each of the demographics should pay for their use of the system;
  • Each of the demographics should feel that they are not over-paying;
  • Fraud should be discouraged and/or made unprofitable;
  • Neither travellers nor staff should be put to unnecessary effort.

To keep the costs down (important for goal #2), existing technology should be used whereever feasible. A smart card of some description allows a convenient touch-on/touch-off pattern, high throughput in peak time, and can be easily implemented. The current MyKi system is a little slow in its definition of "touch" (hold your card on the reader for a few seconds), but this can be improved upon.

The four demographics can be served well by a dual system: the smart card for those who wish to keep their long-term costs down, and some other system for those who care only about this one trip. For the latter system, cost is important. Anyone should be able to buy a bus ticket on the bus they're about to ride, a train ticket from the station where they intend to board, a tram ticket on the tram; and they should not have to "buy into" a system they're not going to use. This can easily be dealt with by having very simple (possibly coin-only) ticket vending machines which, rather than providing a smart card, provide a simple token valid for a single trip - a uniquely-numbered, cryptographically-signed slip of fax paper would suffice. The slip can identify the date and time of issue, and the fleet number of the vehicle on which it was bought, making fraud difficult; in the unlikely event of deliberate, systematic fraud, the cryptographic signature can be used to verify its origin.

For the longer-term travellers, a smart card makes a lot of sense. Have some kind of up-front cost (possibly just that you must buy a minimum of, say, $10 of credit) to cover the cost of production, and then reduce the per-trip costs to the point where no sane person would regularly travel on single-trip tickets (goal #4: don't make bus drivers fiddle with coins all the time). The card itself should store a very small amount of information; a card identifier and an account identifier would be sufficient (or even just the card identifier, with account information centralized too). A single account could plausibly have multiple cards associated with it (a company could have a fleet of tickets, all paid for on one bill), or a lost/stolen card could be disabled and a replacement sent out at a reasonable cost.

The smart card does not need to store much information, but it may be beneficial to have it cache some information. However, this cache would be invalidated by fleet ticketing and external top-up, and should not be relied upon. Important: For security, the card itself must NOT be the primary location for any significant data, especially not financial data. Such data must be stored centrally, to prevent unauthorized modification. As the Boston/MIT ticketing hack demonstrated, storing financial data on the card itself is inherently insecure; even encrypted, it is accessible, and brute-force attacks are straightforward when there is nothing to prevent rapid testing of keys.

The obvious consequences of central storage of all data are the significant server load, and the communication requirement. Server load can be handled by industry-standard database engines*. Requiring that all validators be networked, though, may pose a problem. It would require that every bus, every tram, and every railway station have a way to query the server and get back a response in no more than 250ms; any more than that, and the passenger has already moved away from the validator and the next person is trying to touch on/off. The card need not remain in contact with the reader for the full quarter-second, however; if the server reports insufficient funds in that account, the reader can beep a warning. With validator-controlled barriers, the networking requirement is slightly tighter; ideally, 100ms response time.

For railway stations, the solution is simple: Run a wired network along with all the other railway infrastructure. This will give easily enough throughput for 100ms response time. Trams may also be able to add infrastructure. For buses, however, it's not as easy. One option would be to use the 3G (mobile phone) network; another would be to have points at every stop where the bus can communicate back with base. Either way, the system must cope with the possibility of dropouts, by queueing requests and delivering them when able. With touching on, this may permit small-scale fraud; it would be possible for someone to board at a place known to have no coverage - by the time the driver is aware that someone is on the bus without a valid ticket, he's moved on, and probably doesn't know who of his passengers it is. However, it would not be difficult to "blacklist" any card (by its ID) when it goes below, say, negative $10, and have the blacklist cached on each validator, so that it will be refused if it attempts to validate when there is no contact.

Adding money to a card is of course crucial to the utility of the system. People should be able to recharge their cards at any railway station, via the internet, and possibly other ways. Of utmost importance is that this process be secure; if a fraudulent recharge station puts money onto a card without accepting money, the entire system collapses. The simplest and best solution here is to use public/private key encryption; digitally sign and encrypt all communication, and have every recharge station use a unique key pair. If anything is found to be suspect, that station can be configured "down", and it will then deny and be denied. This encryption and verification will add load to the servers, but recharging is a far less common operation than validating, and the comparative load would be insignificant.

There are a number of ways in which a malicious person might attempt to subvert this system. With sufficient logging, all can be recognized before they become a major problem, and dealt with or undone.

  • Attempting to edit one's own smart card will merely result in the invalidation of the cached records.
  • Sending forged recharge requests will be impossible without a genuine recharge station's private key.
  • Tampering with a recharge station to make it give free recharges would be detected as soon as its money box is counted, and could result in it being administratively disabled.
  • Recharging ridiculously small amounts (eg 20c at a time) in order to clog the servers could be prevented with a simple minimum recharge amount.
  • Randomly "zapping" other people's cards (eg by passing a validator through a crowded train) with a direct-charge marker (eg one used for buying a daily newspaper, as per MyKi's proposal) would log entries with the vendor's identification, similarly to current behaviour with credit cards.
  • Failing to validate could be conveniently checked for with hand-held validators.

In conclusion, there is nothing required which is beyond the realm of current technology; Melbourne deserves better than either Metcard or Myki has delivered - so far.

* In peak time, there will be roughly 2000 validations/minute; each validation might result in 1-2 database updates. A good database engine, running on a well-built cluster server, should be able to handle this without problems; see for example in which MySQL and connected open source software achieves 200,000 TPS, which would be enough for probably 50,000 validations per second, or 300K per minute. This is an extreme, but even with simpler and cheaper hardware, 5000 queries per second is an easy target for any good database engine; it may even be possible to manage on a single (non-cluster) server.

Wednesday, 5 January 2011

Problems fscking Ubuntu 10.10

Most of this evening was spent battling a problem with an ext4 hard drive, and Google didn't turn up as much hit quality as it usually does, so I'm documenting the process for future reference.

The most important thing to note is that the entire problem was caused by improperly shutting the computer down. Powering down in the middle of bootup is a bad idea. Obviously there are times when it's unavoidable, but let this wasted evening be a cautionary note: If you can do an orderly shutdown, do.

In theory, the system should fsck (that's File System ChecK, not profanity) on boot up; for some reason, this wasn't working. I haven't plumbed the reason for this, but presumably the normal thing is to simply have a delay on next bootup, and things proceed as normal.

Instead of booting normally, it dropped me into a repair/recovery console. Unfortunately this console did not have an fsck command available, and attempting to mount /dev/sdc1 (the root partition) simply froze the computer.

Next attempt: Boot the Ubuntu 10.10 install CD, and hit Ctrl-Alt-F1 to go to tty1. It had the fsck command, but whenever I tried to use it, it told me that the file system was unavailable, possibly because it's mounted. It did not seem to be, though, and I have no idea why it would have been. Attempting to mount the drive caused the exact same hang as from the recovery console. A slight variant though, I discovered:

$ sudo -i
# mkdir harddrive
# mount /dev/sdc1 ./harddrive
Ctrl-Alt-F2 to go to tty2
$ ps -A
$ kill [pid of bash that's running the sudo -i]

It was possible to kill bash and leave mount running. But it still wasn't possible to do anything with mount. Similarly, proceeding to the next step in the installer caused it to attempt to mount the hard drive, causing the same hang.

Ultimately, I found a solution: Boot some Linux _other than_ Ubuntu. I used a Slax ISO from to get a workable console, then was able to fsck /dev/sda1 (for some reason it became the first drive, whereas under Ubuntu it kept being the third - possibly something to do with how the BIOS handled PATA vs SATA, and presumably inconsequential). It found some errors, fixed them, then dropped me back to the shell. I tried mounting the file system to see if it looked alright, but Slax's mount told me that ext4 was an unknown partition type, and flat out refused. Why would it be known to fsck and unknown to mount? Doesn't make sense!

But fortunately, that Slax fsck was enough to get the drive functional again, and everything booted quite happily after that. It does worry me, though, that (a) Ubuntu couldn't fix the problem, and (b) the Ubuntu installer was unable to simply format the disk and start fresh (which, ultimately, should always be possible - no matter how much file system damage you have). Googling for 'ubuntu mount hang' was surprisingly unproductive, so I've posted this here in case anyone else has similar trouble.

We return you now to your regularly scheduled programming.

Sunday, 2 January 2011


Since it's been mentioned in a couple of places, I feel I ought to explain myself on this one. What's reiplophobia, why do I have it, and where did the term come from?

I'll start with the easy one. Reiplophobia is the fear of rebooting a computer.

Why? What's so bad about a reboot? Several reasons. Firstly, rebooting is the mainstay of apathetic, careless, or incompetent tech support. "You have a problem? Reboot your computer." Since this frequently achieves the short-term goal, it's held up as the ultimate solution to all problems. This is a bad thing for the long-term health of the system, because the problem has only been dodged, not solved.

Secondly, restarting a computer destroys all transient data. This is more serious in some situations than in others, but if you have any unsaved data and you need to reboot, you have to either delay the reboot while you deal with it, or lose whatever hadn't been saved. If all you're doing is editing files, you might think that periodically pressing Ctrl-S will ensure that you lose nothing; but most editors maintain a lot of less obvious state, such as cursor position and undo stack, which can be extremely valuable in figuring out where you were up to in some major task. Being able to set something aside for a week and pick it up where you left off is a valuable thing, and most editors won't fully support this if they've been restarted.

But most importantly, a reboot means downtime. Yes, downtime! and you shuddered at the very word. Actually, you probably didn't, because you don't have the same pathological hatred of outages that a network admin does. (Unless you do, in which case, I applaud you sir.) To my mind, a service should always be available. That means that a server should always be up. Imagine if the internet was built on a 99% uptime rule - every little piece would just randomly be unavailable 1% of the time. You'd be forever frustrated; first your ISP drops you out for a few minutes, then you can't find the site you want, and then you find the site but it's decided to go out for a smoke (you DO know that computers run on smoke, right? Let any of it out and they stop working) and won't respond to you till it's back. Completely unacceptable. So the only option is to aim for 100% uptime. No downtime. Ever. And that means... never reboot. If a problem can be fixed without a reboot, then fix it without a reboot, and the server will keep running all through.

(Technical note: It would be more accurately described as an aversion, not a fear, but everyone understands what a phobia is.)

So what's with the name? Reiplophobia? Did it come up in your alphabet soup this morning?

Not quite; it came up in my CONFIG.SYS a few years ago. The term "REIPL" is from IBM; I first met it in a directive which has for years been my favorite example of IBM-speak:

It's perfectly obvious what this does; it causes fatal errors to be logged to a file called C:\POPUPLOG.OS2 - why didn't you understand that right away?

IPL stands for Initial Program Load, and is IBM's term for, well, booting a computer. Yeah. IBM don't want to call it "boot", it has to be "IPL". So "reIPL" means "reboot" (and I've seen this term in a couple of places, but I'm pretty sure they all derive from IBM). If there's no REIPL= directive in CONFIG.SYS, then fatal errors will be dumped to the screen and the system halted; specifying a drive letter (that's what the C means) causes the system to log to a file on that drive, and then automatically reboot. It makes some (a little) sense that way, but it's still a fairly incomprehensible construct. You have to know what it does before you can understand it.

So there you have it. Reiplophobia: the fear of rebooting a computer.