Lessons to be learned from the Linode Bitcoin incident
Firstly let me say that I like Linode a lot. They had a promotion running a little while ago which got me going with my first virtual private server (VPS), and I only moved off to somewhere from lowendbox after the promotion because my needs are small (and I wanted to match my spend accordingly). This tweet probably sums things up perfectly:
I first heard about the incident via Hacker News, where somebody had posted a blog post from one of the victims. The comments on both sites make for some interesting reading about security and liability. Shortly later Linode posted a statement. The true details of what went down, and whether it was an inside job as some speculate, or ‘hackers’ remains to be seen. Clearly whoever perpetrated the attack knew exactly what they were after and went straight for it – what law enforcement would normally call a ‘professional job’.
So… what can we learn from this? Here are some of my initial thoughts:
Physical access === game over
You have to be able to trust your service provider with your data. If that data has a cash equivalent of thousands of dollars then you have to be able to trust them a lot. There’s a special sort of service provider that we normally use for this – one that’s heavily regulated and where the customer (normally) gets reimbursed if mistakes are made – we call these banks. Of course regular banks haven’t got into servicing novel cryptocurrencies like bitcoin. Ian Grigg does a pretty good job of explaining why (and indeed why Bitcoin will slip into a sewer of criminality where this incident is but one example).
Bottom line – if you can’t trust the people that have physical access, then don’t do it.
Admin access === game over
If a service provider provides out of band management tools that provide the equivalent of physical access then you also need to trust whoever has access to that too. whilst it’s not clear yet who perpetrated the attack, it is pretty clear that it was done by subverting the management tools. This is particularly true when the management tool has direct control over security functionality, such as the ability to reset the root password.
In this case mileage may vary. Some VPS admin tools provide the ability to reset passwords, whilst others don’t.
Bottom line: Management tools might be convenient for service providers and their users, but can present a massive security back door for any measures taken on the machine itself.
Passwords === game over
If I look in the logs of any of my VPSs then I see a constant flood of password guessing attacks. This is why I either turn off passwords altogether with passwd -l account or disable password login in the SSH daemon.
It seems that at least some of the victims had chosen long, hard to guess (or brute force) passwords, which can raise the bar on how long an attack takes. Few people are that disciplined though. Passwords (on their own) are evil, and should be avoided at all costs.
Of course SSH keys aren’t a panacea. Private keys need to be looked after very carefully.
Bottom line: Disable passwords, use SSH keys, look after the private key.
It seems to have become best practice in the IaaS business to build machines that only work with SSH keys, and where the management console (and the API under it) don’t have security features than can be used to subvert anything done to secure the machine. These two steps go a long way towards ensuring that security in a VM/VPS has a sound foundation.
There is nothing that can be done about physical access (at least until homomorphic encryption becomes a reality) – so if you can’t trust your service provider (or at least get contractual recompense for any incident) then think again.
 Apart from some experimentation I pretty much never actually do much with any VPS that I run. They’re just used as end points for SSH and OpenVPN tunnels for when I want to swerve around some web filters (or keep my traffic from the prying eyes of those running WiFi).
 When I was running a VPS on Linode I took the precaution of adding two factor authentication (2FA) using Google Authenticator.
 Most management functions have serious implications for integrity and availability, but if they can’t hurt confidentiality then that’s a good start.
Filed under: security | 4 Comments
Tags: admin, Bitcoin, console, iaas, Linode, management, password, security, SSH, VM, VPS