As seems to be a common theme, I was getting a 403 error when trying to change the name of this blog. The solution was in this thread – it was the mod_security module of Apache getting in the way, and adding these lines to Apache:

SecFilterEngine Off
SecFilterScanPOST Off

… disabled mod_security for my domain, and let me change the blog name. Personally, I removed them afterwards – I like having mod_security enabled.