The load you're talking about is insignificant -- even 4M entries is going to fit into RAM, so you should be able to perform lookups at wire speed.
The main concern is MYSQL is the single point of failure.
Don't make it a single point of failure, then. Set up several boxes. Heck, don't use MYSQL -- your update rate is going to be low (even if you have 10Gbps of incoming traffic, that's only one new GB file per second), so you could run the file-location service over DNS if you wanted.
Exactly what I was thinking. On a box with a decent amount of memory, MySQL's query cache will easily handle 5k/sec or, probably, quite a bit more. From fuzzy memory of a friend's test earlier this year I think he got about 30k/sec over a socket connection and a little less over tcp; sounds about right to me. And that's nothing but MySQL, default installation, on an iMac. Just make sure you have plenty of memory, that's always the key.
SPOF concerns could be alleviated by having a standard master/slave failover setup. You might have other needs you haven't mentioned but otherwise sounds like MySQL can handle your needs for now, and keeping the setup as simple and off-the-shelf as possible is always a good thing ..
wicked that makes me feel better about it. I was worried a lot about one server being the focus of such a lot of activity. I know it should handle a lot of traffic but it's always best to sanity check thoughts like that (when a crashed server ultimately = lost revnue :P)
Well don't take anyone's word for it, make sure to test it yourself! But I am fairly confident you'll be reassured by the results of those tests. Just make sure you're not actually testing the language the driver is in ...
Do share more details of the setup you end up with eventually, though, it's an interesting topic and I for one will be curious to see what you came up with.
The main concern is MYSQL is the single point of failure.
Don't make it a single point of failure, then. Set up several boxes. Heck, don't use MYSQL -- your update rate is going to be low (even if you have 10Gbps of incoming traffic, that's only one new GB file per second), so you could run the file-location service over DNS if you wanted.