Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Erasure encoding does not greatly increase storage space consumption. Consider a 2n or 3n replication factor in systems like hdfs, Cassandra and Riak. In those cases storage space consumption is 200% and 300%, respectively. Erasure encoding allows you to select the percentage increase in space consumed for some level of durability. You can, for instance, confirgure the system to encode a given blob of data at 1.5x (150%) space consumption and then stripe that data across 10 or so locations, only requiring 6 or 7 of them to reconstruct the original blob. Facebook put a post out about this some while back re. their long term picture storage.


You're correct, I mistyped & intended to say what you wrote.


Erase encoding was my first thought too. Here's a link to the paper, for anyone who's interested:

https://research.fb.com/publications/xoring-elephants-novel-...




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: