My observation is that it is not completely true. I constantly have a close to full partition in ext4 and there is no secrets when there is no space the filesystem WILL create a bazillion of small extents because, it has no physical choice! and everytime you have a non maxed out extent, it will create an overhead I don't see how magically you can avoid that. And it is not like a few, on my main working partition, extends of a few KB by the thousands per files are created instead of the optimal size of 128MB. If you just use defrag in read mode on it, it is red from top to bottom, you cannot find a block of few MB with no fragmented file in it.
What actually triggered me to do that was to be able to test repeatably my other open source project in rust were the compile target directory can go to hundreds of gigabytes of files and at the same time I need to benchmark it on large linear slabs we do create for memory mapped files and I definitely noted the impact. I can only guess but non-optimal extent length create a repeated round trip to the metadata in main memory instead of "linear reads" (as consecutive logical numbers) + maybe the fact that those SSD are probably super smart and prefetch the next sectors? (see the jitter numbers I get even on a slow device in other comments)
Yes: I discovered that tested performance on copper-rs a high performance OS for robotics when I run it on ext4 and what is annoying in robotics is the max latency & jitter. Especially that we allocate large slabs so this is not helping at all.
(fragmented relative to baseline)
bandwidth: 100.65 -> 94.74 MiB/s (-5.86%)
elapsed time: 339.550 -> 360.699 s (+6.23%)
mean latency: 9935.7 -> 10554.6 us (+6.23%)
p50 latency: 6715.2 -> 6838.9 us (+1.84%)
p95 latency: 19394.4 -> 20599.9 us (+6.22%)
p99 latency: 131498.2 -> 131988.3 us (+0.37%)
max latency: 183713.2 -> 624392.9 us (+239.87%)
jitter stddev: 17823.1 -> 19806.0 us (+11.13%)
jitter CV: 179.38 -> 187.65% (+4.61%)
Also check your nvme granularity with fstrim -D if you are on 4KB your nvme is so fine grain that it doesn't matter much but if it is 64KB like my main one. Ouch, those small files in the middle of the 64KB won't magically go away.
I discovered that tested performance on copper-rs a high performance OS for robotics when I run it on ext4 and what is annoying in robotics is the max latency & jitter. Especially that we allocate large slabs in copper so this is not helping at all.
This is on a slow device but it might be worse on a fast one as the extra allocation of
small extents start to hit harder on the host side:
(fragmented relative to baseline)
bandwidth: 100.65 -> 94.74 MiB/s (-5.86%)
elapsed time: 339.550 -> 360.699 s (+6.23%)
mean latency: 9935.7 -> 10554.6 us (+6.23%)
p50 latency: 6715.2 -> 6838.9 us (+1.84%)
p95 latency: 19394.4 -> 20599.9 us (+6.22%)
p99 latency: 131498.2 -> 131988.3 us (+0.37%)
max latency: 183713.2 -> 624392.9 us (+239.87%)
jitter stddev: 17823.1 -> 19806.0 us (+11.13%)
jitter CV: 179.38 -> 187.65% (+4.61%)
Also check your nvme granularity with fstrim -D if you are on 4KB your nvme is so fine grain that it doesn't matter much but if it is 64KB like my main one. Ouch, those small files in the middle of the 64KB won't magically go away.
This is true if you can scale out (ie you can add resources to your system). But for a robot for example, just adding a GPU can just flop your product completely: you need more battery, more weight, suddenly your unit economics is out of the window... Your next hardware iteration will be very slow to come and very expensive.
So here, you better not have a system wasting too much resources pretty early on after the prototype phase.
It is more like a "design by committee" issue to me. The decisional structure needs to be built for some opinionated decision. When you have a committee there is not one clear thing they are solving for as everyone has an agenda to tug the language towards their own interests. The result of that can certainly be inaction because it is easier to say no than yes.
but check my other comments, it makes a difference, even on a slow nvme. Probably the overhead of the wasted small extent, and maybe prefetching?
reply