that's the tldr. we used fuse and we learnt we shouldn't for a sandbox filesystem

interested. is the split for dedup, parallel pulls, or lazy loading specific files? maybe all.

we've played with some chunking ideas on our end but haven't landed on a format. drop a link when it's out.

All of the above, plus being able to reflink to skip copies of large files, plus not having to round trip from disk a few times for tar layers, plus a number of other side benefits. Only using lazy loading for buildkit right now, as it does require FUSE and I want it to be opt in (for robotics contexts, for instance, you never want to lazy load).

fair point. that's the title we used on our X article and i copy-pasted. updated the article's actual title.

Wow. Just seeing this. I've not done proper benchmarking yet but rn we are lagging behind in file I/O for the OverlayFS impl

There was a period where NFS was faster, particularly on windows and OSX where you were paying a double indirection.

Overlays are always tough because docker doesn’t like you writing to the filesystem in the first place. The weapon if first result is deflection; tell them not to do it.

I had to put up with an old docker version that leaked overlay data for quite a while before we moved off prem.

They are Linux VMs and you can host any executable that can work on that. The python/node environment you see is part of what makes the SDK work. Really, it's very similar to Docker in use.

thank you. Is there any "docker host" or centralized repo where I can pull VMs from?

We support just Docker hub for now. Let me know if you want any other OCI-compatible registry.

PS: microsandbox will likely have its own OCI registry in the future

Your statement initially went over my head. Sorry lol. You can always download the installer script and audit yourself. I will set up proper distribution later.

In case you're interested when you set up proper distribution, I'm working on an open source solution aiming to improve security of downloads from the internet. Our first step is maintaining a mirror of checksums published in GitHub releases at https://github.com/asfaload/checksums/. If you publish a checksums file in your releases it can automatically be mirrored. The checksums mirror is not our end game, but it already protects against changes of released files from the time the mirror was taken. For anyone interested: https://asfaload.com/asfald/

.. did exactly that and also changed the BINDIR and LIBDIR to another location. BTW, amazing project from initial glance. Will give it a detailed look this weekend!

It will be maintained as I will be using it for some other product. And it will be audited in the future but it still early days.

> can you share some thoughts on how you compare or future direction?

Microsandbox does not offer a cloud solution. It is self-hosted, designed to do what E2B does, to make it easier working with microVM-based sandboxes on your local machine whether that is Linux, macOS or Windows (planned) and to seamlessly transition to prod.

> Do you also use Firecracker under the hood?

It uses libkrun.

Self-hosting is definitely something we are keen to explore as most of the cloud solutions have resource constrains (ie, total active MicroVMs and/or specs per VM) and managing billing gets complicated even with hibernation features. Great project and we'll definitely take it for a spin

They can. I need to improve the doc. Working on that right now

You are right. We leverage libkrun. Libkrun uses virtio-mmio transport for block, vsock and virtio-fs to keep overhead minimal so we basically depend on any perf improvement made upstream.

Firecracker is no different btw and E2B uses that for agentic AI workloads. Anyway, I don't have any major plan except fix some issues with the filesystem rn.