I'm really not a fan of the decision to overhaul Vim script. It seems this is driven from a desire for Vim script to be faster [1]. I've not needed Vim script to be "faster" but I don't use very complex plugins. That said a lot of Vim script is written in sub-optimally. Using a more functional approach is faster (though still not "fast"), e.g.
$ cat loop.vim
let sum = 0
for i in range(1, 2999999)
let sum += i
endfor
quit
$ cat fold.vim
call eval(join(range(1, 2999999), '+'))
quit
$ time vim -u NONE -S loop.vim
vim -u NONE -S loop.vim 7.08s user 0.09s system 99% cpu 7.198 total
$ time vim -u NONE -S fold.vim
vim -u NONE -S fold.vim 2.03s user 0.13s system 99% cpu 2.166 total
I think its cryptic reputation comes form the use of normal mode commands and regex within scripts. There certainly are warts, like the user's 'ignorecase' setting permeating to many places in the runtime. Which I'd love to see fixed and whilst may also represent a breaking change for some scripts would be far more easily adoptable.
Meanwhile, on a computer that’s benchmarking within 20% of your figures:
• CPython 3.9.1 segfaults on a direct port of fold.vim after a few seconds. (It generates the 22,888,887-character `1+2+3+…+2999999` string in mostly under a second, then hits a stack overflow in the eval() call, presumably in parsing.)
• CPython 3.9.1 completes it with a loop in under a second;
• CPython 3.9.1 completes it with functools.reduce in half a second;
• CPython 3.9.1 completes it using sum() in about 150ms;
• Node.js 15.6.0 completes it with a loop in about 150ms;
• Node.js 15.6.0 with --jitless completes it with a loop within 300ms;
• Perl 5.32 completes it with a C-style loop in under 300ms;
• Perl 5.32 completes it with a foreach loop on a range in about 160ms;
• Perl 5.32 completes it with List::Util::sum in about 600ms.
(I didn’t try porting fold.vim to JavaScript because it lacks the primitives to make that a sane operation; or Perl, because my Perl skills are rusty enough that it would take a few minutes to sort it out, and I was lazy.)
So yeah, other scripting languages’ alternatives are pretty much just 5–20× as fast across the board, and often even more than that—and worse still, the performance pitfalls in Vimscript are often not what someone familiar with performance might expect—few would expect your fold.vim to be faster than your loop.vim.
Vimscript is routinely painfully slow to the point that it can easily be a serious bottleneck on anything involved, e.g. a fancy indent script, or some script that you use to process a file.
I know nothing about Vim script, but since they're overhauling it, wouldn't it be a better option to rely on python scripting by providing a plugin API? Of course there's backwards compatibility, but they could always support both for a few years. Having to support your own language always seemed like a waste of time to me.
See the "PHASING OUT INTERFACES" section of the link in my comment for Vim's reasoning of moving away from this. Personally I broadly agree with it as I've seen many people struggle setting up plugins that depend on other things like python.
That said the approach Neovim took of shipping Lua built-in addresses the same portability/ease/reliability of use. In retrospect I think this is a better course of action.
I agree. I haven't given Neovim a chance yet and I'm not a huge fan of Lua but it would certainly be a pretty good upgrade compared to clunky Vimscript.
Vimscript feels like a configuration file format gone wild (probably because it's sort of what it is) and it's really annoying to use in my experience. Coming from Emacs it was really a bad surprise: say what you want about elisp, it at least feels like a proper programming language.
Overhauling vim script feels forced by Neovim's successful integration of Lua. But instead of adopting a sane language, or adopting Lua so Vim and Neovim can continue to be compatible, he decided to write yet another homebrew language.
I'm on the fence about this, I don't think Bram or other Vim devs are malicious towards Neovim. But the repeated implementation of equivalent features with incompatible APIs (terminals, async, and now faster scripting) doesn't look good.
I think it is useful to keep in mind that Vim has been around since 1991. And in light of that I can see how Bram and the other devs may have a very different view on where the future of Vim should be from where the NeoVim authors think that NeoVim should go.
Personally I think the fork to NeoVim was a good thing, giving us an alternative to Vim, and that convergence in itself is not necessarily a goal.
Just like with how I am happy that macOS, FreeBSD, and Linux all exist for me to use, and that Windows exists for others to use. Imagine if the whole world decided that only Windows should exist. I would not want to go back to Windows. Yet at the same time I recognize that Windows is the best choice for many other people.
Same way here, I switched to NeoVim, but I am happy that Vim continues to exist. Just like I am happy that CLion exists which I also use, and I am happy that other people have emacs and Visual Studio and VS Code and all of the other editors and IDEs.
[1] https://github.com/brammool/vim9/blob/master/README.md