IMO, if you're rerunning gating tests then there's no point in having the tests in the first place. The whole purpose of tests is to find value in their failing, so when you have flakes and rerun multiple times to pass, your tests have lost their value and you have no trust in your codebase.
also... rerunning is annoying when you have to figure out if the problem is the SUT or a brittle test (one upside for me is having spare time in the afternoons to make coffee while waiting for CI lol)
On small teams or projects, this attitude is probably fine.
One of the most frustrating experiences as an engineer is running up against a deadline and just staring at my CI with my finger on the re-run button because my coworkers can't be bothered to believe me when I tell them their tests are flaky.
Fixing your flaky tests is like putting your cart away - a little effort on your part makes a lot of peoples lives a little better.
Yes, we sponsored them because we have a commercial product that uses ESLint and we want to contribute back to open source. I don't see why this is an issue.
We also wanted to show them our metalinter, which we believe has genuine use cases in open source. That's why it's free for open-source repos (all public repos).
The CLI tool is also just free for everyone, for ever.
Well, we built Trunk Check to address some of these issues. Maybe it'll suit your org better.
- We support hold-the-line: we only lint on diffs so you can refactor as you go. Gradual adoption.
- Use existing configs: use standard OSS tools you know. Trunk Check runs them with standardized rules and output format.
- Better config management: define config within each repo and still let you do shared configs across the org by defining your own plugin repos.
- Better ignores: You can define line and project level ignores in the repo
- Still have nightly reporting: We do let you run nightly on all changes and report them to track code base health and catch high-risk vulnerabilities and issues. There's a web app to view everything.
