Then maybe caveat your posts with that

yeah i really don't like the corpus of statements and it makes me doubt lenz. consider

> “Artificial intelligence will cause widespread job loss among software engineers.”

https://lenz.io/c/ai-software-engineers-job-loss-impact-05e4...

this is a statement about the future. who knows? dataset also includes

> Robots will not replace human teachers in schools in the near future.

or

> Papua New Guinea has very few female members of parliament.

what counts as very few?

> “Taurine supplementation supports mood and emotional health in humans.”

why is this labeled as misleading? i'm not even sure when I'm supposed to use the misleading label

> Anaximander was the first scientist in recorded history.

this is a judgement call as the term scientist didn't exist.

the claims that feel actually solidly answerable seem to have much better LLM performance

Agree that some of the claims are forward-looking. The messiness of the real-world and real-user fact checks. No ground-truth verdicts are provided or used in the study though. It only measures the level of agreement between the selected models, not which one is right on which claim. I.e. none of the claims is actually labelled.

were you involved in making the study? your bio says you work for them so you should probably indicate that in your comments.

lack of agreement when there is no singular correct answer (or any answer at all) isn't a useful metric

I ran into a lot of these kinds of issues when working on the Citation Needed WMF project (and related extensions). Truth is so often very nuanced.

They introduced themselves as the study author here: https://news.ycombinator.com/item?id=48307887#48307899

ah. I missed that.

Most jeeps never go off-road

I think the problem isn't that people don't care. It's that checking is expensive. "Only $15" isn't trivial when there's tons of claims floating around. And even when you do it people return with complaints and you'll have to redo it (see the other comments here).

jython has been basically unmaintained for quite some time

Well, they never made the jump to Python 3. But shipping 2.7 interpreters in 2024 was quite an achievement on its own. So their users already know this pain. And from my experience in academia, python 2.7 and java 8 will probably be used for another 20 years before the last machine running that stuff burns out.

Their variable cost is (basically) the number of tokens. They increased that. I don't get how that saves them money

You can freeze and concentrate a substance without chemically altering it

I think b/w film has a different grain than color. It isn't identical to grayscale color

many people do not think the science against them are credible. foods containing those things are staples of many diets.