It's actually worse if it is plausibly trustworthy for "99.9%", since that's enough that naive users will get accustomed to believing the verification badge is authentic.
When a motivated malicious user (who doesn't actually need that much resources) will be able to convince people something is authentic because the verification passes when it shouldn't since naive users are primed to believe it by default.
Read the rest of my comment please. Is the single motivated malicious user able to do as much damage as all of the blocked attempts put together? Probably not, since if there's really all that much riding on it, people will point out it can be bypassed.
Should we also abolish Pangram, because it's not 100% accurate? Someone might be convinced a text is not AI-generated when it actually is! We should get rid of it rather than fool people into thinking it can be determined accurately. What about antivirus? We should abolish it as well rather than fool people into thinking that their software is ever 100% safe. What about HTTPS? We shouldn't call it "secure" shell because the computer you're connecting to could be compromised! I could go on and on and on.
The median instance of AI image generation isn't evidence in a court case. It's cyberbullying, or deepfakes, or fake news. It's called "slop" because there's a lot of it being churned out at low effort.
> Is the single motivated malicious user able to do as much damage as all of the blocked attempts put together?
Yes, absolutely. Probably moreso. The whole point of these proposals is to try to solve for the "motivated malicious user" who is engaging in actual high-stakes fraud. There is no point in applying techniques that suppress inconsequential pranks while making serious crimes easier to get away with.
This really seems like a rehash of the perennial DRM argument: DRM restrictions provably do not reduce large-scale motivated copyright infringement, they just annoy legitimate paying users. This is the same class of solution, in that it is effective only where the stakes are low and the impact is minimal.
I don't suspect there is a uniform spectrum between those two. I think this is something that's going to be clinal, which we see in a lot of other comparable social contexts. The number of people actually willing to cross a moral threshold into outright crime is relatively small, but those are precisely the people who cause the most damage when they get away with their behavior.
Bur I don't even really think that's really relevant anyway, because whatever the density of "malicious" motivations is, the point here is that the fact that it only is an effort/motivation threshold that allows this technique to "block" malicious uses, and the motivation to overcome that threshold correlates directly with the stakes involved in the malicious use.
In other words, the more malicious the abuse is, the less effective this solution will be: the boundary of its usefulness will be wherever the line between pranksters and actual criminals happens to lie.
Your idea of a criminal seems to be hypercompetent and think of everything. These do exist, but most criminals are not very smart. Smart, dedicated, technical people can typically make more money legally.
Your argument applies to any imperfect security technology -- aka practically all of them.
> Your idea of a criminal seems to be hypercompetent and think of everything.
No, my idea of a criminal is someone who is motivated to commit crime, and I feel that we've already established in this thread that the approaches we're discussing are motivation gates far more than competence gates.
> Smart, dedicated, technical people can typically make more money legally.
Then who's been running all the botnets, writing cryptolocker malware, and running phishing scams for the past couple of decades?
We've always had script kiddies, and now we have people using AI itself to do malicious things. Technical skill has never been an obstacle for sufficiently motivated scammers.
> Your argument applies to any imperfect security technology -- aka practically all of them.
Ultimately, everything has weaknesses, and with enough effort, most measures can be circumvented. But how much effort is enough varies wildly between solutions.
There's a huge gulf between a "no trespassing" sign, on the one hand, and a concrete wall topped with barbed wire, on the other. The "no trespassing" sign only keeps out people willing to obey it; the concrete wall keeps out anyone who isn't willing and able to accept the time, effort, and risk necessary to climb over it or knock it down.
And the point is that using digital signatures to distinguish AI-generated media from hand-made media is much closer to the "no trespassing" side of things than it is to the wall. Maybe it's analogous to a gate with a latch you can open from the other side if you reach over in just the right spot.
I'm afraid it doesn't. To the contrary, treating effective security in terms of how it influences the attacker's cost-benefit tradeoffs, and evaluating proposed measures on whether the effort threshold they create is enough, is quite literally the opposite of black-and-white thinking.
And in this case we can clearly see that a solution that (a) does not substantially increase their costs -- and in fact, as I've pointed out above, only really filters by motivation, not by time, money or effort, and (b) doesn't target their potential benefits at all, is one that isn't likely to be effective.
You're conflating the effectiveness of the mechanism with its systemic impact.
* Pangram: Yes we should really be discouraging people from putting trust in tools like this because they can't be made totally reliable.
* Antivirus: We should be building application environments with robust security models so that malicious software has a limited blast radius (like we do on mobile, like the Linux ecosystem is trying to do with Flatpak, etc).
* HTTPS: HTTPS is a strict upgrade from HTTP so we should be using it everywhere possible. The UI symbols to indicate to users the security expectations they're getting are good practice.
* ssh: This is just an inappropriate comparison.
The HTTPS comparison would make more sense if actually 0.1% of the time when their browser said they were using HTTPS it was just lying.
Pangram: I'm not arguing against encouraging skepticism; I'm arguing that the technology is not useless. It's good for people to know the limitations, but it's still evidence.
Antivirus: "Actually, we should build this hypothetical better thing" is a cop-out.
HTTPS: C2PA is a strict upgrade from unsigned photographs, so it should be used wherever possible.
SSH: Totally appropriate, the entire point of the discussion is whether it's permissible to the user that they might be more secure.
You're missing all of the points that there could be by focussing on random people.
While it is always an individual tragedy when people treat each other badly (e.g. through deepfakes and all), the real threat does not exist on that level.
This is about misinformation and disinformation, so we're talking state actors. And with that, the 99.9% hypothesis does not hold true.
It's funny how people always say something is "a tragedy at the individual level" when they mean "it's not my problem." It's even crazier to dismiss the value of a security feature, just because it might make people feel more secure. That's true of every security feature! Very little of the technology that the web is built on is proof against state actors.
I like being contrarian as much as the next guy, but "Actually, having security is worse for security" is taking it a little too far.
I am repeating myself, but this is about systems, and not about people.
It is however in the interest of the people to keep the systems running in an untainted way.
As said, on the individual level it's a tragedy, but one that can be absorbed somewhat. Democracy itself failing otoh is kinda hard to absorb.
C2PA is not "having security". It is "having an illusion of security for compliance and CYA reasons, that can be fairly trivially exploited by nation state actors".
Banality of evil. Again.
___
Actually, come to think of it, "security" is the wrong term there. Signatures don't secure anything. They attest.
Those are different things.
Argh and I ran with your term aah
This is pascal's mugging. Democracy itself might fail! You're just inflating the stakes of your hypothetical bad outcome until it can overwhelm any positive upside.
Not to mention, democracy is under just as much or more threat from "banal" fake news created by citizens. People gonna people. They don't need the DPRK lying to them to fool themselves.
Yeah I think any additional security is good. At worst this could help in a lot of court cases. Someone presents photo evidence - it could be manipulated - it could be not. This happens already. Then someone produces an original higher quality version (like a raw photo - which I take even on my phone at all times now) and experts can verify that as the original.
And I agree on state actors. If a major one is invested in something like this they might have well compromised the signing project itself, the verification process, or even the court system or media. That seems like a rare and extremely high bar to guard against.
No biological process is 100% precise. DNA copying being imperfect is the driver of evolution, which is the prime example of this. What I assume they're referring to is that their ribosomes just "make fewer mistakes" when creating proteins from mRNA.
These "mistranslated proteins" are normally infrequent and benign, and their material is recycled eventually. But that whole process wastes energy, so making a "better ribosome" makes all cellular processes more energy efficient and allows the energy budget to be reallocated.
The thing with Nostr is that the protocol spec expressly forbids relays from forwarding messages to each other.
What this means is that users trying to reach each other need to shotgun messages to many relays, or congregate around specific ones. User profile data will list the relays they listen on, but this suffers from the problem of sticky defaults and makes client authors the kingmakers.
There's lots of centralization pressures like this that the protocol maintainers don't have a good answer to. They tout the simplicity of the protocol, which is often a virtue, but they overdid it and made the protocol too simple to achieve its goals.
> protocol spec expressly forbids relays from forwarding messages to each other.
That statement is false and if you disagree: please provide a source.
There is no such restriction on the NIP (protocol guidelines). I have been writing NOSTR software since years and there was NEVER such restriction in place. In fact, wouldn't even make sense because some relays (e.g. Primal) are super-aggregators for smaller relays.
> users trying to reach each other need to shotgun messages to many relays
This is a false statement. NIP 65 provides a list of which servers the users declares to be using. This way readers for that user know at which door (server) to knock and ask for updates.
there are relays built exactly for this, for rebroadcasting. You can do whatever you want in nostr btw.
this is also not entirely needed since you publish a list of relays you use, and so clients publish notes for you to them and you read from the relays of people you interact with.
there are many different people/teams working on different aspects of nostr, there are no protocol maintainers.
it doesn't explicitly forbdit it. It just doesn't spec it out, because it doesn't need it for the protocol to work. There are already many relays and clients that do just this.
I think GP's point (which I agree with) is that the functionality is almost essential. When it's not part of the spec, then you end up with differing off-spec implementations, and again, centralization risk.
This would be like the HTTP protocol not defining `POST`, and leaving it up to servers and clients to implement it based on however it feels like.
I really like the ideals behind Nostr, but I think its implementation and execution could be better.
It would be more analog to HTTP not specing out how CDNs should work, or Usenet not specing out how DejaNews is going to work. It's infrastructure stuff neither the client nor the simple server has to care about.
The Nostr spec covers what matters, cryptographic identities and unique message ids, that make dumb relays that duplicating messages from elsewhere possible (an area where HTTP or Activity Pub fail at).
But part of the protocol is that user profiles also list the relays where to find their content (where their posts go to) and a second user connects to those relays to fetch their content directly?
I couldn't disagree more. Juniors have very poor design sense and can't guide the AI to land in the right spot. Consistently on my team the developers who are the most reliant on AI are causing me the most trouble. They produce a lot of code but constantly make the same mistakes and can't seem to learn and improve their own design skills, or are doing it at a snail's pace.
A help with this is the sun is to the left and it seems to be midday, so you could answer the "cardinal direction" question just from the picture with "west ish", which is what it turns out to be.
There's a reason for this. Rob Pike was asked about it and said that syntax highlighting reminds him of the bright colors of children's toys and he personally disables it so that he can focus on the text.
I don't know why it's still like that but that's the original reasoning.
> Syntax highlighting is juvenile. When I was a child, I was taught
arithmetic using colored rods
(http://en.wikipedia.org/wiki/Cuisenaire_rods). I grew up and today I
use monochromatic numerals.
The reason traffic lights are colored is because they are showing distinct states of the same thing and the color is the means of differentiating.
Same for transit maps: different routes are colored to distinguish them from other routes, which is especially useful if they overlap.
But that’s not what syntax highlighting does.
The equivalent of your examples would be to not highlight the syntax at all, but only use color coding to distinguish variables.
The equivalent of how syntax highlighting currently works for your examples would be if the light fixture was one color, and the light pole was another, but then all the actual lights were the same color.
I actually think highlighting only the variables with distinct colors could be extremely valuable. Would certainly help avoid mistakes with nested i/j loop counters.
Edit to add: come to think of it, it would have been even more valuable in Go, until recently anyway. The variable color coding would expose the common loop variable instance bugs, because a programmer would be instantly puzzled by the unexpected coloring.
That does exist, it's called semantic highlighting.
In any case, my analogy was not perfect, but neither was Russ's! The point is it's totally normal and not "childish" to use colours to help distinguish things. Traffic lights do not technically need colours (you can use the position of the lights - I assume that's what badly colourblind people do). Nor do transit maps technically need colours - you could just label the lines, or use patterns.
Your comment makes me think of LabVIEW - if you're not familiar with it, it uses a visual programming language ("G") in which data flows down wires. The color of the wire indicates the wire's data type (blue for ints, orange for floats, green for bools, pink for strings) and the width of the wire indicates the dimensionality of the data (thin line = scalar, thick line = 1D array, double-thick = 2D array). The color is more than decorative or even assistive - it's essential to understanding the program.
I still wonder why every open-source visual programming language is either a toy for teaching or straight up awful, often not implementing but even loops, when LabVIEW has been doing it right for decades.
Despite its huge size and it installing several services that constantly run in the background, it's still one of my favorite "languages" of all time. It's the only one I've ever seen people going from never having programmed before to making simple but meaningful contributions in within a single day.
There are a bunch of identical things on my screen called lines. They contain a bunch of mostly identical things called tokens. Syntax highlighting uses different colors to identify the different purposes of all of these
tokens blasted onto my screen.
> Gofmt was written to reduce the number of pointless discussions about code formatting. It succeeded admirably. I'm sad to say it had no effect whatsoever on the number of pointless discussions about syntax highlighting, or as I prefer to call it, spitzensparken blinkelichtzen.
> When I was a child, I used to speak like a child, think like a child, reason like a child; when I became a man, I did away with childish things.
I sincerely hope Rob Pike was being sarcastic/ironic, because otherwise, he sounds insufferable
He sounds insufferable because he's calling syntax highlighting childish. Not simply politely saying it's not his thing. It's like he can't understand why anyone else would use it, or isn't bothering to. Maybe there's more nuance to what he said, but it's not obvious from the quote.
"Everyone who uses syntax highlighting is a child"
I mean, maybe he was genuinely trying to make a joke, but I'm well past the point of assuming random strangers are joking when they're being offensively dumb.
Unlike formatting preferences, the colours I use in my editor don't effect others. That said I might not mind living in a world where everyone used the same highlighting scheme, as long as it was reasonable ;)
That makes sense as a personal preference for him, but it's odd for that to still be the company/project stance. Like, surely he knows he's the minority for not wanting highlighting?
Apart from the fact that your comment is against the rules: please don't trivialize that quote. It has a profound meaning, and is completely out of place here.
There is a thread linked where we literally see gophers express the belief that they do not like syntax highlighting, and mock those who do - because Great Leader - born in 1956 - does not like syntax highlighting. Its called double think.
that's an extremely odd explanation and it makes me think that he has some hidden PTSD. it's also insane that one person's preference trumps the rest of the world's.
I understand and respect this position. I think syntax highlighting
is a highly subjective matter, bordering on personal preference with regard to shell interactions, editor configurations, bindings, shortcuts, snippets, and the like. It's also... insignificant somehow, like quibbles over formatting rules that Go settled once and for all with `go fmt`.
I often prefer not to enable syntax highlighting just for color. Occasionally I'd choose some minimal theme that only highlights string literals and keywords. So it has two or three colors. But some of the color schemes I see are a festival of lights where every special element of syntax has its own color.
I don't understand how that is supposed to help me parse anything and why the rules are complex. The `range` keyword needs to be purple, and `chan` must be navy blue. Why exactly? And every site has a different color scheme? There is no consensus, and there shouldn't be.
For a serious community-driven project like Go, dealing with the question of syntax highlighting is strange. The creators deliberately avoided the questions of IDEs and editors for Go, leaving them to the community. I think the same principle applies here.
> The creators deliberately avoided the questions of IDEs and editors for Go, leaving them to the community. I think the same principle applies here.
Exactly. This is fundamentally about accessibility (in the broadest sense). Do people have opinionated screenreader settings they like to force on others too?
There is such thing as criminal negligence. Not saying that's what happened here, but people do sometimes go to prison for being uniquely bad at their jobs.
Debugging concurrency issues isn't a syntactic process so they have to resort to println debugging. This works but isn't exhaustive and burns a lot of tokens.
Rust's std::sync::mpsc contains most of the hazards of Go's channel, and arguably adds some because the Go runtime provides deadlock detection that Rust lacks.
Intrinsics usually come after new instructions have existed long enough for the higher level pattern across several architectures to establish a common pattern. If specific hardware is being targeted you may need to use those instructions before intrinsics exist.
intrinsics are nicer in every way, assuming they exist. but some instructions are inherently non-portable. performance instruction like my popcnt example are good candidates since they can be implemented at varying costs on other architectures. but for systems programming there are control register and mode switch instructions that really aren't. some of those can be put into separate asm routines, but there are some that are poorly suited. segment long jumps on x86 are maybe an example. rdtsc is another one potentially.
its also true that when I unwrap my new spin with fancy new instructions its unlikely to have a robust set of instrinsics around them.
inline asm is a real mess, I always regret tussling with it, but its kind of pragmatically necessary if you're actually working at the metal in a high performance or embedded context unless you're doing the whole thing in assembly.
When a motivated malicious user (who doesn't actually need that much resources) will be able to convince people something is authentic because the verification passes when it shouldn't since naive users are primed to believe it by default.
reply