> On Linux, by default only the superuser can set the append-only flag. That severely limits its usefulness for many applications
Ah, good point.
> Get one to add it, and convince the others to copy it. If you can do that, then getting it added to the POSIX standard is likely to be easy.
In actuality it usually works in reverse: Red Hat (now IBM), which has the most dominate presence on the committee, convinces POSIX to add or modify something, and then everybody else adopts it. Examples: asprintf, fmemopen, stpcpy, vdprintf. I can't really think of an example that worked the other way around, though there may be some in the next specification.
The fact that nobody is certified to V7 is beside the point. Nobody is certified to the latest HTML5 standard, either, AFAIK. The point is to provide a shared target. Few people want to copy Linux or glibc because the semantics are invariably underspecified and in part accidental, and people would prefer to avoid those aspects even if they have no other choice but to nominally adopt the interface. Standardization provides a chance to clarify behavior and to fix the bounds of what a portable program can expect long term. If I have to support epoll + kqueue in perpetuity (and I assume I do), I'll structure my program differently.[1] However, if I want to use pselect (or the forthcoming ppoll), I'll target the POSIX-defined semantics, provide a best effort wrapper (or none at all), and tell users on slower evolving platforms to complain to their vendor.
The Linux kernel strives for strong ABI backward compatibility, but it hardly has a perfect track record in that regard (e.g. sysctl(2)), and the future is even more unclear given the various directions it's being pulled. Even POSIX can change course, but it does so more methodically than with a Github code search. And while Linux doesn't make promises regarding POSIX compliance, it's less likely to break POSIX semantics than its own semantics, ceteris paribus. Whether that's because kernel developers value POSIX compliance, or merely because POSIX-vetted semantics are narrower and less accidental is irrelevant.
[1] Also, I'll take the bet that epoll is never standardized, precisely because of its accidental and sometimes very undesirable semantics as compared to kqueue. At best we'll get a greatest common denominator interface that can be built around both--or possibly taking into consideration Solaris ports.
> In actuality it usually works in reverse: Red Hat (now IBM), which has the most dominate presence on the committee, convinces POSIX to add or modify something, and then everybody else adopts it. Examples: asprintf, fmemopen, stpcpy, vdprintf.
I'm not convinced your timeframe is right. stpcpy was added to FreeBSD in 2001, didn't officially become part of POSIX until 2008. Red Hat didn't invent stpcpy either; it was invented on the Amiga in the 1980s, and the GNU project adopted it from there and then Linux acquired it from the GNU project. I think if we studied the history of the other functions you mention, we'd also find that their formal addition to POSIX wasn't the primary cause of their spread, just a formal recognition of an existing de facto reality.
> This function was added to POSIX.1-2008. Before that, it was not part of the C or POSIX.1 standards, nor customary on UNIX systems. It first appeared at least as early as 1986, in the Lattice C AmigaDOS compiler, then in the GNU fileutils and GNU textutils in 1989, and in the GNU C library by 1992. It is also present on the BSDs.
OpenBSD didn't adopt stpcpy until 5.1, circa 2012 (https://man.openbsd.org/stpcpy) and NetBSD until 6.0, also circa 2012 (https://man.netbsd.org/stpcpy.3). In that time frame there was a flurry of activity in both projects regarding POSIX compliance.
My experience with the others was similar, though not uniformly. Support on OpenBSD, NetBSD, macOS, and AIX typically post dated their addition to POSIX. Whether they would have been added independent of POSIX I can't say, but there are plenty of GNU extensions that remain unsupported, and some, like strerror_r (specifically the return type and behavior on error), that likely will never be.
While not specifically on point, I think it's noteworthy that the proposal to add stpcpy to C2X was made by Martin Sebor, a Red Hat employee: http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2352.htm Red Hat doesn't share the seemingly pervasive cynicism regarding standardization. Precisely what motivates that, I'm hesitant to speculate. Like most things I'm sure there are mixed motives.
macOS's man page says "The stpcpy() function first appeared in FreeBSD 4.4". I believe FreeBSD 4.4 was released in September 2001. So that's approximately 7 years before it was added to POSIX. Given that timeframe, it seems unlikely that POSIX has much to do with its presence in FreeBSD. A much more likely explanation is to ease porting of GNU projects (and software developed on Linux) to FreeBSD.
From checking opensource.apple.com, I conclude that OS X added it in 10.3, released in 2003, so that's around 5 years before POSIX 2008. Again, given the timeframe, it seems hard to argue that POSIX triggered Apple's action here.
Given Linux, FreeBSD and Darwin all already supported it, I wonder to what extent NetBSD/OpenBSD's decision was motivated by formal POSIX conformance versus improving compatibility with Linux/FreeBSD/Apple. It is hard to say, but I think the later was likely just as important as the former, and given the importance of the later, they might still have added it even if it had never formally made it into POSIX.
> Given Linux, FreeBSD and Darwin all already supported it, I wonder to what extent NetBSD/OpenBSD's decision was motivated by formal POSIX conformance versus improving compatibility with Linux/FreeBSD/Apple. It is hard to say, but I think the later was likely just as important as the former, and given the importance of the later, they might still have added it even if it had never formally made it into POSIX.
One of the benefits of standardization is, "here is a definitive list of things that need to be added". With rare exception (e.g. async I/O), there's no hand-wringing regarding whether a POSIX interface should be added. Search for the keyword "POSIX" among the OpenBSD 5.1 (circa 2012) changes at https://www.openbsd.org/plus51.html Now search for Linux. For whatever reason stpcpy isn't listed (a tweak is listed in plus52.html), but you'll see where the emphasis lay.
Linux POSIX conformance is incredibly good, particularly at the libc level notwithstanding lack of formal certification[1], so invariably it's tough to say, absent direct confirmation, what was on people's minds. But at least on the OpenBSD mailing-lists, more often than not the explicit reason is POSIX compliance. And I get the same impression from reading NetBSD changelogs. FreeBSD is unique because they've been far more proactive with not only adding POSIX interfaces, but also adding their own extensions. And historically they were a bigger player and in many ways the heir to the "BSD" mantle; certainly more so than NetBSD. But pre-existing support doesn't negate my point (see plus51.html for evidence concerning the immediate motive behind adding stpcpy), nor does the fact that an interface supported by multiple vendors is more likely to be adopted by POSIX. I could likewise dredge up some SysV and other extensions that weren't supported in Linux or glibc until they were adopted by POSIX.
Which isn't to say platforms don't adopt Linux interfaces (whether or not they originated with Linux) with an eye toward Linux compatibility; of course they do, and do so increasingly. For example, I know that OpenBSD refactored their getpeerid interface as a wrapper around newly added getsockopt+SO_PEERCRED support, which I've always assumed was a nod to Linux compatibility, or at least an admission of the obscurity of getpeereid. OpenBSD also adopted MSG_NOSIGNAL (POSIX, SysV'ish, Linux) instead of the SO_NOSIGPIPE that already existed on FreeBSD, macOS, and NetBSD (likewise for F_SETNOSIGPIPE and O_NOSIGPIPE supported by macOS and NetBSD), which goes to show that OpenBSD doesn't merely add whatever interface might marginally aid portability.[2] MSG_NOSIGNAL was a no-brainer as compared to the alternatives, and that would have been true regardless of whether Linux supported MSG_NOSIGNAL. If you want to add an interface, the default answer is "yes" if it's POSIX; if it's not POSIX the default answer is "tell me more". POSIX is self-justifying. That difference in friction might seem de minimis, but ask anyone who has maintained or tried to contribute to a large open source project. The ability to appeal to a standard model which already has buy-in by the project is a major convenience in terms of everybody being on the same page.[3] In that way it can drive behavior unintentionally, which isn't an accident as it pertains to the the salience of a formal standard. Of course, Linux is a de facto target for most projects, and one of far more concern than nominal POSIX conformance, but that doesn't diminish the value of POSIX targeting, and IME neither has it diminished the interest in POSIX conformance among the people for whom it actually matters--those interested in portability, which tend to be disjointed from the set of people for whom POSIX is a dirty word.
[1] IME it's better than macOS even though macOS is UNIX03 certified. glibc and musl take POSIX very seriously, both the letter and the spirit. Developers from both projects actively file tickets on https://austingroupbugs.net to remediate errors and omissions in the text, and problems with actual semantics, explicit and accidental. Various BSD developers are also active there, but I get the sense of a pecking order, if only because of Red Hat's large presence (literally and figuratively).
[2] Contrast that with kqueue. FreeBSD, macOS, NetBSD, and OpenBSD kernels have diverged significantly since kqueue was adopted. You can't copy+paste kqueue-related kernel code across them. But they still look to each other for prior art when it comes to filling gaps or extending the behavior of kqueue, and uniformity is given significant weight. (Especially across *BSD. macOS extensions don't always make the most sense, like with macOS's poorly considered EV_OOBAND. See https://sandstorm.io/news/2015-04-08-osx-security-bug).
[3] "Running a successful open source project is just Good Will Hunting in reverse, where you start out as a respected genius and end up being a janitor who gets into fights." https://diff.substack.com/p/working-in-public-and-the-econom... Fights over whether or not to support POSIX are relatively rare. The ones that happen are infamous because they're the exception.
Ah, good point.
> Get one to add it, and convince the others to copy it. If you can do that, then getting it added to the POSIX standard is likely to be easy.
In actuality it usually works in reverse: Red Hat (now IBM), which has the most dominate presence on the committee, convinces POSIX to add or modify something, and then everybody else adopts it. Examples: asprintf, fmemopen, stpcpy, vdprintf. I can't really think of an example that worked the other way around, though there may be some in the next specification.
The fact that nobody is certified to V7 is beside the point. Nobody is certified to the latest HTML5 standard, either, AFAIK. The point is to provide a shared target. Few people want to copy Linux or glibc because the semantics are invariably underspecified and in part accidental, and people would prefer to avoid those aspects even if they have no other choice but to nominally adopt the interface. Standardization provides a chance to clarify behavior and to fix the bounds of what a portable program can expect long term. If I have to support epoll + kqueue in perpetuity (and I assume I do), I'll structure my program differently.[1] However, if I want to use pselect (or the forthcoming ppoll), I'll target the POSIX-defined semantics, provide a best effort wrapper (or none at all), and tell users on slower evolving platforms to complain to their vendor.
The Linux kernel strives for strong ABI backward compatibility, but it hardly has a perfect track record in that regard (e.g. sysctl(2)), and the future is even more unclear given the various directions it's being pulled. Even POSIX can change course, but it does so more methodically than with a Github code search. And while Linux doesn't make promises regarding POSIX compliance, it's less likely to break POSIX semantics than its own semantics, ceteris paribus. Whether that's because kernel developers value POSIX compliance, or merely because POSIX-vetted semantics are narrower and less accidental is irrelevant.
[1] Also, I'll take the bet that epoll is never standardized, precisely because of its accidental and sometimes very undesirable semantics as compared to kqueue. At best we'll get a greatest common denominator interface that can be built around both--or possibly taking into consideration Solaris ports.