The go flag package has a nice API, but the user interface it creates is straight from its Rob Pike's Plan9 roots, and Linux users familiar with getopt command line interfaces will find it strange and perhaps even ugly.
I wanted to create command line programs that look like traditional GNU/Unix utilities, so I wrote the https://github.com/ogier/pflag package. It's a drop-in replacement for the flag package, just change the import line. It also adds some basic features, such as the common pattern of a flag with a long-hand form usable with a double --dash and a shorthand character with a single -d.
The Go flag package has nothing to do with Plan 9. Like Unix, Plan 9 had single-dash single-letter options ONLY. The long option syntax of getopt is a BSD/GNU-ism, not from Unix.
The Go flag package's command lines adopt the observation made by the Google C++ command line flag parser (https://code.google.com/p/gflags): the long/short --/- two-names-for-every-flag dichotomy was useful for backwards compatibility when recreating original Unix commands, but for new programs, it doesn't pull its weight. At the cost of requiring that each option be specified with a separate command-line argument, you can drop the distinction between - and -- entirely. (This is similar to what X11 programs do, although I believe they do not admit --x as a synonym for -x.)
It's not clear to me what is "ugly" about not having to keep alternating between - and -- as you spell out options on the command line.
> the long/short --/- two-names-for-every-flag dichotomy was useful for backwards compatibility when recreating original Unix commands, but for new programs, it doesn't pull its weight.
Short is more usable for users on the command line. Long is more maintainable in scripts. That's why it makes sense to have both.
> At the cost of requiring that each option be specified with a separate command-line argument, you can drop the distinction between - and -- entirely.
Again, that cost is high for users on the command line, one of the few places where every keystroke does matter. It's a lot more pleasant to be able to do:
$ rm -rf
Than:
$ rm -r -f
Or, worse:
$ rm --recursive --force
Orthogonality is definitely an important principle, but when it comes to usability, it often makes sense to offer multiple ways to accomplish the same goal for users in different circumstances.
Ultimately, you're interfacing with a greater primate. Even with all of our nice features, we still don't support USB or web standards. Instead, it's all lossy OCR and weird muscles. You should expect a certain amount of... squishiness... when interfacing with something like that.
Re command line vs scripts, I am not convinced that the mental overhead of knowing two names for each flag and flipping back and forth when you context switch between the command line and a shell script really makes that much sense. Certainly common flags should be short and less common flags need not be, but personally I'd rather have one way to spell each. Certainly having just one name per flag is more in keeping with Go trying to keep things simple.
My officemate is a long-time Git user but was surprised the other day to see me type "git pull -r". He didn't know that -r was short for --rebase, and in fact I didn't know that --rebase was long for -r. This kind of mutual incomprehensibility is a tiny variant of the observation that every C++ programmer uses only a small subset of C++, but the problem is that each programmer chooses a different subset. Not having two names for a flag can be a feature.
Re typing, I am also not convinced that the overhead of rm -r -f vs rm -rf is really that high. Of course, rm is a bad example: if you are implementing rm, you MUST use the Unix conventions and accept -rf. But for new commands not burned into decades-old muscle memory, it's not clear that it's needed.
I think there are multiple valid design decisions that could be made here, and we've tried to tilt the balance toward simplicity, but of course other balances could be struck, and maybe in other contexts it might make sense to strike a different one.
I tried to make sure it was possible that someone could write a package (say, flg) with the following API (and nothing else):
// Package flg implements a getopt-compatible command line flag parser.
package flg
// Parse parses the command-line flags defined in package flag.
// In contrast to flag.Parse, Parse imposes getopt semantics:
// - single letter flag names must be specified with a single dash: -x
// - longer names must be specified with a double dash: --long
// - the argument to a single-letter flag can follow it immediately:
// -xfoo means -x foo when -x takes an argument.
// - multiple short flags can be combined: -xyz means -x -y -z,
// when neither -x nor -y takes an argument.
// - name aliases can be introduced by calling Alias before Parse
func Parse()
// Alias records new as an alias for the flag named old.
// Typically old is a long name and new is a single-letter name or vice versa.
// For example, Alias("r", "recursive").
func Alias(new, old string)
I believe it is, although I haven't seen one. That would let programs still use package flag as the data definition, so that they can reuse any other "plug ins" for the package (like custom flag.Values) but still provide a getopt-style parser if desired.
I would argue (at the risk of being down-voted) that if you are scripting tools then it is important to use the long option, not just because future-you might forget what the options are (relevant XKCD http://xkcd.com/1168/), but also for searchability, it's rather difficult in almost any engine (except the wonderful new code-orientated ones that are cropping) up to search for `-xvzf`, but rather simpler to search for `extract verbose zip filename +tar`, for example.
That's really what munificent was arguing too though -- that the long-form options are preferable for scripting, but that no one prefers "tar --extract --verbose --compressed --file" to "tar -xzvf" when you just have to extract a file on the command line (in fact, many people don't even bother with the "-", "v", or "z", opting for "tar xf"). This optional brevity is important for utilities.
> It's not clear to me what is "ugly" about not having to keep alternating between - and -- as you spell out options on the command line.
It's about the principle of least surprise.
Regardless of whether double-dash opts came from UNIX or GNU (and obviously you are correct in that they came from BSD/GNU), they're the convention nowadays and it's less confusing for end users to just embrace the convention.
Go has bucked the (recent) tradition on a lot of different fronts, which I love it for, but the flag package -- the interface to the users of the software, not the authors -- is one place I don't really think it's quite as appropriate.
Certainly, Google is able to make these sort of changes in its internal world, and reach a high enough percentage of software that the new choice becomes normal, but Go will probably never have enough mindshare in the extraGoogle for its CLI parsing to not be "a little weird".
---
As an addendum, I happily use the builtin flag package for software that is consumed primarily by my team. For software I expect other teams to use, who don't necessarily care about the fact that it was written in Go, I prefer pflag.
People insisting that "XYZ is not UNIX", Unix or however you want to capitalize it, is kinda pointless. For all practical purposes Linux is Unix, FreeBSD, OpenBSD and OSX are Unix.
One would think that if any OS would foster the idea of sameness it would be those belonging to the Unix family.
As for old vs new style command line options: that's now Unix in much the same way that Dave Gilmour was a proper part of Pink Floyd and not merely "the new guy".
The original OS we call Unix has not been relevant to the discussion for 20-25 years.
Unix now refers to a family of operating systems -- not a single OS.
Thinking of Unix as a specific OS and a specific trademark in 2015 is just odd since the majority of people using various forms of Unix today were not even born when the distinction was even relevant.
And just as with other families, the peculiarities of a single member does not automatically apply to the entire family.
There are so many existing scripts, books and guides that use "rm -rf", "du -hs", "ls -la" and so on that those will have to be supported forever.
For many people, those sorts of commands are 80% of what they run on the command line. Having it so that the other 20% consisting of new programs works different is confusing and annoying.
There is a difference between writing systems in an environment where you have a centralized flag parser and flag inheritance across google3, and shipping a binary to users who expect things to behave like getopt. Also, let's be honest, flags in google3 are a nightmare. They are certainly well-engineered but lost any semblance of sanity a long time before you and me got there. The kernel patch springs to mind.
Having to rethink my entire career of something simple like "how to pass flags to a binary" had better bring significant improvement, and flag does not. flag regresses a lot of convention about the operator interface to a program. This:
I understand the argument for self-explanatory command lines but at the cost of unexpected (and far more verbose) behavior is a tough trade off. One thing I haven't seen discussed is how "required" options are encouraged by flag because of how difficult positional arguments are. That's in addition to everything in the article, which I agree with.
Python argparse gets this nearly perfect, subcommands and all. flag is near the top of what I hate about Go as an SRE. Java too.
The maximum command line size allowed by the Linux kernel used to be 128kB. This might sound crazy, but that limit was a real constraint for some Google services. My guess is that this is referring to a kernel patch to increase the maximum size.
This is a somewhat educated guess. A long time ago at Google I used to work on the program that was used to launch most jobs on the clusters. To help mitigate the command line length problems, the launcher preferred generating command lines with -nofoo instead of --foo=false, and -foo instead of -foo=true. This saved a few characters per boolean flag. Of course it also caused no end of trouble for people who'd defined their own "three-state boolean" flags that could e.g. be true/false/autodetect. Not the finest hour of engineering, in retrospect.
Shouldn't programs simply support a @args.txt argument which references a file containing more arguments to the program? Most build tools I've seen do that and it makes command-line length limits mostly a moot point.
Packages like this are where I find the debate between idiomatic Go usage and general use cases difficult to side on. I would err on the idiomatic side by using stdlib flag.
"Use stdlib unless you can name a reason good enough to incur the cost of importing another package, which shouldn't be underestimated" is probably the short version of the idiomatic Go position. If you've only got three flags, there probably isn't any reason to use anything other than stdlib; you can't be winning big enough to pull in another package. If you're solely being driven by external scripts, it's harder to win than if you expect interactive usage. Of course, if you're writing a grep replacement and you've got half the alphabet in both capitalizations being used and half of them take arguments and a couple of things switch "modes"... well, by all means don't write that yourself, go get a package that does it well. Most of the stdlib is designed to hit the 95-99% use case, but if you're really pushing something hard you may need something else.
Probably the only truly controversial bits of the Go standard answers are A: pulling in external packages isn't free and extremely frequently underestimated over the life of the product, they better be bringing some real value beyond merely preventing the developer's nose from crinkling a bit [1] and B: if you've only got three flags right now, don't go leaping to the conclusion you're in the process of writing a grep replacement; wait until you actually have the situation I described before dropping another package in (YAGNI).
[1]: Contrast this with something like Node or Ruby that seems to encourage pulling in dozens of libraries, many of which may literally be single-digit lines of payload code, and it is considered totally worth it for developer tastes to alter significant elements of the language/runtime/stdlib. And I really do mean contrast this with... I am currently expressing no opinion as to which philosophy is "better".
Of course if the Go stdlib produced something which looked more like almost every other Unix-like tool out there, then we wouldn't be having this discussion - but it doesn't, and leaving your app's commandline parsing until it becomes a catastrophe to manage (or your users complain) is technical debt whichever way you spin it.
For a language like Go I also strongly question why keeping with the stdlib is necessary. There's value in a language like Python where you the environment is dynamic - Go produces statically linked binaries. Who cares what libraries you build them with? That's kind of the point.
"leaving your app's commandline parsing until it becomes a catastrophe to manage (or your users complain) is technical debt whichever way you spin it."
I wouldn't deny that.
But I do agree with the Go philosophy (now I am expressing an opinion) that says you should also not underestimate the ongoing technical price of taking on a 3rd party library. Most developers consider it "free". In practice, it isn't. It is entirely possible for the value to exceed the cost, but developers rarely make a sensible decision... they just account the costs as zero and go grab the library, and often don't even notice what they're paying, and certainly don't notice what they're costing the team.
(Bear in mind we're talking "stdlib" vs. "grab a library" here, so NIH-syndrome is not currently on the table.)
And you're also really prejudicing the discussion, and indeed probably your own mental visualization, by speaking as if every Go program is a budding "command-line catastrophe" waiting to happen, when in fact BY FAR the most common case is that your program takes zero command line parameters, and the vast bulk of the remainder is five or fewer with no complicating factors like "modes" or something. Those who are programming the remainder are free to grab any of the 5 libraries I saw mentioned in the discussion yesterday (and probably more since I last looked), and nobody is going to complain.
"Who cares what libraries you build them with? That's kind of the point."
The maintainer. Go is generally focused on "serious" programs who will be maintained by teams of developers over periods of time that typically involve 100% turnover (though, hopefully, not all at once). If you don't have that problem, well, Go is going to be less appealing to you and I invite you to consider other options. If you do have that problem, you either A: already know how big a problem that is and will tend to appreciate some of what Go has to say, even if you don't quite agree with the solution or B: in another five or six years will fit into A.
In general I am inclined to agree with you. Given two equivalent libraries, if one is standard and one is pulled from github, the standard library should be preferred. If it were just a matter of developer ergonomics, I would agree with you in this case too.
But the command line interface is the fundamental user-facing piece of a command-line program. If it's not intuitive and clean, the usability of the whole program suffers. The internal API is secondary (this is why I didn't try to "improve" anything here -- go's flag package is good enough and I'm not trying to reinvent the wheel). It's all about a good experience for users, and I don't think Go's stdlib flag package provides that.
Commandline programs written in Go shouldn't feel like idiomatic Go programs, they should feel like idiomatic commandline programs.
If idiomatic Go has you preferring using packages that provide terrible, confusing, non-standard[1] UI to users of your program, then either idiomatic Go is wrong, or needs better interpretation.
It's interesting that you acknowledge that struggle but settle on what I'd consider to be a poor choice for your users. Potato/potahto, I guess.
[1] Whether you like it or not, GNU/BSD-style getopt/getopt_long is the standard these days, by a wide margin.
I wanted to create command line programs that look like traditional GNU/Unix utilities, so I wrote the https://github.com/ogier/pflag package. It's a drop-in replacement for the flag package, just change the import line. It also adds some basic features, such as the common pattern of a flag with a long-hand form usable with a double --dash and a shorthand character with a single -d.