I've used bazel pretty extensively and I have to say it's not fully baked unless you're using it the way Google does internally (vendor all dependencies) and you're using a language Google does internally.
Additionally, it wants to take over your entire build process end to end. Get used to prefixing every shell command with `bazel run` or `bazel build` because everything goes into a little sandbox and bazel likes to be in the middle of everything.
These tradeoffs make sense at Google scale (the goal of bazel is to have very repeatable builds), but in my opinion it's a huge drag with not much benefit for smaller shops (to be clear, "Oh you only have 200 developers?" Still a small shop by these standards)
I am a minority I feel but everywhere I have ever worked I have always pushed to keep your dependencies close. There was never a way to reproduce a build without doing so.
I know other smart people want badly to trust in outside sources ... but I am old and jaded.
And never skipped a build because npm was down... or before that if I could get a jar from a remote source ... and so on.
Vendoring is totally a thing you can do, but keeping them up to date with upstream (and keeping your bazel BUILD files current) is a full time job. You have to hire someone to do that work, it's not completely automatable.
Absolutely agree. But I meant maintaining BUILD files for code you didn't write can be very time consuming. Even a security-only update can include new source files that you now have to tell Bazel about. It's real work
Bazel encourages you to list geographically diverse sources to download from for workspace setup. That said, you could run a local proxy with something like Squid or Varnish, turn on caching in a remote build server, or set up a mono-repo and never use npm (as Google does), each has advantages and tradeoffs... you could also archive and backup your Bazel folders to hermetically reproduce workspaces and builds if patching is necessary, though you’d have to also archive a copy of your host or build environment if you’re paranoid about host tools or configurations changing.
Thank you for those comments. For my own projects at home I mostly use Haskell, Python, and Java, but sometimes also Typescript, SBCL Common Lisp, Racket, Chez Scheme, and Scala.
I was imagining getting set up to build everything with Bazel after reading the article but your comments are on point and made me reconsider. I used to vender locally Ruby, Merb, Rails, and most Java libraries I used - but I stopped doing that about ten years ago.
Bazel, from the outset, wasn't designed for open source projects that tend to have dependencies with incompatible build systems. It isn't flexible enough.
Some features that I find elegant in Bazel, but probably don't have that much impact on small projects/teams is the whole concept of "visibility"[1]. For large projects, being able to control who can take a dependency on your code is really nice.
This mixes really well with automatically adding Required reviewers in Pull Requests based on which folder gets modified [2][3].
It's also incredibly nice for having a clean deprecation process.
Make visibility private except to current blaze deps, and then deprecate or delete as normal. You can be certain no one new will depend on the old stuff.
As a Googler I am very satisfied with Blaze (the internal version of Bazel) at work. But I just don't want to manage such a complex beast for my own personal projects. Besides the large amount of dependencies to install for a build tool, it demands me to manually rewrite the build script for almost all third party libraries. At Google someone has most likely done that for me, but outside it, I have to do it myself.
This is the catch with Bazel. All of its integration with npm, pypi, crates.io, etc are hacks that essentially vendor the source of your dependencies and attempt to generate BUILD files for those deps with some heuristics. For a lot of packages the heuristics work, but not all. Ultimately bazel needs you to vendor and hand-maintain BUILD files to get a seamless experience.
Unrelated, but the GitHub logo in the corner covers a significant portion of the content for me on mobile. This isn’t too annoying because I can go into reader mode, but I thought I might mention it in the hopes that it gets fixed.
I cannot believe how popular TypeScript has become. Is Microsoft running a huge ad campaign?
I've used TypeScript for frontend work (Angular 2) for about 6 months. I did not renew my contract because I did not enjoy constantly fixing TypeScript configuration, versioning and source mapping issues.
My new company switched to TypeScript for backend work and development across all teams has slowed down noticeably. I spend a lot of my time just discussing with colleagues what types should be called and whether or not we should merge two types together into one or whether or not it makes sense to have two different types to represent essentially the same data in order to enforce different constraints on the user of the data depending on the kinds of functionality that we want to expose.
Then, as we build the project (in an agile way), we have to keep renaming and combining types ALL the time because otherwise we end up with too many different types that sound almost the same and have almost all the same properties but are slightly different semantically.
Also, we spend a lot of time just thinking about what the intention of the user should be and all the ways in which we can impose contraints on them instead of correctly acknowledging that the user will always find some way to do whatever the heck they want and that all this strict typing adds no real value once you introduce the human factor.
Ironically, the so called 'stupidity' which TypeScript aims to protect the code from will find even more compex, more creative and less obvious ways to express itself within that code.
I've had exactly the opposite experience. Our typical JS apps turn into giant hairballs of ad hoc data. Refactoring is basically impossible. We just did our first all-TS project and the difference in maintainability has been night and day. If I have anything to say about it we'll never do another JS project without types.
If you're having this much trouble defining the types in your app I suspect your data structures and flow might need some rethinking. There are way more large codebases written in typed languages than untyped languages so the idea that types are an obstacle to development just isn't convincing.
What are you talking about? Lost productivity compared to what, JS? Merge types? You can just use `&` to combine them if need be? Discussing types? Does it matter that much, and I mean refactoring them isn't a big deal if need be. Two different types represent essentially the same data? You've tried extending interfaces or using generics?
Seems like you lack an experienced TS developer in your team and are contributing your lack of experience as a fault of the language itself. The typing has saved my ass maany times and makes the code in our projects actually maintainable for us and for someone outside of our team's bubble. It might take a while longer to code the app, but it'll take less hours in the long run to keep it safe and working.
> It might take a while longer to code the app, but it'll take less hours in the long run to keep it safe and working.
I'd argue it's faster to market, not just faster to fix, with TypeScript over vanilla JS. Static type checking means significantly better auto completion and the immediate elimination of the vast majority of "stupid me" type JS errors from typos, misnamed arguments, and more.
>> "stupid me" type JS errors from typos, misnamed arguments
That is true but these errors are very easy to fix in any language.
The difficult and most important part of programming is getting the system design/structure/architecture right.
Avoiding bad design is the real challenge of programming e.g. code paths are too long and traverse too many different files, it's not clear where objects are coming from in the code (bad DI), files are poorly named, directory structure makes no sense, exceptions are not thrown or handled correctly, logging is absent or not detailed enough, testing is focused on the least important functionality, validation logic for external data is wrong or hard to follow, system doesn't scale, components don't have clear separation of concerns, component abstractions are leaking lower level concepts into the higher level logic, access control layer has gaps, etc... Static typing doesnt add any value for any of these things.
Static typing can help find class or function definitions and usages in the code more easily but only if the classes, functions and methods are poorly named to begin with (if classes and methods have good, highly relevant names, then they're always easy to find in any language).
For a senior developer; design problems are the real hard problems. The rest is minor stuff and we shouldn't spend all our effort on those.
> That is true but these errors are very easy to fix in any language.
No they're not. Language without static type checks end up with unit tests as an ersatz type check.
> The difficult and most important part of programming is getting the system design/structure/architecture right.
TypeScript is literally JavaScript with typing added. The underlying runtime is the exact same as whatever will be running your non-tranpiled-from-TypeScript JavaScript. The big changes for class syntax and enums are simply more tools in your belt but you do not have to use them. Any valid pure-JS design translates immediately to usage in TypeScript. It's strictly superior.
> Avoiding bad design is the real challenge of programming e.g. code paths are too long and traverse too many different files, it's not clear where objects are coming from in the code (bad DI), files are poorly named, directory structure makes no sense, exceptions are not thrown or handled correctly, logging is absent or not detailed enough, testing is focused on the least important functionality, validation logic for external data is wrong or hard to follow, system doesn't scale, components don't have clear separation of concerns, component abstractions are leaking lower level concepts into the higher level logic, access control layer has gaps, etc... Static typing doesnt add any value for any of these things.
> Static typing can help find class or function definitions and usages in the code more easily but only if the classes, functions and methods are poorly named to begin with (if classes and methods have good, highly relevant names, then they're always easy to find in any language).
Static typing isn't just grep on steroids. It catches entirely different classes of errors as well. Being able to immediately hop from a usage to its declaration is convenient, going in the other direction is amazing.
Refactoring the signature, not just body, of a function and feeling confident that all usages have been updated to reflect the change even before you run the app is empowering. It lowers the bar to change and allows your code to evolve more quickly.
> For a senior developer; design problems are the real hard problems. The rest is minor stuff and we shouldn't spend all our effort on those.
Exactly why analysis of problems that computers can solve better than humans (ex: "Is variable fooBarXyz in scope here?") should be deferred to computers.
>> No they're not. Language without static type checks end up with unit tests as an ersatz type check.
Not so. Checking that an object has a certain property or method is very different from checking if the object is of a specific type. Also, in any case, these kinds of assertions are best inside the test logic than inside the source code. Source code should be about raw logic - verification of the correctness of that logic is the job of tests. So dynamic typing has better separation of concerns.
I started coding with ActionScript 2 and then switched to ActionScript 3 (ECMAScript with types; basically the same as TypeScript) for years starting at a very early age. Then I did Java and C++ while at uni. Then I used JavaScript for several years (and did some half year stints with TypeScript in between) so I have a more balanced experience with both statically and dynamically typed languages than the vast majority of developers (I have over 15 years of experience in total: Slightly more with dynamically typed languages). I also maintain several open source projects in my own time so I work many hours per week. Also, I'm currently working on a TypeScript project (second time in the past 2 years).
Also, I've spoken to several very senior developers (including some influential individuals) who also have broad experience and who share my point of view.
So that's why I feel confident when I say that TypeScript does not add any actual safety. I don't have skin on either side (well actually my current employer uses TypeScript so saying bad things about TypeScript does not align with my career interests) and I really did give both a really fair, thorough trial with an open mind. As I said, I even switched back and forth between them for extended periods to gain more perspective.
Typescript solves a few small but obvious problems and introduces multiple new problems that are worse but less obvious (it makes the details of the code better, but the overall foundation and the process worse).
I am curious, so do you then prefer JS over TS? It still feels to me that you're doing something wrong, and are finding any reason to blame the language. I had very negative views about TS when I did some hobby project with it was it 3 years ago. I felt it only added confusing boilerplate, that hindered my ability to code. But a lot of it was just my own inexperience with the type system, and I was getting stuck many times because I didn't know how to use it. Now it feels like a breeze and I take shortcuts when I need them, and add the interfaces when it feels necessary.
It was a bitter thing to swallow, to admit to myself that I just hadn't understood TS and reverse my views once I got hang of it. Sometimes when you first have a bad experience with well basically anything, you'll be viewing it with the same negative emotion throughout your life. If so many people, experienced and inexperienced developers alike, are saying good things about TS maybe there's something there that you just haven't discovered?
>Typescript solves a few small but obvious problems and introduces multiple new problems that are worse but less obvious (it makes the details of the code better, but the foundation worse).
- Discourages using external libraries because of type name conflicts. Interface and class names must be universal so if two libraries use the same name for different types, then that makes things difficult. It's also a problem if two libraries use different type names to represent the same concept. This encourages developers to reinvent the wheel as opposed to relying on battle-tested external libraries. There is a workaround for name conflicts but it's not ideal.
- Type definitions of third party modules are likely to be or to become out of date.
- In terms of code foundation; the way all the logic fits together in TypeScript is rigid and tightly interlocked around arbitrary labels (type names) that developers have invented (and each developer tends to have a different idea about the meaning and use cases of each type). As project requirements evolve, the underlying logic starts to diverge from the meaning of the labels to the point that they lose their meaning completely. Most large TypeScript projects end up with many types that have slightly different meanings but are meant to be used in different ways and impose different constraints on developers (e.g. User, UserInfo, UserConfig, Account, UserAccount, ReadonlyUser...). You might say that this could be avoided by good programming patterns, but if you look at most large popular TypeScript projects, you will see a lot of strangely named types which only make sense to developers who have been working on the project for a long time.
I assume the point is that creating new types is very simple, so if you need a new one, then you can just make one, call it what you like, put whatever you need inside it, and start to use it. You don't need to have interminable discussions about this.
If you need more stuff in the type later, you can add it. If the name turns out to be wrong, you can change it. If you need to remove something, you can do that. If you just decide that it was a bad idea, you can get rid of it entirely. In each case, you can just make the change, and the compiler will show you all the code that has to change because of it.
If this is some public-facing or versioned thing then some discussion would be warranted - but you would have had to do that already, types or no! (But even if the worst happens, and some new type gets added in, and it's kind of wrong, and it gets exposed anyway: at least you know about it.)
You gave a better answer than I could have done myself :). Yes, more or less so. Sometimes there can be tricky typing issues but in those cases somebody wiser often comes in and the problem is solved together. At times it's better for individual developers to try to solve say a typing problem on their own instead of somebody else giving them an immediate answer. Maybe it decreases productivity a little bit, but in the long run it's better for developers' individual growth. Though incase the problem is too difficult, I can't emphasize enough the importance of the wisdom of experienced developers.
What is your argument? If we are not "discussing with colleagues what types should be called" we are only using TypeScript for no other reason than momentary whim? For his next points, sure you have to give it a pause before you decide to implement your interfaces. I do not know what type of app he is building, but I myself and our team have clear established patterns that we use. And if something bothers someone, they'll be discussed in a code review or be refactored at some point. But without more context, I can't give a better answer.
Anyone had any luck embedding Bazel in another build system, like CMake? Tensorflow's C++ API seems to only support building with Bazel[1], which makes it painful to integrate into a standard C++ build pipeline (don't want the person maintaining the build to come after me with an axe). I'm seriously considering just rewriting to Torch for low latency deployment, which is nicely packaged in the standard manner of library + header file rather than trying to force users to incorporate a heavyweight, esoteric build system just to use the library.
1. https://www.tensorflow.org/guide/extend/cc - "The C++ API is only designed to work with TensorFlow bazel build. If you need a stand-alone option, use the C API" (and the C API page notes that it's "designed for simplicity and uniformity rather than convenience", making it unsuitable as a wholesale replacement for the C++ API).
Eventually they’ll refactor the C++ and Java hard-codes functionality into Starlark rules but until then, they don’t support exporting headers from Bazel C++ yet— https://github.com/bazelbuild/bazel/issues/5192
Have you tried Meson? It's also a breath of fresh air after CMake, but while Bazel is a "corporate" build system (wants to take everything over, assumes one consistent environment), Meson is a "very Unix" build system, it aims for portability, configurability and reproducibility.
It doesn't take everything over. You can use repository rules to build dependencies in whatever way you want.
Also, it's not corporate and not bloated. If you understand the system well enough, you can create support for a language yourself from scratch without spending huge amount of time on it.
I always enjoy reading about build systems. I wish there was a run-down on how internal global build systems work across large companies (eg. Brazil vs. Blaze/Bazel vs. Pants vs. etc etc)
For more examples of how to use Bazel with TypeScript or JavaScript, have a look at the code samples I shared in this somewhat unrelated GitHub issue: https://github.com/bazelbuild/rules_typescript/issues/344 (near the bottom I post a few more realistic ones as I experimented with getting webpack working, etc.)
It’s a possibility that some of this post (the TypeScript-Rollup chaining) might have been inspired by this issue (maybe), but as I note in various places there, it’s been tricky to get these three separate projects to work together — rules_nodejs was recently updated to better work with npm with an escape hatch if your node or compiler code cannot be rewritten to allow Bazel to manage your npm modules - but while rules_typescript was rewritten to support rules_nodejs, the otherwise excellent rules_closure is not updated to support such npm modules.
With few dependencies, and with caching of individual modules, your builds can go lightning fast so you don’t need the watch mode that webpack promotes so heavily. However, ibazel lets you watch your codebase for changes and rebuilds, it just doesn’t inject into your browser and host a web server for you the way webpack does.
I’d say at this point Bazel deserves your experimentation, but until it supports all common JS build tools and scenarios, including linting and IDE integration (there’s some but currently it’s all maintained by groups at Google and not the IDE companies themselves the way other build tools are integrated),so it’ll be a hard sell for most companies to adopt it wholesale, unless, like Dropbox and others, you have a diverse set of programming languages and targets that have shared dependencies that, when changed, you want to trigger partial rebuilds of multiple projects. But the dependencies have to be supported by the rules you’re using and right now, there are some limits to what first-party rules support.
The tools are at a state where a couple people on your team will need to learn Starlark, the Python-like language used in lower-level .bzl files (which can define rules and macros that the BUILD.bazel files that can then refer to) and especially the internals of Bazel including reading source code, a lot of it. This is because the debugger tools are there but not fully bug-free yet, they don’t provide the rich debugging experience you’ve grown to expect with other languages. It’s kind of like debugging and writing Go code early on — you knew once you found the right syntax that it would be obvious but until then it was a lot of experimentation and copying other examples. It’s obvious the teams at Google are pushing Bazel’s development hard, but the Angular team right now is most visible in trying to make things easier for third-party, non-Google development workflows using Bazel. And I’m sure the project will be one to watch in 2019.
From what I gather, bazel is used to build Angular itself but users of Angular mostly use angular-cli which is a webpack wrapper that does not use Bazel.
Bazel seems to be too resource intensive and complicated for use outside of Google - at least for JS projects.
I am not very familiar with build systems but how difficult would it be to make one that wraps the native ones whenever and provides modern features, unlike make.
Additionally, it wants to take over your entire build process end to end. Get used to prefixing every shell command with `bazel run` or `bazel build` because everything goes into a little sandbox and bazel likes to be in the middle of everything.
These tradeoffs make sense at Google scale (the goal of bazel is to have very repeatable builds), but in my opinion it's a huge drag with not much benefit for smaller shops (to be clear, "Oh you only have 200 developers?" Still a small shop by these standards)