> Unfortunately, the last step is a “draw the rest of the owl” moment. For us, this is deeply entwined in our collab layer, so we don’t have helpful code to share here.
By "the last step" here, do you mean the "do a three-way merge yourself" step? If you do, I would still be interested in hearing about why its "deeply entwined" even without sharing code.
Would for you to expand more on why you're bullish on synthetic hydrocarbons (also called e-fuels or power-to-gas). Is there a specific company you're bullish on, such as Prometheus Fuels or Terraform Industries?
Just that there's going to be a lot of cheap renewable energy available and finding ways to turn that into something useful will likely involve generation of various chemicals like methanol, ammonia, methane and sustainable aviation fuel from green hydrogen at prices that fossil fuels won't be able to match when externalities are included, and in some cases even without.
IRA subsidies have kick started this in the USA already. The main input cost is green hydrogen and the main cost input to that is electricity and I'm very bullish on wind and solar continuing to get cheaper and cheaper.
I wasn't aware that was the current consensus, due to a lack of knowledge around this domain. Prometheus Fuels' process to generate ethanol doesn't see to require green hydrogen as an input step, so I assumed other production processes were also going to get around needing that input?
Don't feel obligated, but I would appreciate a quick link to something I could read about green hydrogen uses other than "burn this instead of fossil fuels"
What are some examples of the "enormous tool stack" required for XML? I ask, because I came into software development after everyone adopted JSON. When I do need to parse XML, there was a library I could use, although I will admit that needing xpath was a bit annoying.
If your XML is written the way people write JSON, then the stack isn't enormous. But XML is usually wrapped in layers of additional complexity. SOAP envelopes and namespaces they require, XSLT that someone invariably used to write an XML transformer, etc.
Also broken parsers. So many broken parsers. Honestly this is what keeps JSON going: the parser was simple because the language was simple, and as a result any JSON you got basically worked the same way.
Not so with XML: all the parsers were insanely complex with the namespacing and whatnot feature support and possible external URLs and everything else...and as a result however no XML library was ever adequate to interface with anything. On multiple occasions generally the best way to build XML for something was to take a working copy, and then glue text together so you would exactly replicate whatever that specific application wanted, rather then trying to use anyone's library for it.
This was before my time, but I believe the WS-* series of specifications is an example.
> Like with the original J2EE spec, which sought to complicate the basic mechanics of connecting databases via HTML to the internet, this new avalanche of specifications under the WS-* umbrella sought to complicate the basic mechanics of making applications talk to each other over the internet. With such riveting names as WS-SecurityPolicy, WS-Trust, WS-Federation, WS-SecureConversation, and on and on ad nauseam, this monstrosity of complexity mushroomed into a cloud of impenetrable specifications in no time. All seemingly written by and for the same holders of those advanced degrees in enterprisey gibberish.
> When I do need to parse XML, there was a library I could use, although I will admit that needing xpath was a bit annoying.
It sounds a bit like someone paved a garden path for you by that point. One of the reasons for the "enormous tool stack" wasn't just depth of tools needed ("tool X feeds tool Y which needs tool Z to process namespace A, but tool B to process namespace C, …"), but also the breadth. I recall there were at least six types of parsers to choose from with all sorts of trade-offs in memory utilization, speed, programming API: a complicated spectrum from forward-only parsers that read a node at a time very quickly but had the memory of a goldfish through to HTML DOM-like parsers that would slowly read an entire XML document all at once and take up a huge amount of memory for their XML DOM but you could query through the DOM beautifully and succinctly. (ETA: Plus or minus if you needed XSD validation at parsing time, and if you wanted the type hints from XSD to build type-safe DOMs, etc.)
A lot of XML history was standards proliferation in the xkcd 927 way: https://xkcd.com/927/
XPath tried to unify a lot of mini-DSLs defined for different DOM-style XML parsers.
XSLT tried to unify a bunch of XML transformation/ETL DSLs.
The things XPath and XSLT were designed to replace lingered for a while after those standards were accepted.
Eventually quite a few garden paths were paved from best practices and accepted "best recommended" standards and greenfield projects start to look easy and a simple number of well-coordinated tools. But do enough legacy Enterprise work and you can find all sorts of wild, brownfield gardens full of multiple competing XML parsers using all sorts of slightly different navigation and transformation tools.
The last time I worked with XML, using an external library wasn't really a great option. I ended up writing my own parser in C++. It took about a week to get all the features required for my purpose.
Honestly, if you just need a one-off transformer, VB.Net is probably one of the better options. The .Net XML library is pretty good in the box, and VB.Net has XML literal support on the top... if you just need to read, then C# is a better language imo.
VB.NET is masochism, and XML literals are a dumb, confusing, and unnecessary feature. Creating XML elements with regular code is quite easy, and at scale, you probably want templates.
Yes, YAML does seem to be the "Cloud Native" tax. For all but the simplest of Workflows, it is necessary to use a templating language or one of the Argo SDKs.
We use jsonnet for building and manipulating YAML at the moment and are reasonably happy with it.
I don't understand type systems very well. When you say "check correctness" do you mean something beyond static linting with type-hints, like in Python? Or do you mean something deeper, like in functional languages like Elm and F#?
Also, is there always a trade-off between types and flexible meta-programming? Like, OCaml has meta-programming capabilities, but they make type-checking way harder, according to my PhD friend who's written extensively in Scheme and OCaml.
I mean, I'd like them to work like Python, Ruby and TypeScript, but you're right to say I can't describe why I want this.
Is there some guide I could read about structuring a large Julia project? It was pretty easy to intuit with Python, wherein I would put related files in a folder. But with Julia, everything is everywhere and I'm baffled.
I think that `/src/Name.jl` must have the main module, and `/test/runtests,jl` tests. And the package manager cares about `Project.toml`. But beyond this there are no real rules enforced, although there really seems to be one way to do things.
Here's a much bigger project, organised the same way. `include(file.jl)` literally copies in the text, and it's somewhat conventional to collect all imports & exports in the main file:
Still no sub-modules. No files included in mysterious locations. Methods being defined for functions from elsewhere are all qualified, like `Base.show(io::IO, t::OptimizationState) = ...`
> But with Julia, everything is everywhere and I'm baffled.
This is exactly it. Julia allows you to import and include anything, anywhere. You open a file and it doesn't say anything about where the dependencies are coming from and where this particular piece of code will go. Both of those are defined at the place where this file is included, which itself could be anywhere. It could be a different directory, different file, tucked away in a module. It could be in a dozen other files, or no files at all, and you can't tell from looking at just the source of the file.
Data Scientist implementing Machine Learning algorithms (time series forecasting/regression/analysis, reinforcement learning). Also an educator able to create engaging curriculums and intuitive documentation.
I'm a Data Scientist with 5 years experience researching and implementing Machine Learning algorithms (time-series forecasting, reinforcement learning and others) turning theoretical insights into meaningful applications.
By "the last step" here, do you mean the "do a three-way merge yourself" step? If you do, I would still be interested in hearing about why its "deeply entwined" even without sharing code.