Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I'm not saying that grammar rules are supposed to work for only 99% of cases; that comment was referring to the fact that most binary data of interest for binding (counts etc) will be stored in the document as integers. But I'm still not understanding what you're saying.

All counts, repetitions, aggregate sizes etc are going to come from limited precision data in the document being examined (except in cases where you're passing in a numeric literal). In some cases these values will be used directly to produce expressions via functions and repetition, and in others they will be used in calculations that are then passed to functions or repetition to produce expressions.

The grammar rules only match expressions, which are bit patterns, not numbers. Some of those bit patterns are arrived at through functions such as `float()`, taking a number range to give an expression composed of a set of possible (ieee754 binary) bit patterns. The bit patterns that the `float(32,1~10)` expression will match are far more numerous than the bit patterns that the `uint(32,1~10)` expression will match (0x00000001, 0x00000002, ... 0x0000000a), even though they both are taking in the same numeric range (each picks out only the values it can represent as discrete bit patterns from the range). `uint(32,1.5)` is a malformed grammar because it violates the invariant of `uint()`, whereas `uint(32,2)` is fine (giving an expression of the big endian bit pattern 00000000000000000000000000000010).



The problem is slightly subtler and, I need to reiterate, at the grammar level.

Suppose you define some constants a, b, c, etc. with integer values. First of all, it isn't obvious that these constants do not overflow the mantissa size of whatever floating point type you process them as; but we can generously assume that they are representable exactly. Then you use these fake integers to compute something that should be an actual integer, for example the number of occurrences of something as

  (a*b)/c
in one place and

  (a/c)*b
in another place (probably obfuscated by layers of variables and very simple computations that would be harmless with exact arithmetic), and the two values might differ (and they might be both wrong).

So without even getting into the bit patterns of valid "example" values you are unable to specify your grammar reliably. Floating point arithmetic in the grammar is an unnecessary nightmare that you are inflicting to your users.


But these are reals, not floats. With reals, commutativity holds, even though for ieee754 floats it does not.

There's nothing forcing an implementation to calculate (2*6)/3 or (2/3)*6 using binary floats rather than some other method that it can guarantee will give a correct result. These are implementation details, which the grammar doesn't concern itself with. An analyzer could easily discover that the ultimate destination of the calculation is an unsigned integer, and rejig the calculation as necessary to produce the expected integer result (or produce a no-match expression if the result of the calculation would violate the unsigned integer invariant).

Computerized math is hard no matter what you do (rounding, range, precision, overflow behavior, impossible calculations, infinities, etc), and those will still exist whether the grammar prescribes a particular computerized approach or not. So it's better to not force implementation details when you don't have to.

Another thing to consider is that implementations of this metalanguage won't even have to be 100% correct or even handle crazy complex calculations, because real-world data formats won't do such things since they want speed and accuracy in the codecs that don't fall over on platform subtleties. A real world format won't expect the precise bits 00111110100110011001100110011010 (~0.3 in ieee754 binary float 32) for anything, and even if (god forbid) it did, one could just as easily write uint(32,0x3e99999a) instead to make sure there's no mistake (subnormals notwithstanding). You could have provably correct (but slow) implementations, and less-correct-but-super-fast-and-actually-useful-for-the-real-world implementations. A performant implementation might even require for example that calculations whose destination is an integer encoding must be calculable solely using integer math - i.e. (a * b) / c, not (a / c) * b. Nothing wrong with that if it allows you to maximize performance.

On a side note, even the float() function is fraught with subtleties. Different algorithms exist for converting decimal strings to binary floats, which produce subtly different bit patterns depending on the value. We can't get away from that, but once again for the real world it almost never matters because we don't need that level of precision so we just live with it (which is why ieee754 binary has enjoyed such success, and one reason among many why ieee754 decimal is slow to catch on).

The math is pure and should remain pure (especially in the documentation, which this metalanguage is designed for). Making it actually work in silicon is a job for a computer.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: