Hmm good point. My plan was to allow concepts such as inf, NaN and negative 0 to be available in the abstract, and then convertible to a concrete implementation such as ieee754. But in actual fact it's mostly muddying the waters, and those kinds of values would be better represented as functions that produce ieee754 values rather than abstract concepts that don't really work abstractly anyway (I was thinking that as an abstract concept it could be passed to a general function for any given floating type implementation such as binary float, decimal float, bfloat, etc).

>> unsigned: limited to positive integers and 0 > signed: limited to positive and negative integers, and 0 (but excluding -0)

> These are floating point values pretending to be integers. You should have actual unlimited precision integers with constraints like explicit maximum and minimum values.

The idea here is to have all numbers be treated as reals (in the mathematical sense) for abstract calculation purposes, and then use the pseudo-types "signed" and "unsigned" to represent common invariants on those reals (such as "can only be 0 or an integer" or "can only be 0 or a positive integer"). For example, ('a'~'z'{chars_per_record}){char_count / chars_per_record} (where char_count and chars_per_record are bound from another part of the document) is a common enough method used to envelope data in binary formats. But if char_count = 100 and chars_per_second = 3, you can't accept a fractional count (i.e. it would be an indication that the data in this document is corrupt or malformed).

Only the functions `uint`, `sint`, and `float` would convert a number into an actual expression (provided their invariants are respected).

> The idea here is to have all numbers be treated as reals (in the mathematical sense)

No. You can't represent, compare etc. most real numbers, so they aren't any good for computational purposes.

Practically usable number types include integers and things that have sufficiently number-like behaviour (e.g. fields) and can be represented with a finite number of integers: rational numbers, rational numbers with field extensions, vector spaces, finite sets, intervals, etc.

In a language for grammars you can probably stop at integers; general purpose programming languages might want more (for instance, Python has rational numbers) but advanced number types can usually be implemented with libraries.

I think maybe I'm not understanding you correctly...

In most cases for binary formats an integer type will be expected for 99% of cases. Something like `uint(8,a/b)` will only generate an alternates set within the realm of integers, so a rule containing such an expression as this could only match when this particular byte matches a/b, which can only happen when a/b results in an integer from 0 to 255. For all a/b where this isn't true, the rule will not match.

But support for matching float encodings is still needed, such as for `reading = float(32,0~1);`, which would happily match an ieee754 32-bit binary float approximation of 0.3 or 0.8 or anything from 0 to 1 that can be represented in this float format.

You are talking about the limited precision numbers in data, not about appropriate numbers for use in your language to express counts, repetitions, aggregate sizes etc. They are two related but categorically different kinds of entity, like screws and screwdrivers or sheet music and audio files.

Grammar rules aren't supposed to work "for 99% of cases" depending on complex invisible constraints such as whether finite precision causes floating point arithmetic to depart from exact integer arithmetic.

I'm not saying that grammar rules are supposed to work for only 99% of cases; that comment was referring to the fact that most binary data of interest for binding (counts etc) will be stored in the document as integers. But I'm still not understanding what you're saying.

All counts, repetitions, aggregate sizes etc are going to come from limited precision data in the document being examined (except in cases where you're passing in a numeric literal). In some cases these values will be used directly to produce expressions via functions and repetition, and in others they will be used in calculations that are then passed to functions or repetition to produce expressions.

The grammar rules only match expressions, which are bit patterns, not numbers. Some of those bit patterns are arrived at through functions such as `float()`, taking a number range to give an expression composed of a set of possible (ieee754 binary) bit patterns. The bit patterns that the `float(32,1~10)` expression will match are far more numerous than the bit patterns that the `uint(32,1~10)` expression will match (0x00000001, 0x00000002, ... 0x0000000a), even though they both are taking in the same numeric range (each picks out only the values it can represent as discrete bit patterns from the range). `uint(32,1.5)` is a malformed grammar because it violates the invariant of `uint()`, whereas `uint(32,2)` is fine (giving an expression of the big endian bit pattern 00000000000000000000000000000010).

The problem is slightly subtler and, I need to reiterate, at the grammar level.

Suppose you define some constants a, b, c, etc. with integer values. First of all, it isn't obvious that these constants do not overflow the mantissa size of whatever floating point type you process them as; but we can generously assume that they are representable exactly. Then you use these fake integers to compute something that should be an actual integer, for example the number of occurrences of something as

  (a*b)/c

in one place and

  (a/c)*b

in another place (probably obfuscated by layers of variables and very simple computations that would be harmless with exact arithmetic), and the two values might differ (and they might be both wrong).

So without even getting into the bit patterns of valid "example" values you are unable to specify your grammar reliably. Floating point arithmetic in the grammar is an unnecessary nightmare that you are inflicting to your users.

But these are reals, not floats. With reals, commutativity holds, even though for ieee754 floats it does not.

There's nothing forcing an implementation to calculate (2*6)/3 or (2/3)*6 using binary floats rather than some other method that it can guarantee will give a correct result. These are implementation details, which the grammar doesn't concern itself with. An analyzer could easily discover that the ultimate destination of the calculation is an unsigned integer, and rejig the calculation as necessary to produce the expected integer result (or produce a no-match expression if the result of the calculation would violate the unsigned integer invariant).

Computerized math is hard no matter what you do (rounding, range, precision, overflow behavior, impossible calculations, infinities, etc), and those will still exist whether the grammar prescribes a particular computerized approach or not. So it's better to not force implementation details when you don't have to.

Another thing to consider is that implementations of this metalanguage won't even have to be 100% correct or even handle crazy complex calculations, because real-world data formats won't do such things since they want speed and accuracy in the codecs that don't fall over on platform subtleties. A real world format won't expect the precise bits 00111110100110011001100110011010 (~0.3 in ieee754 binary float 32) for anything, and even if (god forbid) it did, one could just as easily write uint(32,0x3e99999a) instead to make sure there's no mistake (subnormals notwithstanding). You could have provably correct (but slow) implementations, and less-correct-but-super-fast-and-actually-useful-for-the-real-world implementations. A performant implementation might even require for example that calculations whose destination is an integer encoding must be calculable solely using integer math - i.e. (a * b) / c, not (a / c) * b. Nothing wrong with that if it allows you to maximize performance.

On a side note, even the float() function is fraught with subtleties. Different algorithms exist for converting decimal strings to binary floats, which produce subtly different bit patterns depending on the value. We can't get away from that, but once again for the real world it almost never matters because we don't need that level of precision so we just live with it (which is why ieee754 binary has enjoyed such success, and one reason among many why ieee754 decimal is slow to catch on).

The math is pure and should remain pure (especially in the documentation, which this metalanguage is designed for). Making it actually work in silicon is a job for a computer.