Fix std::float16_t rounding of subnormal ties and fast-path overflow

[jadidbourbaki]

Overview

This addresses #300.

compute_float rounded subnormal ties up instead of to even. For float and double this never shows because no subnormal tie fits in 19 digits. For std::float16_t the ties (2k+1) * 2^-25 up to 33 * 2^-25 do. So 2.98023223876953125e-8, which is exactly 2^-25, parsed as the smallest subnormal instead of zero. Nine more ties with an even lower neighbour parsed as the odd neighbour. The same strings padded with zeros past 19 digits were wrong too, because the too_many_digits check compares compute_float(w) with compute_float(w + 1) and both rounded up. I added the round-to-even test from the normal branch to the subnormal branch. I also widened min_exponent_round_to_even for float16 from -22 to -26, which is where the subnormal ties sit. The test is gated on a compile-time predicate, subnormal_ties_possible(), so that it only exists in the float16 instantiation.

Clinger's fast path can overflow for float16 because 2^11 * 10^4 exceeds 65504. Strings such as 656e2 returned infinity with std::errc() while every other path reports result_out_of_range. I added binary_format<T>::fast_path_can_overflow(), which is true only for float16. The fast path now hands such inputs to the slow path. The check folds to false for the other types.

The double max_mantissa table skipped the 5^14 entry, so indices 14 to 22 were one power of five too small. Results were unaffected. The non-nearest fast path was taken less often than intended. I added the missing entry.

I went through every binary_format constant for the four types and wrote down the condition each one must satisfy. Three were conservative and I tightened them: smallest_power_of_ten for float16 (-27 to -26) and bfloat16 (-60 to -59), and min_exponent_fast_path for float16 (0 to -4) and bfloat16 (0 to -3). The division fast path is safe for the 16-bit types even when the compiler evaluates the quotient in float, double or x87 long double. I checked that exhaustively for every w up to 2^(p+1) and every k. max_digits is one larger than necessary for double, float and bfloat16 and I left it alone.

script/format_parameters.py reads the constants out of float_common.h and checks each condition with exact integer arithmetic, in the spirit of script/mushtak_lemire.py. It also runs a bit-exact model of compute_float for the 16-bit types against a ties-to-even reference on every (w, q) within three of a representable value, a tie or the overflow threshold. It reruns the Mushtak-Lemire convergent search over each type's exponent range. bfloat16 has no subnormal tie under 20 digits and no wrong results.

Testing

• tests/exhaustive16_midpoint.cpp parses the exact decimal expansion of every finite float16 and bfloat16 value, every midpoint, strings just above and below each midpoint, and midpoints padded past 19 digits. The expected bits come from the bit pattern that generated the string. It runs in under a second and is registered under FASTFLOAT_FIXEDWIDTH_TESTS. Against the unmodified headers it reports 18 failures.
• I added cases to tests/basictest.cpp for the ties and for the overflow error code. The constexpr variants of the tie cases fail to compile against the unmodified headers.
• I compared the library against a GMP reference on 10.9 million float16 and 20.1 million bfloat16 strings, the same candidate set the script uses, with no mismatch in value or error code.
• basictest, fixedwidthtest and the rest of the suite pass with GCC 16 in C++23 mode. exhaustive32, exhaustive32_midpoint and exhaustive32_64 pass, so float is unchanged by the shared compute_float edit. The headers build under Apple clang in C++11, 17 and 20.
• python3 script/format_parameters.py reports zero failures for all four types.

Benchmarks

Apple M4 Pro, macOS 26.5, base commit a8a02f7 against this branch, interleaved runs, minimum of 1000 repetitions per run.

For float and double I compiled a translation unit calling from_chars with both header sets and diffed the assembly (Apple clang 16 and GCC 16, -O3). The only differences are the corrected max_mantissa constants. realbenchmark on canada and mesh agrees, with every difference inside the run-to-run noise:

Apple clang, Mfloat/s	base	this branch
canada, double	142.2	142.4
canada, float	142.2	142.1
mesh, double	221.8	241.9
mesh, float	222.4	208.5

realbenchmark has no float16 path, so I timed from_chars<std::float16_t> with the same loop shape (GCC 16, -O3) on two inputs: every finite positive float16 printed as its shortest decimal (31,743 lines), and 100,000 normally distributed values rounded to float16, also printed shortest. Best of three interleaved runs:

GCC 16, Mfloat/s	base	this branch
all float16 values, float16	126.5	130.6
all float16 values, bfloat16	126.0	131.2
weights, float16	164.5	169.2
weights, bfloat16	165.6	163.1

The float control in the same binaries moved by up to 4 percent between runs, so I read these as no change. Few of these inputs take the division fast path anyway: a shortest float16 decimal usually has four or five significant digits, and the fast path needs w <= 2048.

Future Work

I am working on a formal analysis in Lean of compute_float for the 16-bit types, with the truncation and tie lemmas machine-checked instead of argued in comments. The constants that reduce to inequalities on small integers could also become static_asserts.