mirrors/meta-openembedded
1
0
Fork 0
mirror of https://github.com/openembedded/meta-openembedded.git synced 2026-09-22 11:00:49 +00:00
Code Issues Projects Releases Wiki Activity

python3-fastnumbers: fix denoise of large floats when built with clang

Browse Source 
float_as_int_without_noise() returns early when the floored value fits
in a long, checking it with floor_val == static_cast<long>(floor_val).
For values outside the range of long the conversion is undefined
behavior; clang -O2 folds the round trip into "floor_val is integral",
which is always true, so the denoising is skipped:

  FAIL: tests/test_fastnumbers_examples.py:test_try_real
  FAIL: tests/test_fastnumbers_examples.py:test_try_forceint
  assert 3452999999999999737856 == 3453000000000000000000

Add a patch that only does the conversion when the value is in range.
A reduced reproducer returns the wrong answer with the recipe's clang++
at -O2 and the right one with the fix (and with -O0 or gcc).

AI-Generated: Uses Claude Code
Signed-off-by: Khem Raj <khem.raj@oss.qualcomm.com>
This commit is contained in:
Khem Raj
2026-09-12 16:52:16 -07:00
parent 549be445de
commit 9bfe6f0ea9
2 changed files with 47 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
meta-python/recipes-devtools/python/python3-fastnumbers/0001-parser-avoid-undefined-double-to-long-conversion-in-.patch
+46
View File
@@ -0,0 +1,46 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Khem Raj <khem.raj@oss.qualcomm.com>
Date: Sat, 12 Sep 2026 00:00:00 -0700
Subject: [PATCH] parser: avoid undefined double to long conversion in
float_as_int_without_noise
float_as_int_without_noise() returns early when the floored value fits
in a long:
if (floor_val == static_cast<long>(floor_val))
For values outside the range of long (e.g. 3.453e21) the conversion is
undefined behavior. clang (-O2) takes advantage of that and folds the
round trip into "floor_val is integral", which is always true, so the
denoising is skipped and try_real(3.453e21, denoise=True) returns
3452999999999999737856 instead of 3453000000000000000000:
tests/test_fastnumbers_examples.py::test_try_real FAILED
tests/test_fastnumbers_examples.py::test_try_forceint FAILED
Only do the conversion when the value is within the range of long.
Upstream-Status: Pending
Signed-off-by: Khem Raj <khem.raj@oss.qualcomm.com>
---
src/cpp/parser.cpp | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
--- a/src/cpp/parser.cpp
+++ b/src/cpp/parser.cpp
@@ -92,8 +92,14 @@
// If the given float can fit a C long without loss then no need
// to go through the below rounding steps.
+ // Converting a double that is outside the range of long is undefined
+ // behavior, and clang uses that to fold this check into "floor_val is
+ // integral", which is always true and skips the denoising below for large
+ // values. Make sure the value is in range before converting.
const double floor_val = std::floor(val);
- if (floor_val == static_cast<long>(floor_val)) {
+ constexpr double long_limit = -static_cast<double>(std::numeric_limits<long>::min());
+ if (floor_val >= -long_limit && floor_val < long_limit
+ && floor_val == static_cast<long>(floor_val)) {
return val_int;
}
meta-python/recipes-devtools/python/python3-fastnumbers_5.2.0.bb
+1
View File
@@ -4,6 +4,7 @@ LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=d32431d1b650010945da4e078011c8fa"
SRC_URI += "file://0001-parser-avoid-undefined-double-to-long-conversion-in-.patch"
SRC_URI[sha256sum] = "07266a2fca9e08eeb5a6c70be4c8db3637db9b930ea5198facbbb9b0b31d4d03"
inherit pypi python_setuptools_build_meta ptest-python-pytest