gh-100687: Reduce frequency of overallocation in x_add()

x_add() allocated a digit for the carry out of the top digit on every
call. In most cases that digit is zero and long_normalize() drops it
from the digit count, but the allocation keeps its size, so the result
uses more memory than its value needs for as long as it lives.

Allocate the carry digit only when the top digits of the operands sum to
at least PyLong_MASK, the only case in which a carry out of the top digit
is possible. When no carry digit was allocated the top digit of the
result is at least the top digit of the larger operand, so the result is
already normalized and is returned directly.

This applies to same-sign additions and opposite-sign subtractions, which
go through x_add(); x_sub() is unchanged.

Extends PR 100688 by Mark Dickinson, which contributed the size test and
the tests.

Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Author: 3 people

Lib/test/test_long.py

@@ -1698,6 +1698,21 @@ class MyInt(int):
         # GH-117195 -- This shouldn't crash
         object.__sizeof__(1)
 
+    def test_long_add_overallocate(self):
+        # see gh-100687
+        x = (MASK//2) * (MASK+1)
+        x2 = (MASK//2 + 1) * (MASK+1)
+        z = x + x2
+        self.assertEqual(x + x2, MASK * (MASK + 1))
+
+    def test_karatsuba_single_digit_parts(self):
+        # gh-100687: k_mul() adds the halves of its operands with x_add(),
+        # and those halves can be single digits.
+        a = 1 + (1 << (SHIFT * 70))
+        b = 1 << (SHIFT * 139)
+        self.assertEqual(a * b, (1 << (SHIFT * 139)) + (1 << (SHIFT * 209)))
+        self.assertEqual(b * a, a * b)
+
     def test_hash(self):
         # gh-136599
         self.assertEqual(hash(-1), -2)

Misc/NEWS.d/next/Core_and_Builtins/2023-01-02-14-54-12.gh-issue-100687.yenIag.rst

@@ -0,0 +1,2 @@
+Reduce frequency of overallocation in some cases of multidigit integer
+additions and subtractions.

Objects/longobject.c

@@ -3760,7 +3760,13 @@ x_add(PyLongObject *a, PyLongObject *b)
             size_a = size_b;
             size_b = size_temp; }
     }
-    z = long_alloc(size_a+1);
+    assert(size_a >= 1);
+    /* A carry out of the top digit is only possible if the top digits sum
+       to at least PyLong_MASK; only then allocate a digit for it. */
+    digit top_sum = a->long_value.ob_digit[size_a - 1]
+        + (size_b == size_a ? b->long_value.ob_digit[size_b - 1] : (digit)0);
+    int extra_digit = top_sum >= PyLong_MASK;
+    z = long_alloc(size_a + extra_digit);
     if (z == NULL)
         return NULL;
     for (i = 0; i < size_b; ++i) {
@@ -3773,8 +3779,13 @@ x_add(PyLongObject *a, PyLongObject *b)
         z->long_value.ob_digit[i] = carry & PyLong_MASK;
         carry >>= PyLong_SHIFT;
     }
-    z->long_value.ob_digit[i] = carry;
-    return long_normalize(z);
+    if (extra_digit) {
+        z->long_value.ob_digit[i] = carry;
+        return long_normalize(z);
+    }
+    assert(carry == 0);
+    assert(z->long_value.ob_digit[i - 1] != 0);
+    return z;
 }
 
 /* Subtract the absolute values of two integers. */