pFad - Phone/Frame/Anonymizer/Declutterfier! Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

URL: http://github.com/python/cpython/commit/7861dfd26a41e40c2b4361eb0bb1356b9b4a064b

9241e157469407.css" /> gh-111140: Adds PyLong_AsNativeBytes and PyLong_FromNative[Unsigned]B… · python/cpython@7861dfd · GitHub
Skip to content

Commit 7861dfd

Browse files
zoobazooba
authored
gh-111140: Adds PyLong_AsNativeBytes and PyLong_FromNative[Unsigned]Bytes functions (GH-114886)
1 parent a82fbc1 commit 7861dfd

File tree

14 files changed

+533
-26
lines changed

14 files changed

+533
-26
lines changed

‎Doc/c-api/long.rst‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,28 @@ distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate.
113113
retrieved from the resulting value using :c:func:`PyLong_AsVoidPtr`.
114114
115115
116+
.. c:function:: PyObject* PyLong_FromNativeBytes(const void* buffer, size_t n_bytes, int endianness)
117+
118+
Create a Python integer from the value contained in the first *n_bytes* of
119+
*buffer*, interpreted as a two's-complement signed number.
120+
121+
*endianness* may be passed ``-1`` for the native endian that CPython was
122+
compiled with, or else ``0`` for big endian and ``1`` for little.
123+
124+
.. versionadded:: 3.13
125+
126+
127+
.. c:function:: PyObject* PyLong_FromUnsignedNativeBytes(const void* buffer, size_t n_bytes, int endianness)
128+
129+
Create a Python integer from the value contained in the first *n_bytes* of
130+
*buffer*, interpreted as an unsigned number.
131+
132+
*endianness* may be passed ``-1`` for the native endian that CPython was
133+
compiled with, or else ``0`` for big endian and ``1`` for little.
134+
135+
.. versionadded:: 3.13
136+
137+
116138
.. XXX alias PyLong_AS_LONG (for now)
117139
.. c:function:: long PyLong_AsLong(PyObject *obj)
118140
@@ -332,6 +354,50 @@ distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate.
332354
Returns ``NULL`` on error. Use :c:func:`PyErr_Occurred` to disambiguate.
333355
334356
357+
.. c:function:: Py_ssize_t PyLong_AsNativeBytes(PyObject *pylong, void* buffer, Py_ssize_t n_bytes, int endianness)
358+
359+
Copy the Python integer value to a native *buffer* of size *n_bytes*::
360+
361+
int value;
362+
Py_ssize_t bytes = PyLong_CopyBits(v, &value, sizeof(value), -1);
363+
if (bytes < 0) {
364+
// Error occurred
365+
return NULL;
366+
}
367+
else if (bytes > sizeof(value)) {
368+
// Overflow occurred, but 'value' contains as much as could fit
369+
}
370+
371+
*endianness* may be passed ``-1`` for the native endian that CPython was
372+
compiled with, or ``0`` for big endian and ``1`` for little.
373+
374+
Return ``-1`` with an exception raised if *pylong* cannot be interpreted as
375+
an integer. Otherwise, return the size of the buffer required to store the
376+
value. If this is equal to or less than *n_bytes*, the entire value was
377+
copied.
378+
379+
Unless an exception is raised, all *n_bytes* of the buffer will be written
380+
with as much of the value as can fit. This allows the caller to ignore all
381+
non-negative results if the intent is to match the typical behavior of a
382+
C-style downcast.
383+
384+
Values are always copied as twos-complement, and sufficient size will be
385+
requested for a sign bit. For example, this may cause an value that fits into
386+
8 bytes when treated as unsigned to request 9 bytes, even though all eight
387+
bytes were copied into the buffer. What has been omitted is the zero sign
388+
bit, which is redundant when the intention is to treat the value as unsigned.
389+
390+
Passing *n_bytes* of zero will always return the requested buffer size.
391+
392+
.. note::
393+
394+
When the value does not fit in the provided buffer, the requested size
395+
returned from the function may be larger than necessary. Passing 0 to this
396+
function is not an accurate way to determine the bit length of a value.
397+
398+
.. versionadded:: 3.13
399+
400+
335401
.. c:function:: int PyUnstable_Long_IsCompact(const PyLongObject* op)
336402
337403
Return 1 if *op* is compact, 0 otherwise.

‎Doc/whatsnew/3.13.rst‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -587,6 +587,7 @@ Tier 2 IR by Mark Shannon and Guido van Rossum.
587587
Tier 2 optimizer by Ken Jin.)
588588

589589

590+
590591
Deprecated
591592
==========
592593

@@ -1526,6 +1527,11 @@ New Features
15261527

15271528
(Contributed by Victor Stinner and Petr Viktorin in :gh:`110850`.)
15281529

1530+
* Add :c:func:`PyLong_AsNativeBytes`, :c:func:`PyLong_FromNativeBytes` and
1531+
:c:func:`PyLong_FromUnsignedNativeBytes` functions to simplify converting
1532+
between native integer types and Python :class:`int` objects.
1533+
(Contributed by Steve Dower in :gh:`111140`.)
1534+
15291535

15301536
Porting to Python 3.13
15311537
----------------------
@@ -1585,7 +1591,6 @@ Porting to Python 3.13
15851591
platforms, the ``HAVE_STDDEF_H`` macro is only defined on Windows.
15861592
(Contributed by Victor Stinner in :gh:`108765`.)
15871593

1588-
15891594
Deprecated
15901595
----------
15911596

‎Include/cpython/longobject.h‎

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,40 @@
44

55
PyAPI_FUNC(PyObject*) PyLong_FromUnicodeObject(PyObject *u, int base);
66

7+
/* PyLong_AsNativeBytes: Copy the integer value to a native variable.
8+
buffer points to the first byte of the variable.
9+
n_bytes is the number of bytes available in the buffer. Pass 0 to request
10+
the required size for the value.
11+
endianness is -1 for native endian, 0 for big endian or 1 for little.
12+
Big endian mode will write the most significant byte into the address
13+
directly referenced by buffer; little endian will write the least significant
14+
byte into that address.
15+
16+
If an exception is raised, returns a negative value.
17+
Otherwise, returns the number of bytes that are required to store the value.
18+
To check that the full value is represented, ensure that the return value is
19+
equal or less than n_bytes.
20+
All n_bytes are guaranteed to be written (unless an exception occurs), and
21+
so ignoring a positive return value is the equivalent of a downcast in C.
22+
In cases where the full value could not be represented, the returned value
23+
may be larger than necessary - this function is not an accurate way to
24+
calculate the bit length of an integer object.
25+
*/
26+
PyAPI_FUNC(Py_ssize_t) PyLong_AsNativeBytes(PyObject* v, void* buffer,
27+
Py_ssize_t n_bytes, int endianness);
28+
29+
/* PyLong_FromNativeBytes: Create an int value from a native integer
30+
n_bytes is the number of bytes to read from the buffer. Passing 0 will
31+
always produce the zero int.
32+
PyLong_FromUnsignedNativeBytes always produces a non-negative int.
33+
endianness is -1 for native endian, 0 for big endian or 1 for little.
34+
35+
Returns the int object, or NULL with an exception set. */
36+
PyAPI_FUNC(PyObject*) PyLong_FromNativeBytes(const void* buffer, size_t n_bytes,
37+
int endianness);
38+
PyAPI_FUNC(PyObject*) PyLong_FromUnsignedNativeBytes(const void* buffer,
39+
size_t n_bytes, int endianness);
40+
741
PyAPI_FUNC(int) PyUnstable_Long_IsCompact(const PyLongObject* op);
842
PyAPI_FUNC(Py_ssize_t) PyUnstable_Long_CompactValue(const PyLongObject* op);
943

@@ -50,7 +84,7 @@ PyAPI_FUNC(PyObject *) _PyLong_FromByteArray(
5084
*/
5185
PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v,
5286
unsigned char* bytes, size_t n,
53-
int little_endian, int is_signed);
87+
int little_endian, int is_signed, int with_exceptions);
5488

5589
/* For use by the gcd function in mathmodule.c */
5690
PyAPI_FUNC(PyObject *) _PyLong_GCD(PyObject *, PyObject *);

‎Lib/test/test_capi/test_long.py‎

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
import unittest
22
import sys
3+
import test.support as support
34

45
from test.support import import_helper
56

@@ -423,6 +424,150 @@ def test_long_asvoidptr(self):
423424
self.assertRaises(OverflowError, asvoidptr, -2**1000)
424425
# CRASHES asvoidptr(NULL)
425426

427+
def test_long_asnativebytes(self):
428+
import math
429+
from _testcapi import (
430+
pylong_asnativebytes as asnativebytes,
431+
SIZE_MAX,
432+
)
433+
434+
# Abbreviate sizeof(Py_ssize_t) to SZ because we use it a lot
435+
SZ = int(math.ceil(math.log(SIZE_MAX + 1) / math.log(2)) / 8)
436+
MAX_SSIZE = 2 ** (SZ * 8 - 1) - 1
437+
MAX_USIZE = 2 ** (SZ * 8) - 1
438+
if support.verbose:
439+
print(f"SIZEOF_SIZE={SZ}\n{MAX_SSIZE=:016X}\n{MAX_USIZE=:016X}")
440+
441+
# These tests check that the requested buffer size is correct
442+
for v, expect in [
443+
(0, SZ),
444+
(512, SZ),
445+
(-512, SZ),
446+
(MAX_SSIZE, SZ),
447+
(MAX_USIZE, SZ + 1),
448+
(-MAX_SSIZE, SZ),
449+
(-MAX_USIZE, SZ + 1),
450+
(2**255-1, 32),
451+
(-(2**255-1), 32),
452+
(2**256-1, 33),
453+
(-(2**256-1), 33),
454+
]:
455+
with self.subTest(f"sizeof-{v:X}"):
456+
buffer = bytearray(1)
457+
self.assertEqual(expect, asnativebytes(v, buffer, 0, -1),
458+
"PyLong_AsNativeBytes(v, NULL, 0, -1)")
459+
# Also check via the __index__ path
460+
self.assertEqual(expect, asnativebytes(Index(v), buffer, 0, -1),
461+
"PyLong_AsNativeBytes(Index(v), NULL, 0, -1)")
462+
463+
# We request as many bytes as `expect_be` contains, and always check
464+
# the result (both big and little endian). We check the return value
465+
# independently, since the buffer should always be filled correctly even
466+
# if we need more bytes
467+
for v, expect_be, expect_n in [
468+
(0, b'\x00', 1),
469+
(0, b'\x00' * 2, 2),
470+
(0, b'\x00' * 8, min(8, SZ)),
471+
(1, b'\x01', 1),
472+
(1, b'\x00' * 10 + b'\x01', min(11, SZ)),
473+
(42, b'\x2a', 1),
474+
(42, b'\x00' * 10 + b'\x2a', min(11, SZ)),
475+
(-1, b'\xff', 1),
476+
(-1, b'\xff' * 10, min(11, SZ)),
477+
(-42, b'\xd6', 1),
478+
(-42, b'\xff' * 10 + b'\xd6', min(11, SZ)),
479+
# Extracts 255 into a single byte, but requests sizeof(Py_ssize_t)
480+
(255, b'\xff', SZ),
481+
(255, b'\x00\xff', 2),
482+
(256, b'\x01\x00', 2),
483+
# Extracts successfully (unsigned), but requests 9 bytes
484+
(2**63, b'\x80' + b'\x00' * 7, 9),
485+
# "Extracts", but requests 9 bytes
486+
(-2**63, b'\x80' + b'\x00' * 7, 9),
487+
(2**63, b'\x00\x80' + b'\x00' * 7, 9),
488+
(-2**63, b'\xff\x80' + b'\x00' * 7, 9),
489+
490+
(2**255-1, b'\x7f' + b'\xff' * 31, 32),
491+
(-(2**255-1), b'\x80' + b'\x00' * 30 + b'\x01', 32),
492+
# Request extra bytes, but result says we only needed 32
493+
(-(2**255-1), b'\xff\x80' + b'\x00' * 30 + b'\x01', 32),
494+
(-(2**255-1), b'\xff\xff\x80' + b'\x00' * 30 + b'\x01', 32),
495+
496+
# Extracting 256 bits of integer will request 33 bytes, but still
497+
# copy as many bits as possible into the buffer. So we *can* copy
498+
# into a 32-byte buffer, though negative number may be unrecoverable
499+
(2**256-1, b'\xff' * 32, 33),
500+
(2**256-1, b'\x00' + b'\xff' * 32, 33),
501+
(-(2**256-1), b'\x00' * 31 + b'\x01', 33),
502+
(-(2**256-1), b'\xff' + b'\x00' * 31 + b'\x01', 33),
503+
(-(2**256-1), b'\xff\xff' + b'\x00' * 31 + b'\x01', 33),
504+
505+
# The classic "Windows HRESULT as negative number" case
506+
# HRESULT hr;
507+
# PyLong_CopyBits(<-2147467259>, &hr, sizeof(HRESULT))
508+
# assert(hr == E_FAIL)
509+
(-2147467259, b'\x80\x00\x40\x05', 4),
510+
]:
511+
with self.subTest(f"{v:X}-{len(expect_be)}bytes"):
512+
n = len(expect_be)
513+
buffer = bytearray(n)
514+
expect_le = expect_be[::-1]
515+
516+
self.assertEqual(expect_n, asnativebytes(v, buffer, n, 0),
517+
f"PyLong_AsNativeBytes(v, buffer, {n}, <big>)")
518+
self.assertEqual(expect_be, buffer[:n], "<big>")
519+
self.assertEqual(expect_n, asnativebytes(v, buffer, n, 1),
520+
f"PyLong_AsNativeBytes(v, buffer, {n}, <little>)")
521+
self.assertEqual(expect_le, buffer[:n], "<little>")
522+
523+
# Check a few error conditions. These are validated in code, but are
524+
# unspecified in docs, so if we make changes to the implementation, it's
525+
# fine to just update these tests rather than preserve the behaviour.
526+
with self.assertRaises(SystemError):
527+
asnativebytes(1, buffer, 0, 2)
528+
with self.assertRaises(TypeError):
529+
asnativebytes('not a number', buffer, 0, -1)
530+
531+
def test_long_fromnativebytes(self):
532+
import math
533+
from _testcapi import (
534+
pylong_fromnativebytes as fromnativebytes,
535+
SIZE_MAX,
536+
)
537+
538+
# Abbreviate sizeof(Py_ssize_t) to SZ because we use it a lot
539+
SZ = int(math.ceil(math.log(SIZE_MAX + 1) / math.log(2)) / 8)
540+
MAX_SSIZE = 2 ** (SZ * 8 - 1) - 1
541+
MAX_USIZE = 2 ** (SZ * 8) - 1
542+
543+
for v_be, expect_s, expect_u in [
544+
(b'\x00', 0, 0),
545+
(b'\x01', 1, 1),
546+
(b'\xff', -1, 255),
547+
(b'\x00\xff', 255, 255),
548+
(b'\xff\xff', -1, 65535),
549+
]:
550+
with self.subTest(f"{expect_s}-{expect_u:X}-{len(v_be)}bytes"):
551+
n = len(v_be)
552+
v_le = v_be[::-1]
553+
554+
self.assertEqual(expect_s, fromnativebytes(v_be, n, 0, 1),
555+
f"PyLong_FromNativeBytes(buffer, {n}, <big>)")
556+
self.assertEqual(expect_s, fromnativebytes(v_le, n, 1, 1),
557+
f"PyLong_FromNativeBytes(buffer, {n}, <little>)")
558+
self.assertEqual(expect_u, fromnativebytes(v_be, n, 0, 0),
559+
f"PyLong_FromUnsignedNativeBytes(buffer, {n}, <big>)")
560+
self.assertEqual(expect_u, fromnativebytes(v_le, n, 1, 0),
561+
f"PyLong_FromUnsignedNativeBytes(buffer, {n}, <little>)")
562+
563+
# Check native endian when the result would be the same either
564+
# way and we can test it.
565+
if v_be == v_le:
566+
self.assertEqual(expect_s, fromnativebytes(v_be, n, -1, 1),
567+
f"PyLong_FromNativeBytes(buffer, {n}, <native>)")
568+
self.assertEqual(expect_u, fromnativebytes(v_be, n, -1, 0),
569+
f"PyLong_FromUnsignedNativeBytes(buffer, {n}, <native>)")
570+
426571

427572
if __name__ == "__main__":
428573
unittest.main()

‎Misc/NEWS.d/next/C API/2024-02-05-17-11-15.gh-issue-111140.WMEjid.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Adds :c:func:`PyLong_AsNativeBytes`, :c:func:`PyLong_FromNativeBytes` and
2+
:c:func:`PyLong_FromUnsignedNativeBytes` functions.

‎Modules/_io/textio.c‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2393,7 +2393,7 @@ textiowrapper_parse_cookie(cookie_type *cookie, PyObject *cookieObj)
23932393
return -1;
23942394

23952395
if (_PyLong_AsByteArray(cookieLong, buffer, sizeof(buffer),
2396-
PY_LITTLE_ENDIAN, 0) < 0) {
2396+
PY_LITTLE_ENDIAN, 0, 1) < 0) {
23972397
Py_DECREF(cookieLong);
23982398
return -1;
23992399
}

‎Modules/_pickle.c‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2162,7 +2162,8 @@ save_long(PicklerObject *self, PyObject *obj)
21622162
pdata = (unsigned char *)PyBytes_AS_STRING(repr);
21632163
i = _PyLong_AsByteArray((PyLongObject *)obj,
21642164
pdata, nbytes,
2165-
1 /* little endian */ , 1 /* signed */ );
2165+
1 /* little endian */ , 1 /* signed */ ,
2166+
1 /* with exceptions */);
21662167
if (i < 0)
21672168
goto error;
21682169
/* If the int is negative, this may be a byte more than

‎Modules/_randommodule.c‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -342,7 +342,8 @@ random_seed(RandomObject *self, PyObject *arg)
342342
res = _PyLong_AsByteArray((PyLongObject *)n,
343343
(unsigned char *)key, keyused * 4,
344344
PY_LITTLE_ENDIAN,
345-
0); /* unsigned */
345+
0, /* unsigned */
346+
1); /* with exceptions */
346347
if (res == -1) {
347348
goto Done;
348349
}

‎Modules/_sqlite/util.c‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -162,7 +162,7 @@ _pysqlite_long_as_int64(PyObject * py_val)
162162
sqlite_int64 int64val;
163163
if (_PyLong_AsByteArray((PyLongObject *)py_val,
164164
(unsigned char *)&int64val, sizeof(int64val),
165-
IS_LITTLE_ENDIAN, 1 /* signed */) >= 0) {
165+
IS_LITTLE_ENDIAN, 1 /* signed */, 0) >= 0) {
166166
return int64val;
167167
}
168168
}

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad © 2024 Your Company Name. All rights reserved.




Check this box to remove all script contents from the fetched content.



Check this box to remove all images from the fetched content.


Check this box to remove all CSS styles from the fetched content.


Check this box to keep images inefficiently compressed and original size.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy