Notes on Python cookbook 3rd (3.5): packing and unpacking bytes to large integers

Giant ship 2020-11-17 00:24:45
notes python cookbook 3rd rd

Packing and unpacking bytes to large integers

problem

You have a byte string and want to decompress it into an integer . perhaps , You need to convert a large integer into a byte string .

solution

Suppose your program needs to deal with a program that has 128 Bit long 16 A byte string of elements . such as ：

``````data = b'\x00\x124V\x00x\x90\xab\x00\xcd\xef\x01\x00#\x004'
``````

In order to bytes Resolve to integer , Use int.from_bytes() Method , And specify the byte order as follows ：

``````>>> len(data)
16
>>> int.from_bytes(data, 'little')
69120565665751139577663547927094891008
>>> int.from_bytes(data, 'big')
94522842520747284487117727783387188
>>>
``````

To convert a large integer into a byte string , Use int.to_bytes() Method , And specify the number and order of bytes as follows ：

``````>>> x = 94522842520747284487117727783387188
>>> x.to_bytes(16, 'big')
b'\x00\x124V\x00x\x90\xab\x00\xcd\xef\x01\x00#\x004'
>>> x.to_bytes(16, 'little')
b'4\x00#\x00\x01\xef\xcd\x00\xab\x90x\x00V4\x12\x00'
>>>
``````

Discuss

Conversion between large integers and byte strings is not common . However , In some application areas, sometimes , such as cryptography perhaps The Internet . for example , IPv6 Use a network address 128 The integer representation of bits . If you want to extract such values from a data record , You're going to have a problem like this .

As an alternative , You may want to use 6.11 In the section struct Module to decompress bytes . It also works , But use struct Module to decompress is limited to the size of integers . therefore , You may want to decompress multiple byte strings and merge the results into the final result , It looks like this ：

``````>>> data
b'\x00\x124V\x00x\x90\xab\x00\xcd\xef\x01\x00#\x004'
>>> import struct
>>> hi, lo = struct.unpack('>QQ', data)
>>> (hi << 64) + lo
94522842520747284487117727783387188
>>>
``````

Byte order rules (little or big) It only specifies the low and high order arrangement of bytes when constructing integers . We constructed it from below 16 It's easy to see in the representation of base numbers ：

``````>>> x = 0x01020304
>>> x.to_bytes(4, 'big')
b'\x01\x02\x03\x04'
>>> x.to_bytes(4, 'little')
b'\x04\x03\x02\x01'
>>>
``````

If you try to package an integer as a byte string , Then it doesn't fit , You will get a mistake . If necessary , You can use int.bit_length() Method to determine how many bytes are needed to store the value .

``````>>> x = 523 ** 23
>>> x
335381300113661875107536852714019056160355655333978849017944067
>>> x.to_bytes(16, 'little')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: int too big to convert
>>> x.bit_length()
208
>>> nbytes, rem = divmod(x.bit_length(), 8)
>>> if rem:
... nbytes += 1
...
>>>
>>> x.to_bytes(nbytes, 'little')
b'\x03X\xf1\x82iT\x96\xac\xc7c\x16\xf3\xb9\xcf...\xd0'
>>>
``````