1、 Sticky package
Sticking phenomenon ： When multiple messages are sent, receiving becomes one or receiving is inaccurate
（1）、 There are two cases of sticking
<1> Packet sticking happens at the sender ： The sender needs to wait for the buffer to fill before sending it out , Caused by sticky package
The interval between two messages is short , Short in length , It'll put the two messages together before they're sent
Save every time you send a message reply to the network resources
<2> Packet sticking occurs at the receiving end ： The receiver does not receive the packets of the buffer in time , Causes multiple packets to be received together
Multiple messages are sent to the cache side , But it wasn't received in time , Or the received length is less than the length of one transmission
<3> The essence of the phenomenon of enveloping stickiness ： There is no boundary between each piece of data sent
Sticking occurs only in tcp Agreement ：
- On the face of it , The problem of packet sticking is mainly due to the caching mechanism of sender and receiver 、tcp The characteristics of the protocol are stream oriented .
- actually , Mainly because the receiver does not know the boundaries between messages , Not knowing how many bytes of data to extract at once .
Generally speaking, there is no boundary between data and data , So there's a sticking phenomenon .
（2）、 Cause of sticking
<1> TCP Data transfer in protocol
tcp The unpacking mechanism of the protocol ：
When the length of the sender buffer is greater than that of the network card MTU when ,tcp The data will be sent in several packets . MTU yes Maximum Transmission Unit Abbreviation . It means the largest packet on the network .MTU In bytes . Most network devices MTU All are 1500. If this machine has MTU Than gateway's MTU Big , Large packets will be split and sent , This creates a lot of packet fragmentation , Increase packet loss rate , Reduce network speed .
Flow oriented communication features and Nagle Algorithm ：
TCP（transport control protocol, Transmission control protocol ） It's connection-oriented , Stream oriented , Provide high reliability service . Sending and receiving ends （ Client side and server side ） They all have to come in pairs socket, therefore , The sending end sends multiple packets to the receiving end , Send it to the other person more effectively , An optimization approach is used （Nagle Algorithm ）, Data that will be spaced several times at small intervals and have a small amount of data , Merge into one large data block , Then the packet is sealed . such , The receiver , It's hard to tell , Scientific unpacking mechanisms must be provided . That is, flow - oriented communication is message - free . For empty messages ：tcp It's based on data flow , So the message sent and received cannot be empty , This requires adding a handling mechanism for empty messages on both the client and the server , Prevent program from getting stuck , and udp It's based on datagrams , Even if it's empty （ Directly enter ）, It can also be sent ,udp The agreement will encapsulate the message and send it to you . It's reliable tcp agreement ：tcp Protocol data will not be lost , I didn't finish the bag , Next time you receive , I'm going to pick up where I left off , The end is always receiving ack The buffer contents are not cleared until . The data is reliable , But it sticks .
be based on tcp The cause of the phenomenon of sticking packets in the characteristics of the protocol :
The sender can be 1K 1K Geodesic data , The applications on the receiving end can be two K two K Pick up the data , Of course, it is possible to take it away at one time 3K or 6K data , Or just a few bytes of data at a time . in other words , The data that the application sees is a whole , Or a stream （stream）, How many bytes of a message are invisible to the application , therefore TCP Protocols are stream-oriented protocols , This is also prone to sticky package problems . and UDP It's a message-oriented protocol , Every UDP Each segment is a message , The application must extract the data as a message , You cannot extract arbitrary bytes of data at once , This and TCP It's very different . How do you define a message ？ Consider the other person one-time write/send Is a message , The need to understand is when the other side send A message , No matter how segmented the bottom layer is ,TCP The protocol layer sorts the data segments that make up the entire message before rendering them to the kernel buffer . For example, based on tcp Socket client to upload files to the server , The file content is sent as a byte stream segment by segment , On the receiving side , It is not at all known where the byte stream of the file begins , Where to end . Besides , Sender - caused sticky packets are caused by TCP The agreement itself ,TCP To improve transmission efficiency , Senders often collect enough data to send one TCP paragraph . If needed several times in a row send There is very little data , Usually TCP The data will be combined according to the optimization algorithm TCP After the segment is sent out , The receiver then receives the packet data .
<2> UDP It won't stick
UDP（user datagram protocol, User datagram protocol ） It's disconnected , Message oriented , Efficient service delivery . Block merge optimization algorithm is not used ,, because UDP It supports a one-to-many model , So on the receiving end skbuff( Socket buffer ） A chain structure is used to record each arrival UDP package , At every UDP The header is in the package （ Source address , Port and other information ）, such , For the receiver , It's easy to differentiate . That is, message - oriented communication is message - protected . For empty messages ：tcp It's based on data flow , So the message sent and received cannot be empty , This requires adding a handling mechanism for empty messages on both the client and the server , Prevent program from getting stuck , and udp It's based on datagrams , Even if it's empty （ Directly enter ）, It can also be sent ,udp The agreement will encapsulate the message and send it to you . Unreliable and non stick udp agreement ：udp Of recvfrom It's blocked , One recvfrom(x) It has to be the only one sendinto(y), The end x Two bytes of data is done , if y;x Data is lost , It means udp It doesn't stick at all , But you lose data , unreliable .
# Additional explanation ： use UDP When the protocol is sent , use sendto The maximum length of data that the function can send is ：65535- IP head (20) – UDP head (8)＝65507 byte . use sendto Function sends data , If the sent data length is greater than this value , The function returns an error .（ Discard this bag , Don't send ） use TCP When the protocol is sent , because TCP It's a data stream protocol , So there is no packet size limit （ The size of the buffer is not taken into account at this time ）, This means using send Function time , The data length parameter is unlimited . But in fact , The specified piece of data is not necessarily sent at one time , If the data is long , It will be sent in segments , If it's short , May wait to be sent with the next data .
2、 Solve the sticking problem
The root of the sticking problem is , The receiver does not know the length of the byte stream to be transmitted by the sender , So the solution to the sticky package is around , How to let the sender know the total size of the byte stream before sending data , Then the receiver receives all the data in an endless loop .
import socket sk = socket.socket() sk.bind(('192.168.12.26',9001)) sk.listen() conn,addr = sk.accept() msg1 = b'hello' msg2 = b'world' len_msg = len(msg1) str_len = str(len_msg) proto_len = str_len.zfill(10) # Turn the length into a cross conn.send(proto_len.encode()) # Send the length to the other party conn.send(msg1) # Send the content to the other party len_msg = len(msg2) str_len = str(len_msg) proto_len = str_len.zfill(10) conn.send(proto_len.encode()) conn.send(msg2)
import socket sk = socket.socket() sk.connect(('192.168.12.26',9001)) for i in range(1000000):2*i # Manufacturing delays len_msg = sk.recv(10).decode('utf-8') # Receive the cross len_msg = int(len_msg) # Convert the received byte to a number msg1 = sk.recv(len_msg) # Receive the length of bytes sent print(msg1) len_msg = sk.recv(10).decode('utf-8') len_msg = int(len_msg) msg2 = sk.recv(len_msg) print(msg2)
The problem is ：
The program runs faster than the speed of network transmission , So before you send a stanza , First use send To send the length of the byte stream , This approach amplifies the performance cost of network latency .
3、 Advanced solution of sticking package
struct The module can put a type , Convert to fixed length bytes
import struct ret = struct.pack('i',197274000) print(ret) # It can take a data of any size Convert to fixed 4 Bytes # -2147483648 ~ +2147483647 # 2g res = struct.unpack('i',b'\x90)\xc2\x0b') print(res)
（2）、 Use struct Solve the sticky package
With the help of struct The module can convert the length of the data to be sent into fixed bytes , In this way, each time the client receives a message, it only needs to receive the content of the fixed length byte to see the size of the information to be received next , Then the final received data will stop as long as it reaches this value , You can just receive the complete data .
<1> The process ：
Calculate the length of data bytes to be sent
Write the length of a byte into 4 byte Range （-2 * * (4 * 8) ~ 2 * * (4 * 8) -1）
send out 4 byte
|When sending||Reception time|
|Send... First struct Convert good data length 4 byte||Accept first 4 Bytes use struct Convert to a number to get the length of data to be received|
|Send data again||Then receive the data according to the length|
import struct import socket def proto_send(msg): msg = msg.encode('utf-8') len_msg = len(msg) proto_len = struct.pack('i', len_msg) # Programming the length of bytes 4 byte ,i representative int conn.send(proto_len) conn.send(msg) sk = socket.socket() sk.bind(('192.168.12.26',9001)) sk.listen() conn,addr = sk.accept() msg1 = 'hello' msg2 = 'world' proto_send(msg1) proto_send(msg2)
import struct import socket sk = socket.socket() def proto_recv(): len_msg = sk.recv(4) len_msg = struct.unpack('i', len_msg) msg = sk.recv(len_msg) return msg sk.connect(('192.168.12.26',9001)) for i in range(1000000):2*i msg1 = proto_recv() print(msg1) msg2 = proto_recv() print(msg2)
5、 File transfer
import socket sk = socket.socket() sk.bind(('192.168.12.26',9002)) sk.listen() conn,addr = sk.accept() with open(r'D:\Python\ Content .py','rb') as f: # Open the file to be transferred content = f.read() # Read the bytes conn.send(content) # Send what you read conn.close() sk.close()
import socket sk = socket.socket() sk.connect(('127.0.0.1',9002)) content = sk.recv(40960) # The received size is 40960 byte with open(r' Content 2.py','wb') as f: # Create a new file f.write(content) # Write the received content to a new file sk.close()