Talk about how JMeter executes Python scripts concurrently

AirPython 2021-09-15 06:58:43
talk jmeter executes python scripts


1. Preface

Hello everyone , I'm Ango !

Recently, a little friend left me a message in the background , Say you use it Django Wrote a big file to upload Api Interface , Now I want to test the stability of interface concurrency locally , Ask me if I have a good plan

This article takes file upload as an example , Chat  Jmeter Concurrent execution Python The complete flow of the script

2. Python File upload

Large file upload contains 3 A step , Namely :

  • Get file information and number of slices

  • Sectioning , And upload  - API

  • File merge  - API

  • File path parameterization

2-1   Get file information and number of slices

First , Get file size

then , Get the total number of segments using the preset slice size

Last , Get the file name and md5 value

import os
import math
import hashlib
def get_file_md5(self, file_path):
""" retrievable md5 value """
with open(file_path, 'rb') as f:
data = f.read()
return hashlib.md5(data).hexdigest()
def get_filename(self, filepath):
""" Get the original name of the file """
# File name with suffix
filename_with_suffix = os.path.basename(filepath)
# file name
filename = filename_with_suffix.split('.')[0]
# Suffix name
suffix = filename_with_suffix.split('.')[-1]
return filename_with_suffix, filename, suffix
def get_chunk_info(self, file_path):
""" Get segmentation information """
# Get the total file size ( byte )
file_total_size = os.path.getsize(file_path)
print(file_total_size)
# Total number of segments
total_chunks_num = math.ceil(file_total_size / self.chunk_size)
# file name ( With suffix )
filename = self.get_filename(file_path)[0]
# Of documents md5 value
file_md5 = self.get_file_md5(file_path)
return file_total_size, total_chunks_num, filename, file_md5
 Copy code 

2-2   Slice and segment upload

Utilize the total number of segments and segment size , Slice the file , Call segmented file upload interface

import requests
def do_chunk_and_upload(self, file_path):
​ """ Segment the document , And upload """
file_total_size, total_chunks_num, filename, file_md5 = self.get_chunk_info(file_path)
# Traverse
for index in range(total_chunks_num):
print(' The first {} File upload '.format(index + 1))
if index + 1 == total_chunks_num:
partSize = file_total_size % chunk_size
else:
partSize = chunk_size
# File offset
offset = index * chunk_size
# Generate fragment id, from 1 Start
chunk_id = index + 1
print(' Start preparing to upload files ')
print(" Fragmentation id:", chunk_id, " File offset :", offset, ", Current slice size :", partSize, )
# Upload files in sections
self.__upload(offset, chunk_id, file_path, file_md5, filename, partSize, total_chunks_num)
def __upload(self, offset, chunk_id, file_path, file_md5, filename, partSize, total):
""" Upload files in batches """
url = 'http://**/file/brust/upload'
params = {'chunk': chunk_id,
'fileMD5': file_md5,
'fileName': filename,
'partSize': partSize,
'total': total
}
# According to the file path and offset , Read file binary data
current_file = open(file_path, 'rb')
current_file.seek(offset)
files = {'file': current_file.read(partSize)}
resp = requests.post(url, params=params, files=files).text
print(resp)
 Copy code 

2-3   Merge files

Finally, call the merge file interface. , Combine segmented small files into large files

def merge_file(self, filepath):
""" Merge """
url = 'http://**/file/brust/merge'
file_total_size, total_chunks_num, filename, file_md5 = self.get_chunk_info(filepath)
​ payload = json.dumps(
{
"fileMD5": file_md5,
"chunkTotal": total_chunks_num,
"fileName": filename
}
)
print(payload)
headers = {
"Content-Type": "application/json"
}
resp = requests.post(url, headers=headers, data=payload).text
print(resp)
 Copy code 

2-4   File path parameterization

For concurrent execution , Parameterize file upload path

# fileupload.py
...
if __name__ == '__main__':
filepath = sys.argv[1]
# Size of each slice (MB)
chunk_size = 2 * 1024 * 1024
fileApi = FileApi(chunk_size)
# Segment upload
fileApi.do_chunk_and_upload(filepath)
# Merge
fileApi.merge_file(filepath)
 Copy code 

3. Jmeter Concurrent execution

In the use of Jmeter Before creating a concurrent process , We need to write batch scripts

among , When executing a batch script , You need to follow the file path

# cmd.bat
@echo off
set filepath=%1
python C:\Users\xingag\Desktop\rpc_demo\fileupload.py %*
 Copy code 

then , Create a new one locally CSV file , Write multiple file paths

# Prepare multiple file paths (csv)
C:\\Users\\xingag\\Desktop\\charles-proxy-4.6.1-win64.msi
C:\\Users\\xingag\\Desktop\\V2.0.pdf
C:\\Users\\xingag\\Desktop\\HBuilder1.zip
C:\\Users\\xingag\\Desktop\\HBuilder2.zip
 Copy code 

next , You can use Jmeter Create concurrent process

The complete procedure is as follows :

  • Create a test plan , Next, add a thread group

    Here, the number of thread groups can be consistent with the number of files above

  • Under thread group , add to 「  Synchro timer  」

    Synchronization timer 「  The number of simulated user groups  」 It is consistent with the number of parameters above

  • add to CSV Data file settings

    Point to the one prepared above csv Data files , Set file format to UTF-8, The variable name is set to  file_path, Finally, set the thread sharing mode to 「  Current thread group  」

  • Add debug sampler , Convenient debugging

  • add to OS Process sampler

    Select the batch file created above , The command line parameter is set to 「 ${file_path} 」

  • Add view results

4. Last

Run the created above Jmeter Concurrent process , In the result count, you can view the results of concurrent upload files

Of course , We can increase the number of concurrency to simulate real usage scenarios , It just needs to be modified CSV Data sources and Jmeter Parameters can be

If you think the article is good , Please   give the thumbs-up 、 Share 、 Leaving a message.   Next , Because this will be the strongest driving force for me to continue to output more quality articles !

版权声明
本文为[AirPython]所创,转载请带上原文链接,感谢
https://pythonmana.com/2021/09/20210909124112663z.html

  1. Take you to learn more about nginx basic login authentication: generating passwords using Python
  2. 超硬核Python避坑学习方案奉上!入门到就业一篇就搞定!
  3. Talk about how JMeter executes Python scripts concurrently
  4. Talk about how JMeter executes Python scripts concurrently
  5. Talk about how JMeter executes Python scripts concurrently
  6. python内置函数通过字符串的方式来执行函数代码块,类似java的反射机制相当强大!
  7. python内置函数通过字符串的方式来执行函数代码块,类似java的反射机制相当强大!
  8. python內置函數通過字符串的方式來執行函數代碼塊,類似java的反射機制相當强大!
  9. Les fonctions intégrées Python exécutent des blocs de code de fonction à travers des chaînes, et les mécanismes de réflexion comme Java sont assez puissants!
  10. Python module 1
  11. Python tip: use namedtuple instead of manually created classes
  12. Python - poetry(3)配置项详解
  13. Python - poetry(3)配置项详解
  14. Python - poetry(3)配置項詳解
  15. Python - poetry(3)配置項詳解
  16. Python - détails de l'élément de configuration Poetry (3)
  17. Python - détails de l'élément de configuration Poetry (3)
  18. Python案例实战,pygame模块,Python实现字母代码雨
  19. Python calculation vector angle code
  20. Python基础面试题解读|《Python面试100层》|第1层
  21. 面对小白的pandas命令手册+练习题【三万字详解】
  22. 面對小白的pandas命令手册+練習題【三萬字詳解】
  23. Face au Manuel de commande pandas de Xiaobai + question d'exercice [30 000 mots pour plus de détails]
  24. Interprétation des questions d'entrevue de base Python | 100 couches d'entrevue Python | couche 1
  25. Python data structure and algorithm (17) -- merge sort
  26. Les fonctions intégrées Python exécutent des blocs de code de fonction à travers des chaînes, et les mécanismes de réflexion comme Java sont assez puissants!
  27. Python笔记-uiautomator2截图点击,OpenCV找图
  28. Python文档阅读笔记-OpenCV中Template Matching
  29. Python笔记-利用OpenCV的matchTemplate屏幕找图并使用pyautogui点击
  30. Python筆記-利用OpenCV的matchTemplate屏幕找圖並使用pyautogui點擊
  31. Notes python - utilisez l'écran matchtemplate d'OpenCV pour trouver des images et cliquez sur
  32. Notes de lecture de documents python - Matching de modèles dans OpenCV
  33. Notes python - capture d'écran de l'automate 2 Cliquez pour ouvrir la vue
  34. python链接云服务器的mysql8
  35. python鏈接雲服務器的mysql8
  36. Mysql8 pour les serveurs Cloud liés Python
  37. Python资源大集合,要的话可以拿走!
  38. ️万字【Python基础】保姆式教学️,小白快速入门Python!
  39. ️萬字【Python基礎】保姆式教學️,小白快速入門Python!
  40. Wanzi [base Python] Baby - sitting Teaching, Little White Quick Start Python!
  41. Realizing the function of sending e-mail automatically with Python
  42. Smtpauthenticationerror in Python: solution
  43. 8 steps to teach you how to solve Sudoku in Python! (including source code)
  44. Python change la vie | identifier facilement des centaines de numéros de livraison
  45. Python change life | utilisation de modèles reconnus par ocr
  46. Bibliothèques Python utiles et intéressantes - - psutil
  47. 3. Traitement des données pandas
  48. 【Python编程基础】控制流之链式比较运算符
  49. MFC uses Python scripting language
  50. 【Python編程基礎】控制流之鏈式比較運算符
  51. 【 base de programmation python】 opérateur de comparaison de chaîne pour le flux de contrôle
  52. Python game development, pyGame module, python implementation of Xiaole games
  53. Mise en œuvre du Code de vérification unique (OTP) avec le cadre de repos Django
  54. Python - eval ()
  55. Python - Programmation orientée objet - _Rapport()
  56. Différence entre python - rep (), Str ()
  57. Python - Programmation orientée objet - _Appel()
  58. Python calling matlab script
  59. Python - Programmation orientée objet - _Nouveau() et mode Singleton
  60. Python - Programmation orientée objet - méthode magique (méthode de double soulignement)