Crawler + Django to create personalized API interface

mushroom27 2021-01-21 13:13:41
crawler django create personalized api


sketch

Today, my colleagues are also doing the development of wechat applets , Need music interface test , But it's troublesome to use Netease cloud's open interface , And you can't test it , I also told you about it here , So I wrote a simple Netease cloud song with crawler URL The reptiles of , Store data in mysql database , recycling django The package contains a simple API Interface , Test for colleagues .

principle

establish django project , Do a good job of basic configuration , stay views There are two ways in writing , One is from mysql Look up the data in the database and encapsulate it into API, One is the crawler method , After the data is picked up , adopt django Of ORM Insert data into mysql In the database .
The route here also corresponds to two , One is the crawler's request routing ( It's running the route ), One is interface routing ,MODEL It's also for convenience , Just set up two fields , One is the name of the song , One is URL Address .

The code is as follows

views File code

from django.shortcuts import render,HttpResponse
import requests
from lxml import etree
from .models import Api
# Create your views here.
def api_wy(request):
api = Api.objects.all()
return render(request, "index.html",locals())
def pc(request):
url = 'https://music.163.com/discover/toplist?id=3779629'
headers = {
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.25 Safari/537.36 Core/1.70.3861.400 QQBrowser/10.7.4313.400'
}
data = requests.get(url=url, headers=headers)
html = etree.HTML(data.text)
music_list = html.xpath('//ul[@class="f-hide"]/li/a')
music_lis = [] # Store song information
for music in music_list:
music_name = music.xpath('./text()')[0] # Get the song name
music_id_all = music.xpath('./@href')[0] # obtain a Label content
music_id = music_id_all.split('=')[-1] # take a Label content for data cleaning , Extract the content of the song id
download_music = music_name + ' ' + f'http://music.163.com/song/media/outer/url?id={music_id}.mp3' # Name the song and url Splicing
music_lis.append(download_music)
print(download_music)
for url in music_lis:
try:
url_name = url.split(' ')[0] # Get the name
url_music = url.split(' ')[1] # obtain url
Api.objects.create(name=url_name,url=url_music)
print(" Inserting data ")
except:
print("charushibai")
return HttpResponse(" Downloading ")

URL Routing file

from django.contrib import admin
from django.urls import path
from api.views import api_wy,pc
urlpatterns = [
path('admin/', admin.site.urls),
path('api/',api_wy),
path("pc/",pc),
]

Models level

from django.db import models
# Create your models here.
class Api(models.Model):
name = models.CharField(' Song name ', max_length=100)
url = models.CharField(" Song address ",max_length=300)
class Meta:
verbose_name = ' song API'
verbose_name_plural = verbose_name
def __str__(self):
return self.name

There's nothing else to say , It's also a relatively simple test requirement , Just to save something
Okay , That's it today , Bye-bye

版权声明
本文为[mushroom27]所创,转载请带上原文链接,感谢
https://pythonmana.com/2021/01/20210121131314027D.html

  1. LeetCode | 0508. 出现次数最多的子树元素和【Python】
  2. Leetcode | 0508
  3. LeetCode | 0530. 二叉搜索树的最小绝对差【Python】
  4. LeetCode | 0515. 在每个树行中找最大值【Python】
  5. Leetcode | 0530. Minimum absolute difference of binary search tree [Python]
  6. Leetcode | 0515. Find the maximum value in each tree row [Python]
  7. 我来记笔记啦-搭建python虚拟环境
  8. Let me take notes - building a python virtual environment
  9. LeetCode | 0513. 找树左下角的值【Python】
  10. Leetcode | 0513. Find the value in the lower left corner of the tree [Python]
  11. Python OpenCV 泛洪填充,取经之旅第 21 天
  12. Python opencv flood fill, day 21
  13. Python爬虫自学系列(二)
  14. Python crawler self study series (2)
  15. 【python】身份证号码有效性检验
  16. [Python] validity test of ID number
  17. Python ORM - pymysql&sqlalchemy
  18. Python ORM - pymysql&sqlalchemy
  19. centos7 安装python3.8
  20. centos7 安装python3.8
  21. Centos7 installing Python 3.8
  22. Centos7 installing Python 3.8
  23. Django——图书管理系统(六)
  24. Django——图书管理系统(五)
  25. Django -- library management system (6)
  26. Django -- library management system (5)
  27. python批量插入数据小脚本
  28. Python batch insert data script
  29. ZoomEye-python 使用指南
  30. Zoomeye Python User's Guide
  31. 用Python写代码,一分钟搞定一天工作量,同事直呼:好家伙 - 知乎
  32. Using Python to write code, one minute to complete a day's workload, colleagues call: good guy - Zhihu
  33. Python 上的可视化库——PyG2Plot
  34. Pyg2plot: a visualization library on Python
  35. Python 上的可视化库——PyG2Plot
  36. Python实用代码-无限级分类树状结构生成算法
  37. Pyg2plot: a visualization library on Python
  38. Python utility code - infinite classification tree structure generation algorithm
  39. 奇技淫巧,还是正统功夫?Python推导式最全用法
  40. Pandas 的这个知识点,估计 80% 的人都得挂!
  41. 前后端分离有什么了不起,手把手教你用Python爬下来!
  42. 在 Azure 上执行一些简单的 python 工作
  43. 推荐 :利用Python的混合集成机器学习(附链接)
  44. Cunning or orthodox Kung Fu? The most complete usage of Python derivation
  45. It's estimated that 80% of pandas people have to hang up!
  46. What's so great about the separation of front and rear ends? Hand in hand teach you to climb down with Python!
  47. Doing some simple Python work on azure
  48. Recommendation: hybrid integrated machine learning using python (link attached)
  49. Learning PPO algorithm programming from scratch (Python version)
  50. Python OpenCV 图片模糊操作 blur 与 medianBlur
  51. Python OpenCV image blur operation blur and mediablur
  52. 成功解决cv2.error: OpenCV(4.1.2) C:\projects\opencv-python\opencv\modules\imgproc\src\color.cpp:182: err
  53. Cv2.error solved successfully: opencv (4.1.2) C:: (projects / opencv Python / opencv modules / imgproc / SRC)\ color.cpp:182 : err
  54. Python 中使用 virtualenv 管理虚拟环境
  55. Using virtualenv to manage virtual environment in Python
  56. 如何使用Python执行系统命令?Python学习教程!
  57. How to use Python to execute system commands? Python tutorial!
  58. 快速掌握Python中的循环技术
  59. Quickly grasp the loop technology in Python
  60. Python主流Web框架之Tornado