爬虫+django,打造个性化API接口

mushroom27 2021-01-21 13:13:22
爬虫 django 打造 个性化 个性


简述

今天也是同事在做微信小程序的开发,需要音乐接口的测试,可是用网易云的开放接口比较麻烦,也不能进行测试,这里也是和我说了一下,所以就用爬虫写了个简单网易云歌曲URL的爬虫,把数据存入mysql数据库,再利用django封装装了一个简单的API接口,给同事测试使用。

原理

创建django项目,做好基础的配置,在views里写两个方法,一个是从mysql数据库中查数据然后封装成API,一个是爬虫方法,数据扒下来以后,通过django的ORM把数据插入到mysql数据库中。
这里的路由也是对应两个,一个是爬虫的请求路由(就是运行路由),一个是接口路由,MODEL层里也是为了方便,就设了两个字段,一个是歌曲名称,一个是URL地址。

代码如下

views文件代码

from django.shortcuts import render,HttpResponse
import requests
from lxml import etree
from .models import Api
# Create your views here.
def api_wy(request):
api = Api.objects.all()
return render(request, "index.html",locals())
def pc(request):
url = 'https://music.163.com/discover/toplist?id=3779629'
headers = {
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.25 Safari/537.36 Core/1.70.3861.400 QQBrowser/10.7.4313.400'
}
data = requests.get(url=url, headers=headers)
html = etree.HTML(data.text)
music_list = html.xpath('//ul[@class="f-hide"]/li/a')
music_lis = [] # 存放歌曲信息
for music in music_list:
music_name = music.xpath('./text()')[0] # 获取歌曲名称
music_id_all = music.xpath('./@href')[0] # 获取a标签内容
music_id = music_id_all.split('=')[-1] # 将a标签内容进行数据清洗,提取歌曲的id
download_music = music_name + ' ' + f'http://music.163.com/song/media/outer/url?id={music_id}.mp3' # 将歌曲名称和url进行拼接
music_lis.append(download_music)
print(download_music)
for url in music_lis:
try:
url_name = url.split(' ')[0] # 获取名称
url_music = url.split(' ')[1] # 获取url
Api.objects.create(name=url_name,url=url_music)
print("正在插入数据")
except:
print("charushibai")
return HttpResponse("正在下载")

URL路由文件

from django.contrib import admin
from django.urls import path
from api.views import api_wy,pc
urlpatterns = [
path('admin/', admin.site.urls),
path('api/',api_wy),
path("pc/",pc),
]

Models层面

from django.db import models
# Create your models here.
class Api(models.Model):
name = models.CharField('歌曲名称', max_length=100)
url = models.CharField("歌曲地址",max_length=300)
class Meta:
verbose_name = '歌曲API'
verbose_name_plural = verbose_name
def __str__(self):
return self.name

其他的也就没什么可说的了,也是一个比较简单的测试需求,就是为了省点事情才弄得
好了,今天就到这了,拜拜

版权声明
本文为[mushroom27]所创,转载请带上原文链接,感谢
https://www.cnblogs.com/mushroom7/p/14307426.html

  1. LeetCode | 0508. 出现次数最多的子树元素和【Python】
  2. Leetcode | 0508
  3. LeetCode | 0530. 二叉搜索树的最小绝对差【Python】
  4. LeetCode | 0515. 在每个树行中找最大值【Python】
  5. Leetcode | 0530. Minimum absolute difference of binary search tree [Python]
  6. Leetcode | 0515. Find the maximum value in each tree row [Python]
  7. 我来记笔记啦-搭建python虚拟环境
  8. Let me take notes - building a python virtual environment
  9. LeetCode | 0513. 找树左下角的值【Python】
  10. Leetcode | 0513. Find the value in the lower left corner of the tree [Python]
  11. Python OpenCV 泛洪填充,取经之旅第 21 天
  12. Python opencv flood fill, day 21
  13. Python爬虫自学系列(二)
  14. Python crawler self study series (2)
  15. 【python】身份证号码有效性检验
  16. [Python] validity test of ID number
  17. Python ORM - pymysql&sqlalchemy
  18. Python ORM - pymysql&sqlalchemy
  19. centos7 安装python3.8
  20. centos7 安装python3.8
  21. Centos7 installing Python 3.8
  22. Centos7 installing Python 3.8
  23. Django——图书管理系统(六)
  24. Django——图书管理系统(五)
  25. Django -- library management system (6)
  26. Django -- library management system (5)
  27. python批量插入数据小脚本
  28. Python batch insert data script
  29. ZoomEye-python 使用指南
  30. Zoomeye Python User's Guide
  31. 用Python写代码,一分钟搞定一天工作量,同事直呼:好家伙 - 知乎
  32. Using Python to write code, one minute to complete a day's workload, colleagues call: good guy - Zhihu
  33. Python 上的可视化库——PyG2Plot
  34. Pyg2plot: a visualization library on Python
  35. Python 上的可视化库——PyG2Plot
  36. Python实用代码-无限级分类树状结构生成算法
  37. Pyg2plot: a visualization library on Python
  38. Python utility code - infinite classification tree structure generation algorithm
  39. 奇技淫巧,还是正统功夫?Python推导式最全用法
  40. Pandas 的这个知识点,估计 80% 的人都得挂!
  41. 前后端分离有什么了不起,手把手教你用Python爬下来!
  42. 在 Azure 上执行一些简单的 python 工作
  43. 推荐 :利用Python的混合集成机器学习(附链接)
  44. Cunning or orthodox Kung Fu? The most complete usage of Python derivation
  45. It's estimated that 80% of pandas people have to hang up!
  46. What's so great about the separation of front and rear ends? Hand in hand teach you to climb down with Python!
  47. Doing some simple Python work on azure
  48. Recommendation: hybrid integrated machine learning using python (link attached)
  49. Learning PPO algorithm programming from scratch (Python version)
  50. Python OpenCV 图片模糊操作 blur 与 medianBlur
  51. Python OpenCV image blur operation blur and mediablur
  52. 成功解决cv2.error: OpenCV(4.1.2) C:\projects\opencv-python\opencv\modules\imgproc\src\color.cpp:182: err
  53. Cv2.error solved successfully: opencv (4.1.2) C:: (projects / opencv Python / opencv modules / imgproc / SRC)\ color.cpp:182 : err
  54. Python 中使用 virtualenv 管理虚拟环境
  55. Using virtualenv to manage virtual environment in Python
  56. 如何使用Python执行系统命令?Python学习教程!
  57. How to use Python to execute system commands? Python tutorial!
  58. 快速掌握Python中的循环技术
  59. Quickly grasp the loop technology in Python
  60. Python主流Web框架之Tornado