Zoomeye Python User's Guide

osc_ yizwdm15 2021-01-21 08:56:13
zoomeye python user guide


0x01 Preface

ZoomEye As a search engine in cyberspace , Through the browser you can quickly search for network devices . It's not very friendly to the technicians , Technicians are looking forward to a convenient and effective query ZoomEye Information about 、 Data and formatting results can also be used as SDK Integrated into other tools , So there was ZoomEye-python Such a tool .

ZoomEye-python It's based on ZoomEye API Developed Python library , Provides ZoomEye Command line mode , It can also be used as SDK Integrate into other tools . The library makes it easier for technicians to Search for Screening export ZoomEye The data of .

0x02 ZoomEye API

ZoomEye The official provided Search data , The user login , The remaining resources as well as Device history Interface , at present ZoomEye API Open to the outside world (developer) jurisdiction , There is a certain amount of limit . The monthly quota is 10000 strip , The open developer package will reset the request quota of corresponding resources every month . Registered users can get API Right to use the developer package of .

0x03 Installation steps

Directly from pypi Installation :

pip3 install zoomeye-python

It can also be done through github Installation :

pip3 install git+https://github.com/knownsec/zoomeye-python.git

0x04 How to use

ZoomEye-python Provides cli and SDK Two working modes .

1. Use cli

After installation , have access to zoomeye -h Command to verify that the installation was successful .

$ zoomeye -h
usage: zoomeye [-h] {info,search,init} ...
positional arguments:
{info,search,init}
info Show ZoomEye account info
search Search the ZoomEye database
init Initialize the token for ZoomEye-python
optional arguments:
-h, --help show this help message and exit
1. initialization

ZoomEye-python The initialization of is very simple , Use one of the following ways :

APIKEY ( recommend )

$ zoomeye init -apikey "01234567-acbd-00000-1111-22222222222"
successfully initialized
Role: developer
Quota: 10000

or

username/password

$ zoomeye init -username "username@zoomeye.org" -password "password"
successfully initialized
Role: developer
Quota: 10000

There is essentially no difference between the two ways , adopt username/password The method of initialization cli , This way authentication will return JWT-token, It has certain timeliness ( Probably 12 Hours ),JWT-toekn After the failure, the user needs to log in again . and APIKEY It doesn't expire , Users can reset their personal information according to their needs . So we recommend Use APIKEY The way To initialize .

land ZoomEye In personal information (https://www.zoomeye.org/profile) obtain APIKEY .

2. Account resources

ZoomEye API It is stipulated that each account registration can obtain 10000 The amount of data , Remember to check the quota left in your account before using search .

Use command zoomeye info You can query the remaining quota of the account .

$ zoomeye info
Role: developer
Quota: 10000

We need to pay attention to :info The command only shows the quota for the free part , It will be fixed in the next version .

3. Search for
$ zoomeye search -h
usage: zoomeye search [-h] [-num value] [-facet [field]]
[-filter [field=regexp]] [-stat [field]]
[-save [field=regexp]] [-count]
dork
positional arguments:
dork The ZoomEye search keyword or ZoomEye exported file
optional arguments:
-h, --help show this help message and exit
-num value The number of search results that should be returned
-facet [field] Perform statistics on ZoomEye database, field:
[app,device,service,os,port,country,city]
-filter [field=regexp]
Output more clearer search results by set filter
field, field:
[app,version,device,port,city,country,asn,banner,*]
-stat [field] Perform statistics on search results, field:
[app,device,service,os,port,country,city]
-save [field=regexp] Save the search results with ZoomEye json format, if
you specify the field, it will be saved with JSON
Lines
-count The total number of results in ZoomEye database for a
search

search The command provides the ability to Screening Search for export Aggregate statistics The ability of , Let's say Flying to cloud fortress For example ZoomEye-python The function of :

Search data

$ zoomeye search "app:" Flying to cloud fortress ""
ip:port service country app banner
46.*.*.254:80 http Russian Federation Tornado httpd HTTP/1.1 302 FOUND\r\nContent-...
139.*.*.11:6080 http China nginx HTTP/1.1 200 OK\x0d\nServer: n...
180.*.*.202:8880 http China nginx HTTP/1.1 200 OK\r\nServer: ngi...
180.*.*.181:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nEtag: W/"600...
180.*.*.104:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nContent-Type...
180.*.*.195:80 http China HTTP/1.1 200 OK\nContent-Leng...
180.*.*.118:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nCache-Contro...
180.*.*.120:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nDate: Mon, 1...
180.*.*.212:80 http China HTTP/1.1 200 OK\nDate: Mon, 1...
180.*.*.119:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nAccept-Range...
101.*.*.237:8888 http China nginx HTTP/1.1 200 OK\r\nServer: ngi...
175.*.*.71:443 https China nginx HTTP/1.1 200 OK\r\nServer: ngi...
182.*.*.7:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nStrict-Trans...
182.*.*.114:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nDate: Mon, 1...
182.*.*.45:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nAccept-Range...
182.*.*.79:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nDate: Mon, 1...
52.*.*.175:80 http China nginx HTTP/1.1 302 Found\r\nServer: ...
182.*.*.131:80 http China HTTP/1.1 200 OK\nExpires: Tue...
182.*.*.183:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nLast-Modifie...
182.*.*.41:80 http China HTTP/1.1 200 OK\nAccept-Range...
total: 20

The search function is ZoomEye-python The most important feature , Its advantage is that it can not only obtain the data, but also filter the data , Statistics , Import and export .

Tips: Can pass -num Specify the number of displays .

Why isn't this the number of gains ?

because ZoomEye API The minimum number of single queries is 20 strip , therefore -num Parameter consumption quota is 20 Integer multiple .

Whether the quota will be consumed repeatedly the next time the data is acquired ?

It's not , When searching for data ,ZoomEye-python Yes API The returned data is cached , The next fetch will be taken from the cache , Exceed the number of caches from API obtain , The cache time is 5 God , The purpose of this is to save users' quota to a certain extent , It also ensures the accuracy of the data .

Look at the total

$ zoomeye search "app:" Flying to cloud fortress "" -count
7748

Online data aggregation

$ zoomeye search "app:" Flying to cloud fortress "" -facet "country"
country count
China 6921
United States 252
Singapore 152
Japan 129
Russian Federation 114
Asia Pacific Regions 26
Republic of Korea 26
South Africa 24
India 15
Philippines 10

That's how to get ZoomEye API The statistical results of all the data , It can be seen directly Flying to cloud fortress It's used the most in China , Total equipment . In the analysis can provide no small help .

Local data aggregation

zoomeye search "app:" Flying to cloud fortress "" -stat "country,city"
country count
China 19
Russian Federation 1
city count
Beijing 9
Shanghai 6
Kamyshin 1
New Taipei City 1
Hangzhou 1
Chengdu 1
Zhongwei 1

Local data aggregation can make statistics on the data currently obtained .

Filter data

In the face of a lot of data , It often seems more troublesome , To make it easier for us to view the data ,ZoomEye-python Provides a sweet feature —— Screening .ZoomEye-python You can specify fields in the filter ( key ) You can also specify fields and values ( key=value ).value regular expression , Such as :

$ zoomeye search "app:" Flying to cloud fortress "" -filter "city=Beijing,port=8*"
ip city port
180.*.*.181 Beijing 80
180.*.*.104 Beijing 80
182.*.*.7 Beijing 80
182.*.*.114 Beijing 80
182.*.*.45 Beijing 80
182.*.*.79 Beijing 80
182.*.*.131 Beijing 80
182.*.*.183 Beijing 80
182.*.*.41 Beijing 80
total: 9

Save the data

According to different requirements, we need to export data with different formats , stay ZoomEye-python There are two ways to save : One is OK json , The other is ZoomEye API The metadata returned , It also supports screening .

Save row json when , You can specify the fields you want , The grammar is the same as above filter Agreement , Regular expressions are also supported .

# Save the filter data 
$ zoomeye search "app:" Flying to cloud fortress "" -save "city=Beijing,port"
save file to /app: Flying to cloud fortress _9_1610962280.json successful!
$ cat app: Flying to cloud fortress _9_1610962280.json
{'ip': '180.*.*.181', 'city': 'Beijing', 'port': 80}
{'ip': '180.*.*.104', 'city': 'Beijing', 'port': 80}
{'ip': '182.*.*.7', 'city': 'Beijing', 'port': 80}
{'ip': '182.*.*.114', 'city': 'Beijing', 'port': 80}
{'ip': '182.*.*.45', 'city': 'Beijing', 'port': 80}
{'ip': '182.*.*.79', 'city': 'Beijing', 'port': 80}
{'ip': '182.*.*.131', 'city': 'Beijing', 'port': 80}
{'ip': '182.*.*.183', 'city': 'Beijing', 'port': 80}
{'ip': '182.*.*.41', 'city': 'Beijing', 'port': 80}

When no field is specified , Save from ZoomEye API Metadata obtained .

# Save metadata 
$ zoomeye search "app:" Flying to cloud fortress "" -save
save file to /app: Flying to cloud fortress _20_1610962433.json successful!
$ cat app: Flying to cloud fortress _20_1610962433.json
{"total": 7748, "matches": [{"geoinfo": {"city": {"geoname_id": null, "names": {"zh-CN": "\u5361\u6885\u7533", "en": "Kamyshin"}}, "country": {"geoname_id": null, "code": "RU", "names": {"zh-CN": "\u4fc4\u7f57\u65af", "en": "Russian Federation"}}, "isp": "abrikosnet.ru", ......

Import data

ZoomEye-python To be able to save ZoomEye API Metadata is loaded and filtered 、 Search and other operations , Such as :

$ zoomeye search app: Flying to cloud fortress _20_1610962433.json
ip:port service country app banner
46.*.*.254:80 http Russian Federation Tornado httpd HTTP/1.1 302 FOUND\r\nContent-...
139.*.*.11:6080 http China nginx HTTP/1.1 200 OK\x0d\nServer: n...
180.*.*.202:8880 http China nginx HTTP/1.1 200 OK\r\nServer: ngi...
180.*.*.181:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nEtag: W/"600...
180.*.*.104:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nContent-Type...
180.*.*.195:80 http China HTTP/1.1 200 OK\nContent-Leng...
180.*.*.118:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nCache-Contro...
180.*.*.120:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nDate: Mon, 1...
180.*.*.212:80 http China HTTP/1.1 200 OK\nDate: Mon, 1...
180.*.*.119:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nAccept-Range...
101.*.*.237:8888 http China nginx HTTP/1.1 200 OK\r\nServer: ngi...
175.*.*.71:443 https China nginx HTTP/1.1 200 OK\r\nServer: ngi...
182.*.*.7:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nStrict-Trans...
182.*.*.114:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nDate: Mon, 1...
182.*.*.45:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nAccept-Range...
182.*.*.79:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nDate: Mon, 1...
52.*.*.175:80 http China nginx HTTP/1.1 302 Found\r\nServer: ...
182.*.*.131:80 http China HTTP/1.1 200 OK\nExpires: Tue...
182.*.*.183:80 nagios-nsca China Nagios NSCA HTTP/1.1 200 OK\nLast-Modifie...
182.*.*.41:80 http China HTTP/1.1 200 OK\nAccept-Range...
total: 20

2. SDK

stay ZoomEye-python The following interfaces are provided in :

1.login()
Use username/password authentication
2.dork_search(dork, page=0, resource="host", facets=None)
according to dork Search the data of the specified page
3.multi_page_search(dork, page=1, resource="host", facets=None)
according to dork Search multiple pages of data
4.resources_info()
Get current user information
5.show_count()
Get current dork The number of all matching results
6.dork_filter(keys)
Extract the data of the specified field from the search results
7.get_facet()
Get aggregate results of full data from search results
8.history_ip(ip)
Query a ip Historical data information of
9.show_site_ip(data)
Traverse web-search Result set , And output the domain name and ip Address
10.show_ip_port(data)
Traverse host-search Result set , And the output ip Address and port 

##### Using examples

Use username/password initialization

$ python3
>>> import zoomeye.sdk as zoomeye
>>> dir(zoomeye)
['ZoomEye', 'ZoomEyeDict', '__builtins__', '__cached__', '__doc__',
'__file__', '__loader__', '__name__', '__package__', '__spec__',
'fields_tables_host', 'fields_tables_web', 'getpass', 'requests',
'show_ip_port', 'show_site_ip', 'zoomeye_api_test']
>>> # Use username and password to login
>>> zm = zoomeye.ZoomEye()
>>> zm.username = 'username@zoomeye.org'
>>> zm.password = 'password'
>>> print(zm.login())
....JIUzI1NiIsInR5cCI6IkpXVCJ9.....
>>> data = zm.dork_search('apache country:cn')
>>> zoomeye.show_site_ip(data)
213.***.***.46.rev.vo***one.pt ['46.***.***.213']
me*****on.o****e.net.pg ['203.***.***.114']
soft********63221110.b***c.net ['126.***.***.110']
soft********26216022.b***c.net ['126.***.***.22']
soft********5084068.b***c.net ['126.***.***.68']
soft********11180040.b***c.net ['126.***.***.40']
...

Use APIKEY initialization

$ python3
>>> import zoomeye.sdk as zoomeye
>>> zm = zoomeye.ZoomEye()
>>> zm.api_key = "01234567-acbd-00000-1111-22222222222"
>>> zm.dork_search("apache country:cn")
...

Search for

As shown above , Use dork_search() Function to search ,dork_search Provides page,resource,facets Parameter to get the specified number of pages ,Web/Host Equipment and aggregate statistics of the total amount of data . among resource The default is host equipment ,facets by None.

data = zm.dork_search('telnet', facets='app')
zm.get_facet()
{'product': [{'name': '', 'count': 28323128}, {'name': 'BusyBox telnetd', 'count': 10180912}, {'name': 'Linux telnetd', ......

meanwhile SDK It also provides functions to get multi page data multi_page_search ,multi_page_search() And dork_search() The difference is that :multi_page_search Medium page To get the number of pages of data , and dork_search Medium page For which page of data .

Screening

For easier access to data ,SDK Provides functions to help users filter data dork_filter , Used to get the data of the specified field , Such as :

data = zm.dork_search("telnet")
zm.dork_filter("ip,port")
[['180.*.*.166', 5357], ['180.*.*.6', 5357], ......

according to ZoomEye API Provided interface , Respectively /host/search and /web/search , There are some differences between the data returned by these two interfaces . Therefore, when filtering, please fill in the correct fields according to the specified search type .

/web/search The fields included are :app / headers / keywords / title / ip / site / city / country /host/search The fields included are :app / version / device / ip / port / hostname / city / country / asn / banner

0x05 Knownsec 404 Team Star chain project

ZoomEye-python yes Knownsec 404 Team Star chain project A member of the .

“404 Star chain project ” I know Chuangyu 404 Laboratory on 2020 year 8 Plan to start in January , It is designed to be open source or open , Long term maintenance And promote the instrumentalization of different links in various fields of security research , It's like a star chain , Will be based on different security areas 、 Researchers from different security links are linked up .

It's not just about big tools that break through security barriers , It will also include a variety of gadgets involved in optimizing the daily experience , except 404 Besides its own tools open , And we're constantly collecting security research 、 Pain points during penetration testing , Hope to pass “404 Star chain project ” There are many tools in the safety circle 、 The level is uneven 、 Open source, unmanned maintenance of a variety of issues , Create a better and more open security tools to promote and exchange technology atmosphere .

https://github.com/knownsec/404StarLink-Project

0x06 Project address

ZoomEye-python Fully open source , Anyone can modify or submit code on this basis .

GitHub:https://github.com/knownsec/ZoomEye-python


Paper This paper is written by Seebug Paper Release , If you need to reprint, please indicate the source . This paper addresses :https://paper.seebug.org/1461/

版权声明
本文为[osc_ yizwdm15]所创,转载请带上原文链接,感谢
https://pythonmana.com/2021/01/20210121085456233X.html

  1. Python 爬虫进阶 - 前后端分离有什么了不起,过程超详细!
  2. 【python】使用pip提示ModuleNotFoundError
  3. 【python】虚拟环境搭建
  4. Advanced test | Python written test questions
  5. Fire! Open source Python ticket grabbing artifact, come home to see this wave of New Year!
  6. Python crawler advanced - before and after the end of the separation of what great, super detailed process!
  7. [Python] prompt modulenotfounderror with PIP
  8. Building a virtual environment
  9. Serverless 架构下用 Python 轻松搞定图像分类和预测
  10. Easy image classification and prediction with Python under serverless architecture
  11. python协程爬取某网站的老赖数据
  12. Python coroutine crawls Laolai data of a website
  13. 使用Python分析姿态估计数据集COCO的教程
  14. Using Python to analyze the data set coco of attitude estimation
  15. win环境 python3 flask 上手整理 环境搭建(一)
  16. Getting started with win environment python3 flash
  17. Python实现一个论文下载器,赶紧收藏
  18. win环境 python3 flask 上手整理 快速上手-基础操作(二)
  19. Python 中常见的配置文件写法
  20. Python to achieve a paper Downloader, quickly collect
  21. Python批量 png转ico
  22. 使用line_profiler对python代码性能进行评估优化
  23. 使用line_profiler对python代码性能进行评估优化
  24. Getting started with Python 3 flash in win environment
  25. Common ways to write configuration files in Python
  26. Python会在2021年死去吗? Python 3.9最终版本的回顾
  27. Python batch PNG to ICO
  28. Using line_ Profiler evaluates and optimizes the performance of Python code
  29. Using line_ Profiler evaluates and optimizes the performance of Python code
  30. Will Python die in 2021? A review of the final version of Python 3.9
  31. Python3 SMTP send mail
  32. Understanding closures in Python: getting started with closures
  33. Python日志实践
  34. Python logging practice
  35. [python opencv 计算机视觉零基础到实战] 十、图片效果毛玻璃
  36. [python opencv 计算机视觉零基础到实战] 九、模糊
  37. 10. Picture effect ground glass
  38. [Python opencv computer vision zero basis to actual combat] 9. Fuzzy
  39. 使用line_profiler對python程式碼效能進行評估優化
  40. Using line_ Profiler to evaluate and optimize the performance of Python code
  41. LeetCode | 0508. 出现次数最多的子树元素和【Python】
  42. Leetcode | 0508
  43. LeetCode | 0530. 二叉搜索树的最小绝对差【Python】
  44. LeetCode | 0515. 在每个树行中找最大值【Python】
  45. Leetcode | 0530. Minimum absolute difference of binary search tree [Python]
  46. Leetcode | 0515. Find the maximum value in each tree row [Python]
  47. 我来记笔记啦-搭建python虚拟环境
  48. Let me take notes - building a python virtual environment
  49. LeetCode | 0513. 找树左下角的值【Python】
  50. Leetcode | 0513. Find the value in the lower left corner of the tree [Python]
  51. Python OpenCV 泛洪填充,取经之旅第 21 天
  52. Python opencv flood fill, day 21
  53. Python爬虫自学系列(二)
  54. Python crawler self study series (2)
  55. 【python】身份证号码有效性检验
  56. [Python] validity test of ID number
  57. Python ORM - pymysql&sqlalchemy
  58. Python ORM - pymysql&sqlalchemy
  59. centos7 安装python3.8
  60. centos7 安装python3.8