The title is not intended to offend , I think this advertisement is very interesting
List of articles
- Preface
- Welcome to our circle
- Let's start with the last piece of code , clear
- Code interpretation
- The first pit ：find_element_name
- The second pit ： switch_to_frame()
- cookies Bypass login authentication , Around the desert
- Usher in a turnaround
- The bumps and bumps on the road to success
Preface
Previous review ： You have to learn Python（ Eleventh days ）
The last one , The last article said that this one is going to show you selenium, Of course that's right . This is not the same , This one will continue to be updated .
Insert a push ：（ If it's Xiaobai , Take a look at the following paragraph ）
Welcome to our circle
I built a Python Learn how to answer questions , Interested friends can get to know ： If you have difficulties in learning , Looking for one python Learning communication environment , Can join us python circle , Skirt number 947618024, Can claim python Learning materials , It will save a lot of time , Reduce a lot of problems .
This series of articles default that you have certain C or C++ Basics , Because I learned a little C++ After the fur of Python.
This series of articles default you will Baidu , Study ‘ modular ’ The words of this module , Or suggest you have your own editor and compiler , The last article has already made a recommendation for you ？
so what , The catalogue of this series , To be honest, I prefer those two books Primer Plus, So follow their directory structure .
This series will also focus on developing your hands-on skills , After all, I can't tell you all the knowledge , So the ability to solve their own needs is particularly important , So I buried holes in the article, please don't regard them as pits , That's the exercise I left you , Please show your powers , Take care of yourself .
1234567Let's start with the last piece of code , clear
# Local Chrome Browser settings
from selenium import webdriver # The package needed to operate the browser
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC #Xpath The package needed for navigation
import time # The packets needed to delay
from selenium.webdriver.chrome.options import Options
class golden_data:
def __init__(self):
options = Options()
#chrome_options.add_argument('--no-sandbox') # solve DevToolsActivePort File does not exist
#options.add_argument('--disable-gpu') # Google Documents mention the need to add this attribute to circumvent bug
#options.add_argument('blink-settings=imagesEnabled=false') # Don't load pictures , Speed up
#options.add_argument('--headless') # Browsers don't provide visual pages . linux Next, if the system does not support visualization, it will fail to start
self.driver = webdriver.Chrome(options=options,executable_path="D:/Python3.9/chromedriver.exe") # Get Google browser control handle
self.driver.get('https://jinshuju.net/login') # Open the golden data login page
self.wait = WebDriverWait(self.driver, 10) # Xpath Navigation
time.sleep(2)
# Login data
def login_data(self):
# Write to the account column
name_input = self.wait.until(EC.presence_of_element_located((By.XPATH, '//*[@id="auth_key"]')))
name_input.clear()
name_input.send_keys('18039027069') # Write your name here
time.sleep(2)
# Click on “ next step ”
next_step = self.wait.until(EC.presence_of_element_located((By.XPATH, '//*[@id="login_form"]/div/div[3]/button')))
next_step.click()
time.sleep(2)
# Enter the password
pwd_input = self.wait.until(EC.presence_of_element_located((By.XPATH, '//*[@id="password"]')))
pwd_input.clear()
pwd_input.send_keys('123456.sp') # Write your name here
time.sleep(2)
# Click login
login_click = self.wait.until(EC.presence_of_element_located((By.XPATH, '//*[@id="login_form"]/div/div[4]/button')))
login_click.click()
time.sleep(2)
# create form
#def create_table(self):
# Collect form data
def collect_data(self):
# Click on the first questionnaire
first_block = self.wait.until(EC.presence_of_element_located((By.XPATH, '//*[@id="5fa54cff61936cdee3121fa5"]/div/a')))
first_block.click()
time.sleep(2)
# Click on ‘ data ’
table_click = self.wait.until(EC.presence_of_element_located((By.XPATH, '//*[@id="entries_nav"]')))
table_click.click()
time.sleep(2)
# Export data
data_out = self.wait.until(EC.presence_of_element_located((By.XPATH, '/html/body/div[3]/div[1]/div[3]/div/div[2]/table/tbody/tr/td[15]')))
data_out.click()
time.sleep(60) # The waiting time here is a bit out of control
# Data pre download
data_fore_download = self.wait.until(EC.presence_of_element_located((By.XPATH, '//*[@id="export_job_modal"]/div/div/div[3]/div/a[1]')))
data_fore_download.click()
time.sleep(10)
# Data download
data_download = self.wait.until(EC.presence_of_element_located((By.XPATH, '/html/body/div[3]/div[1]/div[3]/div/div[2]/table/tbody/tr/td[15]/table/tbody/tr/td/div[1]/span[2]/a[1]')))
data_download.click()
self.driver.close()
test = golden_data()
test.login_data()
test.collect_data()
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081Don't spray , I know code robustness is not good , Also know that there is no high availability . But it's just the result of my day as a novice , It's not that you can't learn , I'll make up for it later .
Code interpretation
Explain as much as you remember .
The first pit ：find_element_name
At the very beginning , It's not like my code , It's this kind of ：
···
name_in = driver.find_element_by_name('···')
teacher.send_keys('····')
pwd_in = driver.find_element_by_name('···')
assistant.send_keys('····')
time.sleep(1)
button = driver.find_element_by_class_name('···')
time.sleep(1)
button.click()
time.sleep(1)
driver.close()
1234567891011 good heavens , What's wrong with the report ？
NoSuchFrameException: Message: no such frame
good heavens , I haven't been through a lot of battles , It's been dozens of battles, big and small , I decisively opened Baidu , A lot of solutions come into view , I'll be dazzled ？ A ball , I caught the common ground in this messy solution at a glance , They're all repeating a tag name “ifraem”, Ah , Is not important , Anyway, just the combination of these letters .
what did you say? , It's because there's a tab switch , So go to that tab first , Just put that page of id perhaps class Just bring it up and do a guide .
Na , like this ：browser.switch_to_frame(' new iframe')
well , You really don't have to say , And I did it , It's also the starting point of my night of collapse .
The second pit ： switch_to_frame()
wow , This huge pit . As soon as this line of code is pasted, it shows that it has been deleted , But I don't think so , therefore ...
So how is this fat four ？
This function is really obsolete , Under my constant efforts , I found a replacement function ：switch_to.frame()
If this is solved, everything will be fine ？
Please go back to the first question .
Collapse ？ When you think you're passing the border , All the way up , Suddenly found that he was still in the same place .
I can't come out of it , It's one or two o'clock in the evening , I decided to go to sleep .
Before going to bed , I think about it again and again , Get up tomorrow and change cookies Well .
cookies Bypass login authentication , Around the desert
use cookies I won't say much about it , It won't go around. Please go back to “ Seventh days ” Start .
Let's just say that I came up with something after going around ：
D:\pythonProject3\Scripts\python.exe C:/Users/asus/PycharmProjects/pythonProject3/main.py
<Response [200]>
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="utf-8"/>
<title> Gold data - Page not found </title>
<meta content="IE=Edge,chrome=1" http-equiv="X-UA-Compatible"/>
<link href="https://gd-assets.jinshujucdn.com/assets/favicon-62fe2f27ea9d532a13fc76ed0e8b5e68bc2f61dde4a7935f54ff5dc3e3a727b2.ico" rel="shortcut icon" type="image/x-icon"/>
<link href="https://gd-assets.jinshujucdn.com/assets/blank-layout-67aac9b8f147aa0bf1d9b85c3683738452c3f41160e169b85e61130171db5992.css" media="screen" rel="stylesheet"/>
</head>
<body>
<div class="main-content blank-container">
<div class="page-alert">
<div class="status-code">404</div>
<h2> I'm sorry , The page you are looking for does not exist </h2>
<p> Are you sure it's this URL ？ perhaps <a href="/"> Back to the home page </a></p>
</div>
</div>
<footer>
<a class="powered-by" data-no-turbolink="true" href="/">
<p><span>Powered By </span><i class="gd-icon-logo powered-logo"></i><span> Gold data </span></p>
</a> </footer>
<script>
var _hmt = _hmt || [];
(function() {
if(document.querySelectorAll("script[src*='hm.baidu.com']").length === 0){
var hm = document.createElement("script");
hm.src = "//hm.baidu.com/hm.js?47cd03e974df6869353431fe4f4d6b2f";
var s = document.getElementsByTagName("script")[0];
s.parentNode.insertBefore(hm, s);
}
})();
</script>
</body>
</html>
12345678910111213141516171819202122232425262728293031323334353637 You know what? ？ When I bypass login verification , I still have a burst of joy in my heart , well , Sample , It's not that I've been through yet .
then ,‘pia’
And then I thought about it , Or come back to use selenium 了 .
Why? ？ First of all, there must be my consideration . Secondly, there are my considerations , Finally, it is because other modules in the project need to use selenium Well , Sooner or later we have to face , To die sooner or later is to die .
Usher in a turnaround
At the end of the day , I have a try mentality , Direct search “selenium Operating gold data ”..
A dead horse is a living horse doctor , Guess what ？
Sure enough, the ball was not found. Ha ha ha ha ha
Last , forced , I went to change the course ,“selenium Automated test operations Chrome”, Ah , Over and over again , Find a lazy script left by a senior of RTVU for his students , My day, , Finally opened it for me （ Be careful , I didn't step out before , even UI I can't feel it ）！！！
therefore , The code at the beginning of the article appears .
In fact, the code is not simple , It's not to say that a change is over , It's so simple that it's been done for a long time .
The bumps and bumps on the road to success
Actually, I didn't write it like this at the beginning , At first, I wrote C In language form , Although I know this one is also C stylistic , Who would tell me this is object-oriented , Then I suggest you go back and learn design patterns .
# Local Chrome Browser settings
class golden_data:
def __init__(self):
# These lines are used to hide the browser , But it can't be downloaded , So put it on hold
#options = Options()
#chrome_options.add_argument('--no-sandbox') # solve DevToolsActivePort File does not exist
#options.add_argument('--disable-gpu') # Google Documents mention the need to add this attribute to circumvent bug
#options.add_argument('blink-settings=imagesEnabled=false') # Don't load pictures , Speed up
#options.add_argument('--headless') # Browsers don't provide visual pages . linux Next, if the system does not support visualization, it will fail to start
Get handle , Sign in , skip
# Collect form data
def collect_data(self):
# Click on the first questionnaire
There's a need to improve scalability
first_block = self.wait.until(EC.presence_of_element_located((By.XPATH, '//*[@id="5fa54cff61936cdee3121fa5"]/div/a')))
first_block.click()
time.sleep(2)
# Click on ‘ data ’
table_click = self.wait.until(EC.presence_of_element_located((By.XPATH, '//*[@id="entries_nav"]')))
table_click.click() It used to be hard to live here , Later I found that it was because I forgot to put first_block Change to table_click 了
time.sleep(2)
# Export data
data_out = self.wait.until(EC.presence_of_element_located((By.XPATH, '/html/body/div[3]/div[1]/div[3]/div/div[2]/table/tbody/tr/td[15]'))) There was no way to export data here , direct copy Xpath It can't meet the needs of the business
notes （1）
data_out.click()
time.sleep(60) # The waiting time here is a bit out of control
There was a time-out when the function writing method was changed to the class writing method , The reason is that I let go 10 second
So the speed here is very affected by the network , This should be try···throw···
# Data pre download
data_fore_download = self.wait.until(EC.presence_of_element_located((By.XPATH, '//*[@id="export_job_modal"]/div/div/div[3]/div/a[1]')))
data_fore_download.click()
time.sleep(10)
# Data download
data_download = self.wait.until(EC.presence_of_element_located((By.XPATH, '/html/body/div[3]/div[1]/div[3]/div/div[2]/table/tbody/tr/td[15]/table/tbody/tr/td/div[1]/span[2]/a[1]')))
The problem here is the same as above , Unable to meet business requirements , But with the above as a cushion , So it's a quick solution here
data_download.click()
self.driver.close()
test = golden_data()
test.login_data()
test.collect_data()
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051notes
in my opinion ,copy full Xpath More stable , I don't know if it's Xpath It's faster , After all, the absolute path is compared to the relative path , The relative path is faster .
First come here. , In fact, the previous form is automatically generated, and I can write it fast OK 了 , But keep some suspense , Add in slowly , Yes , It's in this article , This article is constantly updated ！！！
One more sentence at the end , Want to learn Python Please contact Xiaobian , Here's my own set python Learning materials and routes , Anyone who wants this information can enter q skirt 947618024 receive .
The material of this article comes from the Internet , If there is infringement, please contact to delete .