限时免费试用:欢迎注册 api.bigmodel.org ,快速体验大模型 API 接入服务。

Python

python 采集斗图啦xpath

import requests; import re; import os; import parsel; # 1.请求网页 header = { "user-agent":'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrom

作者
王明昌
发布时间
import requests;
import re;
import os;
import parsel;
# 1.请求网页
header = {
    "user-agent":'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36'
}
response = requests.get('https://www.doutula.com/photo/list/',headers=header)
html =  response.text
# 2.解析网页
selector = parsel.Selector(html)
# print(selector)

dir_name = 'doutu_list'
if not os.path.exists(dir_name):
    os.mkdir(dir_name)

result_list = selector.xpath('//a[@class="col-xs-6 col-sm-3"]')
for result in result_list:
    title = result.xpath('./img/@alt').get()
    img_url = result.xpath('./img/@data-original').get()
    ext = img_url.split('.')[-1]
    file_name = title+'.'+ext
    print(title,img_url)
    # 保存
    response = requests.get(img_url,headers=header)
    with open(dir_name+'/'+file_name,'wb') as f:
        f.write(response.content)

0/500
友善分享您的看法。
反馈邮箱