Day 28 烂番茄影评网爬取

今天的影片内容为爬取一个非常有名的影评网—Rotten Tomatoes(烂番茄)
还会介绍网页图片的下载及储存，非常的实用呦～

以下为影片中有使用到的程序码

#烂番茄爬取
import requests, bs4

url = "https://editorial.rottentomatoes.com/guide/2021-best-movies/"
headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36'}
htmlfile = requests.get(url, headers = headers)
objsoup = bs4.BeautifulSoup(htmlfile.text, 'lxml')


movies = objsoup.find_all('div', class_ = 'row countdown-item')

for movie in movies:

    number = movie.find('div', class_ = 'countdown-index')
    print("电影编号:", number.text)
    
    name_1 = movie.select('h2 > a') #搜寻所有<h2>内的<a>元素，中间没有其他元素
    name_2 = name_1[0].text
    print("Movie:", name_2)
    
    tomatoes = movie.find('span', class_ = 'tMeterScore') #寻找烂番茄指数
    print("Grade:", tomatoes.text)
    
    consensus = movie.find('div', class_ = 'info critics-consensus') #寻找评论
    print(consensus.text)
    
    synopsis = movie.find('div', class_ = 'info synopsis') #寻找剧情摘要
    print(synopsis.text)
    
    starring = movie.find('div', class_ = 'info cast') #寻找主演影星
    print(starring.text)
    
    directed_by = movie.find('div', class_ = 'info director') #寻找导演
    print(directed_by.text)
    
    print("="*100)

#电影海报下载
#请依个人喜好新增photo资料夹，并将C:\\Users\\ASUS\\Desktop修改为photo资料夹的路径
import requests, bs4

url = "https://editorial.rottentomatoes.com/guide/2021-best-movies/"
headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36'}
htmlfile = requests.get(url, headers = headers)
objsoup = bs4.BeautifulSoup(htmlfile.text, 'lxml')

fn = "C:\\Users\\ASUS\\Desktop\\photo\\"
number = 176 #网站影评数量会缓慢增加，执行前建议先查看原网页

movies = objsoup.find_all('div', class_ = 'row countdown-item')

for movie in movies:
    picture_element = movie.find('img')
    picture_url = picture_element.get('src') 
    print(picture_url)
    
    picture = requests.get(picture_url)
    picture.raise_for_status() #查看下载情形
    print("图片下载成功!")
    
    number -= 1
    
    img = fn + str(number) + ".png" #设定档名
    
    with open(img, 'wb') as file: 
	    file.write(picture.content) #写入档案
	    file.flush() #清空暂存资料
    file.close() #关闭档案
    
    print("图片" + str(number) + "储存成功!")

本篇影片及程序码仅提供研究使用，请勿大量恶意地爬取资料造成对方网页的负担呦！
如果在影片中有说得不太清楚或错误的地方，欢迎留言告诉我，谢谢您的指教。