BeautifulSoup is not an HTTP client

BeautifulSoup不支持直接打开链接,使用urllib3库来获取链接内容再实例化BeautifulSoup对象

import ssl
from bs4 import BeautifulSoup
import urllib3

url = "https://www.baidu.com"
urllib3.disable_warnings()
http = urllib3.PoolManager(cert_reqs=ssl.CERT_NONE, assert_hostname=False)
response = http.request("GET", url)
bs = BeautifulSoup(response.data, "lxml")

相关问答