BeautifulSoup is not an HTTP client
BeautifulSoup
不支持直接打开链接,使用urllib3
库来获取链接内容再实例化BeautifulSoup
对象
import ssl
from bs4 import BeautifulSoup
import urllib3
url = "https://www.baidu.com"
urllib3.disable_warnings()
http = urllib3.PoolManager(cert_reqs=ssl.CERT_NONE, assert_hostname=False)
response = http.request("GET", url)
bs = BeautifulSoup(response.data, "lxml")