使用 BeautifulSoup 从未关闭的特定元标记中提取内容 [英] Extracting contents from specific meta tags that are not closed using BeautifulSoup
问题描述
我正在尝试从特定元标记中解析出内容.这是元标记的结构.前两个以反斜杠结束，但其余没有任何结束标记.一旦我获得了第三个元标记，就会返回 标记之间的全部内容.我也试过 soup.findAll(text=re.compile('keyword')) 但它没有返回任何东西，因为关键字是元标记的一个属性.
I'm trying to parse out content from specific meta tags. Here's the structure of the meta tags. The first two are closed with a backslash, but the rest don't have any closing tags. As soon as I get the 3rd meta tag, the entire contents between the <head> tags are returned. I've also tried soup.findAll(text=re.compile('keyword')) but that does not return anything since keyword is an attribute of the meta tag.
<meta name="csrf-param" content="authenticity_token"/>
<meta name="csrf-token" content="OrpXIt/y9zdAFHWzJXY2EccDi1zNSucxcCOu8+6Mc9c="/>
<meta content='text/html; charset=UTF-8' http-equiv='Content-Type'>
<meta content='en_US' http-equiv='Content-Language'>
<meta content='c2y_K2CiLmGeet7GUQc9e3RVGp_gCOxUC4IdJg_RBVo' name='google-site- verification'>
<meta content='initial-scale=1.0,maximum-scale=1.0,width=device-width' name='viewport'>
<meta content='notranslate' name='google'>
<meta content="Learn about Uber's product, founders, investors and team. Everyone's Private Driver - Request a car from any mobile phone—text message, iPhone and Android apps. Within minutes, a professional driver in a sleek black car will arrive curbside. Automatically charged to your credit card on file, tip included." name='description'>
代码如下:
import csv
import re
import sys
from bs4 import BeautifulSoup
from urllib.request import Request, urlopen
req3 = Request("https://angel.co/uber", headers={'User-Agent': 'Mozilla/5.0')
page3 = urlopen(req3).read()
soup3 = BeautifulSoup(page3)
## This returns the entire web page since the META tags are not closed
desc = soup3.findAll(attrs={"name":"description"})
推荐答案
按照@Albert Chen 的建议添加了区分大小写的正则表达式.
Edited: Added regex for case sensitivity as suggested by @Albert Chen.
Python 3
from bs4 import BeautifulSoup
import re
import urllib.request
page3 = urllib.request.urlopen("https://angel.co/uber").read()
soup3 = BeautifulSoup(page3)
desc = soup3.findAll(attrs={"name": re.compile(r"description", re.I)})
print(desc[0]['content'])
虽然我不确定它是否适用于每个页面:
Although I'm not sure it will work for every page:
from bs4 import BeautifulSoup
import re
import urllib
page3 = urllib.urlopen("https://angel.co/uber").read()
soup3 = BeautifulSoup(page3)
desc = soup3.findAll(attrs={"name": re.compile(r"description", re.I)})
print(desc[0]['content'].encode('utf-8'))
产量:
Learn about Uber's product, founders, investors and team. Everyone's Private Dri
ver - Request a car from any mobile phoneΓÇötext message, iPhone and Android app
s. Within minutes, a professional driver in a sleek black car will arrive curbsi
de. Automatically charged to your credit card on file, tip included.
这篇关于使用 BeautifulSoup 从未关闭的特定元标记中提取内容的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
