使用Python中的正则表达式从文本中删除html标签 [英] Removing html tags from a text using Regular Expression in python

问题描述

我试图查看一个html文件并从中删除所有标签，以便只留下文本，但是我的正则表达式出现问题。这是我到目前为止。

  import urllib.request，re 
 def test（url）：
 html = str（urllib.request.urlopen（url）.read（））
 print（re.findall（'< [\w\ / \..\w] *>' ，html））
  

html是一个带有几个链接和文本的简单页面，但是我的正则表达式赢得了' t pick up！DOCTYPE HTML PUBLIC - // W3C // DTD HTML 4.01 Transitional // ENand'a href =....tags。任何人都可以解释我需要在我的正则表达式中更改什么？使用 解决方案 /问题/标记/ beautifulsoup> BeautifulSoup 。使用 lxml 。请勿使用正则表达式来解析HTML。

---

编辑2010-01-29：这对于lxml来说是一个合理的起点：

  from lxml.html import fromstring $ b $ from lxml.html.clean import Cleaner 
导入请求
 
 url =https://stackoverflow.com/questions/2165943/removing-html-tags-from-a-text-using-regular-expression-in-python
 html = requests.get（url ）.text 
 
 doc = fromstring（html）
 
 tags = ['h1'，'h2'，'h3'，'h4'，'h5'，'h6 '，
'div'，'span'，
'img'，'area'，'map'] 
 args = {'meta'：False，'safe_attrs_only'：False，'' page_structure'：False，
'scripts'：True，'style'：True，'links'：True，'remove_tags'：tags} 
 cleaner = Cleaner（** args）
 
 path ='/ html / body' 
 body = doc.xpath（path）[0] 
 
 print clean.clean_html（body）.text_content（）。encode（'ascii'，'ignore'）
  

您想要内容，所以想必您不需要任何JavaScript或CSS。另外，据推测，你只需要内容而不是头部的HTML。阅读 lxml.html.clean ，看看您可以轻松剥离。比正则表达式更聪明，没有？

另外，注意unicode编码问题。您可以很容易地结束HTML，您无法打印。

---

$ b 2012-11-08：从使用urllib2更改为< a href =http://docs.python-requests.org/en/latest/ =nofollow noreferrer>请求。只需使用请求！

I'm trying to look at a html file and remove all the tags from it so that only the text is left but I'm having a problem with my regex. This is what I have so far.

import urllib.request, re
def test(url):
html = str(urllib.request.urlopen(url).read())
print(re.findall('<[\w\/\.\w]*>',html))

The html is a simple page with a few links and text but my regex won't pick up !DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" and 'a href="...." tags. Can anyone explain what I need to change in my regex?

解决方案

Use BeautifulSoup. Use lxml. Do not use regular expressions to parse HTML.

---

Edit 2010-01-29: This would be a reasonable starting point for lxml:

from lxml.html import fromstring
from lxml.html.clean import Cleaner
import requests

url = "https://stackoverflow.com/questions/2165943/removing-html-tags-from-a-text-using-regular-expression-in-python"
html = requests.get(url).text

doc = fromstring(html)

tags = ['h1','h2','h3','h4','h5','h6',
       'div', 'span', 
       'img', 'area', 'map']
args = {'meta':False, 'safe_attrs_only':False, 'page_structure':False, 
       'scripts':True, 'style':True, 'links':True, 'remove_tags':tags}
cleaner = Cleaner(**args)

path = '/html/body'
body = doc.xpath(path)[0]

print cleaner.clean_html(body).text_content().encode('ascii', 'ignore')

You want the content, so presumably you don't want any javascript or CSS. Also, presumably you want only the content in the body and not HTML from the head, too. Read up on lxml.html.clean to see what you can easily strip out. Way smarter than regular expressions, no?

Also, watch out for unicode encoding problems. You can easily end up with HTML that you cannot print.

---

2012-11-08: changed from using urllib2 to requests. Just use requests!

这篇关于使用Python中的正则表达式从文本中删除html标签的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！