如何使用Python从网站获取脚本标签变量 [英] How to Get Script Tag Variables From a Website using Python
本文介绍了如何使用Python从网站获取脚本标签变量的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
我正在尝试使用Python在脚本标签中提取一个名为meta的变量.我以前曾经用硒来做到这一点，但是硒对于我要完成的工作来说太慢了.还有其他方法吗?
I am trying to pull a variable called meta in a script tag using Python. I have used selenium to do this before, but selenium is too slow for what I am trying to accomplish. Is there any other way of doing this.
我尝试使用BeautifulSoup，但是我被卡住了……代码在下面
I have tried using BeautifulSoup, but I'm stuck... code is below
这是我试图从以下位置获取元变量的脚本标签:
Here is the script tag I'm trying to get the meta variable from:
<script>window.ShopifyAnalytics = window.ShopifyAnalytics || {};
window.ShopifyAnalytics.meta = window.ShopifyAnalytics.meta || {};
window.ShopifyAnalytics.meta.currency = 'USD';
var meta = {"product"{"id":2006141861957,"vendor":"Nike","type":"Sneakers","variants": [{"id":19039563677765,"price":16000,"name":"Nike React Element '87 - Light Orewood Brown \/ Laser Orange \/ Volt Glow - 4","public_title":"4","sku":"191888228157"}, {"id":19039563710533,"price":16000,"name":"Nike React Element '87 - Light Orewood Brown \/ Laser Orange \/ Volt Glow - 4.5","public_title":"4.5","sku":"191888228164"},{"id":19039563743301,"price":16000,"name":"Nike React Element '87 - Light Orewood Brown \/ Laser Orange \/ Volt Glow - 5","public_title":"5","sku":"191888228171"},{"id":19039563776069,"price":16000,"name":"Nike React Element '87 - Light Orewood Brown \/ Laser Orange \/ Volt Glow - 5.5","public_title":"5.5","sku":"191888228188"},{"id":19039563808837,"price":16000,"name":"Nike React Element '87 - Light Orewood Brown \/ Laser Orange \/ Volt Glow - 6","public_title":"6","sku":"886059750741"},{"id":19039563841605,"price":16000,"name":"Nike React Element '87 - Light Orewood Brown \/ Laser Orange \/ Volt Glow - 6.5","public_title":"6.5","sku":"886059750758"},{"id":19039563874373,"price":16000,"name":"Nike React Element '87 - Light Orewood Brown \/ Laser Orange \/ Volt Glow - 7","public_title":"7","sku":"886059750765"},{"id":19039563907141,"price":16000,"name":"Nike React Element '87 - Light Orewood Brown \/ Laser Orange \/ Volt Glow - 7.5","public_title":"7.5","sku":"886059750772"},{"id":19039563939909,"price":16000,"name":"Nike React Element '87 - Light Orewood Brown \/ Laser Orange \/ Volt Glow - 8","public_title":"8","sku":"886059750789"},{"id":19039563972677,"price":16000,"name":"Nike React Element '87 - Light Orewood Brown \/ Laser Orange \/ Volt Glow - 8.5","public_title":"8.5","sku":"886059750796"},{"id":19039564005445,"price":16000,"name":"Nike React Element '87 - Light Orewood Brown \/ Laser Orange \/ Volt Glow - 9","public_title":"9","sku":"886059750802"},{"id":19039564038213,"price":16000,"name":"Nike React Element '87 - Light Orewood Brown \/ Laser Orange \/ Volt Glow - 9.5","public_title":"9.5","sku":"886059750819"},{"id":19039564070981,"price":16000,"name":"Nike React Element '87 - Light Orewood Brown \/ Laser Orange \/ Volt Glow - 10","public_title":"10","sku":"886059750826"},{"id":19039564103749,"price":16000,"name":"Nike React Element '87 - Light Orewood Brown \/ Laser Orange \/ Volt Glow - 10.5","public_title":"10.5","sku":"886059751038"},{"id":19039564136517,"price":16000,"name":"Nike React Element '87 - Light Orewood Brown \/ Laser Orange \/ Volt Glow - 11","public_title":"11","sku":"886059751045"},{"id":19039564169285,"price":16000,"name":"Nike React Element '87 - Light Orewood Brown \/ Laser Orange \/ Volt Glow - 11.5","public_title":"11.5","sku":"886059751052"},{"id":19039564202053,"price":16000,"name":"Nike React Element '87 - Light Orewood Brown \/ Laser Orange \/ Volt Glow - 12","public_title":"12","sku":"886059751069"},{"id":19039564234821,"price":16000,"name":"Nike React Element '87 - Light Orewood Brown \/ Laser Orange \/ Volt Glow - 12.5","public_title":"12.5","sku":"886059751076"},{"id":19039564267589,"price":16000,"name":"Nike React Element '87 - Light Orewood Brown \/ Laser Orange \/ Volt Glow - 13","public_title":"13","sku":"886059752448"}]},"page":{"pageType":"product","resourceType":"product","resourceId":2006141861957}};
for (var attr in meta) {
window.ShopifyAnalytics.meta[attr] = meta[attr];
}</script>
这是我尝试过的:
bs = soup(r.text, "html.parser")
scripts = bs.findAll('script')
for s in scripts:
if 'var meta' in s.text:
print(s)
这就像字符串一样返回:
This returns like a string:
<script>window.ShopifyAnalytics = window.ShopifyAnalytics || {};
window.ShopifyAnalytics.meta = window.ShopifyAnalytics.meta || {};
window.ShopifyAnalytics.meta.currency = 'USD';
var meta = {"product":{"id":2008798101573,"vendor":"Adidas","type":"Sneakers","variants":[{"id":19054898249797,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 3","public_title":"3","sku":"98197426"},{"id":19054898282565,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 3.5","public_title":"3.5","sku":"98197427"},{"id":19054898315333,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 4","public_title":"4","sku":"98197428"},{"id":19054898348101,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 4.5","public_title":"4.5","sku":"98197429"},{"id":19054898380869,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 5","public_title":"5","sku":"191525030983"},{"id":19054898413637,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 5.5","public_title":"5.5","sku":"191525030952"},{"id":19054898446405,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 6","public_title":"6","sku":"191525030938"},{"id":19054898479173,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 6.5","public_title":"6.5","sku":"191525030914"},{"id":19054898511941,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 7","public_title":"7","sku":"191525030907"},{"id":19054898544709,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 7.5","public_title":"7.5","sku":"191525030976"},{"id":19054898577477,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 8","public_title":"8","sku":"191525031010"},{"id":19054898610245,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 8.5","public_title":"8.5","sku":"191525030990"},{"id":19054898643013,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 9","public_title":"9","sku":"191525031027"},{"id":19054898675781,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 9.5","public_title":"9.5","sku":"191525030921"},{"id":19054898708549,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 10","public_title":"10","sku":"191525030969"},{"id":19054898741317,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 10.5","public_title":"10.5","sku":"191525030945"},{"id":19054898774085,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 11","public_title":"11","sku":"191525031003"},{"id":19054898806853,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 11.5","public_title":"11.5","sku":"98197443"},{"id":19054898839621,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 12","public_title":"12","sku":"98197444"},{"id":19054898872389,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 12.5","public_title":"12.5","sku":"98197445"},{"id":19054898905157,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 13","public_title":"13","sku":"98197446"},{"id":19054898937925,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 13.5","public_title":"13.5","sku":"98197447"},{"id":19054898970693,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 14","public_title":"14","sku":"98197448"},{"id":19054899003461,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 14.5","public_title":"14.5","sku":"98197449"},{"id":19054899036229,"price":13000,"name":"adidas Originals WMNS Falcon - Ash Pearl \/ Off White - 15","public_title":"15","sku":"98197450"}]},"page":{"pageType":"product","resourceType":"product","resourceId":2008798101573}};
for (var attr in meta) {
window.ShopifyAnalytics.meta[attr] = meta[attr];
}</script>
我想做的是返回meta变量，这样我就可以从meta变量中提取数据，例如，"product"的所有"id"
What I want to do is return the meta variable so that I have the capability to pull the data from the meta variable, for example, all the "id"s of the "product"
推荐答案
在没有获得该输出的完整代码的情况下，我在这里有点猜测.但是，如果您可以抓取文本，那么只需使用json，就应该能够获取该数据.
Without having the full code to get that output, I'm guessing a bit here. But If you can grab the text, then just use json, you should be able to get that data.
因此，我将使用您以前的问题之一的示例，该示例基本上具有相同的格式:
So I'll use an example of one of your previous questions, that essentially has this same format:
除了我们将提取可利用 json.loads()的字符串部分之外，其他内容实际上没有什么不同.然后，您将获得一个不错的json类型的字典和列表，您可以提取该产品的ID:
There's really nothing different, except we're going to extract the part of the string that can utilize json.loads(). Then you have a nice json type of dictionaries and lists that you can extract the id's of the product:
import requests
import bs4
import json
url = 'https://packershoes.com/products/copy-of-adidas-predator-accelerator-trainer'
r = requests.get(url)
bs = bs4.BeautifulSoup(r.text, "html.parser")
scripts = bs.find_all('script')
jsonObj = None
for s in scripts:
if 'var meta' in s.text:
script = s.text
script = script.split('var meta = ')[1]
script = script.split(';\nfor (var attr in meta)')[0]
jsonStr = script
jsonObj = json.loads(jsonStr)
for value in jsonObj['product']['variants']:
print ('ID: '+ str(value['id']))
输出:
ID: 14189113049177
ID: 14189122912345
ID: 14139452129369
ID: 14139452194905
ID: 14139452227673
ID: 14139452293209
ID: 14139452325977
ID: 14139452391513
ID: 14139452424281
ID: 14189321715801
ID: 14139452457049
ID: 14139909505113
这篇关于如何使用Python从网站获取脚本标签变量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
