Python 请求 401 错误，但 url 在浏览器中打开 [英] Python requests 401 error but url opens in browser

问题描述

我正在尝试从这个位置提取 json -https://www.nseindia.com/api/option-chain-指数?symbol=BANKNIFTY

I am trying to pull the json from this location - https://www.nseindia.com/api/option-chain-indices?symbol=BANKNIFTY

这在我的浏览器中打开正常，但在 python 中使用请求会引发 401 权限错误.我尝试添加具有不同参数的标题，但无济于事.有趣的是，此页面上的 json 也不会在浏览器中打开，直到 https://www.nseindia.com 单独打开.我相信它需要某种身份验证，但很惊讶它可以在没有任何身份验证的浏览器中运行.

This opens fine in my browser, but using requests in python throws a 401 permission error. I have tried adding headers with different arguments, but to no avail. Interestingly, the json on this page does not open in the browser as well until https://www.nseindia.com is opened separately. I believe it requires some kind of authentication, but surprised it works in the browser without any.

有没有办法从这个网址中提取信息?非常感谢任何帮助.

Is there a way to extract the information from this url? Any help is much appreciated.

这是我的实现 -

import requests

url = 'https://www.nseindia.com/api/option-chain-indices?symbol=BANKNIFTY'

# This throws a 401 response error
page = requests.get(url, headers={"User-Agent": "Mozilla/5.0"})

# This throws a 'Connection aborted' error
page = requests.get(url, headers={"User-Agent": "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8 GTB7.1 (.NET CLR 3.5.30729)"})

推荐答案

要获取正确的数据，首先使用 requests.get() 从其他 URL 加载 cookie，然后执行 Ajax 请求加载JSON:

To get the correct data, first load cookies from other URL with requests.get() and then do Ajax request to load the JSON:

import json
import requests
from bs4 import BeautifulSoup


headers = {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:80.0) Gecko/20100101 Firefox/80.0'}
url = 'https://www.nseindia.com/api/option-chain-indices?symbol=BANKNIFTY'

with requests.session() as s:

    # load cookies:
    s.get('https://www.nseindia.com/get-quotes/derivatives?symbol=BANKNIFTY', headers=headers)

    # get data:
    data = s.get(url, headers=headers).json()

    # print data to screen:
    print(json.dumps(data, indent=4))

打印:

{
    "records": {
        "expiryDates": [
            "03-Sep-2020",
            "10-Sep-2020",
            "17-Sep-2020",
            "24-Sep-2020",
            "01-Oct-2020",
            "08-Oct-2020",
            "15-Oct-2020",
            "22-Oct-2020",
            "29-Oct-2020",
            "26-Nov-2020"
        ],
        "data": [
            {
                "strikePrice": 18100,
                "expiryDate": "03-Sep-2020",
                "CE": {
                    "strikePrice": 18100,
                    "expiryDate": "03-Sep-2020",
                    "underlying": "BANKNIFTY",
                    "identifier": "OPTIDXBANKNIFTY03-09-2020CE18100.00",
                    "openInterest": 1,
                    "changeinOpenInterest": 1,
                    "pchangeinOpenInterest": 0,
                    "totalTradedVolume": 2,
                    "impliedVolatility": 95.51,
                    "lastPrice": 6523.6,
                    "change": 2850.1000000000004,
                    "pChange": 77.58540901048048,
                    "totalBuyQuantity": 2925,
                    "totalSellQuantity": 2800,
                    "bidQty": 25,
                    "bidprice": 6523.6,
                    "askQty": 25,
                    "askPrice": 6570.3,
                    "underlyingValue": 24523.8
                },
                "PE": {
                    "strikePrice": 18100,
                    "expiryDate": "03-Sep-2020",
                    "underlying": "BANKNIFTY",

...and so on.

这篇关于Python 请求 401 错误，但 url 在浏览器中打开的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！