刮取json请求`ak`参数值 [英] Scraping a json request `ak` parameter value

问题描述

我正在尝试抓取嵌入式m3u8 url路径进行自学。

到目前为止，我设法确定了请求产生json响应m3u8信息。

例如，

虽然有点令人生畏，但似乎无法转换此内容到Python程序，这样你就可以通过混淆任何 space_id 和域<来获得 ak / you c $ c> youtube-dl中你需要的。祝你好运。

I'm currently trying to scrape embedded m3u8 url paths for self-study.

So far, I managed to pin down the request produces a json response with the m3u8 information.

For example, the https://headlines.yahoo.co.jp/videonews/ann?a=20190527-00000051-ann-soci page would make the following request:

https://feapi-yvpub.yahooapis.jp/v1/content/1579522?appid=dj0zaiZpPVZMTVFJR0FwZWpiMyZzPWNvbnN1bWVyc2VjcmV0Jng9YjU-&output=json&space_id=2078710307&domain=headlines.yahoo.co.jp&ak=e25b66ca8b37b7a383feecf4e084fe95&device_type=1100&thumb_width=1204&thumb_height=676&thumb_priority=l&thumb_bd=0

In order to make a successful request, the contentid (1579522), appid, space_id, device_type, and ak parameters are required. I managed to scrape the id and device_type values but have no idea where to get the ak value, which I guess stands for access key. Any idea on how to get this value?

Side note: I accessed the page through different browsers and they all produced the same ak parameter value in the request, so I'm guessing the value is not uniquely linked to each session, but I could be wrong.

Thanks in advance!

解决方案

The mystery argument ak is created by first combining space_id and domain into a single string such as "2078710307_headlines.yahoo.co.jp". Then it is run through an obfuscator which is located in player.script.js.

The obfuscator looks a bit complicated and seems to be made hard to understand on purpose. Below are debugger screenshots from the obfuscator main function, and a subroutine that it calls.

While a bit daunting, it doesn't seem impossible to convert this to a Python program so that you could then get ak by obfuscating any space_id and domain you needed in youtube-dl. Good luck.

这篇关于刮取json请求`ak`参数值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！