从使用javascript的页面下载（动态生成） [英] Download from page which uses javascript (dynamically generated)
问题描述
如何从此页面下载数据？
http://www.cmegroup.com/trading/agricultural/spot-call-data.html
I尝试了下面的内容，但它似乎只给出了标题。
如果可以使用批处理文件下载它会很棒。否则.NET很好。我正在使用Visual Studio 2010.
表格下的值是根据所选交易日期使用Java脚本获取的，因此它们是动态值，因此在写字板上打开时无法看到。
如何获取桌面上的值？
我尝试过：
Dim uriString As String =http：//www.cmegroup。 com / trading / agricultural / spot-call-data.html
创建一个新的WebClient实例。
Dim myWebClient As New WebClient（）
Console.WriteLine（（Searching+ uriString +.......））
myWebClient .DownloadFile（uriString，C：\ temp \ searchresult.htm）
未经许可访问网站数据不符合道德标准。但我会解释如何用于教育目的。
数据存储在部分下载中。要查看这些内容，请在Google Chrome中加载该页面，然后打开开发人员工具并选择网络标签。这样您就可以观看流量了。现在更改下拉列表的选择并等待数据更改。完成后，滚动时间线，直到看到轮询的数据。选择时间轴上的条目，现在您可以看到查询。他们应该看起来像这样：
http://www.cmegroup.com/CmeWS/mvc/xsltTransformer.do?xlstDoc=/XSLT/md/spotcall_Cheese。 xsl& url = / da / Clearing / SpotCallProducts？date = 09/07/2017
http://www.cmegroup.com/CmeWS/mvc/xsltTransformer.do?xlstDoc=/XSLT/md /spotcall_NonfatDryMilk.xsl&url=/da/Clearing/SpotCallProducts?date=09/07/2017
http://www.cmegroup.com/CmeWS/mvc/xsltTransformer.do?xlstDoc=/ XSLT / md / spotcall_Butter.xsl& url = / da / Clearing / SpotCallProducts？date = 09/07/2017
现在您可以打开一个新的Google Chrome标签并尝试输出每个链接，现在您可以访问格式化的数据。你需要从数据中去掉html元素。
要知道数据刷新的时间，你需要轮询原始页面并从drop中提取日期。
以下是Google搜索：如何使用c＃从html页面提取数据 - Google搜索 [ ^ ] - 这看起来会有所帮助： c＃ - 从网页中提取数据，解析特定部分的数据，以及显示它 - Stack Overflow [ ^ ]。
这是一个VB搜索版本：如何从html页面中提取数据使用vb -
Google搜索 [ ^ ]给我们这个：提取来自vb.net中html源代码（网站）的特定html字符串 - Stack Overflow [ ^ ]
How do I download data from this page?
http://www.cmegroup.com/trading/agricultural/spot-call-data.html
I tried the following, but it only seems to give headers.
If it can be downloaded using a batch file that would be great. Otherwise .NET is fine. I'm using Visual Studio 2010.
The values under the table are fetched using a Java script based on the trade date selected, so they are dynamic values and so can’t be seen when opened on the textpad.
How can I get the values on the table?
What I have tried:
Dim uriString As String = "http://www.cmegroup.com/trading/agricultural/spot-call-data.html"
' Create a new WebClient instance.
Dim myWebClient As New WebClient()
Console.WriteLine(("Searching " + uriString + " ......."))
myWebClient.DownloadFile(uriString, "C:\temp\searchresult.htm")Accessing website data without permission is not ethical. But I will explain how to do it for educational purposes.
The data is stored in partial downloads. To see these, load the page in Google Chrome, then open the developer tools and select the Network Tab. This is so that you can watch the traffic. Now change the selection of the drop down and wait for the data to change. Once completed, scroll the timeline until you see the data polled. Select the entries on the timeline and now you can see the queries. They should look something like this:
http://www.cmegroup.com/CmeWS/mvc/xsltTransformer.do?xlstDoc=/XSLT/md/spotcall_Cheese.xsl&url=/da/Clearing/SpotCallProducts?date=09/07/2017 http://www.cmegroup.com/CmeWS/mvc/xsltTransformer.do?xlstDoc=/XSLT/md/spotcall_NonfatDryMilk.xsl&url=/da/Clearing/SpotCallProducts?date=09/07/2017 http://www.cmegroup.com/CmeWS/mvc/xsltTransformer.do?xlstDoc=/XSLT/md/spotcall_Butter.xsl&url=/da/Clearing/SpotCallProducts?date=09/07/2017
Now you can open a new Google Chrome tab and try out each link, now you have access to the formatted data. You will need to strip the html elements from the data.
To know when the data refreshes, you will need to poll the original page and extract the dates from the drop down.
Here is a Google Search to: how to extract data from html page using c# - Google Search[^] - this looks like it will help: c# - Pulling data from a webpage, parsing it for specific pieces, and displaying it - Stack Overflow[^].
And here is a VB search version: how to extract data from html page using vb -
Google Search[^] that gives us this: Extract specific html string from html source code(website) in vb.net - Stack Overflow[^]
这篇关于从使用javascript的页面下载（动态生成）的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
