火狐 - 收集由LAB.min.js使用的数据 [英] Firefox - Collecting the data used by LAB.min.js

问题描述

我有一个装置 - 斯宝亚创热泵 - 回家，我可以用一个web界面跟踪。

我只能访问previous月和当月。
但我想，以保持所有数据，在数字的方式，而不是像图像。

解决方案

首先，我必须得到原始数据。我知道如何与Firefox和Firebug的插件做手动复制/粘贴数据到一个文件...。它采用 angularjs.1.0.6 ，我从获取显示由于萤火虫之一获取原始数据：

  GET的https：// thesite / API /数据/ 1036493 / heatEnergy /今天noCacheDummyValue = 1459148624909
GET的https：// thesite / API /数据/ 1036493 / outTemp /今天noCacheDummyValue = 1459148624905
GET的https：// thesite / API /数据/ 1036493 / heatEnergy / lastMonth noCacheDummyValue = 1459148708330
GET的https：// thesite / API /数据/ 1036493 / outTemp / lastMonth noCacheDummyValue = 1459148708321
 

这yould会更好，如果我能做到这一点
  - 通过直接入侵到LAB.min.js数据？
  - 通过调用卷曲与我平时登录的帖子，然后收集上述

与 GET s的数据

下面是如何我卷曲登录 POST 电话如下：

<？pre> 卷曲-o myraw.data --data-urlen code的userName =蓬蓬裙和放大器;密码= XX的https：// thesite / API /登录noCacheDummyValue = 1458825357449 '\\
     -H主机：thesite'-H的User-Agent：Mozilla的/ 5.0（Windows NT的6.1; WOW64; RV：45.0）的Gecko / 20100101火狐/ 45.0\\
     -H接受：应用/ JSON，纯文本/ * / *-H的Accept-Language：EN-US，EN; Q = 0.5'-H接受编码：gzip，紧缩，BR'\\
     -HX-要求 - 由于：XMLHtt prequest'\\
     -H内容类型：应用程序/ JSON的;字符集= utf-8'\\
     -H'的Referer：HTTPS：//thesite/mobile/app/app.html'\\
     -H内容长度：44'-H'的Cookie：JSESSIONID = 1m5aputqimremdk95fgbtmgo9'\\
     -H代理授权：基本?? -HDNT：1 \\
     -H'连接：保持活动


不过，我什么也没得到。现在，我很高兴与萤火虫 复制响应体/粘贴功能。

现在我得到我的原始数据，寻找像一些JSON的事情我想：

  [{价值观：
[{选项：空，webId：空，值：8.9，日期：1458842750000，writeValue：NULL，单位：空}
，选项：空，webId：空，值：9.1，日期：1458842451000，writeValue：NULL，单位：空}
{选项：空，
...
,{\"options\":null,\"webId\":null,\"value\":\"5.5\",\"date\":1458774051000,\"writeValue\":null,\"unit\":null}],\"unit\":\"°C\"}]
 

现在我得到了我的数据，我想这会很容易。但我越来越头疼试图做这个JSON数据东西...我知道有一些人已经尝试：<一个href=\"http://stackoverflow.com/questions/22534203/does-libreoffice-calc-support-json-file-importing-sorting/36210005#36210005\">Does LibreOffice的计算器支持JSON文件导入/排序？

我只是在我的JSON规律，最近（重新）发现 SED 。所以，我用我的新动力（和它是多么困难找到我必须在我的替代值增加一个真正的换行符）：

 猫rawdata.txt | TR -d的'\\ n'| SED的/}，{/ \\
/ G'| SED的/选项：空，webId：空，价值：//| \\
SED的/，writeValue：NULL，单位：空//'| SED的/，日期：/; /'| \\
SED的/ \\ [{价值观：\\ [{//'| sed的'S /} \\]，单位：°C} \\] / \\
    /| SED的/} \\]，单位：千瓦时} \\] / \\
    /'＆GT; pureData.csv
 

和得到像温度 / 时间戳我的数据：

 菊菊@ HP-康柏DC7900小外形：〜/ INSTALL / MyProj文$猫pureData.csv
8.9; 1458842750000
9.1; 1458842451000
9.3; 1458842150000
9.4; 1458841851000
9.7; 1458841550000
9.9; 1458841251000
10.0; 1458840950000
...
 

...在这一点上太高兴了！当我绘制它，它看起来像什么，我在我的Firefox窗口！

注：我估计我的时间轴是基于的自毫秒数01.01.1970 的

如果有兴趣的人，我写的东西检索与数据卷曲有：<一href=\"http://stackoverflow.com/questions/36315280/how-to-use-linux-curl-to-login-and-retreive-data-once-logged/36461565#36461565\">How使用Linux卷曲登录从其获取一次记录的数据？

I have a device - Stiebel Eltron heat pump - home and I can track it with a web interface.

I can only access the previous month and the current month. But I would like to keep the all the data, in a numeric way rather than having it like images.

解决方案

In the first place I must get the raw data. I know how to do it with Firefox and Firebug addin to copy/paste the data to a file... manually. It uses angularjs.1.0.6, and I get the raw data from one of the GETs displayed thanks to Firebug:

GET https://thesite/api/data/1036493/heatEnergy/today?noCacheDummyValue=1459148624909
GET https://thesite/api/data/1036493/outTemp/today?noCacheDummyValue=1459148624905
GET https://thesite/api/data/1036493/heatEnergy/lastMonth?noCacheDummyValue=1459148708330
GET https://thesite/api/data/1036493/outTemp/lastMonth?noCacheDummyValue=1459148708321

It yould be better if I could do it - by hacking directly into the LAB.min.js data? - by calling curl with my usual login post, then collect data with the GETs above

Here is how my curl login POST call looks like:

    curl -o myraw.data --data-urlencode "userName=tutu&password=xx"  'https://thesite/api/login?noCacheDummyValue=1458825357449' \
     -H 'Host: thesite' -H 'User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0' \
     -H 'Accept: application/json, text/plain, */*' -H 'Accept-Language: en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate, br' \
     -H 'X-Requested-With: XMLHttpRequest' \
     -H 'Content-Type: application/json;charset=utf-8' \
     -H 'Referer: https://thesite/mobile/app/app.html' \
     -H 'Content-Length: 44' -H 'Cookie: JSESSIONID=1m5aputqimremdk95fgbtmgo9'  \
     -H 'Proxy-Authorization: Basic ??' -H 'DNT: 1'   \
     -H 'Connection: keep-alive' 

But I get nothing. For now, I am happy with the Firebug Copy Response Body / Paste functionality.

Now I get my raw data looking like some JSON thing I guess:

[{"values":
[{"options":null,"webId":null,"value":"8.9","date":1458842750000,"writeValue":null,"unit":null}
,"options":null,"webId":null,"value":"9.1","date":1458842451000,"writeValue":null,"unit":null}
,{"options":null, 
...
,{"options":null,"webId":null,"value":"5.5","date":1458774051000,"writeValue":null,"unit":null}],"unit":"°C"}]

Now I got my data, I thought it would be easy. But I am getting headache trying to do something with this JSON data... I know some did try already: Does LibreOffice Calc support JSON file importing/sorting?

I just have regular pattern in my JSON, and recently (re)discovered sed. So I use my new power (and how difficult it was to find I had to add a real line break in my substitute value):

cat rawdata.txt | tr -d '\n' | sed 's/},{/\
/g' | sed 's/"options":null,"webId":null,"value"://' |\
sed 's/,"writeValue":null,"unit":null//' |sed 's/,"date":/;/' |\
sed 's/\[{"values":\[{//' |sed 's/}\],"unit":"°C"}\]/\
    /' | sed 's/}\],"unit":"kWh"}\]/\
    /' > pureData.csv

And get my data like "temperature" / "timestamp":

ju@ju-HP-Compaq-dc7900-Small-Form-Factor:~/INSTALL/myproj$ cat pureData.csv
"8.9";1458842750000
"9.1";1458842451000
"9.3";1458842150000
"9.4";1458841851000
"9.7";1458841550000
"9.9";1458841251000
"10.0";1458840950000
...

... so happy at this point! When I plot it, it looks like what I have in my Firefox window!

NB: I calculate that my time axis is based on a number of milliseconds since 01.01.1970.

If anyone interested, I wrote something to retrieve data with curl there: How to use linux curl to login and retreive data once logged?

这篇关于火狐 - 收集由LAB.min.js使用的数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！