如何通过命令行将爬虫数据发送到PHP? [英] How to send crawler data to PHP via command line?
本文介绍了如何通过命令行将爬虫数据发送到PHP?的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我可以发送结果而不是将结果存储在JSON文件中,然后将其发送到PHP吗?
Can I send the results rather than stored in the JSON file, send it to PHP?
我有这两个文件
settings.json
settings.json
{
"outputFile" : "C:\\wamp\\www\\drestip\\admin\\crawls\\mimshoes.json",
"logFile" : "C:\\wamp\\www\\drestip\\admin\\crawls\\mimshoes.tsv",
"pause" : 1,
"local" : false,
"connections" : 3,
"cookiesEnabled" : false,
"robotsDisabled" : false,
"advancedMode" : true,
"crawlTemplate" : [ "www.mimshoes.com/" ],
"startUrls" : [ PAGES ],
"maxDepth" : 10,
"dataTemplate" : [ "www.mimshoes.com/{alpha}-{alpha}_{alpha}-{alpha}$" ],
"destination" : "JSON",
"connectorGuid" : "xxxxxxxxxxxxxxxxxxxxxxxx",
"canonicalDisabled" : false
}
user.json
user.json
{
"userGuid": "xxxxxxxxxxxxxxxxxxxx",
"apiKey": "xxxxxxxxxxxxxxx"
}
命令行:
C:\Users\creatingweb03\AppData\Roaming\import.io\import.ioc.exe -crawl settings.json user.json
推荐答案
如果在settings.json中使用"target"参数,则可以将结果直接发布到API端点.
If you use the "target" parameter in your settings.json, you can POST the results directly to an API endpoint.
// The url that crawled data will be HTTP POSTed to
"target" : "http://localhost:9200/index/datatype",
这里有更多信息:
这篇关于如何通过命令行将爬虫数据发送到PHP?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文