网页抓取 - 谷歌地图网站 - 是否可以抓取? [英] Web Scraping - Google Map Website - is it possible to scrape?

问题描述

刚刚加入 SO 所以我想知道你是否可以帮助我解决这个问题.我们曾经爬过一个网站，获取美国/世界运动量健身馆的所有联系信息，因为这些信息在那里非常暴露.但是，现在他们已将其网站更改为 map.crossfit.com，因此信息被嵌入到谷歌样式的地图中，因此您实际上只能通过缩放来获取每个健身房的信息(名称、地址、电话号码等)一个一个地选择它们，这需要我花很长时间才能获得所有美国的(大约 6,000 个).

Just joined SO so I was wondering if you can help me with this issue. We used to scrape a website and get all the contact information for crossfit gyms in the US/world as the information was pretty exposed out there. Now, however, they have changed their website to map.crossfit.com so the information is embedded within a google style map, so you can only actually get the information for each gym (name, address, phone #, etc.) by zooming in and choosing them one by one, which would take me forever just to get all the US ones (approximately 6,000).

我不是编程专家，所以我假设如果信息仍然存在，应该有一种方法来抓取它.你们能告诉我这是否可能，并可能给我一些提示吗?

I'm not an expert in programming so I'm assuming that if the information is still there, there should be a way to scrape it. Can you guys give tell me if that is possible and possibly give me some hints on that?

非常感谢您的帮助！瑞克

Really appreciate your help! Rick

推荐答案

您好，您可以使用下一个命令=

Hello you can use the next command=

curl 'https://map.crossfit.com/getAffiliateInfo?aid=9347'

{"name":"CrossFit Radiate","website":"http://www.crossfitradiate.com/","address":"149 S. Fowler St","city":"Bishop","state":"CA","zip":"93514","country":"United States","cfkids":true,"phone":"[phone]","courses":[]}

{"name":"CrossFit Radiate","website":"http://www.crossfitradiate.com/","address":"149 S. Fowler St","city":"Bishop","state":"CA","zip":"93514","country":"United States","cfkids":true,"phone":"[phone]","courses":[]}

你会得到一个包含健身房所有信息的 json...

yo will get a json with all the information about the gym...

"name":"Golden State CrossFit","website":"http://goldenstatecrossfit.com/","address":"11174 La Grange Ave","city":"洛杉矶","state":"CA","zip":"90025","country":"美国","cfkids":false,"phone":"[phone]","courses":[]}

"name":"Golden State CrossFit","website":"http://goldenstatecrossfit.com/","address":"11174 La Grange Ave","city":"Los Angeles","state":"CA","zip":"90025","country":"United States","cfkids":false,"phone":"[phone]","courses":[]}

并将值更改为 +1 的值

And change the value adding +1 to the value

--可以将信息从json解析为csv，或者excel，或者

--The info can be parsed from json to csv, or excel, or

这篇关于网页抓取 - 谷歌地图网站 - 是否可以抓取?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！