如何使用 Chrome 浏览器使用 VBA 抓取 Twitter? [英] How to web scrape Twitter with VBA using Chrome browser?

问题描述

我正在尝试使用 VBA 抓取 Twitter.该代码适用于使用 Internet Explorer 的其他网站，但由于 Internet Explorer 无法打开 Twitter 网站，我尝试将其替换为 Chrome 浏览器.我找到了如何在 Chrome 中打开 URL，但不知道应该放什么来从 HTML 文档中检索数据.下面，我保留了与 Internet Explorer 一起使用的代码，并添加了打开 Chrome 浏览器的代码.我的主要问题是我应该放什么而不是?????????"?在以下代码中:

I am trying to scrape Twitter with VBA. The code worked fine for other website with the Internet Explorer, but as the Internet Explorer does not open Twitter website, I am trying to replace it with the Chrome browser. I found how to open a URL in Chrome but do not know what should I put to retrieve the data from the HTML document. Below, I kept the code that worked with Internet Explorer and added the codes opening Chrome browser. My main question is what I should put instead of "?????????" in the following code:

Sub GetData()
    
    Dim objIE As InternetExplorer
    Dim itemEle As Object
    Dim desc As String, a As String, title As String, titleDate As String
    
    Dim y As Integer
    Dim sURL As String
    Dim lastrow As Long
 
    Dim chromePath As String
    
    chromePath = """C:\Program Files (x86)\Google\Chrome\Application\chrome.exe"""
    
    lastrow = Sheet1.Cells(Rows.Count, "A").End(xlUp).Row
    y = 2
    
    For i = 2 To lastrow
    
  
    sURL = Sheet1.Cells(i, 1)
    'Set objIE = New InternetExplorerMedium
    'objIE.Visible = True
 
    Shell (chromePath & Sheets("profileLinks").Cells(i, 1))
 
    'objIE.navigate sURL
    'Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
    
    On Error GoTo err_clear
    Application.Wait (Now + TimeValue("0:00:10"))
    
    
    For Each itemEle In ?????????.document.getElementsByClassName("css-901oao css-bfa6kz r-111h2gw r-18u37iz r-1qd0xha r-a023e6 r-16dba41 r-ad9z0x r-bcqeeo r-qvutc0")
        Text = itemEle.getElementsByTagName("span")(0).innerText
    
        Sheets("Outcome").Range("A" & y).Value = Text

        y = y + 1
    Next
    
    'objIE.Quit
    Next i
    
err_clear:
        If Err <> 0 Then
        Err.Clear
        Resume Next
        End If
    
End Sub

请随时提出任何其他解决方案.提前致谢.

Please feel free to suggest any other solution. Thank you in advance.

推荐答案

我认为您需要额外的对象变量来保存对来自 html 对象库的对象的引用.比如在参考了微软的 HTML Object 库之后，可以这样添加:

I think you would need additional object variables that can hold references to objects from the html object library. For example, after referencing to the Microsoft HTML Object library, you can add on sth like this:

Dim HTMLDocument As MSHTML.HTMLDocument
Dim itemEle As MSHTML.IHTMLElement
Dim itemCollection AS MSHTML.IHTMLElementCollection

Set HTMLDocument = objIE.Document
Set itemCollection = HTMLDocument.getElementsByClassName("...")

For Each itemEle in itemCollection
'extra code here
Next itemEle

我希望这能回答您的问题！

I hope this answers your question!

这篇关于如何使用 Chrome 浏览器使用 VBA 抓取 Twitter?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！