使用webclient c#检查URL是否是下载链接 [英] check to see if URL is a download link using webclient c#

问题描述

我正在从历史数据库中读取数据，对于每个读取的URL，我都将其下载并将数据存储到字符串中.我希望能够确定该链接是否为下载链接，例如，.exe或.zip.我假设我需要阅读标题才能确定这一点，但是我不知道如何使用WebClient来做到这一点.有什么建议吗?

I am reading from the history database, and for every URL read, I am downloading it and storing the data into a string. I want to be able to determine if the link is a download link, i.e. .exe or .zip for e.g. I am assuming I need to read the headers to determine this, but I don't know how to do it with WebClient. Any suggestions?

while (sqlite_datareader.Read())
{
    noIndex = false;

    string url = (string)sqlite_datareader["url"];

    try
    {
        if (url.Contains("http") && (!url.Contains(".pdf")) && (!url.Contains(".jpg")) && (!url.Contains("https")) && !isInBlackList(url))
        {

            WebClient client = new WebClient(); 
            client.Headers.Add("user-agent", "Only a test!");


            String htmlCode = client.DownloadString(url);
        }
    }
}

推荐答案

我将发出HEAD请求，而不是在链接后面加载完整的内容.

Instead of loading the complete content behind the link, I would issue a HEAD request.

HEAD方法与GET相同，不同之处在于服务器必须在响应中不返回消息正文.响应HEAD请求的HTTP标头中包含的元信息应该与响应GET请求发送的信息相同.此方法可用于获取有关请求所隐含的实体的元信息，而无需转移实体主体本身.此方法通常用于测试超文本链接的有效性，可访问性和最新修改.

The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response. The metainformation contained in the HTTP headers in response to a HEAD request SHOULD be identical to the information sent in response to a GET request. This method can be used for obtaining metainformation about the entity implied by the request without transferring the entity-body itself. This method is often used for testing hypertext links for validity, accessibility, and recent modification.

http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html

有关C#示例的信息，请参见这些问题

See these questions for C# examples

• 如何检查System.Net.WebClient.DownloadData是否正在下载二进制文件?

这篇关于使用webclient c#检查URL是否是下载链接的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！