HttpClient 请求类似浏览器 [英] HttpClient Request like browser

问题描述

当我通过 HttpClient 类调用网站 www.livescore.com 时，我总是收到错误500".可能是服务器阻止了来自 HttpClients 的请求.

When I calling site www.livescore.com by HttpClient class I always getting error "500". Probably server blocked request from HttpClients.

1)还有其他方法可以从网页中获取html吗?

1)There is any other method to get html from webpage?

2)如何设置标题以获取 html 内容?

2)How I can set the headers to get html content?

当我像在浏览器中一样设置标题时，我总是得到 stange 编码的内容.

When I set headers like in browser I always get stange encoded content.

    http_client.DefaultRequestHeaders.TryAddWithoutValidation("Accept", "text/html,application/xhtml+xml,application/xml");
    http_client.DefaultRequestHeaders.TryAddWithoutValidation("Accept-Encoding", "gzip, deflate");
    http_client.DefaultRequestHeaders.TryAddWithoutValidation("User-Agent", "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0");
    http_client.DefaultRequestHeaders.TryAddWithoutValidation("Accept-Charset", "ISO-8859-1");

3) 我该如何解决这个问题?有什么建议吗?

3) How I can slove this problem? Any suggestions?

我在 C# 和 HttpClientClass 中使用 Windows 8 Metro Style App

I using Windows 8 Metro Style App in C# and HttpClientClass

推荐答案

给你 - 注意你必须解压缩 gzip 编码的结果 - 你得到的结果 按照 mleroy:

Here you go - note you have to decompress the gzip encoded-result you get back as per mleroy:

private static readonly HttpClient _HttpClient = new HttpClient();

private static async Task<string> GetResponse(string url)
{
    using (var request = new HttpRequestMessage(HttpMethod.Get, new Uri(url)))
    {
        request.Headers.TryAddWithoutValidation("Accept", "text/html,application/xhtml+xml,application/xml");
        request.Headers.TryAddWithoutValidation("Accept-Encoding", "gzip, deflate");
        request.Headers.TryAddWithoutValidation("User-Agent", "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0");
        request.Headers.TryAddWithoutValidation("Accept-Charset", "ISO-8859-1");

        using (var response = await _HttpClient.SendAsync(request).ConfigureAwait(false))
        {
            response.EnsureSuccessStatusCode();
            using (var responseStream = await response.Content.ReadAsStreamAsync().ConfigureAwait(false))
            using (var decompressedStream = new GZipStream(responseStream, CompressionMode.Decompress))
            using (var streamReader = new StreamReader(decompressedStream))
            {
                return await streamReader.ReadToEndAsync().ConfigureAwait(false);
            }
        }
    }
}

这样调用:

var response = await GetResponse("http://www.livescore.com/").ConfigureAwait(false); // or var response = GetResponse("http://www.livescore.com/").Result;

这篇关于HttpClient 请求类似浏览器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！