从 URL 获取 HTML - StreamReader 使用其他字符编码?

Question

我想从这个网址获取 HTML：https://store.steampowered.com/app/513710/SCUM/

这应该很容易，但由于SSL/TLS错误，我无法做到这一点。

所以我使用了这个问题中的代码：使用 c# Webclient 通过 https 请求 html

最后我可以填充我的 StreamReader，但是当我尝试将 ReadToEnd(( 与字符串一起使用时，我得到一个损坏的字符串，如下所示：">

这一定是关于字符编码的，但是如果您打开：https://store.steampowered.com/app/513710/SCUM/

然后打开你的浏览器控制台，你可以在开头看到：

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

在提供的代码中：

webClient.Headers["Accept-Charset"] = "ISO-8859-1,utf-8;q=0.7,*;q=0.7";

你有utf-8，所以我只是不知道为什么我会遇到这个问题。我试图替换：

StreamReader(webClient.OpenRead(steamURL));

跟：

StreamReader(webClient.OpenRead(steamURL), Encoding.UTF8, true);

但它仍然没有得到正确的文本。我试图添加所有我能添加的信息，如果您需要任何其他信息，我会编辑问题。

感谢您抽出宝贵时间，祝您有美好的一天。

问候

大卫

PS：这是我现在的代码：

private StreamReader getStreamReader(string steamURL, WebClient webClient)
{
return new StreamReader(webClient.OpenRead(steamURL), Encoding.UTF8, true);
}
private void getSteamCosts()
{
// When I try to access an Steam HTML, SSL error appears
// We need an specific security protocol
// I check all, just in case
ServicePointManager.ServerCertificateValidationCallback =
new RemoteCertificateValidationCallback(
delegate
{
return true;
});
using (WebClient webClient = new WebClient())
{
webClient.Headers["User-Agent"] = "Mozilla/5.0 (Windows;"
+ " U; Windows NT 6.0; en-US; rv:1.9.2.6) Gecko/20100625"
+ " Firefox/3.6.6 (.NET CLR 3.5.30729)";
webClient.Headers["Accept"] = "text/html,application/xhtml+"
+ "xml,application/xml;q=0.9,*/*;q=0.8";
webClient.Headers["Accept-Language"] = "en-us,en;q=0.5";
webClient.Headers["Accept-Encoding"] = "gzip,deflate";
webClient.Headers["Accept-Charset"] = "ISO-8859-1,utf-8;q=0.7,*;q=0.7";
StreamReader sr = null;
string steamURL = "https://store.steampowered.com/app/513710/SCUM/";
try
{
// This one should work
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
sr = getStreamReader(steamURL, webClient);
lbFinalSteam.Text = "TLS12Final";
}
catch (Exception) // Bad coding practice, just wanted it to work
{
// If that's not the case, I try the rest
try
{
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls;
sr = getStreamReader(steamURL, webClient);
lbFinalSteam.Text = "TLSFinal";
}
catch (Exception)
{
try
{
ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3;
sr = getStreamReader(steamURL, webClient);
lbFinalSteam.Text = "SSL3Final";
}
catch (Exception)
{
try
{
ServicePointManager.SecurityProtocol =
SecurityProtocolType.Tls11;
sr = getStreamReader(steamURL, webClient);
lbFinalSteam.Text = "TLS11Final";
}
catch (Exception)
{
lbFinalSteam.Text = "NoFinal";
}
}
}
}
if (sr != null)
{
string allLines = sr.ReadToEnd();
}
}
}

编辑：也许问题是如何将流阅读器转换为字符串？我的意思是这行：

string allLines = sr.ReadToEnd();

我应该使用其他东西吗？

Answer 1

正如 https://stackoverflow.com/users/246342/alex-k 已经写过的，问题不在于编码，而是我得到了一个压缩的Gzimp。我刚刚删除了这个：

webClient.Headers["Accept-Encoding"] = "gzip,deflate";

它有效！谢谢亚历克斯K！:D