多个Web请求，在等待最后一个请求时启动另一个请求.C＃

Question

我正在尝试从网站上执行多个Web请求，这些网站需要很长时间才能加载完整的响应（实际上很长的日志文件）。我想实现一种方法，该方法将在等待其他程序的响应并加快过程中创建Web请求。

尝试实现异步等待方法，但我认为不是在创建同时调用，而是同步运行。我对如何实施异步等待，我很困惑，我查看了其他类似的问题，但仍然不太清楚，我该如何修复此代码以运行异步？
这是我的代码：

        public static async Task<String> MakeRequestAsync(String url)
    {
        String responseText = null;
            try
            {
                HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
                request.Proxy = null;
                WebResponse response = request.GetResponse();
                Stream responseStream = response.GetResponseStream();
                string sr = new StreamReader(responseStream).ReadToEnd();
                response.Close();
                responseStream.Close();
                responseText = sr;
            }
            catch (Exception e)
            {
                Console.WriteLine("Error: " + e.Message);
            }
        return responseText;
    }

..

private void button_test_Click(object sender, EventArgs e)
    {
         for (int n = 0; n < 5; n++)  //multiple requests
            {
                String a = AsyncTest().Result;
            }
    }
    private async Task<String> AsyncTest()
    {
        String b = await MakeRequestAsync(url);
        return b;
    }

还有关我如何加快此过程的任何建议。

Answer 1

您的IO呼叫都不是async。WebRequest也没有async/await模式，您可以使用AsyncCallback使用BeginGetResponse获得async行为，但是还有更好的方法。使用HttpClient而不是，这是进行HTTP调用的推荐方法。

即

public static async Task<String> MakeRequestAsync(string url)
{
    using(var client = new HttpClient()) // don't do this, inject it as a singleton
    {
       using (var response = await client.GetAsync(url)
          return await response.Content.ReadAsStringAsync();
    }  
}

然后将您的Click处理程序更改为async void：

private async void button_test_Click(object sender, EventArgs e)
{
     var tasks = new List<Task<string>>();
     for (int n = 0; n < 5; n++)  //multiple requests
     {
        tasks.Add(MakeRequestAsync(url));
     }
     await Task.WhenAll(tasks); // wait for all of them to complete;
     foreach(var task in tasks)
     {
         var str = await task; // or task.Result, won't block, already completed;
     }
}

Answer 2

您应该使用Microsoft的反应性框架（又称RX）-Nuget System.Reactive并添加using System.Reactive.Linq;-然后您可以这样做：

var query =
    from n in Observable.Range(0, 5)
    from r in Observable.Using(
        () => new WebClient(),
        wc => Observable.Start(() => wc.DownloadString($"{url}{n}")))
    select new { n, r };
IDisposable subscription =
    query
        .ToArray()
        .Select(xs => xs.OrderBy(x => x.n).Select(x => x.r).ToArray())
        .Subscribe(rs =>
        {
            /* Do something with the results */
        });

都是多线程和非障碍物。

如果您不在乎返回结果的顺序，请执行此操作：

var query =
    from n in Observable.Range(0, 5)
    from r in Observable.Using(
        () => new WebClient(),
        wc => Observable.Start(() => wc.DownloadString($"{url}{n}")))
    select r;
IDisposable subscription =
    query
        .Subscribe(r =>
        {
            /* Do something with each result */
        });

呼叫subscription.Dispose()如果您需要取消查询。

---

您可以写下MakeRequestAsync方法：

public static async Task<String> MakeRequestAsync(String url)
{
    return await
        Observable.Using(
            () => new WebClient(),
            wc => Observable.Start(() => wc.DownloadString(url)));
}