Goroutines和HTTP请求中的超时

Question

我正在检查服务器的状态。服务器的睡眠超过15秒，我正在检查超时。

package main
import (
    "fmt"
    "net/http"
    "time"
)
var urls = []string{
    "http://site-centos-64:8080/examples/abc1.jsp",
    }
type HttpResponse struct {
    url      string
    response *http.Response
    err      error
}
var ch = make(chan *HttpResponse, 100) // buffered
var count int   
func asyncHttpGets(urls []string) []*HttpResponse {
    responses := []*HttpResponse{}
    count:=0
    timeout := make(chan bool, 100)
    for i:=0;i<500;i++{
        go func(){
                for _, url := range urls {
                    resp, err := http.Get(url)
                    count++;
                    go func() {
                    time.Sleep(1 * time.Second)
                    timeout <- true
                    }() 
                    ch <- &HttpResponse{url, resp, err}
                    if err != nil {
                        return
                    }
                    resp.Body.Close()
                }
        }()
    }
    for {
        select {
        case r := <-ch:
            responses = append(responses, r)
            if count == 500 {
                return responses
            }
        case <-timeout:
                fmt.Println("Timed Out")
                if count == 500 {
                return responses
            }
        }
    }
    return responses
}
func main() {
    now:=time.Now()
    results := asyncHttpGets(urls)
    for _, result := range results {
        fmt.Printf("%s status: %sn", result.url,result.response.Status)
    }
    fmt.Println( time.Since(now))
}

但是发生的事情是，它最初打印一个"定时出版"，但最后150-200的请求显示了" 200个好"状态，不应该。同样，在尝试进行1000次尝试时，它显示"恐慌：运行时错误：无效的内存地址或零指针删除"

Answer 1

在启动超时goroutine之前，您正在执行resp, err := http.Get(url)。这将导致一切都阻塞，直到响应准备就绪，然后同时在两个通道上发送。

在发送请求之前，只需将超时goroutine启动到行，就可以了。即：

  for _, url := range urls {

                go func() {
                   time.Sleep(1 * time.Second)
                   timeout <- true
                   count++;
                }() 
                resp, err := http.Get(url)
                count++; //I think this is what you meant, right?
                ch <- &HttpResponse{url, resp, err}
                if err != nil {
                    return
                }
                resp.Body.Close()
            }

btw尝试使用原子增量来计数，也许使用waitgroup和 time.After频道而不是睡眠。

Answer 2

如果您想避免将并发逻辑与业务逻辑混合，我写了此库https://github.com/shomali11/paralliers，以帮助您解决这个问题。它封装了并发逻辑，因此您不必担心它。

所以在您的示例中：

package main
import (
    "github.com/shomali11/parallelizer"
    "fmt"
)
func main() {
    urls := []string{ ... }
    results = make([]*HttpResponse, len(urls)
    options := &Options{ Timeout: time.Second }
    group := parallelizer.NewGroup(options)
    for index, url := range urls {
        group.Add(func(index int, url string, results *[]*HttpResponse) {
            return func () {
                ...
                results[index] = &HttpResponse{url, response, err}
            }
        }(index, url, &results))
    }
    err := group.Run()
    fmt.Println("Done")
    fmt.Println(fmt.Sprintf("Results: %v", results))
    fmt.Printf("Error: %v", err) // nil if it completed, err if timed out
}