如何从HTTP响应中读取附件

Question

我正在尝试从URL下载一些CSV数据。原始响应看起来像这样

HTTP/1.1 200 OK
Server: Europa-4
X-Varnish: 33948791
Vary: Accept-Encoding, X-UA-Device
X-Cache: MISS
Cache-Control: no-cache, no-cache, no-store, proxy-revalidate, must-revalidate, max-age=0
Content-Type: application/octet-stream
P3p: CP="CAO PSA OUR"
Date: Fri, 01 Sep 2017 19:53:27 GMT
X-Server: web03
Expires: Fri, 01 Sep 2017 19:53:26 GMT
X-XSS-Protection: 1; mode=block
Transfer-Encoding: chunked
Accept-Ranges: bytes
X-Content-Type-Options: nosniff
Content-Disposition: attachment; filename="GooglePLAv1US.txt"
Via: 1.1 varnish-v4
Connection: keep-alive
Last-Modified: Fri, 01 Sep 2017 19:53:27 +0000
X-Frame-Options: sameorigin
X-UA-Device: desktop
Age: 0
X-Modified-Url: /amfeed/main/get/file/GooglePLAv1US/?___store=ca_en
id      title   description     google_product_category  .....
20649074       ......
20652632       ......
.
.
.

现在，我意识到这并不是真正的多部分响应，但它具有Content-Disposition: attachment; filename="GooglePLAv1US.txt"标头，该标头说它需要由浏览器将其视为下载。

当我尝试阅读响应的主体时，它会引发错误 unexpected EOF。我如何读取这些数据，并不是任何部分？

代码

http.DefaultClient.Timeout = time.Second * 30
resp, err := http.Get(ht.Creds.AccessData.URL)
if err != nil {
    return nil, err
}
defer resp.Body.Close()
d, err := ioutil.ReadAll(resp.Body)
if err != nil {
    return nil, errors.Wrap(err, "Error reading HTTP response body")
}

这会产生错误

Error reading HTTP response body: unexpected EOF

Answer 1

尝试net/textproto，这是一个小例子：

package main
import (
    "bufio"
    "fmt"
    "io"
    "net/http"
    "net/http/httptest"
    "net/textproto"
    "os"
)
func main() {
    ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintf(w, "%sn%sn%sn%sn",
            "col1,col2,col3",
            "1,2,3",
            "a,b,c",
            "x,y,x",
        )
    }))
    defer ts.Close()
    client := &http.Client{}
    res, err := client.Get(ts.URL)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        return
    }
    defer res.Body.Close()
    reader := bufio.NewReader(res.Body)
    tp := textproto.NewReader(reader)
    for {
        if line, err := tp.ReadLine(); err != nil {
            if err == io.EOF {
                // if file is emtpy
                return
            }
            return
        } else {
            fmt.Printf("%snn", line)
        }
    }
}

https://play.golang.org/p/fee4b5mh35

这是基于您最初有关" CSV"的最初问题的另一个示例：

package main
import (
    "bufio"
    "fmt"
    "net/http"
    "os"
)
func main() {
    client := &http.Client{}
    res, err := client.Get("http://samplecsvs.s3.amazonaws.com/Sacramentorealestatetransactions.csv")
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        return
    }
    defer res.Body.Close()
    scanner := bufio.NewScanner(res.Body)
    scanner.Split(bufio.ScanBytes)
    for scanner.Scan() {
        c := scanner.Text()
        switch c {
        case "r":
            fmt.Println()
        default:
            fmt.Printf("%s", c)
        }
    }
}

在这种情况下，请注意scanner.Split(bufio.ScanBytes)，希望这可以给您更多想法。