Python: read gzip from stdin

Question

如何逐行从stdin读取gzipped内容

---

我在当前目录中有一个包含UTF-8内容的gzip文件a.gz。

场景1:

使用gzip.open(filename)工作。我可以打印拉开拉链的线条。

with gzip.open('a.gz', 'rt') as f:
for line in f:
print(line)
# python3 my_script.py

场景2:

我想从stdin读取gzipped内容。所以我cat这个gzipped文件作为下面脚本的输入。

with gzip.open(sys.stdin, mode='rt') as f:
for line in f:
print(line)
# cat a.gz | python3 script.py

但对于方法2，我得到以下错误：

Traceback (most recent call last):
File "script.py", line 71, in <module>
for line in f:
File "....../python3.6/gzip.py", line 289, in read1
return self._buffer.read1(size)
File "....../python3.6/_compression.py", line 68, in readinto
data = self.read(len(byte_view))
File "....../python3.6/gzip.py", line 463, in read
if not self._read_gzip_header():
File "....../python3.6/gzip.py", line 406, in _read_gzip_header
magic = self._fp.read(2)
File "....../python3.6/gzip.py", line 91, in read
self.file.read(size-self._length+read)
File "....../python3.6/codecs.py", line 321, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

Answer 1

您想要打开sys.stdin.buffer，而不是sys.stdin，因为后者会将字节透明地解码为字符串。这对我有效：

with gzip.open(sys.stdin.buffer, mode='rt') as f:
for line in f:
print(line)