Python没有读取/写入整个csv文件

Question

我正在做一个项目，我必须解析一个包含500000行的巨大csv文件。下面是代码的一小部分作为示例。它很好地分解了列，但当我需要它遍历所有500000行时，它只读取9132行。csv是用cp1252编码的，我觉得这可能是问题的一部分，但我不确定。这里还有我得到的错误：

UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 4123: character maps to <undefined>

代码：

import csv
outfile = open("newFile.csv", 'w')
with open("ProductFile.csv", "r") as f:
reader = csv.reader(f, delimiter=',')
for row in reader:
Item_ID = row[0]
Sku = row[1]
SKU_ID = row[2]
altpartnum = row[3]
Application = row[4]
Brandcode = row[5]
line = "{},{},{},{},{},{},n".format(
Item_ID, AD_SKU_ID, MemberSku, Application, Brandcode, Application, Brandcode)
outfile.write(line)
outfile.close()

Answer 1

CP1252不支持解码字节0x81，因此编码不是CP1252。它可能是ISO-88591(又称latin1)，但它会将所有字节编码为，所以你可能会得到mojibake:

建议的代码(但如果不是latin1，则使用正确的编码)：

import csv
with (open('ProductFile.csv', 'r', encoding='latin1', newline='') as fin,
open('newFile.csv', 'w', encoding='latin1', newline='') as fout):
reader = csv.reader(fin)
writer = csv.writer(fout)
for row in reader:
writer.writerow(row[:6]) # first 6 columns or whatever you want to write
# The OP code had undefined variables