Python ，如何在不复制循环的情况下将 if 语句移到 for 循环之外

Question

我可以用两种方式编写我的函数。

def output_ip_hist(target, final, stats, table_name, bulk_qty, type = "sql"):
    if(type == "sql"):
        field_names = ",".join(get_field_names(final, table_name))
        count = 0
        stats[table_name] = 0
        values = []
        for comp_name, row in final.items():
            for ip_address, sub_row in row.items():
                for index, ip_hist in enumerate(sub_row):
                    hist_item = ip_hist.replace('"', "'")
                    values.append('("' + comp_name + '", "' + ip_address + '", ' + str(index) + ',"' + hist_item + '")')
                    count += 1
                    if(count == bulk_qty):
                        insert_sql_many(target, count, table_name, field_names, values, stats)
                        count = 0
                        values = []
        if(count != 0):
            insert_sql_many(target, count, table_name, field_names, values, stats)
    elif(type == "csv"):
        for comp_name, row in final.items():
            for ip_address, sub_row in row.items():
                for index, ip_hist in enumerate(sub_row):
                    insert_csv(target, { "computer_name": comp_name, "id": str(index), "ip_address": ip_address, "hist_item": ip_hist.replace('"', "'") }, stats, table_name)

这是第一种方法。这种方式的缺点是循环被写入两次，从而产生一些重复。

第二种方法是在循环内移动最外层的 most if 语句，以便循环只完成一次，但这样做的缺点是 if 语句在每个循环上执行，从而减慢了循环的速度，循环可能循环到 400 万条记录。

我想知道是否有可能两全其美，减少重复并保持循环尽可能快。

谢谢！

Answer 1

你可以定义两个函数：process_sql和process_csv。根据data_type，您可以将process_data设置为第一个函数或第二个函数。

在循环中，您可以使用process_data：

def process_sql(a,b):
    print "SQL"
    return a-b
def process_csv(a,b):
    print "CSV"
    return a+b
data_type = "CSV"
if data_type == "CSV":
    process_data = process_csv
else:
    process_data = process_sql
for a in range(3):
    for b in range(3):
        print process_data(a,b)
#   CSV 
#   0
#   CSV
#   1
#   CSV
#   2
#   CSV
#   1
#   CSV
#   2
#   CSV
#   3
#   CSV
#   2
#   CSV
#   3
#   CSV
#   4

Answer 2

您可以尝试编写生成器来从嵌套循环生成值流。这还允许您使用itertools.islice来简化 SQL 批处理代码。

def my_generator(final):
    for comp_name, row in final.items():
        for ip_address, sub_row in row.items():
            for index, ip_hist in enumerate(sub_row):
               yield comp_name, ip_address, index, ip_hist.replace('"', "'")
def output_ip_hist(target, final, stats, table_name, bulk_qty, type = "sql"):
    items = my_generator(final)
    if type == "sql":
        field_names = ",".join(get_field_names(final, table_name))
        stats[table_name] = 0
        while True:
            values = ['("%s", "%s", "%s", "%s")' % i for i in islice(items, bulk_qty)]
            if not values:
                break                
            insert_sql_many(target, len(values), table_name, field_names, values, stats)
    elif type == "csv":
        for comp_name, ip_address, index, hist_item in items:
            blob = {
                "computer_name": comp_name,
                "id": str(index),
                "ip_address": ip_address,
                "hist_item": hist_item
            }
            insert_csv(target, blob, stats, table_name)

但是，将

实际上两个不同的函数合并到包装if语句的单个函数中，这在某种程度上是一种反模式。

def output_ip_hist_sql(target, final, stats, table_name, bulk_qty):
    field_names = ",".join(get_field_names(final, table_name))
    stats[table_name] = 0
    items = my_generator(final)
    while True:
        values = ['("%s", "%s", "%s", "%s")' % i for i in islice(items, bulk_qty)]
        if not values:
            break                
        insert_sql_many(target, len(values), table_name, field_names, values, stats)
def output_ip_hist_csv(target, final, stats, table_name):
    items = my_generator(final)
    for comp_name, ip_address, index, hist_item in items:
        blob = {
            "computer_name": comp_name,
            "id": str(index),
            "ip_address": ip_address,
            "hist_item": hist_item
        }
        insert_csv(target, blob, stats, table_name)