Python wget 一次下载多个文件



寻找一次下载多个文件的干净Python Wget解决方案。

网址将始终相同:

https://example.com/

到目前为止,我可以这样做:

import wget
print('Beginning file download with wget module')
url = 'https://example.com/new_folder/1.jpg'
wget.download(url)

但是我还需要下载 -2.jpg、-3.jpg、-4.jpg、-5.jpg并将NWZV1WB重命名为 NEWCODE-1.jpg、NEWCODE-2.jpg...


我还需要下载文件夹中的所有内容(22(.jpg文件,并将文件夹局部重命名为NEWCODE之类的名称,但保留文件的原始名称

这里的网址也总是相同的:

import wget
print('Beginning file download with wget module')
url = 'https://example.com/big/1.jpg' #there's 18 jpg inside
wget.download(url)

什么是最好的,wget(找不到很多文章(或请求?任何帮助,不胜感激。

例如:

import wget
import os
import multiprocessing
def run_process(url, output_path):
wget.download(url, out=output_path)
# TODO: you can write your rename logic at here using os.rename

if __name__ == '__main__':
cpus = multiprocessing.cpu_count()
max_pool_size = 4
pool = multiprocessing.Pool(cpus if cpus < max_pool_size else max_pool_size)
base_dir = os.path.dirname(os.path.abspath(__file__))
target = "NEWCODE"
prefix_list = ["NWZV1WB", "AWU3JAD", "NW96MRD"]
download_list = []
name_list = list(range(1, 23))
name_list.extend(["zoom_side", "zoom_sole", "zoom_side-thumb"])
for prefix in prefix_list:
path = os.path.join(base_dir, prefix)
if not os.path.exists(path):
os.mkdir(path)
if not os.path.isdir(path):
exit()
for name in name_list:
download_list.append(['https://img2.tennis-warehouse.com/360/{p}/{n}.jpg'.format(n=name, p=prefix), path])
for url, path in download_list: # change here to download other files
print('Beginning file download with wget module {n}'.format(n=url))
pool.apply_async(run_process, args=(url, path, ))
# add your code here to download other files
pool.close()
pool.join()
print("finish")

最新更新