如何在FreeBSD中使用C代码来限制并行子进程的数量



我想同时限制N个进程(作为参数引入(,但要处理的文件总是多于N个。每个文件一个进程。一次只有N个进程是进程。我知道这个程序必须做什么,但不知道如何实现。

如果我解释得不太好,我很抱歉,我会回答所有需要的细节。在FreeBSD 中使用C代码

for (int i = 3; i < argc; i++) {
fflush(NULL);
if((pid_son = fork()) < 0){
printf("Error");
exit(-1);
}
else if(pid_son == 0){
}
}

目前尚不清楚您是否希望由o/s强制执行限制,也不清楚如果代码达到该限制(如果fork()调用失败(,该怎么办。但是,您可以安排运行命令,直到达到程序员或用户(而不是操作系统(施加的限制,然后等待其中一个命令完成后再启动另一个命令。

以下是一些应该在大多数POSIX系统上运行的C代码。它在GitHub上的SOQ(堆栈溢出问题(存储库中以文件numproc19.c的形式提供,位于src/so-1974-7644子目录中。

numproc19.c--删除了注释

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <time.h>
#include <errno.h>
#include <string.h>
enum { MAX_KIDS = 4 };
enum { DEF_TASKS = 20 };
static _Noreturn void be_childish(int tasknum)
{
srand(getpid());
struct timespec nap = { .tv_sec = rand() % 5, .tv_nsec = (rand() % 1000) * 1000000 };
int millisecs = nap.tv_nsec / 1000000;
printf("PID %5d (PPID %5d): Task %2d - dozing %d.%.3dn",
getpid(), getppid(), tasknum, (int)nap.tv_sec, millisecs);
nanosleep(&nap, 0);
printf("PID %5d (PPID %5d): Task %2d - donen", getpid(), getppid(), tasknum);
exit(tasknum);
}
static size_t dead_kid(pid_t corpse, size_t nkids, pid_t *kids)
{
for (size_t i = 0; i < nkids; i++)
{
if (kids[i] == corpse)
{
kids[i] = kids[--nkids];
return nkids;
}
}
printf("PID %5d exited but was not a known childn", corpse);
return nkids;
}
static int cmp_pid(const void *vp1, const void *vp2)
{
pid_t v1 = *(pid_t *)vp1;
pid_t v2 = *(pid_t *)vp2;
return (v1 > v2) - (v1 < v2);
}
static void print_kids(size_t nkids, pid_t *kids)
{
qsort(kids, nkids, sizeof(kids[0]), cmp_pid);
printf("Kids (%zu):", nkids);
for (size_t i = 0; i < nkids; i++)
printf(" %5d", kids[i]);
putchar('n');
}
int main(void)
{
pid_t kids[MAX_KIDS];
size_t nkids = 0;
setvbuf(stdout, NULL, _IOLBF, 0);
for (size_t task = 0; task < DEF_TASKS; task++)
{
pid_t pid = fork();
if (pid < 0)
{
fprintf(stderr, "failed to fork(): (%d) %sn", errno, strerror(errno));
exit(EXIT_FAILURE);
}
if (pid == 0)
be_childish(task);
kids[nkids++] = pid;
printf("Kid: %5d; Number of kids: %2zun", pid, nkids);
print_kids(nkids, kids);
if (nkids >= MAX_KIDS)
{
int status;
int corpse = waitpid(-1, &status, 0);
if (corpse < 0)
break;
printf("Child %5d exited with status 0x%.4Xn", corpse, status);
nkids = dead_kid(corpse, nkids, kids);
}
}
int corpse;
int status;
while (nkids > 0 && (corpse = waitpid(-1, &status, 0)) > 0)
{
printf("Child %5d exited with status 0x%.4Xn", corpse, status);
nkids = dead_kid(corpse, nkids, kids);
print_kids(nkids, kids);
}
return 0;
}

be_childish()函数执行所需的任何实际工作。在这里,它报告自己的存在,并在0-5秒内随机睡眠。在生产代码中,它要么运行程序中的函数,要么运行执行所需操作的命令。

main()函数跟踪它启动了哪些进程,一旦达到限制,它就会等待进程完成。它认识到自己可能有不知道的孩子,并干净地处理他们(报告他们的终止,并因他们的存在而继续(。

样品运行

Kid: 23402; Number of kids:  1
Kids (1): 23402
Kid: 23403; Number of kids:  2
Kids (2): 23402 23403
PID 23402 (PPID 23401): Task  0 - dozing 4.632
Kid: 23404; Number of kids:  3
Kids (3): 23402 23403 23404
PID 23403 (PPID 23401): Task  1 - dozing 1.881
Kid: 23405; Number of kids:  4
Kids (4): 23402 23403 23404 23405
PID 23404 (PPID 23401): Task  2 - dozing 3.130
PID 23405 (PPID 23401): Task  3 - dozing 0.379
PID 23405 (PPID 23401): Task  3 - done
Child 23405 exited with status 0x0300
Kid: 23406; Number of kids:  4
Kids (4): 23402 23403 23404 23406
PID 23406 (PPID 23401): Task  4 - dozing 2.628
PID 23403 (PPID 23401): Task  1 - done
Child 23403 exited with status 0x0100
Kid: 23407; Number of kids:  4
Kids (4): 23402 23404 23406 23407
PID 23407 (PPID 23401): Task  5 - dozing 4.877
PID 23406 (PPID 23401): Task  4 - done
Child 23406 exited with status 0x0400
Kid: 23408; Number of kids:  4
Kids (4): 23402 23404 23407 23408
PID 23408 (PPID 23401): Task  6 - dozing 1.479
PID 23404 (PPID 23401): Task  2 - done
Child 23404 exited with status 0x0200
Kid: 23409; Number of kids:  4
Kids (4): 23402 23407 23408 23409
PID 23409 (PPID 23401): Task  7 - dozing 3.728
PID 23408 (PPID 23401): Task  6 - done
Child 23408 exited with status 0x0600
Kid: 23410; Number of kids:  4
Kids (4): 23402 23407 23409 23410
PID 23410 (PPID 23401): Task  8 - dozing 0.977
PID 23402 (PPID 23401): Task  0 - done
Child 23402 exited with status 0x0000
Kid: 23411; Number of kids:  4
Kids (4): 23407 23409 23410 23411
PID 23411 (PPID 23401): Task  9 - dozing 2.226
PID 23410 (PPID 23401): Task  8 - done
Child 23410 exited with status 0x0800
Kid: 23412; Number of kids:  4
Kids (4): 23407 23409 23411 23412
PID 23412 (PPID 23401): Task 10 - dozing 4.475
PID 23407 (PPID 23401): Task  5 - done
Child 23407 exited with status 0x0500
Kid: 23413; Number of kids:  4
Kids (4): 23409 23411 23412 23413
PID 23413 (PPID 23401): Task 11 - dozing 1.724
PID 23411 (PPID 23401): Task  9 - done
PID 23409 (PPID 23401): Task  7 - done
Child 23411 exited with status 0x0900
Kid: 23414; Number of kids:  4
Kids (4): 23409 23412 23413 23414
Child 23409 exited with status 0x0700
Kid: 23415; Number of kids:  4
Kids (4): 23412 23413 23414 23415
PID 23414 (PPID 23401): Task 12 - dozing 3.973
PID 23415 (PPID 23401): Task 13 - dozing 0.222
PID 23415 (PPID 23401): Task 13 - done
Child 23415 exited with status 0x0D00
Kid: 23416; Number of kids:  4
Kids (4): 23412 23413 23414 23416
PID 23416 (PPID 23401): Task 14 - dozing 2.824
PID 23413 (PPID 23401): Task 11 - done
Child 23413 exited with status 0x0B00
Kid: 23418; Number of kids:  4
Kids (4): 23412 23414 23416 23418
PID 23418 (PPID 23401): Task 15 - dozing 1.322
PID 23418 (PPID 23401): Task 15 - done
Child 23418 exited with status 0x0F00
Kid: 23419; Number of kids:  4
Kids (4): 23412 23414 23416 23419
PID 23419 (PPID 23401): Task 16 - dozing 3.571
PID 23416 (PPID 23401): Task 14 - done
Child 23416 exited with status 0x0E00
Kid: 23420; Number of kids:  4
Kids (4): 23412 23414 23419 23420
PID 23420 (PPID 23401): Task 17 - dozing 0.820
PID 23412 (PPID 23401): Task 10 - done
Child 23412 exited with status 0x0A00
Kid: 23421; Number of kids:  4
Kids (4): 23414 23419 23420 23421
PID 23421 (PPID 23401): Task 18 - dozing 2.069
PID 23420 (PPID 23401): Task 17 - done
Child 23420 exited with status 0x1100
Kid: 23422; Number of kids:  4
Kids (4): 23414 23419 23421 23422
PID 23422 (PPID 23401): Task 19 - dozing 4.318
PID 23414 (PPID 23401): Task 12 - done
Child 23414 exited with status 0x0C00
PID 23421 (PPID 23401): Task 18 - done
Child 23421 exited with status 0x1200
Kids (2): 23419 23422
PID 23419 (PPID 23401): Task 16 - done
Child 23419 exited with status 0x1000
Kids (1): 23422
PID 23422 (PPID 23401): Task 19 - done
Child 23422 exited with status 0x1300
Kids (0):

因为您询问了如何通过向程序传递参数来限制FreeBSD操作系统的进程,所以一个答案是使用cpuset实用程序,该实用程序的参数与给程序的参数相同:

$ NPROC=4
$ cpuset -l 0-${NPROC} myProgram ${NPROC}

另一种解决方案是使用rctl。另一种解决方案是简单地使用CCD_ 8。

我必须指出,您使用的argc不是一个参数,而是给程序的参数数量。如果要访问参数,必须使用argv但是如果我正确理解您想要做什么,那么您似乎使用argc作为作为参数给定的文件数。在这种情况下,我不确定这是否是实现目标的最佳方式,因为论点可能是错误的、不可读的,等等…所以过程的数量。

如果您想在FreeBSD下以编程方式限制进程的数量,有很多方法可以做到:通过FreeBSDcpuset,或者简单地使用POSIXpthread_affinity_np功能。

在这种情况下,你必须向SO提供一个问题,解释你真正想做什么。在其他情况下,网络会提供很多关于如何使用它的例子。

最新更新