拉拉维尔 4 工匠命令内存问题



我编写了一个工匠命令,用于将练习从数据库导出到可以在 moodle 等电子学习系统中使用的独立包,...

这是大量的练习,一段时间后记忆力耗尽。

我尝试取消设置变量,激活垃圾收集器,禁用查询日志并进行一些分析,但直到现在都没有成功

我在下面附上了我的脚本,我处理的每个练习,内存使用量加起来为 300k知道我能做什么吗?

use IlluminateConsoleCommand;
use IlluminateFilesystemFilesystem;
use SymfonyComponentConsoleInputInputOption;
use SymfonyComponentConsoleInputInputArgument;
set_time_limit(0);
class ExerciseExportCommand extends Command {
    /**
     * The console command name.
     *
     * @var string
     */
    protected $name = 'exercises:export';
    /**
     * The console command description.
     *
     * @var string
     */
    protected $description = 'Export all exercises of a method or a specific exercise.';
    /**
     * Create a new command instance.
     *
     * @return void
     */
    public function __construct()
    {
        parent::__construct();
    }
    /**
     * Execute the console command.
     *
     * @return void
     */
    public function fire()
    {
        try {
            DB::disableQueryLog();
            ini_set('memory_limit','1024M');
            $this->info('Initial: ' . number_format(memory_get_usage(), 0, '.', ',') . " bytesn");
            $base = base_path() . "/export/";
            $playerPath = base_path() . "/public/preview/dist/";
            $uploadsPath = base_path() . "/public/uploads/";
            $methodId = $this->option('method');
            $categoryId = $this->option('category');
            $exerciseId = $this->option('exercise');
            $this->info("Swing baby...");
            $this->info("Let's export some exercises, shall we ?");

            //we make an array which holds all the exercises we have to export
            $exercises = array();
            if($methodId === NULL && $categoryId === NULL && $exerciseId === NULL){
                //we are here now anyways, let's do all exercises at once...
                $this->comment("Nothing specified, let's do all exercises ");
                $exercises2 = Exercise::all();
                foreach ($exercises2 as $exercise){
                    array_push($exercises, $exercise->id);
                }
                unset($exercises2);
            }
            //get all exercises for given methodId
            if($methodId !== NULL){
                $method = Method::with('categories.exercises')->find($methodId);
                if($method == NULL) break;
                $this->comment("We are ready to roll method " . $method->code);
                foreach($method->categories as $category){
                    foreach($category->exercises as $exercise->id){
                        array_push($exercises, $exercise);
                    }
                }
                unset($method);
            }
            //get all exercises for given categoryId
            if($categoryId !== NULL){
                $category = Category::with('exercises')->find($categoryId);
                if($category == NULL) break;
                $this->comment("We are ready to roll category " . $category->name_prefix . " " . $category->name);
                foreach($category->exercises as $exercise->id){
                    array_push($exercises, $exercise);
                }
                unset($category);
            }

            if($exerciseId != null){
                $exercise = Exercise::find($exerciseId);
                if($exercise != NULL) {
                    array_push($exercises, $exercise->id);
                    $this->comment("Exercise added for export: " . $exercise->name_prefix . " " . $exercise->name);
                } else {
                }
                unset($exercise);
            }
            if(empty($exercises)){
                $this->error("No exercises could be found for given method/exercise");
                exit();
            } else {
                $this->comment("Currently counting " . count($exercises) . " exerises to export");
            }
            $fs = new Filesystem();
            //loop the exercises and publish like a charm
            foreach($exercises as $exerciseId){
                $exercise = Exercise::find($exerciseId);
                //determine destination
                $path = $base . $exercise->getPath();
                $this->comment("starting exercise " . $exercise->id);
                //check if path exists, if it does, wipe it out
                if($fs->exists($path)){
                    $fs->deleteDirectory($path, true);
                    $this->comment("wiped out " . $path);
                }
                //copy player files
                //echo "copying " . $path . "<br />";
                $fs->copyDirectory($playerPath, $path);
                $fs->cleanDirectory($path."styles/skins");
                    //copy only necesary skin files to save disk space
                    $method = $exercise->method();
                    if($fs->exists($playerPath."styles/skins/".$method->code)){
                        $fs->copyDirectory($playerPath."styles/skins/".$method->code, $path."styles/skins/".$method->code);
                    } elseif($method->code == "kameleonspelling" || $method->code == "kameleontaalbeschouwing"){
                        $fs->copyDirectory($playerPath."styles/skins/kameleon", $path."styles/skins/kameleon");
                    }
                    if($fs->exists($playerPath."styles/skins/".$method->code.".css")){
                        $fs->copy($playerPath."styles/skins/".$method->code.".css", $path."styles/skins/".$method->code.".css");
                    }
                $this->comment("copied player files to " . $path);
                //copy resources
                //echo "copying resources " . $path . "<br />";
                $fs->copyDirectory($uploadsPath . $exercise->id . "/", $path);
                $this->comment("copied resources to " . $path);
                //copy slide resources
                $slides = Slide::where('exerciseID',"=",$exercise->id)->get();
                mkdir($path."slides/");
                foreach ($slides as $slide) {
                    $image = $slide->image()->first();
                    if($image != NULL){
                        $this->info($uploadsPath."slides/".$image->resourceUri);
                        $this->info($path."slides/".$image->resourceUri);
                        $fs->copy($uploadsPath."slides/".$image->resourceUri, $path."slides/".$image->resourceUri);
                    }
                    unset($image);
                }
                $this->comment("copied slide resources to " . $path);
                //save xml file
                $content = Exercise::getXmlContent($exercise->id);
                $fs->put($path . "exercise.xml", View::make('xml', $content));
                $this->comment("saved xml to " . $path);
                $this->info("finished exercise " . $exercise->id);
                unset($method);
                unset($content);
                unset($slides);
                gc_collect_cycles();
                $this->info('Peak: ' . number_format(memory_get_peak_usage(), 0, '.', ',') . " bytesn");
                $this->info('End: ' . number_format(memory_get_usage(), 0, '.', ',') . " bytesn");
            }
            $this->info("Awesome Possum => finished all exercises ");
            $this->info('Peak: ' . number_format(memory_get_peak_usage(), 0, '.', ',') . " bytesn");
            $this->info('End: ' . number_format(memory_get_usage(), 0, '.', ',') . " bytesn");
        } catch(Exception $e){
            $this->error($e->getMessage());
            $this->comment($e->getTraceAsString());
        }
    }
    /**
     * Get the console command arguments.
     *
     * @return array
     */
    protected function getArguments()
    {
        return array(
            //array('example', InputArgument::REQUIRED, 'An example argument.'),
        );
    }
    /**
     * Get the console command options.
     *
     * @return array
     */
    protected function getOptions()
    {
        return array(
            array('method', null, InputOption::VALUE_OPTIONAL, 'The id of a method  for which all the exercises to export.', null),
            array('category', null, InputOption::VALUE_OPTIONAL, 'The id of a category for which all the exercises to export.', null),
            array('exercise', null, InputOption::VALUE_OPTIONAL, 'The id of an exercise to export.', null),
        );
    }
}

这是我的 xdebug trace 命令的转储,其中包含 20 个最消耗内存的语句:

Showing the 20 most costly calls sorted by 'memory-own'.
                                                                       Inclusive        Own
function                                                       #calls  time     memory  time     memory
-------------------------------------------------------------------------------------------------------
debug_backtrace                                                   646  0.0420 20353496  0.0420 20353496
ComposerAutoloadClassLoader->loadClass                          259  0.1911 17556224  0.1139 13953824
PDOStatement->execute                                             743  0.1184 13729408  0.1184 13729408
array_merge                                                      4051  0.1282  3894816  0.1282  3894816
IlluminateDatabaseEloquentModel->newInstance                  1534  0.4715  3806344  0.0791  3732712
PDOStatement->fetchAll                                            742  0.0323  2364264  0.0323  2364264
IlluminateDatabaseEloquentModel->newBaseQueryBuilder           738  0.6625  2177352  0.0657  1688968
explode                                                          3396  0.1026  1296960  0.1026  1296960
IlluminateDatabaseEloquentModel->newFromBuilder               1534  0.6883  5139552  0.0944  1259576
str_replace                                                     10254  0.3176  1228824  0.3176  1228824
compact                                                           920  0.0339  1181384  0.0339  1181384
PDO->prepare                                                      743  0.1403   816488  0.1403   816488
sprintf                                                          2381  0.0741   802968  0.0741   802968
implode                                                          5586  0.1722   536688  0.1722   536688
array_map                                                         864  0.3164   588512  0.0386   477088
get_class_methods                                                  15  0.0059   472296  0.0059   472296
IlluminateDatabaseEloquentModel->newQuery                      738  0.9783  3044352  0.0656   448488
include                                                           263  6.7525  5732672  0.0468   410416
call_user_func_array                                             1585  0.5734  3937936  0.0659   357056
SplFileInfo->getPathname                                         2724  0.0847   344768  0.0847   344768

事实证明,DB::disableQueryLog();毕竟修复了它。

!!

起初我认为这没有帮助,因为内存不断增加,我每次都手动取消我的脚本。现在,在使用Memtrack进行调试时,我保持命令运行,一段时间后我注意到内存使用量停滞不前。

我猜垃圾收集器在决定有必要之前不会清理内存?

您的服务器memory_limit是什么?

一种解决方案是将命令拆分为较小的命令,并相应地运行它们。也许自动化在完成新命令后运行新命令?

我还建议使用某种队列提供程序。这样,您可以将工作负载拆分为更长的时间。此外,工作负载根本不分布在您的服务器上。

Laravel内置了对Pheanstalk,Amazon SQS和IronMQ的支持。

这是文档链接:Laravel队列文档

最新更新