Why is Laravel or Beanstalkd skipping jobs?

Question

I'm managing audio conversions with Laravel Queues and beanstalkd, monitored by supervisord.

When a user upload an audio file, it goes to AudioController.php that triggers a Queue::push('AudioProcess'), that itself triggers an exec('sh some_script.sh some_audio.mp3') to process the audio and set the application Audio model status to 1 when it's done.

I did a bunch of uploads to test, here are the records

1 means the AudioProcess.php worker has been executed and 0 means the AudioProcess.php worker hasn't been executed.

I guess it might come from either Laravel Queues management or beanstalkd, but I can't find anything relevant in the logs (laravel.log, my supervisord queue.log, php_errors.log).

I'm running a staging and a production environment on the same server, so there are two Laravel applications and therefore two php artisan queue:listen commands running at the same time (each with --env specified), if it has something to do with my issue. It worked well few weeks ago, then I dropped the project for a while and recaught it lately. I did some apt-get update && apt-get upgrade too.

Why is Laravel or beanstalkd not processing all jobs?

Answer 1

The issue was likely happening because the two php artisan queue:listen (for each environment) were sending jobs to the same beanstalkd queue (default). It resulted in some kind of conflict, but I can't tell more...

I fixed it by making each environment's php artisan queue:listen send jobs to a [environment] queue. Therefore I have now two queues, staging and production.

How to do it

• Create a app/config/[environment]/queue.php that overwrites the [connections => [beanstalkd => [queue]]] parameter from default to the name of that environment.

• Specify the name of the queue to Artisan: php artisan queue:listen --queue=[environment]