Difference between revisions of "JReviews Queue Add-on"
(Created page with " __TOC__ == Overview == The Queue Add-on changes the way certain events execute on your site so they are executed asynchronously and run independently of the main program f...") |
(→Failed Jobs) |
||
| (14 intermediate revisions by the same user not shown) | |||
| Line 19: | Line 19: | ||
== Requirements == | == Requirements == | ||
| + | * PHP PDO extension | ||
* JReviews v3 or higher | * JReviews v3 or higher | ||
* A process monitor that can execute queued events. We recommend using Supervisor, a process monitor for Linux that will automatically restart the queue process if it fails. | * A process monitor that can execute queued events. We recommend using Supervisor, a process monitor for Linux that will automatically restart the queue process if it fails. | ||
| + | |||
| + | == Installation == | ||
| + | |||
| + | This is a free '''and optional''' Add-on available to everyone. To install the Queue Add-on go to the JReviews dashboard remote install & update page. You can find the Add-on there and click install. | ||
== Add-on Configuration == | == Add-on Configuration == | ||
| Line 30: | Line 35: | ||
== Supervisor Configuration == | == Supervisor Configuration == | ||
| − | + | '''Before you continue''', it is important that you go to the Add-on configuration screen inside JReviews and save it at least once. This will generate a configuration file that is used by Add-on. | |
| − | + | === Installing Supervisor === | |
| − | To | + | To install Supervisor on Ubuntu, you need to log in through SSH and run the following command: |
<source lang="bash"> | <source lang="bash"> | ||
| Line 42: | Line 47: | ||
=== Configuring Supervisor === | === Configuring Supervisor === | ||
| − | Supervisor configuration files are typically stored in the /etc/supervisor/conf.d directory. Within this directory, you may create any number of configuration files that instruct supervisor how your processes should be monitored. For example, let's create a jreviews-worker.conf file that starts and monitors the JReviews queue. | + | Supervisor configuration files are typically stored in the /etc/supervisor/conf.d directory. Within this directory, you may create any number of configuration files that instruct supervisor how your processes should be monitored. For example, let's create a jreviews-worker.conf file that starts and monitors the JReviews queue. If you have several JReviews sites on the same server and want to use the Queue Add-on on all of them, then you should create a separate configuration file for each site and adjust the paths. |
<source lang="bash"> | <source lang="bash"> | ||
| Line 61: | Line 66: | ||
<source lang="bash"> | <source lang="bash"> | ||
| − | command=php7.0-sp | + | command=php7.0-sp /path/to/queue-addon/worker-many.php |
</source> | </source> | ||
| − | The path to the queue Add-on | + | The path to the queue Add-on must be from the server root and looks like this: |
'''Joomla''' | '''Joomla''' | ||
| − | <pre>/path/to/site/components/com_jreviews_addons/worker-many.php</pre> | + | <pre>/path/to/site/components/com_jreviews_addons/queue/worker-many.php</pre> |
'''WordPress''' | '''WordPress''' | ||
| − | <pre>/path/to/site/wp-content/plugins/jreviews_addons/worker-many.php</pre> | + | <pre>/path/to/site/wp-content/plugins/jreviews_addons/queue/worker-many.php</pre> |
| + | '''Before you continue, you can check if the "command" line is correct by executing it directly in the shell.'''. Instead of worker-many.php, use worker-one.php for testing. | ||
| + | |||
| + | <source lang="bash"> | ||
| + | php7.0-sp /path/to/queue-addon/worker-one.php | ||
| + | </source> | ||
| + | |||
| + | You will need to update both the call to "php" and the path. Once you get that running without errors, then you can replace worker-one.php with worker-many.php and use that as the value for "command" in the config file. | ||
=== Starting Supervisor === | === Starting Supervisor === | ||
| Line 88: | Line 100: | ||
For more information on Supervisor, consult the [http://supervisord.org/index.html Supervisor documentation] | For more information on Supervisor, consult the [http://supervisord.org/index.html Supervisor documentation] | ||
| + | |||
| + | === Checking If Supervior Is Running === | ||
| + | |||
| + | Once you've completed all the steps above you can easily check if Supervisor is running by using the command "supervisorctl". The result will look something like this. | ||
| + | |||
| + | <source lang="bash"> | ||
| + | root@server-name:~# supervisorctl | ||
| + | jreviews-worker:jreviews-worker_00 RUNNING pid 20543, uptime 66 days, 19:23:30 | ||
| + | jreviews-worker:jreviews-worker_01 RUNNING pid 20542, uptime 66 days, 19:23:30 | ||
| + | jreviews-worker:jreviews-worker_02 RUNNING pid 20545, uptime 66 days, 19:23:30 | ||
| + | jreviews-worker:jreviews-worker_03 RUNNING pid 20544, uptime 66 days, 19:23:30 | ||
| + | jreviews-worker:jreviews-worker_04 RUNNING pid 20539, uptime 66 days, 19:23:30 | ||
| + | jreviews-worker:jreviews-worker_05 RUNNING pid 20538, uptime 66 days, 19:23:30 | ||
| + | jreviews-worker:jreviews-worker_06 RUNNING pid 20541, uptime 66 days, 19:23:30 | ||
| + | jreviews-worker:jreviews-worker_07 RUNNING pid 20540, uptime 66 days, 19:23:30 | ||
| + | </source> | ||
| + | |||
| + | It shows the name of the program '''jreviews-worker''' defined in the configuration file and it also shows the number of processes running (8), also defined in the configuration file through '''numprocs=8'''. | ||
| + | |||
| + | == Restarting Supervisor After A JReviews Upgrade == | ||
| + | |||
| + | In order for Supervisor to include code changes it is necessary to restart it whenever you upgrade JReviews. You can run the following command to do that: | ||
| + | |||
| + | <source lang="bash"> | ||
| + | sudo supervisorctl restart all | ||
| + | </source> | ||
== Failed Jobs == | == Failed Jobs == | ||
When an event is added to the queue, we call this a job. If a job fails, it will be attempted later. If the maximum number of attempts is exceeded, the job will be moved to the failed queue table. The Add-on provides a convinient way to look at the current Queue Jobs and the Failed Jobs through the admin dashboard. | When an event is added to the queue, we call this a job. If a job fails, it will be attempted later. If the maximum number of attempts is exceeded, the job will be moved to the failed queue table. The Add-on provides a convinient way to look at the current Queue Jobs and the Failed Jobs through the admin dashboard. | ||
Latest revision as of 13:25, 2 May 2018
Contents
Overview
The Queue Add-on changes the way certain events execute on your site so they are executed asynchronously and run independently of the main program flow. They are executed in a non-blocking scheme, allowing the main action to continue processing without unnecessary delays. This not only improves the user experience due to faster processing, but it also means that an error in one of the events won't affect the main program flow. Lets put all of this into a practical example:
When a listing, review or media is submitted there are many actions that take place, other than saving the review in the database. These include:
- Sending one or more e-mail notifications
- Posting the review to Twitter
- Posting the review to Facebook
- Posting the review to the JomSocial or EasySocial streams (Joomla)
- Triggering native Joomla or WordPress events
These events are all executed in series, one after the other, and it can amount to a substantial delay. Even if one of these events results in error, then the review for submission could hang and the rest of the events will not be executed.
Requirements
- PHP PDO extension
- JReviews v3 or higher
- A process monitor that can execute queued events. We recommend using Supervisor, a process monitor for Linux that will automatically restart the queue process if it fails.
Installation
This is a free and optional Add-on available to everyone. To install the Queue Add-on go to the JReviews dashboard remote install & update page. You can find the Add-on there and click install.
Add-on Configuration
In the Add-on configuration you can choose the types of events (emails, Twitter, Facebook, etc.) that you want to executed through the queue vs having them run directly with the main program flow.
One of the additional advantages of running certain events through the queue is that it is possible to introduce an additional delay before the event is executed. For example, when a listing is submitted and photos are uploaded for this listing right after, if the listing will be posted to Twitter or Facebook, by introducing a delay to these events, the listing can already have a published photo by the time it gets posted to Twitter and Facebook.
Supervisor Configuration
Before you continue, it is important that you go to the Add-on configuration screen inside JReviews and save it at least once. This will generate a configuration file that is used by Add-on.
Installing Supervisor
To install Supervisor on Ubuntu, you need to log in through SSH and run the following command:
sudo apt-get install supervisor
Configuring Supervisor
Supervisor configuration files are typically stored in the /etc/supervisor/conf.d directory. Within this directory, you may create any number of configuration files that instruct supervisor how your processes should be monitored. For example, let's create a jreviews-worker.conf file that starts and monitors the JReviews queue. If you have several JReviews sites on the same server and want to use the Queue Add-on on all of them, then you should create a separate configuration file for each site and adjust the paths.
[program:jreviews-worker] process_name=%(program_name)s_%(process_num)02d command=php /path/to/queue-addon/worker-many.php autostart=true autorestart=true numprocs=8 redirect_stderr=true stderr_logfile=/path/to/folder/below/public/jreviews-worker.err.log stdout_logfile=/path/to/folder/below/public/jreviews-worker.out.log
In this example, the numprocs directive will instruct Supervisor to run 8 processes and monitor all of them, automatically restarting them if they fail.
On some hosts, like DigitalOcean, you may need to specify the PHP version to use in the command.
command=php7.0-sp /path/to/queue-addon/worker-many.php
The path to the queue Add-on must be from the server root and looks like this:
Joomla
/path/to/site/components/com_jreviews_addons/queue/worker-many.php
WordPress
/path/to/site/wp-content/plugins/jreviews_addons/queue/worker-many.php
Before you continue, you can check if the "command" line is correct by executing it directly in the shell.. Instead of worker-many.php, use worker-one.php for testing.
php7.0-sp /path/to/queue-addon/worker-one.php
You will need to update both the call to "php" and the path. Once you get that running without errors, then you can replace worker-one.php with worker-many.php and use that as the value for "command" in the config file.
Starting Supervisor
Once the configuration file has been created, you may update the Supervisor configuration and start the processes using the following commands:
sudo supervisorctl reread sudo supervisorctl update sudo supervisorctl start
For more information on Supervisor, consult the Supervisor documentation
Checking If Supervior Is Running
Once you've completed all the steps above you can easily check if Supervisor is running by using the command "supervisorctl". The result will look something like this.
root@server-name:~# supervisorctl jreviews-worker:jreviews-worker_00 RUNNING pid 20543, uptime 66 days, 19:23:30 jreviews-worker:jreviews-worker_01 RUNNING pid 20542, uptime 66 days, 19:23:30 jreviews-worker:jreviews-worker_02 RUNNING pid 20545, uptime 66 days, 19:23:30 jreviews-worker:jreviews-worker_03 RUNNING pid 20544, uptime 66 days, 19:23:30 jreviews-worker:jreviews-worker_04 RUNNING pid 20539, uptime 66 days, 19:23:30 jreviews-worker:jreviews-worker_05 RUNNING pid 20538, uptime 66 days, 19:23:30 jreviews-worker:jreviews-worker_06 RUNNING pid 20541, uptime 66 days, 19:23:30 jreviews-worker:jreviews-worker_07 RUNNING pid 20540, uptime 66 days, 19:23:30
It shows the name of the program jreviews-worker defined in the configuration file and it also shows the number of processes running (8), also defined in the configuration file through numprocs=8.
Restarting Supervisor After A JReviews Upgrade
In order for Supervisor to include code changes it is necessary to restart it whenever you upgrade JReviews. You can run the following command to do that:
sudo supervisorctl restart allFailed Jobs
When an event is added to the queue, we call this a job. If a job fails, it will be attempted later. If the maximum number of attempts is exceeded, the job will be moved to the failed queue table. The Add-on provides a convinient way to look at the current Queue Jobs and the Failed Jobs through the admin dashboard.