TinyWP.in Infrastructure

Back in 2011, I already wrote a colophon post. Nothing much changed in it in terms of underlying technologies used such as Nginx web server. However, a few things aren’t mentioned in it, but will have a mention here. Basically, I am running most of the services using Google services for this domain (tinywp.in). Even though, I’ve been trying to de-google myself for years, I still use Google services with this domain (and with tinywp.com), mainly to collaborate with those who uses Google services too. Here’s the list of (Google) services that I use for tinywp.in…

Server for site hosting

The server is hosted in Google Cloud (Compute Engine). I’ve been running this under the free tier for years. I still use some paid services for the sack of remembering to use Cloud Engine and other Google services. It’s pretty limited. However, for the kind of traffic this site gets, the free limits are more than enough. :)

Email Hosting

As you may have guessed, I use Google Workspace. It’s been used since its inception too. However, most of my communication has moved to Proton Mail, mainly to improve privacy. Please note that most features that are free with Google are paid in Proton Mail (or are severely limited). To send mass emails, I use Amazon SES, though.

Domain Registry

Domain registrar for tinywp.in is Google Domains. It has changed hands multiple times. Works great most of the time. Offers automated provisioning of SSL / HTTPS for any sub-domain (or root domain). It also offers redirects. I still use redirect.pizza for redirects and for automated SSL, though. Redirect pizza offers analytics that is not offered by Google Domains. I also use redirect.pizza only for the root domain (tinywp.in) to redirect it to www.tinywp.in .

SSL Certificate Authority

Google Trust Services provides SSL for this domain since 2023. Earlier, I used LetsEncrypt and BuyPass.no for SSL certificates. Since, Google’s root certificates have wider compatibility than the rest, I switch to Google’s free SSL.

Backups

Again, I use Google Storage that offers up to 5GB free storage. This is the only service that I use beyond the free limit as my storage requirements are much higher than the free limit. I use one-way backups that helps to improve security.

Version Control

I use Google Source Repositories to keep most of the private repos. I don’t want to keep everything in a single basket (Github). So, using Googe’s only as an alternative. Google doesn’t offer any public repositories. So, it’s just for private repos.

Future course of action

As mentioned earlier, I plan to de-google myself to improve privacy. If any of the above changes in the future, I will update this post accordingly. If I use any additional services too, I will update this post.

But why do I use only free resources?!

You may wondering why I use only free resources (in Google, Amazon SES, etc). Actually, I do pay them. However, it is true that I use mostly free resources on the internet for a specific reason. But, that’s for another post. Stay tuned!

Rate limiting xmlrpc requests on WordPress using Nginx

WordPress based sites are target for most automated bots. Those bots look for various vulnerability in WordPress core, the themes, and the plugins. Then, there are some kids (and their kid bots) that target specific resources in a WP site. “xmlrpc.php” is one such resource. It uses XML-RPC protocol that does many things in WordPress. For example, it helps with remote sites to notify their mentions, to publish (and edit) articles using an app (such as WordPress app for Android / iOS). It is also used by many plugins such as Jetpack.

Naturally, xmlrpc.php file may be called multiple times in a day or in an hour (on a busy site). It may be called multiple times for every minute on a high traffic site. I have seen xmlrpc.php being accessed more frequently even on a site with no traffic too. Those traffic are likely from scan / scam bots, looking for vulnerabilities.

Since, there is no way to cache requests to xmlrpc.php, PHP and MySQL usage tend to go high quickly as every request needs a bit of php and MySQL. As a result, CPU usage spikes up, resulting in a wastage of precise CPU minutes. If we use platforms like AWS EC2, GCP, or Microsoft Azure where every CPU hour is charged, the cost of running a site can increase substantially.

In order to reduce the CPU usage, one solution is to completely block access to xmlrpc.php file. However, since this file is used for genuine purposes too, it is not recommended to disable access to this file. Alternatively, we can rate limit the requests to this file. A genuine request would not call this file multiple times per second. A decent limit is 1 request per second.

Let’s see how to implement rate limiting for xmlrpc in Nginx…

limit_req_zone $binary_remote_addr zone=wp_xmlrpc_limit:10m rate=1r/s;

server {
    server_name example.com;

    location = /xmlrpc.php {
        limit_req zone=wp_xmlrpc_limit;

        fastcgi_split_path_info ^(.+\.php)(/.+)$;

        if (!-f $document_root$fastcgi_script_name) { return 404; }

        # Mitigate https://httpoxy.org/ vulnerabilities
        fastcgi_param HTTP_PROXY "";

        include                     fastcgi_params;
        fastcgi_index               index.php;
        fastcgi_param   SCRIPT_FILENAME    $document_root$fastcgi_script_name;
        fastcgi_pass                fpm;
    }

    # other location blocks such as location / {}
}

Basically, we define a separate location block to process xmlrpc.php file and then insert two lines of code to introduce rate limit. The first line (starting with limit_req_zone) should be defined outside of server block. The other one (starting with limit_req) should be defined inside the newly introduced location block.

In the above code, we limited the requests at 1 request per second. We can fine-tune it depending on our use-case. There are other areas to fine-tune too such as implementing a separate log for xmlrpc requests. That’s for another day!

Happy Hosting!

Disable PHP warnings when running wp-cli

It is not uncommon to test sites on a development environment (locally or in a staging environment where others can see the work-in-progress). On a development environment, usually we have configured WP_DEBUG to be true. Here’s the sample of wp-config.php file in a development / test / staging environment…

<?php

<span class="hljs-function"><span class="hljs-keyword">define(<span class="hljs-string">'WP_CACHE'</span>, <span class="hljs-literal">false</span>)</span></span>;

<span class="hljs-function"><span class="hljs-keyword">define(<span class="hljs-string">'DB_NAME'</span>, <span class="hljs-string">'actual_db'</span>)</span></span>;
<span class="hljs-function"><span class="hljs-keyword">define(<span class="hljs-string">'DB_USER'</span>, <span class="hljs-string">'db_user'</span>)</span></span>;
<span class="hljs-function"><span class="hljs-keyword">define(<span class="hljs-string">'DB_PASSWORD'</span>, <span class="hljs-string">'Super_Secret_Passw0rd'</span>)</span></span>;
<span class="hljs-function"><span class="hljs-keyword">define(<span class="hljs-string">'DB_HOST'</span>, <span class="hljs-string">'localhost'</span>)</span></span>;

<span class="hljs-function"><span class="hljs-keyword">define(<span class="hljs-string">'WP_DEBUG'</span>, <span class="hljs-literal">true</span>)</span></span>;

/<span class="hljs-regexp">/ Other directives such as salts...</span>
<span class="hljs-regexp"></span>

While the above code is perfectly okay, if the site creates PHP warnings, it is a nuisance to see them repeatedly when using wp-cli multiple times. Even if you try configuring error_reporting to various values and turn off everything under the hood, you may still see PHP warning with the above code. It’s because the the warning are configured to be displayed when WP_DEBUG is set to true. In order to disable WP_DEBUG only for wp cli operations, the following modified code can be used…

<?php

define('WP_CACHE', false);

define('DB_NAME', 'actual_db');
define('DB_USER', 'db_user');
define('DB_PASSWORD', 'Super_Secret_Passw0rd');
define('DB_HOST', 'localhost');

if(!defined('$_SERVER["HTTP_HOST"]')) define('WP_DEBUG', false);
if( !defined( WP_DEBUG ) ) define('WP_DEBUG', true);

// Other directives such as salts...

Basically, we added the following code into the original code. The first line checks if it is a visit from a browser or from command line. When we evoke wp from command line, it doesn’t send HTTP_HOST. This way, we can tweak WP_DEBUG depending on the presence (or absence) of HTTP_HOST.

if(!defined('$_SERVER["HTTP_HOST"]')) define('WP_DEBUG', false);
if( !defined( WP_DEBUG ) ) define('WP_DEBUG', true);

The above code eliminates most PHP warnings when running WP-CLI. I hope this helps someone. There are multiple steps involved in getting a perfect development, local, test or staging environment for a WordPress site. If you are looking for a perfect hosting environment or a customized server to meet a better workflow for your developments, please get in touch.

Nginx compatibility for “Cookies for Comments” plugin

Whether you are aware or not, spammers are more interested in your site than anyone else. You’ll understand this more vividly, when your blog starts to become famous and brings more and more visitors week after week, month after month, year after year.

The most annoying thing about spam comments is the amount of time that you need to waste in dealing with it. There are some bright minds in WordPress that help you save your time with spam comments. One such person is Donncha, who put together a nice plugin named Cookies for Comments that blocks the spam at the server level, in such as a way, it doesn’t even reach WordPress. Because, all the work is done by the server itself. Here, we show an example code for Apache and Nginx web server. It could be migrated any web server in general.

Integrating with Apache is straightforward. The code to configure Apache is displayed at the bottom of plugin’s settings page at https://example.com//wp-admin/options-general.php?page=cfc_config. It’d look like this…

# If you're feeling adventerous, you can add the following two lines before
# the regular WordPress mod_rewrite rules in your .htaccess file.
# They will stop comments from spambots before they reach
# the database or are executed in PHP:

RewriteCond %{HTTP_COOKIE} !^.*abcdefghijklmnopqrstuvwxyz0123456789.*$
RewriteRule ^wp-comments-post.php - [F,L]

In the above code, the value of abcdefghijklmnopqrstuvwxyz0123456789 may change for each site. It is also part of the name of the cookie set by this plugin.

In Nginx, the code is little different. Here’s the actual code…

# support for cookies for comments plugin!
location = /wp-comments-post.php {
    if ($http_cookie !~* "abcdefghijklmnopqrstuvwxyz0123456789") { return 403; }
    # rest of the code to process PHP.
}

Considering Akismet can not be used on a commercial site, this solution works great. With Akismet, there is a lot going on behind the scene. With ‘Cookies for Comments’ plugin, a cookie is set for all the visitors and it is checked when a comment is posted by the same visitor. Since, this plugin sets a cookie for all visitors, you may use GDPR consent to include this cookie at the top of every comment form. At least, you could inform about cookies before they comment like how it is done on this site’s comment box…

comment form with cookie warning
Sample comment form showing a warning of cookies being used!

By adding just two lines of code, we can save a lot of trouble and frustration in the long run. If you have any other method to tackle spam, please share it in the comments!

Delete all pending comments at once

If you are presented with a new website with thousands or millions of pending comments and if you know that all pending comments are actually spam comments, you’d want to quickly get rid of all such comments before you do anything further on that site to speed-up your actual workflow. Here’s a quick tip to delete all pending comments.

wp comment delete $(wp comment list --status=hold --format=ids) --force

Word of warning: Never execute any random command found on the internet. NEVER.

Explanation:

Before executing the above command, lets break it up and understand what’s going on.

wp comment list --status=hold --format=ids

The above command creates a list of IDs (of comments) that are under moderation queue (pending).

wp comment delete $(wp comment list --status=hold --format=ids)

The above command will send all the pending comments to trash. Any comment in trash is usually get deleted automatically after 30 minutes. If you’d like to delete a comment permanently skipping trash, then we can use --force flag.

Buypass CA – SSL with 180 Days Validity

Buypass is a Certificate Authority (CA) based on Europe. It offers free SSL certificates with a validity of 180 days. Unlike LetsEncrypt, Buypass CA also offers paid SSL too. So, it is neither a competitor to LetsEncrypt, nor it is a nonprofit. It is a for-profit company that also offers free SSL certificates. There are other CAs that offer free SSL certificates too. However, Buypass CA offers ACME API that is compatible with LetsEncrypt. For example, certbot can be used to authenticate the domain and obtain free SSL certificates.

Starting Afresh

Certbot is the recommended tool / client-side software. However, the procedure for test certificates and live certificates are slightly different, if you have used LetsEncrypt previously.

Here’s the procedure to get started with Buypass CA using certbot…

sudo certbot register --server 'https://api.test4.buypass.no/acme/directory'

The above command would do the following…

  • ask for your email
  • option to agree or disagree with the terms of service
  • an option to share your e-mail address with EFF

If you would like to shorten this long process, you may use the following one-liner, replacing ‘YOUR_EMAIL’ with your actual email address…

sudo certbot register -m 'YOUR_EMAIL' --no-eff-email --agree-tos --server 'https://api.test4.buypass.no/acme/directory'

Once the email is registered, we are free to test drive the domain authentication and fetching the test SSL certificates by running the following command…

sudo certbot certonly --webroot -w /var/www/example.com/public_html/ -d example.com -d www.example.com --server 'https://api.test4.buypass.no/acme/directory'

Please know that the test certificates can not be used on live sites.

The above command issues real certificates for testing purpose, even though the test certificates cant’ be used on live domains. Since, the test certificates are real, we have to remove them before fetching live SSL certificates for live domains. We can remove the test SSL certificates using the following command and selecting correct options when prompted…

sudo certbot delete

Output of the above command would look something similar to the following…

Saving debug log to /var/log/letsencrypt/letsencrypt.log

Which certificate(s) would you like to delete?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: tinywp.dev
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel):

Please be careful on selecting the correct certificates to delete. If you hit “Enter” key without reading the above, you are likely to lose all the certificates listed in it, including the live SSL certificates, if any. If everything goes well, it is time to go live.

Obtaining Live Certficates

Once testing is successful, obtaining the live certificates is likely to go through as expected. The only difference between the test and live environment is the server URL. For live environment, Buypass CA uses “https://api.buypass.com/acme/directory”.

Limitations

While the advantage of using Buypass CA is in the extended validity, compared to LetsEncrypt, there are a few limitations…

  • The free Go SSL certificates from Buypass CA doesn’t allow wildcard. It doesn’t mean wildcard isn’t supported at all. Wildcard is a paid product from Buypass CA.
  • Total number of domains that we can attach to a single SSL certificate is limited to only two (enough for 99% of the sites on the internet). So, it can easily cover the bare / root domain example.com and the www version www.example.com.
  • There is no dry-run. As seen earlier, the testing process is bit complicated than LetsEncrypt where we can do “dry-run” of authentication. However, with Buypass CA, we authenticate the domain/s, and then fetch test SSL certificates that need to be deleted before fetching the live SSL certificates.

Switching from LetsEncrypt

Switching from LetsEncrypt isn’t hard. Delete the existing certificate and do the above steps. If you ever go wrong, you can always go back and re-issue a free SSL certificate from LetsEncrypt.

Conclusion

Overall, SSL certificate with 180-days validity is the main reason to go with Buypass Go SSL. Also, if you are a person like me who doesn’t always depend on a single entity (even it means nonprofit), then this is a real alternative to LetsEncrypt. Compatibility with ACME API makes it easier to switch from existing LetsEncrypt installations where only the bare domain and www version need to be covered under HTTPS.

Version Control

SVN

WordPress uses SVN internally such as for plugins repository. So, a plugin author must use SVN tools to upload a plugin and update the existing plugin/s in wp.org plugins repository. If you are new to SVN, WordPress docs team has a nice guide to get started with SVN and plugins repository. The best practices section is a gem. It is possible to update WP core using SVN. However, things have been changing. Now, you can host your plugin in Github and push it to wordpress.org plugins repo.

Continue reading “Version Control”

Handling WordPress Cron failures using wp-cli

To understand WP cron, let me quote the following text from Cavalcade documentation

wp-cron is not actually a real task scheduler, and doesn’t actually operate like cron. Instead, it’s a pseudo-cron system, which is run as a loopback HTTP call when you access a page on the site (essentially, the page “forks” itself to run scheduled tasks). This is fine for high traffic single-sites, but lower traffic sites might not have their cron activated if the site isn’t viewed.

As you can see, even if a site has moderate traffic, if the site uses aggressive caching techniques, it is possible for real traffic to never hit PHP or WordPress to process a request. When wp-cron fails, a number of things can fail… a scheduled may not get published on time, a newsletter may not be sent as scheduled, an important WordPress security may not be applied, a backup schedule may have been missed, etc.

Cavalcade offers an excellent way to prevent such failures. It is more suitable for multi-site networks with hundreds of cron entries. For single sites, it is easier to handle a failure in wp-cron using wp-cli tool and a server cron.

Most common alternative is to disable internal WP cron and trigger it externally using wp-cron.php file and a server cron. If you are using this alternative method, then please remove cron entry and remove the corresponding entry in wp-config.php file, as wp-cli wouldn’t work if internal cron is disabled.

Let’s make sure we are ready to use the internal wp-cron by running the following command…

wp --path=/path/to/wordpress cron test

You’d see a success message… “Success: WP-Cron spawning is working as expected.“.

Most common error message is… “Error: The DISABLE_WP_CRON constant is set to true. WP-Cron spawning is disabled.” In this case, please disable the line in wp-config.php that disables wp-cron.

Now, it is time to set up a server cron that runs every minute like this…

* * * * * /path/to/wp-cli --path=/path/to/wordpress cron event run --due-now &> /dev/null

 

The 5 stars indicate a schedule to execute the command every minute. The command that follows executes cron events that are due now. Please make sure you provide full path to wp-cli and your WP site installation directory. I have tested the above in multiple sites on various servers. So far, it has been the most effective way to handle the failures in WP-Cron gracefully on single sites. If you run a multi-site network or a complex site with hundreds of crons, I’d highly recommend Cavalcade.