Subversion Command Line Primer

Linux No Comments

Below is an aggregate of my notes on how to use command line Subversion. The syntax may not be 100% correct (may have changed in newer versions) so please adapt accordingly.

Note: For more details, refer to the free book, Version Control with Subversion, written by some of the Subversion developers.

Though I’m using Linux syntax below, the Subversion commands should also work on Windows and macOS.

# Show version installed
svn --version

# ------------ Local Admin ------------

# Create a repository
sudo mkdir /var/repos
sudo svnadmin create /var/repos

# Export a repository
svnadmin dump /var/repos > backup.svn

# Import a repository
svnadmin load /var/repos < backup.svn

# List all projects in the local repository.
svn list file:///var/repos

# ------------ Basic Remote Client ------------

# List all projects in the remote repository by svn+ssh or HTTP URI.
svn list svn+ssh://
svn --username <myuser> --password <mypassword> list

# Import a project into the repository
svn import ./myproject svn+ssh:// -m "Initial Import"

# Check out a local, working copy of a project from the repository
svn co svn+ssh:// ./myproject

# View the working copy's info (no need to input the URI once inside the project)
cd ./myproject
svn info

# Update the project to the latest version
svn update

# Add a new file
svn add newfile.cpp

# Remove a file
svn rm oldfile.cpp

# Move and/or rename a file
svn mv oldfile.cpp subdir/newoldfile.cpp

# Set file to be binary type
svn propset svn:mime-type 'application/octet-stream' myfile.bin

# Have Subversion ignore certain subdirectories or files
svn propedit svn:ignore .

# List the modified files/directories (aka status) made to working copy
svn status
svn st

# Show differences between local files and current revision (usually HEAD)
svn diff

# Commit (aka checkin) all local working changes to the repository
svn commit -m "first commit"
svn ci -m "first commit"

# Show checkin logs (revision number, user, time, checkin comment)
svn log

# List files modified by a particular revision
svn log -v -r <revision_number>

# Show line-by-line differences in a particular revision's changeset
svn diff -c <revision_number>

# ------------ Selective Commit ------------

# Create changelist for selective commit
svn changelist <changelist-name> file1 file2 file3

# Add or remove files from the selective changelist
svn changelist <changelist-name> --add file4
svn changelist <changelist-name> --remove file2

# Show differences in selective changelist
svn diff --changelist <changelist-name>

# Checkin only files listed in selective changelist
svn ci -m "comment" --changelist <changelist-name>

# ------------ Undelete a File ------------

# Find version when deletion took place
svn log verbose

# Undelete by copying back from revision (one prior to deletion revision) to current
svn copy --revision 835 .

# If get "Path not found", then add a revision suffix to the URI
svn copy ‐‐revision 835 .

# ------------ Switch to Previous Revision ------------

# Revert to previous revision
svn up -r [revision_number]

# Show files that have changed
svn st --show-updates

# diff against latest revision (diff by itself won't show any differences)
svn diff -r HEAD
# ------------ Branching and Merging ------------

# Find revision when branched from trunk
svn log -v --stop-on-copy

# Revert back to an earlier revision
svn merge -r 10340:10335 <file or dir>

# Merge just one file
svn merge -r10343:10739 ./myfile.cpp

# Merge from trunk to branch
svn merge --dry-run -r10810:HEAD .
svn merge -r10810:HEAD --accept mine-conflict .

# Merge from branch to trunk (automatically accept theirs-full if conflict)
svn merge --reintegrate --accept theirs-full .

# Update working copy to use a different branch
svn switch http://mydomain/myproject/branches/mybranch .
No Comments

Update to Latest Ubuntu Kernel

Linux No Comments

Recently I got an email from DigitalOcean to update my server’s kernel to protect against the Meltdown and Spectre exploits. The instructions are the normal update commands that I usually run, “apt-get update” and “apt-get dist-upgrade”.

After doing the update, I thought that I would double-check the kernel version. The latest Ubuntu 14.04 kernel is “3.13.0-143-generic”. The kernel that my server was using is “3.13.0-039-generic”. My server was running an extremely old version of the kernel.

Supposedly, the Ubuntu GrubLoader should automatically run the latest installed kernel version after a reboot. However, my server was not doing that. It appears to be an issue with DigitalOcean’s droplet configuration.

The workaround is to use the DigitalOcean web interface to change the kernel from “DigitalOcean GrubLoader v0.2 (20160714)” to “DigitalOcean GrubLoader v0.1 (20160527)” (or vice versa). Then shutdown the server (using the command line) and switch the droplet back on using the DigitalOcean web interface. Somehow, changing the GrubLoader version makes the GrubLoader start working properly again.

Here are the full instructions to ensure that your Ubuntu server is running the latest kernel:

# Show Ubuntu version
lsb_release -a

# Show kernel version (alternatively, run "uname -a" or "cat /proc/version")
uname -r

# Update apt-get repository
sudo apt-get update

# Install any updates, handle dependencies automatically
sudo apt-get dist-upgrade

# Reboot so Grub will load any newly-installed kernel version
sudo reboot

# After reboot, show kernel version
uname -r

# If the kernel version is not the latest "3.13.0-143-generic" or later,
# go to the DigitalOcean web interface and switch the droplet's kernel
# from "DigitalOcean GrubLoader v0.2" to "DigitalOcean GrubLoader v0.1"
# (or vice versa).

# Doublecheck by listing the installed kernel image(s)
dpkg --list | grep linux-image-generic

# Shutdown the server
sudo poweroff

# On the DigitalOcean web interface, refresh the Droplet details until
# you see the power switch (located to right of Droplet name) go to
# the Off position.  Switch it back to the On position.

# After server boots up, show kernel version (you should see latest)
uname -r

# If you can't ssh into your server or you want the later
# "DigitalOcean GrubLoader v0.2" version, just repeat the steps above
# to change the GrubLoader version.

# Remove dependencies and packages which are no longer used
sudo apt-get autoremove

# If you get a "you may need to re-run your boot loader[grub]" warning,
# optionally update Grub's configuration.
sudo update-grub

Info above derived from:

No Comments

Re-subscribe to Let’s Encrypt Renewal Reminder Emails

Linux No Comments

Let’s Encrypt will automatically email expiration notices when your domain’s SSL certificate is coming up for renewal at 20 days, 10 days, and 1 day before the expiration date. (Recently, I received notices at 19 days and again at 10 days.)

I’ve found these renewal reminders very helpful because in the past, they’ve told me that the certificate auto-renewal process I had created (see Free SSL Certificate from Let’s Encrypt for Nginx) was broken. (Initially, I had attempted to setup the auto-renewal process to execute as a non-root user, but frequent updates requiring root access kept breaking it. I ended up configuring the auto-renewal to use root access.)

The renewal email body contains a link with the title “If you want to stop receiving all email from this address, click…” at the end. Unfortunately, the link is very long, taking up 3 to 4 lines of text (on my screen) and making it easy to click on by accident. More unfortunate, clicking on the link will disable the sending of expiration reminders to your email address for all domain certificates, not just the particular domain certificate in question. This change cannot be undone; you cannot re-register the same email address.

However, there is a re-registration workaround documented at Let’s Encrypt’s Expiration Emails page. The workaround takes advantage of how most email services will ignore the plus symbol and whatever follows it in an email address. For example, “” is treated the same as “”.

To re-subscribe your email address, run the command below in your certbot installation directory. (You will be prompted to input your sudo password if necessary.)

$ cd certbot
$ ./certbot-auto register --update-registration --email

Requesting to rerun ./certbot-auto with root privileges...
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Would you be willing to share your email address with the Electronic Frontier
Foundation, a founding partner of the Let's Encrypt project and the non-profit
organization that develops Certbot? We'
d like to send you email about EFF and
our work to encrypt the web, protect its users and defend digital rights.
(Y)es/(N)o: N

 - Your e-mail address was updated to

The community support page, Accidentally Unsubscribed, mentions an alternative workaround using appears to be an external website which periodically checks the expiration date on your domain’s SSL certificate. I haven’t tried it but I don’t see any reason why it wouldn’t work.

No Comments

Securing WordPress

Linux No Comments

Recently, I received a couple of “Password Reset” request emails from my blog. I surmised that someone had tried to log into my WordPress administrative account, was unsuccessfully, and had resorted to using the “Lost your password?” link on the login page. Disconcertingly, the email indicated that my admin username was being used. I checked my blog and thankfully, it looked like nothing was wrong.

It was time to make my WordPress installation more secured.

Disable Password Reset

I decided to remove the “Lost your password?” link on the WordPress login page. I doubted that I would forget my admin password and even if I did, I could always go directly to the MySQL database to change it. Because I didn’t need to support more than one user (my admin user), I decided to use the simpler manual method from How to Remove the Password Reset / Change option from WordPress, instead of the Plainview Protect Passwords plugin.

Here are the steps to globally disable the password reset option:

  1. Create a file named “disable-password-reset.php” with the following content:
     * Plugin Name: Disable Password Reset
     * Description: Disable password reset functionality. Only users with administrator role will be able to change passwords from inside admin area.
     * Version: 1.0
     * Author: WPBeginner
     * Author URI:

    class Password_Reset_Removed
      function __construct()
        add_filter( 'show_password_fields', array( $this, 'disable' ) );
        add_filter( 'allow_password_reset', array( $this, 'disable' ) );
        add_filter( 'gettext',              array( $this, 'remove' ) );
      function disable()
        if ( is_admin() ) {
          $userdata = wp_get_current_user();
          $user = new WP_User($userdata->ID);
          if ( !empty( $user->roles ) && is_array( $user->roles ) && $user->roles[0] == 'administrator' )
            return true;
        return false;
      function remove($text)
        return str_replace( array('Lost your password?', 'Lost your password'), '', trim($text, '?') );
    $pass_reset_removed = new Password_Reset_Removed();
  2. Upload this file to your server’s WordPress plugins directory (“/var/www/wordpress/wp-content/plugins”).
  3. Log into WordPress, go to Plugins, and activate the “Disable Password Reset” plugin. The password reset option will now be disabled for all users, including administrators.

As a precaution, I also changed the admin username because it had been compromised. This was accomplished by creating a new admin user in WordPress, logging into the new user, and then deleting the old user.

Force Login to Require Secure HTTPS

When I configured my VPS (Virtual Private Server), I allowed the WordPress login page and administrative areas to be accessible by unsecured HTTP. So anyone sniffing data packets on the Internet could see, in plain text, the info my browser was sending to my server. This may explain how my admin username was known to whomever was attempting to access my WordPress site.

To prevent the above, I decided to force the WordPress login and administration pages to require secure HTTPS access. I found the simplest method at WordPress SSL Settings and How to Resolve Mixed Content Warnings, which was to enable the “FORCE_SSL_ADMIN” WordPress option.

Edit the WordPress configuration file (“/var/www/wordpress/wp-config.php”) and add the text below before the “/* That’s all, stop editing! Happy blogging. */” statement at the bottom of the file:

 * Secure (force HTTPS) entire wp-admin area
 * Note: Includes FORCE_SSL_LOGIN which secures login.php script

define('FORCE_SSL_ADMIN', true);

/* That's all, stop editing! Happy blogging. */

After the change above, even if I were to access the login page using unsecured HTTP, it would automatically redirect me to the secure HTTPS login page.

Your Connection Is Not Fully Secure

Normally, when accessing a website using secure HTTPS, the browser would display a padlock next to the URL to indicate that the connection is secure. (Chrome shows a yellow padlock before the URL. Internet Explorer shows a grey padlock after the URL.) However, when I accessed my WordPress site using secure HTTPS, the padlock was not shown. Under Chrome, when I clicked on the info icon next to the URL, a message “Your connection to this site is not fully secure” was displayed.

I tested my site using Why No Padlock? and it found two issues:

  1. Server supports SSLv3, may be vulnerable to POODLE attack. It is suggested to disable the SSLv3 protocol.
  2. Number of insecure items: 15. (A.k.a. the mixed content issue. All 15 items were unsecured HTTP image links.)

The first issue can be resolved by disabling the SSL v3 protocol:

  1. Edit the Nginx server block file (“/etc/nginx/sites-available/default”). Locate and add/modify the “ssl_protocols” directive like below (the first line is commented out):
    #       ssl_protocols SSLv3 TLSv1;
            ssl_protocols TLSv1.2 TLSv1.1 TLSv1;
  2. Restart the Nginx service so the changes above will take effect:
    sudo service nginx restart

Note: When I restarted Nginx, I saw warnings in the Nginx error log (“/var/log/nginx/error.log”) that looked like the following:

2017/09/02 01:15:24 [warn] 20631#0: duplicate value "TLSv1.2" in /etc/nginx/sites-enabled/mydomain:128
2017/09/02 01:15:24 [warn] 20631#0: duplicate value "TLSv1.1" in /etc/nginx/sites-enabled/mydomain:128
2017/09/02 01:15:24 [warn] 20631#0: duplicate value "TLSv1" in /etc/nginx/sites-enabled/mydomain:128

I found that it was caused by having two “ssl_protocols” directives in the Nginx server block file. Once I removed the duplicate directive, the warnings disappeared.

If you have more than one domain (resulting in more than one Nginx server block file), you can add the “ssl_protocols” directive to the Nginx configuration file (“/etc/nginx/nginx.conf”), instead of to all the server block files. Directives in the Nginx configuration file will apply to all server block files.

The second issue, “insecure items” or mixed content, was caused by unsecured image link URLs in each blog post’s content. Fixing it would require editing all image links to use the secured URL instead. I plan to replace the unsecured image link URLs with secured URLs over time. Eventually, I hope to see the padlock on secured HTTPS access to my WordPress site. (By default, WordPress inserts an image link using an unsecured URL. So you will need to edit the pasted image link manually to use the secured URL.)

Note: I was concerned that browsers would not cache images downloaded by secure HTTPS. However, after some research, I found out that modern browsers (as of 2010) will cache HTTPS content by default.

WordPress Debug Mode

If you haven’t already done so, you might consider temporarily enabling the WordPress debug mode to see if there are issues with the theme or plugins. Setting debug mode will return a wealth of log messages (all errors, notices, and warnings) that might point out potential issues with your WordPress installation.

To set debug mode, edit the WordPress configuration file (“/var/www/wordpress/wp-config.php”) and add the following directives before the “/* That’s all, stop editing! Happy blogging. */” statement:

// Enable WP_DEBUG mode
define( 'WP_DEBUG', true );

// Enable Debug logging to the /wp-content/debug.log file
define( 'WP_DEBUG_LOG', true );

// Disable display of errors and warnings inside the page HTML
define( 'WP_DEBUG_DISPLAY', false );
@ini_set( 'display_errors', 0 );

// Use dev versions of core JS and CSS files (only if modifying them)
define( 'SCRIPT_DEBUG', true );

/* That's all, stop editing! Happy blogging. */

Once you are satisfied, don’t forget to disable the debug mode because it may adversely affect the performance of your WordPress site.

// Enable WP_DEBUG mode
define( 'WP_DEBUG', false );

I found only notices and PHP strict warnings concerning some plugins, but thankfully, no errors.

No Comments

Free SSL Certificate from Let’s Encrypt for Nginx

Linux No Comments

See my previous post in the unmanaged VPS (virtual private server) series, Automate Remote Backup of WordPress Database, on how to create and schedule a Windows batch script to backup the WordPress database.

Update: Let’s Encrypt has made changes that have broken my pre-existing certificate renewal. First, the new certbot package has replaced the original letsencrypt package. Second, the certbot package does not recognize the pre-existing certificates in the “/etc/letsencrypt” directory (generated by the old letsencrypt package). If you have the old letsencrypt package, I recommend deleting it, the “~/.local” directory, and the “/etc/letsencrypt” directory before installing the new certbot package. I’ve updated the instructions below to use the new certbot package.

I’ve been meaning to enable HTTPS/SSL access to this WordPress site since I heard that Google had started giving ranking boosts to secure HTTPS/SSL websites; however, the thing stopping me was the expensive and yearly cost of a SSL certificate. (Unfortunately, self-signed SSL certificates wouldn’t work because browsers would throw security warnings when encountering them.) But now, there is a new certificate authority, Let’s Encrypt, which provides free SSL certificates.

The only catch is that the SSL certificates would expire in 90 days. But that’s okay because Let’s Encrypt provides a command line client which can create and renew the certificates. Some elbow grease and a weekly Cron job should automatically renew any expiring SSL certificates.

Note: StartSSL provides a free, non-commercial SSL certificate which can be manually renewed after a year. I learned about it at the same time as Let’s Encrypt, but decided to go with Let’s Encrypt because of the possibility of automation and no restrictions on usage.

Below are the steps I took to create and install the SSL certificates on the Nginx server running on my unmanaged DigitalOcean VPS.

Create SSL Certificate

Ubuntu didn’t have the certbot package, so we will need to build it from source. Secure shell into your VPS server and run these commands:

# Install Git version control (alternative to Subversion)
sudo apt-get install git

# Double-check that Git is installed by getting version
git --version

# Download the certbot client source code (to the home directory)
git clone

# Install dependencies, update client code, build it, and run it using sudo
cd certbot
./certbot-auto --help
# Input the sudo password if requested to

# Get a SSL certificate for
./certbot-auto certonly --webroot -w /var/www/wordpress -d -d
# Input your email address (for urgent notices and lost key recovery)

# Get another SSL certificate for
./certbot-auto certonly --webroot -w /var/www/mydomain2 -d -d

Note: The Let’s Encrypt Ubuntu Nginx install instructions suggest using the wget command to get the latest available certbot version. I think “git clone” is a better method because it provides a more powerful way to update the certbot package, as we will see later.

Running the “certbot-auto” script will do the following:

  1. Install any missing dependencies including the GCC compiler and a ton of libraries.
  2. Update the certbot client source code.
  3. If necessary, build or update the certbot client, located at “~/.local/share/letsencrypt/bin/letsencrypt”. (The name switch from letsencrypt to certbot is not complete and thus a little confusing.)
  4. Run the certbot client using sudo; thus, you may be prompted to input the sudo password.

Note: If you want to speed it up by avoiding the extra update steps, you can just run the “sudo ~/.local/share/letsencrypt/bin/letsencrypt” command directly, instead of the “~/certbot/certbot-auto” script.

When running the “certbot-auto certonly –webroot” certificate generation option, the following (with some guesses on my part) occurs:

  1. The certbot client will create a challenge response file under the domain’s root directory (indicated by the “-w /var/www/wordpress” parameter); for example, “/var/www/wordpress/.well-known/acme-challenge/Y8a_KDalabGwur3bJaLfznDr5vYyJQChmQDbVxl-1ro”. (The sudo access is required to write to the domain’s root web directory.)
  2. The certbot client will then call the ACME server, passing in necessary credential request info such as the domain name (indicated by the “-d -d” parameters).
  3. The ACME server will attempt to get the challenge response file; for example, by browsing to “”. This verifies that the domain has valid DNS records and that you have control of the domain.
  4. The ACME server passes the generated SSL certificate back to the certbot client.
  5. The certbot client writes the SSL certificate to the “/etc/letsencrypt” directory, including the private key. If you can only backup one thing, it should be the contents of this directory.
  6. The certbot client deletes the contents of the “.well-known” directory; for example, leaving an empty “/var/www/wordpress/.well-known” directory once done. You can manually delete the “.well-known” directory.

Note: It is possible to create a multi-domain certificate containing more than one domain, but I recommend keeping it simple. Multi-domain certificates are bigger to download and may be confusing to the user when viewed.

Configure Nginx

The directory where the SSL certificates are located under, “/etc/letsencrypt/live”, require root user access, so we will need to copy the certificate files to a directory which Nginx can read.

# Copy out the SSL certificate files
sudo cp /etc/letsencrypt/live/ /etc/nginx/ssl/mydomain-fullchain.pem
sudo cp /etc/letsencrypt/live/ /etc/nginx/ssl/mydomain-privkey.pem
sudo cp /etc/letsencrypt/live/ /etc/nginx/ssl/mydomain2-fullchain.pem
sudo cp /etc/letsencrypt/live/ /etc/nginx/ssl/mydomain2-privkey.pem

# Double-check that the files exist and are readable by group and others
ls -l /etc/nginx/ssl

Because our website will behave the same under HTTPS as under HTTP, we will only need to make minimal changes to the existing HTTP server configuration.

Edit Nginx’s server block file for mydomain (“sudo nano /etc/nginx/sites-available/wordpress”) and add the “#Support both HTTP and HTTPS” block to the “server” section:

server {
        #listen   80; ## listen for ipv4; this line is default and implied
        #listen   [::]:80 default ipv6only=on; ## listen for ipv6

        #Support both HTTP and HTTPS
        listen 80;
        listen 443 ssl;
        ssl_certificate /etc/nginx/ssl/mydomain-fullchain.pem;
        ssl_certificate_key /etc/nginx/ssl/mydomain-privkey.pem;

        root /var/www/wordpress;
        #index index.php index.html index.htm;
        index index.php;

        # Make site accessible from http://localhost/
        #server_name localhost;

If you wish to have this server block be the default to serve (for both HTTP and HTTPS) when your server is accessed by IP address, change the “listen” directives to the following:

server {
        #Support both HTTP and HTTPS
        listen 80 default_server; # default_server replaces older default
        listen 443 ssl default_server;

Note: Make sure that only one of your server block files is set to be the default for IP address access. Also, when you browse to the IP address using HTTPS, you will still get a security warning because the IP address won’t match the domain name.

Repeat the above modifications for any other domain’s server block file.

Note: If you want HTTPS to behave differently than HTTP, leave the HTTP server section alone, uncomment the “# HTTPS server” section (at bottom of the server block file) and make your updates there.

Once you are done updating the server block files, tell Nginx to reload its configuration:

# Reload Nginx service
sudo service nginx reload

# If Nginx throws an error, look at the error log for clues
sudo tail /var/log/nginx/error.log

To test, browse to your server using the HTTPS protocol; for example, “”.

Renew SSL Certificate

To renew all your SSL certificates, run this command 30 days before the expiration date:

~/certbot/certbot-auto renew

Note: If you run it earlier than 30 days before the expiration date, no action will be taken.

Cron Job To Renew

To automate the SSL certificate renewal, we will use a Cron job that runs weekly under the root user. Running the job weekly is sufficient to guarantee a certificate renewal within the 30 days before expiration window.

First, create a script by running “nano ~/” and inputting the content below (make sure to replace “mynewuser” with your actual username):


# Run this script from root user's crontab

# Log file

# Print the current time
echo $(date)

# Try to renew certificates and capture the output
#/home/mynewuser/certbot/certbot-auto renew --no-self-upgrade > $LOGFILE 2>&1
/home/mynewuser/certbot/certbot-auto renew > $LOGFILE 2>&1

# Check if any certs were renewed
if grep -q 'The following certs have been renewed:' $LOGFILE; then
  # Copy SSL certs for Nginx usage
  cp /etc/letsencrypt/live/ /etc/nginx/ssl/mydomain-fullchain.pem
  cp /etc/letsencrypt/live/ /etc/nginx/ssl/mydomain-privkey.pem
  cp /etc/letsencrypt/live/ /etc/nginx/ssl/mydomain2-fullchain.pem
  cp /etc/letsencrypt/live/ /etc/nginx/ssl/mydomain2-privkey.pem

  # Reload Nginx configuration
  service nginx reload

Notes about the script:

  • To test the script, run this command:
    sudo sh ~/

    If none of your certificates are renewed, you won’t get a “Reloading nginx configuration” message (outputted by the “service nginx reload” command).

  • The “–no-self-upgrade” argument flag can be passed to certbot to prevent certbot from upgrading itself. At first, because we will be running the script under the root user, I hesitated to allow certbot to update with root permissions automatically. Avoiding an update seemed more secure and definitely faster to execute. However, without the update, by the time the three months expired, the certbot is hopelessly outdated and won’t successfully renew certificates. So, I had to allow certbot to automatic self-upgrade with root privileges to avoid having to do manual updates.
  • To simulate certificate renewals, use the “–dry-run” argument flag to simulate a successful renewal. Change the “certbot-auto” command in the script to the following:
    /home/mynewuser/certbot/certbox-auto renew --dry-run > $LOGFILE

    When you re-run the “”, you will get the “Reloading nginx configuration” message. Don’t forget to remove this change from the script once you are done testing.

  • The script copies out all the SSL certificates, instead of checking for and only copying certificates which have been modified. I don’t think the effort to do the latter is worth it.

Add the script to the root user’s Crontab (Cron table) by running these commands:

# Edit the root user's crontab
sudo crontab -e
  # Insert this line at the end of the file
  @weekly sh /home/mynewuser/ > /tmp/certbot-cron.log 2>&1

# List content of root user's Crontab
sudo crontab -l

# Find out when @weekly will run; look for cron.weekly entry
cat /etc/crontab

Note: Instead of “@weekly”, you may wish to set a specific time that works best for your situation. Refer to these Cron examples for info on how to set the time format.

If you want to test the Cron job, do the following:

# Delete the script's output log file
sudo rm /tmp/certbot-renew.log

# Change "@weekly" to "@hourly" in the Crontab
sudo crontab -e
  # Edit this line at the end of the file
  @hourly sh /home/mynewuser/ > /tmp/certbot-cron.log 2>&1

# Wait more than an hour

# Check if output log files were generated
cat /tmp/certbot-renew.log
cat /tmp/certbot-cron.log

# Change "@hourly" back to "@weekly" in the Crontab
sudo crontab -e
  # Edit this line at the end of the file
  @weekly sh /home/mynewuser/ > /tmp/certbot-cron.log 2>&1

Update Certbot

If you have chosen to disable the certbot self update in the cron script (using the “–no-self-upgrade” argument flag), I recommend manually running the “certbot-auto” command (without any arguments) once a month to make sure that certbot is up-to-date.

If you find that the “certbot-auto” command is unable to self-update or doing the self-update doesn’t solve an issue, you can try to update using the “git pull” command.

# Update the certbot source code using git
cd certbot
git pull

# See status of certbot source code and version
git status

Backup SSL Certs & Keys

Note: Re-issuing the SSL certificates (because of the switch from the letsencrypt to the certbot package) proved to be painless and fast. Thus, I’ve realized that backing up the “/etc/letsencrypt” directory is not necessary. If something goes wrong, just re-issue the SSL certificates.

In Automate Remote Backup of WordPress Database, we created a Windows batch file to download MySQL dump files and other files from the server. Let us add an additional command to that Windows batch script to download a zip archive of the “/etc/letsencrypt” directory.

Originally, I added an ssh command to the Windows batch file to zip up the “/etc/letsencrypt” directory. Unfortunately, accessing that directory requires sudo privileges which causes the script to prompt for the sudo password. I looked at two solutions to running sudo over SSH without interruption. The first involved just echo’ing the sudo password (in plaintext) to the ssh command. The second involved updating the sudoers file to allow running a particular file without requiring the password. I didn’t actually test the two solutions, but they didn’t look secure so I decided go with a very simple solution: run the zip command in the “~/” script.

First, edit the Cron script (“nano ~/”) and add the tar zip command after reloading the Nginx server:

# Check if any certs were renewed
if grep -q 'The following certs have been renewed:' $LOGFILE; then

  # Reload Nginx configuration
  service nginx reload

  # Zip up the /etc/letsencrypt directory
  tar -zcvf /tmp/letsencrypt.tar.gz /etc/letsencrypt

Note: We are using the tar command, instead of gzip, because gzip doesn’t handle the symbolic links under the “/etc/letsencrypt/live” directory correctly.

There is a security issue because “/tmp/letsencrypt.tar.gz” is readable by others; if this is a concern, you can adjust access permissions by adding the following commands to the “~/” script:


  # Zip up the /etc/letsencrypt directory
  tar -zcvf /tmp/letsencrypt.tar.gz /etc/letsencrypt

  # Change owner and restrict access to owner
  chown mynewuser /tmp/letsencrypt.tar.gz
  chmod 600 /tmp/letsencrypt.tar.gz

Second, edit the Windows batch script file “C:\home\myuser\backups\backup_wordpress.bat” and add the following to the end:

REM Download the /etc/letsencrypt tar file

mkdir \home\myuser\backups\letsencrypt
cd \home\myuser\backups\letsencrypt
rsync.exe -vrt --progress -e "ssh -p 3333 -l mynewuser -v" %date:~10,4%.%date:~4,2%.%date:~7,2%-letsencrypt.tar.gz

And we are done with the backup. In the future, if you ever need to restore the contents of “/etc/letsencrypt”, upload the tar archive to the server’s “tmp” directory and run the following on the server:

# Unzip the tar file
cd /tmp
tar -xvzf letsencrypt.tar.gz
# Will uncompress everything to /tmp/etc/letsencrypt

# Copy the contents to its original location
sudo cp -r /tmp/etc/letsencrypt /etc/

Redirect HTTPS to HTTP

If you have a domain which you don’t wish to provide HTTPS access to (i.e., go through the trouble of creating a SSL certificate for), you can configure Nginx to redirect HTTPS requests to HTTP. Uncomment the “# HTTPS server” section in the domain’s server block file and add a redirect statement:

# HTTPS server
server {
        listen 443 ssl;

        #To redirect, use return instead of rewrite (no longer recommended):
        #rewrite ^(.*) http://$host$request_uri;
        return 302 http://$host$request_uri;
        #Use 302 for temporary redirect vs 301 for permanent redirect.

Because we did not set the “ssl_certificate” and ssl_certificate_Key” values in the server_block above, Nginx will use the default_server’s SSL certificate instead. Unfortunately, the browser will show a security warning because the domain name in the default_server’s SSL certificate won’t match the requested domain name. If the user agrees to proceed, the redirection to non-secure HTTP access would correctly take place.

Info above derived from:

No Comments

Automate Remote Backup of WordPress Database

Linux No Comments

See my previous post, Subversion Over SSH on an Unmanaged VPS, to learn how to set up Subversion on Ubuntu (running on a DigitalOcean VPS). In this post, we will learn how to create a script to backup the WordPress database and copy it from the server to our local Windows client. We’ll also look at copying other files on the server to our local client’s hard drive. Finally, we will automate the execution of the backup script to run at regular intervals on the local client.

Install Windows SSH Tools

The backup script will use Unix tools, like ssh (secure shell) and rsync (remote sync), which are not included with Windows. Fortunately, there are free distributions of these tools for Windows. Let’s install them.

Get the ssh and rsync tools:

  1. Download the version of DeltaCopy without the installer (see “Download Links” located top-right).
  2. Unzip the downloaded “” to “C:\Program Files (x86)\DeltaCopy”.
  3. Add DeltaCopy to the execution path and set the home directory (where we will save the public/private RSA key pair files later):
    1. Open up the “System Properties” dialog by running “Edit system environmental variables” (or “sysdm.cpl”). Click on the Advanced tab. Click on the “Environmental Variables” button near the bottom to launch the “Environmental Variables” dialog.
    2. In the “Environmental Variables” dialog, select “Path” under “System variables” and click the “Edit…” button.
    3. Add “;C:\Program Files (x86)\DeltaCopy” (without the double-quotes) to the end of the existing “Variable value” field. Click Ok to save the change.
    4. Back in the “Environmental Variables” dialog, click “New…” button under “System variables”.
    5. Set “Variable name” to “HOME” and “Variable value” to your home directory like “C:\home\myuser”. Click Ok to save the change.
    6. Click Ok to close the “Environmental Variables” dialog and “Ok” again to close the “System Properties” dialog.

Get the ssh-keygen (secure shell authentication key generation) tool:

  1. Download the free version of cwRsync (click on the Get tab).
  2. Unzip the downloaded “” to a temporary directory like “C:/temp/cwRsync”. We will only need to use ssh-keygen once to generate the public/private RSA key pair.
  3. Besides ssh-keygen, cwRsync includes ssh and rsync which we won’t use; cwRsync’s ssh and rsync is not as Windows-compatible as DeltaCopy. For example, cwRsync’s ssh and rsync require that the RSA key pair files stored on Windows have Unix-like 0600 permissions, which then require the chmod tool (ironically included with DeltaCopy, but not cwRsync). DeltaCopy doesn’t have such issues. (Both DeltaCopy and cwRsync are based on a tiny part of Cygwin and DeltaCopy is the most Windows-friendly option of the three.)

Get the scp (secure copy) tool:

  1. Download the “pscp.exe” file from PuTTY.
  2. Move it into the “C:\Program Files (x86)\DeltaCopy” directory.

Create the “.ssh” directory under the home directory and test the environmental variables by running the “Command Prompt” (or “cmd.exe”) and inputting these commands. (Don’t type the comment lines below that start with the # pound character.)

# Test the HOME variable
echo %HOME%

# Create the .ssh directory
mkdir %HOME%\.ssh

# Test the PATH variable; ssh should be found and executed
ssh -p 3333

Server, Trust Me

To enable the backup script to run without requiring password input from the user, we need to establish trust between the remote server and the local client. To do so, we will create a client public/private RSA key pair and configure the server to trust the client public key. Tools like ssh and rsync can then authenticate against the server using the RSA key pair to avoid requiring the user to input a password.

Open the “Command Prompt” and do the following:

# Go the directory where we unzipped the ssh-keygen tool to
cd /temp/cwRsync/bin

# Generate client RSA key pair (for security, 2048 bits is the new minimum)
ssh-keygen -b 2048

Generating public/private rsa key pair.
# When prompted, select the current directory to write to;
# if you keep the default, it will fail
Enter file in which to save the key (/home/myuser/.ssh/id_rsa): ./id_rsa
# Keep the default; do not input a passphrase
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in ./id_rsa.
Your public key has been saved in ./

# Move client RSA key pair to .ssh directory
move ./id_rsa* /home/myuser/.ssh

# Copy client public key to the server
pscp -P 3333 /home/myuser/.ssh/

# Secure shell into the server; you will be prompted for password
ssh -p 3333

# On server, double-check that we are in the home directory

# Create the .ssh directory
mkdir .ssh

# Create authorized_keys file and append the client public key to it
cat >> .ssh/authorized_keys

# Delete the client public key (no longer needed)

# Optionally, restrict access to .ssh directory
chmod -R 700 .ssh

# Exit the secure shell

# Secure shell into the server again; you won't be prompted for the password
ssh -p 3333

On the last secure shell attempt, you should be able to log into the server without having to input a password.

Create and Schedule Backup Script

Create a file “C:\home\myuser\backups\backup_wordpress.bat” and input the following content:

@echo off

REM Display current date and time

date /t
time /t

REM Dump the wordPress database
REM The -v verbose flag is optional

ssh -p 3333 -v "mysqldump -uwordpress -pmypassword wordpress | gzip -c > /tmp/wordpress.sql.gz"

REM Download the database dump file to local directory.
REM Using rsync over ssh to avoid the need for a rsync server on VPS.
REM The %date:~10...% below helps to date-stamp the file,
REM resulting in a filename like 2015.04.23-wordpress_4.4.2.sql.gz.

mkdir \home\myuser\backups\wordpress
cd \home\myuser\backups\wordpress
rsync -vrt --progress -e "ssh -p 3333 -l mynewuser -v" %date:~10,4%.%date:~4,2%.%date:~7,2%-wordpress_4.4.2.sql.gz

REM Sync all Nginx server block files to local directory
REM Note: Be careful, the --delete flag allows Rsync to delete local files
REM       if they do not exist on the server also!

mkdir \home\myuser\backups\nginx
rsync -vrt --progress --delete -e "ssh -p 3333 -l mynewuser -v"* .

REM Rsync may not set local permissions correctly, so we'll fix with DeltaCopy's chmod.
REM Note: chmod fails for files with Windows-style perms already set, but that is ok.

chmod 660 *

You may wish to add additional rsync commands to download the WordPress configuration file (“/var/www/wordpress/wp-config.php”) and pictures uploaded to WordPress (“/var/www/wordpress/wp-content/uploads/*”).

Schedule the script:

  1. Run the “Task Scheduler”.
  2. Click on “Create Basic Task…” on the right sidebar.
  3. Input a name like “Backup WordPress”. Click Next.
  4. Select your schedule. I recommend “Weekly”. Click Next. Select a specific day and time that works for you. Click Next.
  5. Keep Action as “Start a program”. Click Next.
  6. Input “C:\home\myuser\backups\backup_wordpress.bat” into the “Program/script” box.
  7. Input “> C:\home\myuser\backups\backup_log.txt 2>&1” into the “Add arguments (optional)” box. This will redirect any standard or error outputs from the “backup_wordpress.bat” to “backup_log.txt” for review at your convenience.
  8. Click Next and Finish.
  9. To manually test, click on “Task Schedule Library” on the left sidebar, right-click on the “Backup WordPress” task in the top-center panel (if you don’t see it, click on the Refresh action to the right first), and select Run.

Getting PuTTY to Use Private/Public Key Pair

You may notice that the PuTTY pscp tool still requires a password to be inputted. Unfortunately, PuTTY does not use the RSA key format or the %HOME% environmental variable.

If you wish to use the pscp tool in the backup script, we’ll need to convert the RSA private key to the PPK (PuTTY Private Key) format:

  1. Download the “puttygen.exe” file from PuTTY.
  2. Run it.
  3. Go to menu Conversions and select “Import key”. Browse to the client RSA private key at “C:/home/myuser/.ssh/id_rsa”.
  4. Click the “Save private key” button. Answer Yes to the “Are you sure you want to save this key without a passphrase to protect it?” dialog.
  5. Input filename “id_rsa.ppk” and save to the same location as the original RSA key pair files.

When running the pscp tool in the script, use the “-i” option to tell it where to find the PPK file like so:

REM Copy the WordPress config file to local directory

cd \home\myuser\backups\wordpress
pscp -P 3333 -i /home/myuser/.ssh/id_rsa.ppk .

Hopefully the above will help you to sleep well, knowing that your WordPress data is safe.

See my followup post, Free SSL Certificate from Let’s Encrypt for Nginx, on how to install a free SSL certificate for HTTPS access and as a result, maybe give your Google ranking a boost.

No Comments

DigitalOcean After A Year: Still Good

Linux No Comments

Two weekends ago, on a Friday, my droplet was automatically upgraded to DigitalOcean‘s new cloud. I had received an email about the upgrade but had ignored it, believing that the upgrade would go smoothly. I was on a trip that Friday and the weekend, so did not check my website until Monday morning. Unfortunately, my website was unreachable and had been so since Friday.

Droplet Up, Website Down

050DragonSlayerI logged into DigitalOcean and the web interface said that my droplet was up and working fine. However, I could not ping it or secure shell to it. DigitalOcean’s Console Access web interface did worked and showed the necessary processes running on my droplet. The droplet was working fine but network access to it appeared to be broken.

I contacted support and was eventually routed to second level support (an engineer) who told me that I had to manually power off (run “sudo poweroff” on the command line) and then power on the droplet (using DigitalOcean’s web interface). This fixed the network connection issue and my website was back online. Note that doing a “shutdown -r” command or a “Power Cycle” (using the web interface) did not fix the connectivity problem.

DigitalOcean support was very responsive. Of the three support cases I’ve opened in the year that I’ve been with them, first line support had responded promptly. Of course, support did make use of canned responses (which I didn’t object to because it made sense to filter out beginners). Though I was vexed by the network connectivity issue (my website was offline for 3 days) and my irritation showed in my communications, the support staff always remained very polite and gracious.

Doing such a system-wide upgrade without checking network connectivity to affected droplets concerns me. Checking that upgraded droplets are up and reachable would have been my first validation test after the upgrade, instead of putting the burden on the customer to make sure everything is okay. Then again, this expectation might be acceptable for an unmanaged VPS; though I think it is a grey area because the upgrade was initiated by DigitalOcean. For full disclosure, DigitalOcean did provide a manual upgrade process; which in hindsight, I should have taken advantage of. Lesson learned.

Slow and Slowerer

When I configured my droplet a year ago, I was very impressed by the performance. My website loaded pages within 1 second, as opposed to the 2-4 seconds on my previous shared web hosting service. Recently, I would have been very glad to get back my 2-4 seconds page load time.

Since the past few months, I’ve noticed my website getting slower and slower. Even a simple PHP application I had running (also on the droplet) took longer and longer to process. Like a frog slowly being boiled, I got used to a 4-6 seconds page load time as being “normal”.

Worse, after my droplet was upgraded, the page load time dropped to 8-9 seconds. I installed the “WP Super Cache” WordPress plugin in a quick fix attempt to increase performance and it worked. Once WP Super Cache was activated, page load times moved back to 4-6 seconds.

You know what they say about quick fixes. A week later, the page load times increased to 8-15 seconds. 15 seconds! I disabled WP Super Cache and page load times dropped to 4-6 seconds. I didn’t understand why but at least, the crisis was averted.

Bottleneck? What Bottleneck?

The performance of any VPS (or shared web hosting) is determined by the allocated CPU power, amount of memory, disk access speed, software stack (programs running), and network traffic. The first three can be collectively known as the hardware or virtual hardware. In my website’s case, the software stack is composed of the Ubuntu operating system, LEMP infrastructure, WordPress and its plugins. And though I would love to say that the slowdown was due to increased network traffic to my website, it wasn’t.

When optimizing for performance, it pays to determine where the bottleneck is. For example, you could waste time optimizing the LEMP (by adding Varnish) or WordPress (by adding the WP Super Cache plugin) when the bottleneck is that you are out of memory (Varnish won’t help) or free disk space (WP Super Cache could actually make this worse with its caching mechanism). Having said that, there are ways to optimize LEMP (and WordPress to a lesser extent) to reduce memory usage; but then, it is usually at the cost of performance.

I contacted DigitalOcean support for help. I got a mostly canned reply back. They stated that the fault wasn’t because of network connectivity, hardware, or over-subscription (where there are too many droplets running on the same physical hardware). They had tested loading a couple of static images from my website, which only took 100-200ms each and proved that the problem was not on their end. The canned reply suggested using sysstat to figure out the problem with the droplet.

Sysstat is a collection of utilities to monitor performance under Linux. Here’s how to install and use sysstat:

# Install sysstat
sudo apt-get install sysstat

# Enable sysstat system metrics collection
sudo vi /etc/default/sysstat
  # Change ENABLE="false" to "true"

# Start sysstat
/etc/init.d/sysstat start

# Check CPU usage
sar -u

# Check memory usage
sar -r

Because we have just started the sysstat process, the check CPU and memory usage will only return the current CPU and memory usage. Sysstat will collect system metrics every 10 minutes; so in the future, the “sar” commands above would return the CPU and memory usage collected every 10 minutes in the past. Sysstat has a lot more functionality which I have yet to explore.

My favorite performance monitoring tool is the “top” command. It displays a real-time summary of the CPU, memory, and swap usage with a list of the processes consuming the most CPU. (Note that the Ubuntu image from DigitalOcean has swap disabled by default.) The top command allows me to see what is happening on the system as I load a page.


Right off the bat, I noticed that my CPU usage was around 90% constantly, which was a big red flag. After a day of recording, sysstat returned the same average 90% CPU usage. This might explain why the WP Super Cache plugin, which required more CPU and disk access to do the page caching, made the website performance worse. I didn’t recall seeing the CPU that high when I first configured the droplet a year (it would have concerned me very much then).

Memory also looked alarming with a 98% usage (493424 used / 501808 total); however, it was a false alarm. Evidently, Linux operating systems like Ubuntu will allocate all the free memory for disk caching. Then, when applications need more memory, they get it from the disk cache. So, the important data to look for is the cache size. Here, the cache size was 28% of total memory (144132 cached Mem / 501808 total), which means only about 2/3 of memory was actually used by applications.

Note: The Linux command to display free memory, “free -m”, supports the same conclusion. Look for the reported “cached” number.

What Is Eating My Hard Drive?

Running the Linux command to report file system disk space usage, “df -h”, indicated that 93% of my 20GB quota was used. I remembered that my droplet used much less than 50% of the 20GB a year ago.

Find the space hogs:

cd /
sudo du -h --max-depth=1
16G    ./var

cd var
sudo du -h --max-depth=1
6.0G   ./www
7.4G   ./lib
2.0G   ./log

Note: If your system’s estimate file usage “du” command does not support the “max-depth” flag, then you will need to run this command on each directory one by one like so:

sudo du -sh /var/www
sudo du -sh /var/lib
sudo du -sh /var/log

The “/var/www” directory contained my website content so that was a keeper. The “/var/lib” directory contained important system and application files, so we could not just delete anything in it without a lot of caution. The “/var/lib” directory’s large size was caused primarily by the MySQL database file, “/var/lib/mysql/ibdata1”, which we will examine in detail later. I was certain that I could safely delete the archived log files from the “/var/log” directory though.

# Manually rotate log files (basically create new active log files)
sudo /etc/cron.daily/logrotate

# Delete all gzipped archive files
sudo find /var/log -type f -name "*.gz" -delete

# Delete all next-to-be-archived files
sudo find /var/log -type f -name "*.1" -delete

# Double-check size again
sudo du -sh /var/log
563M   /var/log

Strangely, I found the mail directory, “/var/mail”, taking up 150MB. There were a lot of non-delivery notification emails sent to the website address. (I don’t know why but plan to investigate it at a later time.) I was sure that it is also safe to delete those emails.

# Check usage
sudo du -sh /var/mail
156M   /var/mail

# Truncate all mail files to zero size
sudo find /var/mail -type f -exec truncate {} --size 0 \;

# Double-check usage
sudo du -sh /var/mail
4.0K   /var/mail

Note: I did read a recommendation to enable swap on Ubuntu to guard against out of memory errors (because swap allows disk space to be used as additional memory at the expense of performance); however, because I have 1/3 free memory and a performance problem, I don’t think enabling swap is the appropriate solution for my case.

Die, WordPress Plugin, You Die!

I strongly believed that the bottleneck was the CPU; the top five most CPU-intensive processes were the “php5-fpm” processes (responsible for executing PHP scripts). So, optimizing LEMP by adding Varnish (an additional HTTP accelerator process) would probably not help, and might even harm the performance further. What could be causing so much CPU usage?

According to Google Analytics, the traffic to my website had not changed significantly this past year. Even if it had, the now roughly 200 visitors per day should not cause such a high CPU usage. I had not changed my website in any way (for example, by adding new plugins). The only changes had been software updates to Ubuntu, WordPress and its plugins.

For LEMP infrastructure issues, the recommended step is to check the log files for errors.

# Linux system log
sudo tail /var/log/dmesg

# Nginx log
sudo tail /var/log/nginx/error.log

# PHP log
sudo tail /var/log/php5-fpm.log

# MySQL log
sudo tail /var/log/mysql/error.log

Looking at the Nginx log was eye-opening because I could see hacking attempts against my website using invalid URLs. However, that could not be the cause of the high CPU usage. There were no other errors or clues in the log files.

For WordPress performance issues, the universally-recommended first step is to disable the plugins and see if that fixes the issue. Rather than disabling all the plugins and re-enabling them one by one, my gut told me that the culprit plugin might be the “WordPress SEO”. When I get daily-in-row or even twice-a-day updates to a piece of software, I know that the software is very buggy. WordPress SEO was guilty of that behavior. Disabling the WordPress SEO plugin resulted in an immediate drop in CPU usage to the 30-50% range. Page load times dropped to 2-3 seconds.

Unfortunately, when I checked a few days later, the CPU was back up to 90% and page load times had increased back to 8-10 seconds. The WordPress SEO plugin was a contributor, but it was not the primary cause of my droplet’s performance issue.

MySQL, What Big Eyes You Have

In addition, the “/var/lib” directory had grown another 1.5GB in size and at a total 9GB, had consumed almost half of my 20GB allocation. Digging further, I found that it was the “/var/lib/mysql/ibdata1” file that had grown to over 6GB. The “ibdata1” file was where MySQL (specifically the InnoDB storage engine) stored the database data and while it can grow, unfortunately it can never decrease in size.

A MySQL query on the database size was necessary to investigate further. Log into MySQL as the root user and run this query to show the sizes of the existing databases:

SELECT table_schema "Data Base Name",
sum( data_length + index_length ) / 1024 / 1024 "Data Base Size in MB",
sum( data_free )/ 1024 / 1024 "Free Space in MB"
FROM information_schema.TABLES
GROUP BY table_schema;

I found that my MediaWiki database was over 6GB in size. I had a MediaWiki for personal use. Access to it was restricted by a password-protected directory. I hadn’t used it in a long time (over half a year) so hadn’t paid any attention to it. When I logged into it, I found the main page was blank with a link to an unknown website. A check of the history indicated that multiple unknown revisions had been made to it since February of this year. My MediaWiki had been hacked.

Evidently, someone had gotten past the password-protection and was using the MediaWiki to store 6GB of data. Worse, that someone may have hacked MediaWiki to run their own PHP code (very unlikely but not impossible as I had a very old version of MediaWiki running). This explained the high CPU usage and the low free disk space.

I savaged my personal info from the MediaWiki (using the history to view old page revisions). I then deleted the MediaWiki database and directory containing the MediaWiki PHP code. The CPU usage immediately went down to a few percentages. Page load time dropped to around one second. Hurrah! (I also changed all my passwords just in case.)

MySQL Database Surgery

To reclaim the 6GB in space used by MySQL’s “ibdata1” file required major surgery. I needed to delete the “ibdata1” file which required deleting and re-creating the WordPress database (and my other personal databases).

Before starting, I recommend configuring MySQL to store each InnoDB table in its own separate file, instead of in the “ibdata1” file, to allow more options to manage drive space usage. Doing this will support the MySQL “Optimize Table” command, which can reduce the table’s file size.

sudo nano /etc/mysql/my.cnf
  # Add "innodb_file_per_table" to the [mysqld] section

The change above won’t take effect until we restart MySQL.

We need to do some steps before and after deleting the “ibdata1” file:

# Dump a backup of "wordpress" database (and any other personal database)
mysqldump -u[username] -p[password] wordpress > /tmp/wordpress.sql

# Delete "wordpress" database (and any other database except "mysql" and "performance_schema")
mysql -u root -p
mysql> drop database wordpress;
mysql> quit

# Stop MySQL server
sudo service mysql stop

# Log into root user (necessary to access "/var/lib/mysql" directory)

# Delete subdirectories and files ("ibdata1") under "/var/lib/mysql" except for "/var/lib/mysql/mysql"
cd /var/lib/mysql
ls -I "mysql" | xargs rm -r -f

# Exit root user

# Start MySQL server
sudo service mysql start

# Create "wordpress" database (and any other database)
mysql -u root -p
mysql> create database wordpress;
mysql> quit

# Restore "wordpress" database (and any other database)
mysql -u [username] -p[password] wordpress < /tmp/wordpress.sql

Viewing the “/var/lib/mysql” directory showed a much smaller “ibdata1” file (about 18M). Strangely, my WordPress database was configured to use MyISAM (an alternative storage engine to InnoDB) by default, so it didn’t use the “ibdata1” file. The “/var/lib/mysql/wordpress” directory contained MyISAM .myd storage files. However, my other personal database did use InnoDB and its directory, “/var/lib/mysql/personal_database”, did contain individual InnoDB .ibd storage files (per table).

WordPress On A Diet

While I was poking around WordPress, I decided to optimize the MySQL database by deleting unnecessary data such as previous versions of posts. Rather than manually truncating database tables myself (a very dangerous, though oddly satisfying pastime), I decided to use the “Optimize Database after Deleting Revisions” plugin, which did exactly what its name said it did.

Before running the “Optimize Database after Deleting Revisions” plugin, backup your WordPress MySQL database. Then do the following to manually optimize your database:

  1. Go to “Settings/Optimize Database” in the WordPress administration.
  2. Configured the options. I checked all the “Delete…” options except for “Delete pingbacks and trackbacks”. I did not enable the Scheduler because I only wish to run this plugin manually when I decide to.
  3. Click the “Save Settings” button.
  4. Click the “Go To Optimizer” button.
  5. Click the “Start Optimization” button.

Thoughts on Hardware Upgrade

Had I not fixed the high CPU usage issue (and it had been a valid issue), the next step would have been to look at options to upgrade the hardware. This would mean upgrading to DigitalOcean’s higher-priced plans or even another VPS provider (I have heard that Linode has better performance overall).

Because my bottleneck was the CPU, I would have had to upgrade to DigitalOcean’s $20/month plan which includes a 2 core processor. Upgrading to the $10/month plan, which only included a 1 core processor (the DigitalOcean website didn’t say whether it is a faster processor than the 1 core processor in the $5/month plan), would not have fixed my CPU issue. Had my bottleneck been a memory limitation, I would chose the $10/month plan, which would have doubled the memory size (1GB versus 512MB).

Thinking outside the box, a cheaper option than the $20/month plan would be to get a second $5/month droplet to run a dedicated MySQL server (hosting the WordPress database). The original droplet would run only the WordPress application and talk to the second droplet’s database. This $10/month option with two $5/month droplets would have two 1 core processors, which might be better than a single 2 core processor! Alas, the MySQL process used only 10-15% of the CPU so removing it from the original droplet would not have made much of a difference.

Hopefully documenting my trials and tribulations above will help you to have an easier time with the performance of your unmanaged VPS.

Some info above taken from:

No Comments

Subversion Over SSH on an Unmanaged VPS

Linux No Comments

See my previous post, Upgrade Ubuntu and LEMP on an Unmanaged VPS, to learn how to upgrade LEMP and Ubuntu to the latest versions. In this post, we will install Subversion on the server and learn how to access it using Subversion over SSH (svn+ssh).

Note: Though I’m doing the work on a DigitalOcean VPS running Ubuntu, the instructions may also apply to other VPS providers.

Subversion allows a client to execute svn commands on the server over SSH. As a result, there is no need to have a Subversion server process (svnserve) running or an Apache server configured to support Subversion (mod_dav_svn); one only needs SSH access. Subversion over SSH is simple and sufficient for my needs.

For svn+ssh, access to Subversion is controlled by the Linux user login. To avoid having to input your SSH login password every time you run a svn command, I recommend configuring SSH with public key authentication between your client and the server. For instructions, see the “SSH With Public Key Authentication” section in my previous post, SSH and SSL With HostGator Shared Web Hosting.

To begin, on the server, install the Subversion package and create a repository:

# Install subversion
sudo apt-get install subversion

# Check that subversion is installed
svn --version

# Make a repository directory
sudo mkdir /var/repos

# Create a repository
sudo svnadmin create /var/repos

We need to change the permissions on the newly-created repository directory so that our Linux user can have read-write access. I recommend adding your user to the ‘www-data’ group and giving that group modify access to the repository like so:

# Change mynewuser's primary group to www-data
sudo usermod -g www-data mynewuser

# Check by showing all groups that mynewuser belongs to
groups mynewuser

# Change repository group owner to be www-data
sudo chgrp -R www-data /var/repos

# Add group write permission to repository
sudo chmod -R g+w /var/repos

On the remote client machine, we will use the Subversion client with svn+ssh to access the repository. Because we are using a custom SSH port and the Subversion command line does not provide an option to input the SSH custom port, we have to configure SSH to use the custom port automatically.

Configure SSH to use the custom port when connecting to your server by creating a SSH configuration file located at “~/.ssh/config” (on Mac OS X) or “%HOME%/.ssh/config” (on Windows). Input the following file content:

  Port 3333
  PreferredAuthentications publickey,password

After this, you can run “ssh” instead of “ssh -p 3333” because SSH will use the custom 3333 port automatically when connecting to “”.

Note: On Windows, I am using the DeltaCopy “ssh.exe” client in combination with the CollabNet “svn.exe” Subversion client. On Mac OS X, I am using the built-in ssh client and the svn client (installed using MacPorts).

To test access to the repository, run the following command on the client:

# List all projects in the repository.
svn list svn+ssh://

This command will return an empty line because there are no projects in the repository currently. If you do not see an error, then the command works correctly.

On the client, you can now issue the standard Subversion commands like the following:

# Import a project into the repository
svn import ./myproject svn+ssh:// -m "Initial Import"

# The list command should now show your newly-imported project
svn list svn+ssh://

# Check out a local, working copy of the project from the repository
svn co svn+ssh:// ./myproject2

# View the working copy's info (no need to input the svn+ssh URL once inside the project)
cd ./myproject2
svn info

# Update the project to the latest version
svn update

If you should wish to run Subversion commands locally on the server, you can do so using the “file:///” path instead of “svn+ssh://” URL.

# List all projects in the repository.
svn list file:///var/repos

# Check out a local, working copy of the project from the repository
svn co file:///var/repos/myproject ./myproject2

And we are done. Hopefully the above info will be useful should you ever need to get Subversion working.

See my followup post, Automate Remote Backup of WordPress Database, on how to create and schedule a Windows batch script to backup the WordPress database.

No Comments

Upgrade Ubuntu and LEMP on an Unmanaged VPS

Linux No Comments

See my previous post in my unmanaged VPS (virtual private server) series, Nginx HTTPS SSL and Password-Protecting Directory, to learn how to configure Nginx to enable HTTPS SSL access and password-protect a directory. In this post, I will explore how to upgrade LEMP and Ubuntu.

Upgrade LEMP

While one can upgrade each component of LEMP (Linux, Nginx, MySQL, PHP) separately, the safest way is to upgrade all software components installed on the system to ensure that the dependencies are handled properly.

Upgrade all software packages, including LEMP, by running the following commands:

# Update apt-get repositories to the latest with info
# on the newest versions of packages and their dependencies.
sudo apt-get update

# Use apt-get dist-upgrade, rather than apt-get upgrade, to
# intelligently handle dependencies and remove obsolete packages.
sudo apt-get dist-upgrade

# Remove dependencies which are no longer used (frees up space)
sudo apt-get autoremove

Some changes may require a reboot. To initiate a reboot, execute this recommended command:

# Following command equivalent to: sudo shutdown -r now
sudo reboot

Updating PHP-FPM Breaks WordPress

If the PHP-FPM (FastCGI Process Manager for PHP) package is updated, one may be prompted to overwrite the “/etc/php5/fpm/php.ini” and “/etc/php5/fpm/pool.d/www.conf” configuration files with the latest versions. I recommend selecting the option to show the differences, making a note of the differences (hitting the “q” key to quit out of the compare screen), and accepting the latest version of the files.

After the upgrade, WordPress may be broken because the PHP-FPM is no longer configured correctly. To fix this issue, update the two PHP-FPM configuration files with these changes to ensure that Nginx will successfully integrate with PHP-FPM:

# Fix security hole by forcing the PHP interpreter to only process the exact file path.
sudo nano /etc/php5/fpm/php.ini
   # Add the following or change the "cgi.fix_pathinfo=1" value to:

# Configure PHP to use a Unix socket for communication, which is faster than default TCP socket.
sudo nano /etc/php5/fpm/pool.d/www.conf
   # Keep the following or change the "listen =" value to:
   listen = /var/run/php5-fpm.sock
   # The latest Nginx has modified security handling which requires
   # uncommenting the "listen.owner" and "" properties:
   listen.owner = www-data = www-data
   ;listen.mode = 0660

# Restart the PHP-FPM service to make the changes effective.
sudo service php5-fpm restart

Test by browsing to the “info.php” file (containing the call to “phpinfo” function) to ensure that Nginx can call PHP-FPM successfully. Hopefully, you won’t see the “502 Bad Gateway” error which means it didn’t. If so, look at the Nginx and PHP-FPM error log files for hints on what could have gone wrong.

sudo tail /var/log/nginx/error.log
sudo tail /var/log/php5-fpm.log

Note: If you accidentally select the option to keep the current version of the PHP-FPM configuration files and now wish to the get the latest versions, you will need to uninstall and re-install the PHP-FPM service:

sudo apt-get purge php5-fpm
sudo apt-get install php5-fpm

You will then need to update the two PHP-FPM configuration files per the instructions above.

Upgrade May Break iptables

After a recent upgrade, a “problem running iptables” error message is displayed when logging into the droplet. The whole error is displayed when I attempt to view the firewall status:

~$ sudo ufw status
ERROR: problem running iptables: modprobe: ERROR: could not insert 'ip_tables': Exec format error
iptables v1.4.21: can't initialize iptables table `filter': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

Thanks to this page, problem with iptables and ubuntu Ubuntu 13.10, I found that the issue was caused by the upgrade process switching the kernel to a 64bit version. The problem is that the rest of the system (executables, object code, shared librairies) is 32bit!

# Check the kernel version (x86_64 means 64bit)
~$ uname -a
Linux mydomain 3.13.0-39-generic #66-Ubuntu SMP Tue Oct 28 13:30:27 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

# Check the system executables and libraries (32-bit means 32bit!)
~$ file /sbin/init
/sbin/init: ELF 32-bit LSB  shared object, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=c394677bccc720a3bb4f4c42a48e008ff33e39b1, stripped

To fix the 64bit/32bit mismatch, I did the following:

  1. Browse to the DigitalOcean web interface, drill into my droplet, and select “Kernel” configuration (on left panel).
  2. I then selected the 32bit version of the kernel, which is “Ubuntu 14.04 x32 vmlinuz-3.13.0-39-generic” (only difference from the current kernel “Ubuntu 14.04 x64 vmlinuz-3.13.0-39-generic” is changing “x64” to “x32”). Click the Change button.
  3. Power down the droplet by running the “sudo poweroff” command.
  4. Use the DigitalOcean web interface to power on the droplet.

After doing the above, I no longer see the “problem running iptables” error message. Viewing the firewall status now successfully returns the correct set of rules:

~$ sudo ufw status
Status: active

To                         Action      From
--                         ------      ----
3333/tcp                   ALLOW       Anywhere
80                         ALLOW       Anywhere
25/tcp                     ALLOW       Anywhere
443                        ALLOW       Anywhere

Note: Be patient because the DigitalOcean web interface can take a minute to recognize reflect that the droplet is powered off (and then enable the Power On button). Also, the first two times I tried to power on the droplet, I got timeout errors. The 3rd attempt didn’t do anything. Finally, the 4th attempt successfully powered on the droplet. Whew!

Upgrade Ubuntu

The following is particular to my VPS provider, DigitalOcean, but perhaps it may help provide a general idea on what to expect with your own provider when doing an operating system upgrade.

On logging into my server, I saw the following notice:

New release '14.04.1 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Your current Hardware Enablement Stack (HWE) is no longer supported
since 2014-08-07.  Security updates for critical parts (kernel
and graphics stack) of your system are no longer available.

For more information, please see:

To upgrade to a supported (or longer supported) configuration:

* Upgrade from Ubuntu 12.04 LTS to Ubuntu 14.04 LTS by running:
sudo do-release-upgrade

Update: One does not necessarily have to upgrade to the latest Ubuntu release version when prompted to. However, in the case above, support for the 12.04 LTS release had ended so an upgrade to 14.04 LTS was mandatory. Recently, I got a message to upgrade from release 14.04 LTS to 16.04 LTS. However, I don’t plan to upgrade because the 14.04 LTS release will be supported until 2019.

When I ran “sudo do-release-upgrade”, there was a dire warning about running upgrade over SSH (which I ignored) and many prompts to overwrite configuration files with newer versions (which I accepted after taking note of the differences between the new and old versions). There was also a warning about how the upgrade could take hours to complete, though it ended up taking less than 15 minutes. The upgrade ended with a prompt to reboot, which I accepted.

Note: To be safe, one should run the “sudo do-release-upgrade” command from the Console window (accessible through the DigitalOcean web interface), instead of from a SSH session. I was lucky that nothing went wrong with the release upgrade.

After reboot, I updated the two PHP-FPM configuration files, “/etc/php5/fpm/php.ini”
and “/etc/php5/fpm/pool.d/www.conf”, per the instructions in the above section.

In addition, I had to re-enable sudo permissions for my user by running the following:

# visudo opens /etc/sudoers using vi or nano editor, whichever is the configured text editor.
# It is equivalent to "sudo vi /etc/sudoers" or "sudo nano /etc/sudoers" but includes validation.
   # Add mynewuser to the "User privilege specification" section
   root       ALL=(ALL:ALL) ALL
   mynewuser  ALL=(ALL:ALL) ALL

I found the upgrade process, especially upgrading the Ubuntu operating system, to be a relatively painless experience. Hopefully you will find it to be the same when you do your upgrade.

See my followup post, Subversion Over SSH on an Unmanaged VPS, to learn how to install and use Subversion over SSH (svn+ssh).

Some info above derived from:

No Comments

Nginx HTTPS SSL and Password-Protecting Directory

Linux 1 Comment

See my previous post in my unmanaged VPS (virtual private server) series, Nginx Multiple Domains, Postfix Email, and Mailman Mailing Lists, to learn how to configure multiple domains and get Postfix email and Mailman mailing lists working. In this post, I will configure Nginx to enable HTTPS SSL access and password-protect a directory.

Note: Though I’m doing the work on a DigitalOcean VPS running Ubuntu LTS 12.04.3, the instructions may also apply to other VPS providers.

Enable HTTPS/SSL Acess

I have a PHP application which I want to secure. If I use HTTP, then the information sent back from the server to my browser is in clear text (and visible to anyone sniffing the network). If I use HTTPS (HTTP Secure) with a SSL (Secure Sockets Layer) server certificate, then the information will be encrypted. In the steps below, I will configure HTTPS/SSL to work for a domain and then force HTTPS/SSL access on a particular directory (where the PHP application would be located).

To get HTTPS working, we need a SSL server certificate. While you can get a 3rd party certificate authority to issue a SSL certificate for your domain for about $10 per year, I only need a self-signed certificate for my purpose. A 3rd party issued SSL certificate is convenient because if the browser trusts the 3rd party certificate authority by default, the browser won’t prompt you to accept the SSL certificate like it would for a self-signed certificate (which the browser can’t establish a chain of trust on). If you run a business on your website, I recommend investing in a 3rd party SSL certificate so that your website would behave professionally.

Create a self-signed SSL server certificate by running these commands on the server:

Note: You don’t need to input the lines that start with the pound character # below because they are comments.

# Create a directory to store the server certificate.
sudo mkdir /etc/nginx/ssl

# Change to the newly-created ssl directory.  Files created below will be stored here.
cd /etc/nginx/ssl

# Create a private server key.
sudo openssl genrsa -des3 -out server.key 1024
   # Remember the passphrase you entered; we will need it below.

# Create certificate signing request.
# (This is what you would send to a 3rd party authority.)
sudo openssl req -new -key server.key -out server.csr
   # When prompted for common name, enter your domain name.
   # You can leave the challenge password blank.

# To avoid Nginx requiring the passphrase when restarting,
# remove the passphrase from the server key. (Otherwise, on
# reboot, if you don't input the passphrase, Nginx won't run!)
sudo mv server.key server.key.pass
sudo openssl rsa -in server.key.pass -out server.key

# Create a self-signed certificate based upon certificate request.
# (This is what a 3rd party authority would give back to you.)
sudo openssl x509 -req -days 3650 -in server.csr -signkey server.key -out server.crt

Note: I set the certificate expiration time to 3650 days (10 years); 3rd party certificates will usually expire in 365 days (1 year). The maximum expiration days you can input is dependent upon the OpenSSL implementation. Inputting 36500 days (100 years) would probably fail due to math overflow errors (once you convert 100 years into seconds, the value is too big to store in a 32bit variable). I believe the highest you can go is about 68 years, but I haven’t tested it.

Configure Nginx to use the SSL server certificate we created by editing the server block file for the domain you want to use it on:

sudo nano /etc/nginx/sites-available/domain2

In the “domain2” server block file, find the commented-out “HTTPS server” section at the bottom, uncomment it, and edit it to look like the following:

# HTTPS server
server {
        listen 443;

        root /var/www/mydomain2;
        index index.php index.html index.htm;

        ssl on;
        ssl_certificate /etc/nginx/ssl/server.crt;
        ssl_certificate_key /etc/nginx/ssl/server.key;

#       ssl_session_timeout 5m;
#       ssl_protocols SSLv3 TLSv1;
#       ssl_ciphers ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv3:+EXP;
#       ssl_prefer_server_ciphers on;

        location / {
                try_files $uri $uri/ /index.php;

        # pass the PHP scripts to FPM-PHP
        location ~ \.php$ {
                fastcgi_split_path_info ^(.+\.php)(/.+)$;
                fastcgi_pass unix:/var/run/php5-fpm.sock;
                fastcgi_index index.php;
                include fastcgi_params;

Note: The “HTTPS Server” section looks like the “HTTP Section” we configured previously at the top, except for the addition of “listen 443” (port 443 is the HTTPS port) and the SSL enabling configurations.

Open up the HTTPS port in the firewall and reload Nginx by running these commands on the server:

# Allow HTTPS port 443.
sudo ufw allow https

# Double-check by looking at the firewall status.
sudo ufw status

# Reload Nginx so changes can take effect.
sudo service nginx reload

Test by browsing to “”. When the browser prompts you to accept the self-signed server certificate, answer Yes.

Require HTTPS/SSL Access on a Directory

To require HTTPS/SSL-only access on a particular subdirectory under the domain, we need to add a directive to the domain’s HTTP Server to redirect to the HTTPS Server whenever a browser accesses that directory.

Note: Apache uses a .htaccess file to allow users to configure such actions as redirecting or password-protecting directories. Nginx does not use .htaccess; instead, we will put such directives in the server block files.

Create a secure test directory by running these commands on the server:

# Create a secure test directory.
sudo mkdir /var/www/mydomain2/secure

# Create a secure test page.
sudo nano /var/www/mydomain2/secure/index.html
   # Input this content:
   This page is secure!

# Change owner to www-data (which Nginx threads run as) so Nginx can access.
sudo chown -R www-data:www-data /var/www/mydomain2/secure

Edit the domain’s server block file by running this command on the server:

sudo nano /etc/nginx/sites-available/domain2

Under the “domain2” server block file, in the “HTTP Section” at the top (not the “HTTPS Section” at the bottom), add these lines to do the redirect:

server {
        #listen   80; ## listen for ipv4; this line is default and implied
        #listen   [::]:80 default ipv6only=on; ## listen for ipv6

        # Redirect to port 443.
        # Please put this before location / block as
        # Nginx stops after seeing the first match.
        # Note: ^~ means match anything that starts with /secure/
        location ^~ /secure/ {
                rewrite ^ https://$host$request_uri permanent;

        location / {

Restart Nginx so the changes above can take effect.

sudo service nginx reload

Test by browsing to “http://mydomain2/secure/” and the browser should redirect to “https://mydomain2/secure/”.

Password-Protect a Directory

By password-protecting a directory (aka requiring basic authentication), when a browser accesses that directory, the user will get a dialog asking for the user name and password. To get this functionality working, we will create a user and password file and configure the Nginx server block to require basic authentication based upon that file.

Note: Accessing a password-protected directory over HTTP would result in the user and password being sent in clear text by the browser to the server.

Create a protected test directory by running these commands on the server:

# Create a protected test directory.
sudo mkdir /var/www/mydomain2/protect

# Create a protected test page.
sudo nano /var/www/mydomain2/protect/index.html
   # Input this content:
   This page is password-protected!

# Change owner to www-data (which Nginx threads run as) so Nginx can access.
sudo chown -R www-data:www-data /var/www/mydomain2/protect

We will need a utility from Apache to create the user and password file. Run this command on the server to install and use it:

# Install htpasswd utility from Apache.
sudo apt-get install apache2-utils

# Create a user and password file using htpasswd.
sudo htpasswd -c /var/www/mydomain2/protect/.htpasswd myuser

# Add an additional user using htpasswd without "-c" create parameter.
sudo htpasswd /var/www/mydomain2/protect/.htpasswd myuser2

# Change owner to www-data (which Nginx threads run as) so Nginx can access.
sudo chown www-data:www-data /var/www/mydomain2/protect/.htpasswd

Note: If you move the “.htpasswd” file to another location (say, not under the domain’s document root), make sure that the “www-data” user or group can access it; otherwise, Nginx won’t be able to read it.

Edit the Nginx server block file by running this command on the server:

sudo nano /etc/nginx/sites-available/domain2

In the “domain2” server block file, in the “HTTP Section” at the top (not the “HTTPS Section” at the bottom), add these lines to password-protect the “/protect” directory:

server {
        #listen   80; ## listen for ipv4; this line is default and implied
        #listen   [::]:80 default ipv6only=on; ## listen for ipv6

        # Password-protect directory.
        # Please put this before location / block as
        # Nginx stops after seeing the first match.
        # Note: ^~ means match anything that starts with /protect/
        location ^~ /protect/ {
                auth_basic "Restricted"; # Enable Basic Authentication
                auth_basic_user_file /var/www/mydomain2/protect/.htpasswd;

        location / {

        # Uncomment this section to deny access to .ht files like .htpasswd
        # Recommend to copy this to the HTTPS server below also.
        location ~ /\.ht {
                deny all;


The “^~” in “location ^~ /protect/” above tells Nginx to match anything that starts with “/protect/”. This is necessary to ensure that all files and directories under “/protect/” are also password-protected. Because Nginx stops once it finds a match, it won’t process subsequent match directives, such as the PHP-FPM directive, and PHP scripts won’t execute. If you wish to run PHP scripts under the password-protected directory, you must copy the PHP-FPM directive (and any other directives) under the password-protected location directive like so:

server {

        # Password-protect directory.
        # Please put this before location / block as
        # Nginx stops after seeing the first match.
        # Note: ^~ means match anything that starts with /protect/
        location ^~ /protect/ {
                auth_basic "Restricted"; # Enable Basic Authentication
                auth_basic_user_file /var/www/mydomain2/protect/.htpasswd;

                # pass the PHP scripts to FPM-PHP
                location ~ \.php$ {
                        fastcgi_split_path_info ^(.+\.php)(/.+)$;
                        fastcgi_pass unix:/var/run/php5-fpm.sock;
                        fastcgi_index index.php;
                        include fastcgi_params;

                # deny access to .ht files like .htpasswd
                location ~ /\.ht {
                        deny all;

        # pass the PHP scripts to FPM-PHP
        location ~ \.php$ {

Restart Nginx so the changes above can take effect.

sudo service nginx reload

Test by browsing to “http://mydomain2/protect/” and the browser should prompt you to input a user and password.

Secure Mailman

To run Mailman under HTTPS/SSL, move the “location /cgi-bin/mailman” definition in the server block file, “/etc/nginx/sites-available/mydomain2”, from the HTTP server to the HTTPS server section.

You will also need to modify Mailman to use the HTTPS url:

# Edit Mailman's configuration
sudo nano /etc/mailman/
   # Change its default url pattern from 'http://%s/cgi-bin/mailman/' to:
   DEFAULT_URL_PATTERN = 'https://%s/cgi-bin/mailman/'

# Propagate the HTTPS URL pattern change to all the mailists
sudo /usr/lib/mailman/bin/withlist -l -a -r fix_url

Note: It is not necessary to restart the Mailman service for the changes above to take effect.

If you only want the default URL Pattern change to apply to a specific mailing list, like “”, use this command instead:

sudo /usr/lib/mailman/bin/withlist -l -r fix_url test -u

Take a Snapshot

DigitalOcean provides a web tool to take a snapshot image of the VPS. I can restore using that image or even create a duplicate VPS with it. Because my VPS is now working the way I need it to, it makes sense to take a snapshot at this time.

Unfortunately, performing a snapshot requires that I shutdown the VPS first. More unfortunate, the time required to take the snapshot varies from minutes to over an hour (more on this below). Worst, there is no way to cancel or abort the snapshot request. I have to wait until DigitalOcean’s system completes the snapshot request before my VPS is automatically restarted.

digitalocean_snapshot_stuckI did my first snapshot after getting WordPress working on the VPS. There was about 6GB of data (including the operating system) to make an image of. I shut down the VPS and submitted a snapshot request. The “Processing…” status with zero progress was what I saw for over one hour. During this time, my VPS and WordPress site was offline.

A little over an hour later, the status went from “Processing…” with zero progress to done in a split second. My VPS and WordPress site were back online. I think an hour to backup 6GB of data is excessive. DigitalOcean support agreed. Evidently, there was a backlog on the scheduler and requests were delayed. Because I couldn’t cancel the snapshot request, I had to wait for the backlog to clear in addition to however long it took to do the snapshot.

If I had known more about the snapshot feature, I would have opted to pay for the backup feature, which cost more but doesn’t require shutting down the VPS. Unfortunately, the backup feature can only be enabled during VPS creation so it is too late for me.

The recommended method to shutdown the VPS is to run this command:

# Following command equivalent to: sudo shutdown -h now
sudo poweroff

Update: I just did a snapshot and it only took 5 minutes this time.

See my followup post, Upgrade Ubuntu and LEMP on an Unmanaged VPS, to learn how to upgrade LEMP and Ubuntu to the latest versions.

Most info above derived from:

1 Comment

« Previous Entries