Locking and Unlocking a User in Ubuntu

To lock a user in Ubuntu sudo usermod –expiredate 1 [username] Substitute the user’s username where username is shown below.

sudo usermod --expiredate 1 username

This also can be shortened to

sudo usermod -e 1 username

To unlock a user

sudo usermod -e -1 username

This will disable the user both from accessing via password and from accessing via a private key.

You can also expire a user at a future date using

sudo usermod -e YYYY-MM-DD username

To expire a password and force a user to enter a new password

sudo passwd -e YYYY-MM-DD username

Related: Managing Users in UbuntuReset Root Password on MySQL DatabaseDon’t Copy-Paste Directly from Website to Terminal

How to Manually Run Cron Tasks

To manually run cron tasks you can use the run-parts command in Linux.

So to run your cron-weekly, for example, to test that a fix you just made runs without error (this is what I just did, in fact)

run-parts /etc/cron-weekly

run-parts will run all the executables in a directory (you must point at the directory). So if you have several files in cron-weekly to run, you can’t just point to one of the files.

You may run into environmental differences running the script as a different user than the cron test runs at, so you can run as that user if needed. You need to be aware this is a quick and simple way of testing part of the process but it doesn’t do a perfect job of testing if it works as a cron task. But it will let you catch some failures quickly and fix them in time for the actual cron task to run. So do check that the everything works after the real cron job runs.

This is just the kind of thing I said I would put in this blog. Simple stuff but things I forget – so I put it here to remember and maybe help out others, like me, that need really basic tips.

If you have a cron task item (or have setup the whole task this way) that is just a script and you just want to test that 1 item you may run the script directly. For example (for a Linux shell script):

sh /etc/cron.weekly/your_crontask_script.sh

Related: Updates Needed When Upgrading from Apache 2.2 to 2.4Rsync to copy Files Between Servers and ComputersBash Profile Adjustments for Scrolling HistoryChecklist: Setting Up a New Domain on VPS

Updates Needed When Upgrading from Apache 2.2 to 2.4

I updated from Ubuntu 12.04 to Ubuntu 14.04 on a virtual private server.

When you do that update, Apache is updated from 2.2 to 2.4. Certain changes mean that until you update the site-available configuration files no web sites will work.

The quick checklist of what you have to do for each configuration

  1. update the file to add or edit the directory details (where /var/www/curiouscatnetwork.com/public_html/ is the path to the website files on your server)
    <Directory /var/www/curiouscatnetwork.com/public_html/>
       Require all granted
    </Directory>

    Remember if you have details needed for WordPress those instructions need to remain. I didn’t have a Directory area listed on my non-Wordpress sites.

  2. The files now need a .conf extension so move the rename/move the files to the new name
    sudo mv curiouscatnetwork.com curiouscatnetwork.com.conf
  3. Enable the new configuration
    sudo a2ensite curiouscatnetwork.com.conf
  4. Reload apache to reload the new configuration
    sudo service apache2 reload

More details from Linode.

The upgrade to Ubuntu 14.04 LTS was interrupted (I quit a window using the wrong command – oops). Luckily it wasn’t a big deal. I was able to delete the lock file.

sudo rm /var/lib/dpkg/lock

Then trying to update again

sudo apt-get dist-upgrade

gave a message telling me what command to use to have the upgrade continue.

sudo dpkg --configure -a

which seemed to work fine.

Related: Bash Profile Adjustments, Scrolling HistoryChecklist: Setting Up a New Domain on VPSModSecurity: Adjustments for WordPress

Rsync to copy Files Between Servers and Computers

I found rsync when I wanted to use scp to copy files to a server but not overwrite files already there. Rsync is actually more efficient no matter what (it seems) but it is really great if there are a bunch of duplicate files (Rsync will just skip them).

To copy files from your current computer to a server:

rsync -azv -r ssh [directory] [username]@remotehost:[location]

$ sudo rsync -azv -e ssh directory-on-my-computer/ john@example.com:~/files/go/here

a = archive mode
z = compress file data
v = verbose
r = recurse through subdirectories and copy all of them

Compressing file data saves bandwidth so if that is an issue it is another big win over scp. And in my reading it seems rsync can restart a broken file transfer in process (while scp you have to redo the whole file transfer).

To copy from the server to your computer just reverse the order of the locations. And you can even just put in two addresses not on your current computer and copy between then.

As a reminder, I realize this blog is made up of stuff that is obvious to a large number of people. It is really aimed at me (so I can quickly find what I found before), and to a lessor extent others like me (who use cli some but are not system administrators or programmers to any significant extent).

Related: Making Sure You Don’t Run Out of Space on Your VPSBash Profile Adjustments (Scrolling History)Customizing the Command Prompt for Terminal in Ubuntu

Turn It Off and On Again

I had a WordPress blog lose the database connection. As I tried to troubleshoot it I rapidly got to the point of thinking that maybe just rebooting the server would fix things – since no changes had been made that should cause the database connection to be lost. But also I figured I couldn’t be so lucky that such a simple thing would work.

I tried to log into the database using the cli and it was failing. MySQL was still running – according to top. I have automated security updates setup for the VPS server running Ubuntu. My thought now is those updates somehow messed things up to the extent the server database connections somehow wasn’t working. I sure hoped that was the case.

I rebooted and hoped.

After a bit I was able to see that it was working.

When I went to reboot I noticed I hadn’t rebooted in a long time. My guess is I might have the automated security updates setup wrong on this server (maybe it is installing more than just the security updates), I’ll check. I have not had this problem before with other servers, obviously breaking a database in this way would cause lots of problems on production machines so I tend to think it is more likely I messed something up than this is a likely outcome when using automated security updates (but such things are possible which is why I think places with full time system admins and important servers rely on manual updates with professionals watching everything to be sure nothing obvious breaks).

My feeling is the longer you go with not rebooting the server the more likely some issues are to crop up (but also every reboot is more like to result in some broken thing, right then, than doing nothing so it is a tradeoff). And I could be wrong in that feeling, it is just what I guess without much evidence to support my guess.

Anyway I was very happy turning it off and on again worked. Honestly I went ahead and updated the server before I rebooted, but my guess is just turning it off and on again would have worked.

Related: Making Sure You Don’t Run Out of Space on Your VPSBasic MySQL Performance MonitoringWordPress error: Image could not be processed. Please go back and try again.

Making Sure You Don’t Run Out of Space on Your VPS

Over the years I have had web sites I work on go down because the server ran out of space (when I wasn’t responsible for the system administration). Well today I had it happen for some sites on one of my servers.

I have a remote monitoring service so I was notified when the sites were unavailable. Looking in the error logs I could see I ran out of space, which is a fairly lame error to make, I think. I cleared up some space quickly and things were in working order again.

A few tips from this experience. To see data on space used and available space from the command line (linux/ubuntu) use the df command.

df

You could check that occasionally to see if anything strange is going on. But it is better to use something like Nagios to monitor the server and provide notice if usage goes past certain points is wise (75%, 80%…).

I found a nice way to find large files (that may be possible to delete to free up space)

sudo find . -mount -ls | awk '{print $7, $11}' | sort -rn > large_file_list

If you get an error with the file creation, you can log in as root (su) or create the file first and then it can overwrite the existing file.

It is smart to have log rotate setup for logs. I didn’t have it setup for every log. To create a file to keep track of all your logs for domains you can do the following. Create a file in the logrotate directory so it is run per your settings.

sudo vim /etc/logrotate.d/virtualhosts

This article from Slicehost provide details on options, a simple setup is to include text such as this in the file created in the line above.

/var/www/domain1.com/logs/*log /var/www/domain2.com/logs/*log {
  rotate 14
  daily
  compress
  delaycompress
  sharedscripts
  postrotate
    /usr/sbin/apache2ctl graceful > /dev/null
  endscript
}

Related: System Monitoring Tools for Ubuntu VPSPhusion Passenger Tips and Troubleshooting IdeasChecklist: Moving WordPress site to a New Host

Continue reading

ModSecurity – Adjustments for WordPress

How to secure an Ubuntu 12.04 LTS server – Part 1 The Basics is a great resource to secure a VPS.

Mod_Security is a web application firewall. I found a couple things to add to my servers.

The same site includes a very good guide to installing it (you also may well want to whitelist Googlebot, instructions in the link). However it blocked my access to one of my sites. You end up just getting the message:

“Forbidden

You don’t have permission to access / on this server.”

There are tips on some edits to include when using WordPress on an Apache server with modsecurity.

One simple action is to include

SecRule SERVER_NAME “[your-domain.com]” phase:1,nolog,allow,ctl:ruleEngine=off

SecRule SERVER_NAME "code.curiouscatnetwork.com" phase:1,nolog,allow,ctl:ruleEngine=off

on the bottom of modsecurity.conf which is found /etc/modsecurity (for me, on Ubunutu 12.04). Then restart Apache

sudo service apache2 restart

and see if the problem goes away. If it does then you have a very good indication modsecurity was blocking access and can continue to narrow the scope of the problem by adding the WordPress whitelist rules in the link above.

Another note, service apache2 start, failed in a non-obvious way to me anyway. For me if I use sudo it works fine. If I don’t it gives odd errors which lead me on a 10 minute wild goose chase before remembering to try sudo.

Related: Upgrading VPS Web Server from Ubuntu 10.04 to 12.04Keeping Your Hosted Ubuntu Web Server Software Up to DateLinux/Ubuntu File and Directory Permissions

Upgrading VPS Web Server from Ubuntu 10.04 to 12.04

I am not a “real” system administrator but I manager sever VPS servers for my own web sites. This is what I did to update from Ubunutu LTS 10.04 to Ubunutu VPS 12.04.

  1. I already have all the code and databases backed up, off the server, regularly. I backed up various things to another machine: sites-available files, cron directories, .bashrc, .profile, keys.
  2. I tried running the preferred upgrade command
    do-release-upgrade

    But got the message

    -bash: do-release-upgrade: command not found

    Then I ran

    sudo apt-get install update-manager-core

    Then I was able to run

    sudo do-release-upgrade
  3. Which resulted in this somewhat scary message

    This session appears to be running under ssh. It is not recommended to perform a upgrade over ssh currently because in case of failure it is harder to recover.

    If you continue, an additional ssh daemon will be started at port ‘9004’.
    Do you want to continue?

    I look around on the web for advice. What’s the risk of upgrading over SSH? seems a good recap. I then continued. I then got this message

    To make recovery in case of failure easier, an additional sshd will be started on port ‘1022’. If anything goes wrong with the running ssh you can still connect to the additional one.
    If you run a firewall, you may need to temporarily open this port. As this is potentially dangerous it’s not done automatically. You can open the port with e.g.:
    ‘iptables -I INPUT -p tcp –dport 1022 -j ACCEPT’

  4. 12 minutes after the upgrade started, the download was complete and I got a message to decide if I wanted to manually approve every restart required or just automatically approve them all. I chose the option to have all restart automatically.
  5. Then lots of files were installed and I was asked about various files; where I (or some script) had changed (or deleted) the default file and now the upgrade wanted to replace the existing file. I had to guess what to do in those cases. They let you look at the diff between your existing file and the proposed overwrite. I think I would like it if they default behavior was to create a backup of the file in that same directory (you choosing whether to set your file or the updated file as active).

The entire process took exactly 30 minutes, with the system restarting in Ubuntu 12.04.

Thankfully the upgrade seems to have gone without causing any problems. This is normally the case. But, even with a very small likelihood of encountering issues it is worrisome as those issues might pose some serious problems. Especially for someone with very limited system administration ability.

I am extremely thankful for all the programmers that created the code to make this process so straightforward and reliable.

lsb_release -a

lets you see the current Ubunutu version you are running.

When I updated a server that had Ruby on Rails applications everything almost worked fine. Passenger was unable to load the application talking about

libmysqlclient_r.so.16: cannot open shared object file: No such file or directory – /usr/local/rvm/gems/ruby-1.9.2-p290/gems/mysql2-0.3.11/lib/mysql2/mysql2.so

I was able to find the solution to that issue was to uninstall the mysql2 gem and then run bundle update: which worked great. The uninstall would work for me unless I was actually root (sudo didn’t work), so I used su to login as root and then

gem uninstall mysql2

Then I went back to being my other user ran bundle update

bundle update rake

Then everything was working fine.

Related: Keeping Your Hosted Ubuntu Web Server Software Up to DateAdding a Key to Your Server for SSH LoginChecklist: Setting Up a New Domain on VPS

Set the Web Server to Use the Domain Without www

The main reason to bother with this is to help search rankings. Sub-domains (for example, www.curiouscat.com and curiouscat.com) are treated as separate websites even if you have entirely the same content displayed for both. If 20% of the links to your site use the www and 80% don’t then your sites ranking by search engines is less than it would be if it was just treated as one site.

You can set the domain to use in Google webmaster tools. But that doesn’t do anything for all the other search engines. Also if you have both, some reports some will keep statistics separately for the non-www and www domain (Google Adsense does this, for example).

Using virtual hosts file (sites-enabled)

For Apache you can place the following code in your virtual hosts file (in the sites-available directory under Apache).

Replace curiouscat.com with your domain name.

<Directory /srv/www/curiouscat.com/public_html/>
  RewriteEngine On
  RewriteCond %{HTTP_HOST} ^www.curiouscat.com$ [NC]    
  RewriteRule ^(.*)$ http://curiouscat.com/$1 [R=301,L]
</Directory>

Then you need to capture the update and reload Apache.

Disable the site (in order to enable it with the updates)

sudo a2dissite curiouscat.com

Enable the site

sudo a2ensite curiouscat.com

Reload Apache (new way – Ubuntu 12.04)

sudo service apache2 reload

Old way to reload Apache was > sudo /etc/init.d/apache2 reload

If when you try to reload you get this error message:

“Invalid command ‘RewriteEngine’, perhaps misspelled or defined by a module not included in the server configuration” then enable modrewrite on apache:

sudo a2enmod rewrite

You should then be told to restart apache

sudo /etc/init.d/apache2 restart

Using .htaccess

Or you can include the following in your .htaccess file

  RewriteEngine On
  RewriteCond %{HTTP_HOST} ^www.curiouscat.com$ [NC]    
  RewriteRule ^(.*)$ http://curiouscat.com/$1 [R=301,L]

Related: Checklist for Setting Up a New Domain on VPSPhusion Passenger Tips and Troubleshooting Ideas

If you wanted to force www to be used instead just flip the regular expression around:

<Directory /srv/www/curiouscat.com/public_html/>
  RewriteEngine On
  RewriteCond %{HTTP_HOST} ^curiouscat.com$ [NC]    
  RewriteRule ^(.*)$ http://www.curiouscat.com/$1 [R=301,L]
</Directory>

Customizing the Command Prompt for Terminal in Ubuntu

To customize the look of the command prompt for Terminal in Ubuntu you can insert code into .bashrc (if you are having trouble with updates in .bashrc not working you may want to see if it is set in .profile – thus overriding your .bashrc). Those files are found in your user directory /home/[username]

echo $PS1

will show you the current settings. You can make a change directly from the command line but it will only work for that session. For example:

PS1="\d\w $ "

This is the code I have in my .bashrc related to the terminal prompt

# set variable identifying the chroot you work in (used in the prompt below)
if [ -z "$debian_chroot" ] && [ -r /etc/debian_chroot ]; then
    debian_chroot=$(cat /etc/debian_chroot)
fi

# set a fancy prompt (non-color, unless we know we "want" color)
case "$TERM" in
    xterm-color) color_prompt=yes;;
esac

if [ -n "$force_color_prompt" ]; then
    if [ -x /usr/bin/tput ] && tput setaf 1 >&/dev/null; then
        color_prompt=yes
    else
        color_prompt=
    fi
fi

if [ "$color_prompt" = yes ]; then
    PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '
else
    PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
fi
unset color_prompt force_color_prompt