How to Manually Run Cron Tasks

To manually run cron tasks you can use the run-parts command in Linux.

So to run your cron-weekly, for example, to test that a fix you just made runs without error (this is what I just did, in fact)

run-parts /etc/cron-weekly

run-parts will run all the executables in a directory (you must point at the directory). So if you have several files in cron-weekly to run, you can’t just point to one of the files.

You may run into environmental differences running the script as a different user than the cron test runs at, so you can run as that user if needed. You need to be aware this is a quick and simple way of testing part of the process but it doesn’t do a perfect job of testing if it works as a cron task. But it will let you catch some failures quickly and fix them in time for the actual cron task to run. So do check that the everything works after the real cron job runs.

This is just the kind of thing I said I would put in this blog. Simple stuff but things I forget – so I put it here to remember and maybe help out others, like me, that need really basic tips.

If you have a cron task item (or have setup the whole task this way) that is just a script and you just want to test that 1 item you may run the script directly. For example (for a Linux shell script):

sh /etc/cron.weekly/your_crontask_script.sh

Related: Updates Needed When Upgrading from Apache 2.2 to 2.4Rsync to copy Files Between Servers and ComputersBash Profile Adjustments for Scrolling HistoryChecklist: Setting Up a New Domain on VPS

Updates Needed When Upgrading from Apache 2.2 to 2.4

I updated from Ubuntu 12.04 to Ubuntu 14.04 on a virtual private server.

When you do that update, Apache is updated from 2.2 to 2.4. Certain changes mean that until you update the site-available configuration files no web sites will work.

The quick checklist of what you have to do for each configuration

  1. update the file to add or edit the directory details (where /var/www/curiouscatnetwork.com/public_html/ is the path to the website files on your server)
    <Directory /var/www/curiouscatnetwork.com/public_html/>
       Require all granted
    </Directory>

    Remember if you have details needed for WordPress those instructions need to remain. I didn’t have a Directory area listed on my non-Wordpress sites.

  2. The files now need a .conf extension so move the rename/move the files to the new name
    sudo mv curiouscatnetwork.com curiouscatnetwork.com.conf
  3. Enable the new configuration
    sudo a2ensite curiouscatnetwork.com.conf
  4. Reload apache to reload the new configuration
    sudo service apache2 reload

More details from Linode.

The upgrade to Ubuntu 14.04 LTS was interrupted (I quit a window using the wrong command – oops). Luckily it wasn’t a big deal. I was able to delete the lock file.

sudo rm /var/lib/dpkg/lock

Then trying to update again

sudo apt-get dist-upgrade

gave a message telling me what command to use to have the upgrade continue.

sudo dpkg --configure -a

which seemed to work fine.

Related: Bash Profile Adjustments, Scrolling HistoryChecklist: Setting Up a New Domain on VPSModSecurity: Adjustments for WordPress

Rsync to copy Files Between Servers and Computers

I found rsync when I wanted to use scp to copy files to a server but not overwrite files already there. Rsync is actually more efficient no matter what (it seems) but it is really great if there are a bunch of duplicate files (Rsync will just skip them).

To copy files from your current computer to a server:

rsync -azv -r ssh [directory] [username]@remotehost:[location]

$ sudo rsync -azv -e ssh directory-on-my-computer/ john@example.com:~/files/go/here

a = archive mode
z = compress file data
v = verbose
r = recurse through subdirectories and copy all of them

Compressing file data saves bandwidth so if that is an issue it is another big win over scp. And in my reading it seems rsync can restart a broken file transfer in process (while scp you have to redo the whole file transfer).

To copy from the server to your computer just reverse the order of the locations. And you can even just put in two addresses not on your current computer and copy between then.

As a reminder, I realize this blog is made up of stuff that is obvious to a large number of people. It is really aimed at me (so I can quickly find what I found before), and to a lessor extent others like me (who use cli some but are not system administrators or programmers to any significant extent).

Related: Making Sure You Don’t Run Out of Space on Your VPSBash Profile Adjustments (Scrolling History)Customizing the Command Prompt for Terminal in Ubuntu

Top with Better Display Options

Scout Realtime is a Ruby Gem that allows you to view top in the browser. One huge advantage is to view charts of activity over time.

Scout Realtime is open source and free.

Related: System Monitoring Tools for VPSBasic MySQL Performance MonitoringKeeping Your Hosted Ubuntu Web Server Software Up to Date

Bash Profile Adjustments – Scrolling History

Your bash profile lets you make edits that let you design how the cli works for you. One of my favorite edits is to let me set the scrolling back through my history be based on what I start typing. So, for example, I can type

cd

and then use the up-arrow key and it scrolls through my cli history based only on those that start with cd

To add this to your .profile file (or put it in your .bash_rc file):

bind '"\e[A"':history-search-backward
bind '"\e[B"':history-search-forward

Related: Customizing the Command Prompt for Terminal in UbuntuTesting if Ubuntu server email is working with cliLinux/Ubuntu File and Directory Permissions

Checklist: Setting Up a New Domain on VPS

Two great hosts for Ruby on Rails are Slicehost and Linode. With these hosts you fully manager your virtual private server, installing the operating system, modifying apache (on Ning…), etc.. I use Ubuntu as the operating system and Apache as the web server.

If you are moving a domain from elsewhere it can be wise to reduce the TTL time to say 5 minutes a few days before you make the switch. This is make the change propagate across the internet more quickly.

  1. And DNS entry on your profile (login to your, for example, Linode account)
  2. Add a new file for /etc/apache2/sites-available/your_site_name.com
  3. sudo nano /etc/apache2/sites-available/example.com

    The file would look something like this: with your ip in place of 8.8.8.8

    ServerAdmin is the address Apache will use to send error messages to. Using gmail and the + option lets you use one gmail account and just use rules to filter all your sites.

  4. create the directories needed on your server
  5. enable the site (for apache)
    sudo a2ensite example.com
  6. You should see the file you created /etc/apache2/sites-available/example.com now also at /etc/apache2/sites-enabled/example.com

  7. Test the site out to make sure the setup is working properly. Create a index.html page and just verify the page is displayed. Change your local hosts file to point to your server IP address for the new domain you created. If not, take steps to get this to work, before continuing with the rest of the checklist.
  8. copy over the site – if you are moving the site from elsewhere
  9. remember to move the database over, if the site relies on a database
  10. restart apache
    sudo /etc/init.d/apache2 restart
  11. You can test the site out, before updating the DNS, by changing your local hosts file to point to your server IP address for the new domain you created.

    Resources: Install the Apache 2 Web Server on Ubuntu 10.04 LTS (Lucid)

Replace Text in Files Recursively (Linux)

To replace text in all the files in a directory, recursively, you can use grep.

sudo grep -rl texttoreplace /somedirectory/ | sudo xargs sed -i ‘s/repacethis/withthis/g’

sudo grep -rl oldtext /example/directory/ | sudo xargs sed -i 's/oldtext/newtext/g'

The g on the end, lets it replace all the instances of the text in each file.

More ubuntu and linux tips and code samples

Replacing a Host Key

Host keys are used to security log into remote servers (such as Virtual Private Servers – VPS). With Ubuntu if you are using host keys to sign into servers securely and have asked for strict checking, if you make a change (such as rebuilding your VPS) the host key will change and you cannot login and will get a message like:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is

Please contact your system administrator.

RSA host key for 128.0.0.128 has changed and you have requested strict checking.
Host key verification failed.

if that happens you need to remove your local host key. Then you can sign back in and you will be able to save a new copy of the host key. If you don’t know why the key has changed you should figure that out first as it maybe be an indication of an important security problem. To remove you local key, you can use ssh-keygen -R [ip address of server with the bad key] for example: ssh-keygen -R 128.0.0.128

Then when you try to sign in you will get

The authenticity of host '128.0.0.128 (128.0.0.128)' can't be established.

RSA key fingerprint is ed:...:ea.
Are you sure you want to continue connecting (yes/no)?

And if you know why (such as you made changes to the server) you can say yes and connect and save the new known host key.

Managing Users in Ubuntu

add a user: sudo adduser [newuser]

sudo adduser john

then give password, and setup home folder,when prompted

edit the list of super users

visudo

list users

cat /etc/passwd

change password of a user – sudo passwd [username]

sudo passwd john

to change your password you can just use sudo passwd.

delete user

sudo deluser username

This will not delete the home folder (and subfolder) those must be dealt with separately. A new user created with the old user name would have access to those files.

temporarily lock a user account – Simply locking a user account will not prevent a user from logging into your server remotely if they have previously set up RSA public key authentication.

sudo passwd -l username

To unlock the account

sudo passwd -u username

Groups

Groups are used in to control permissions (see file permissions)
add user to a group
usermod -G [group-name] [username]

usermod -G basketball john

using the -G switch ads the group as a supplemental group. Using -g would make the group that users primary group.

see what groups a user is in

id john

add a new group: groupadd [new_group_name]

groupadd ruby_developers

Linux/Ubuntu File and Directory Permissions

Linux (and therefore Ubuntu) has file permissions on each file and directory for the owner, group and everyone else. Those permissions determine if the file can be viewed, executed or edited.

Only the owner of a file or directory (or a privileged user, root for example) may change its mode.

Ownership of a file

To change the ownership of the file or directory: chown new_owner_username directory

chown john public_html

to change the ownership of directory (and all the files and folders in the directory) and also the group: chown -R new_owner_username:new_groupname directory

chown -R john:developers public_html

to change the ownership of all the files in the current directory and also the group: chown -R new_owner_username:new_groupname *

chown john:developers *

File permissions

The easiest way to set Linux file permissions is using a 3 digit sequence. The first digit designates owner permission; the second, the group permission; and the third, everyone else’s permission.

Read = 4
Write = 2
Execute = 1

The digit is the sum of those. So if you want to grant only read permission you use 4; read and execute 5; read, write and execute = 7.

chmod 775 index.html

That will set the permissions on index.html so the owner, and a user in the group specified can read, write and execute the file and everyone else can read and execute.

chmod -R 755 public_html

That will set the permissions on files and directories (recursively through all subdirectories) so the owner can read, write and execute; members of the group and everyone else can read and execute (but not write).

ls - l

That will give you a list of files and directories, in a directory, with the owner and group settings and the permissions for all 3 (those 2 and everyone else), which will look something like:

-rw-r--r-- 1 root developers   397 2008-05-25 20:33 index.html
-rw-r--r-- 1 mary developers  9177 2010-05-02 22:18 unix_file_permissions.html
...

The lines start with the permissions for the owner, group and then everyone else. There are 9 total characters, 3 for each. Taking the top line above:

rw-r--r--
rw-  (means the owner has read and write permission but not execute)
r--  (means the group has only read permission)
r--  (means everyone else has only read permission)

The next column tells you the number of hard links to the file or directory. Then column tells you the owner, then the group. Then the byte size of the file, the date it was last change and then the file name.

root
means the username of this file is named root

developers
group (means those users in the group named developers have the group permissions indicated)

Related: Ubuntu command line interface syntax examples