Tag Archives: hosting

Web Site Performance Monitoring Tools: Load Testing

Posted on by

Determining the performance of web site and web applications is important. In order to create sites users want to visit they need compelling content but they also need to provide the performance users expect.

One of the wonderful aspects of creating web applications is the availability of software as a service solutions to meet your needs. These include monitoring tools and tools for testing the performance of your sites.

Image of LoadView testing - $9.99/mo + load test costs of each test

In this post I look at load testing using LoadView Stress Testing service. Tests are not cheap. The cost is $9.99 a month for an account plus the costs of each stress test you run. The next image shows the breakdown of a small stress test on this blog which cost $8.25 to

image of a small load test cost breakdown ($8.25 total)

Continue reading →

VPS, Cloud and Server Colocation Options

Posted on by

Finding the right place to host your content is important. Thankfully their are several excellent providers. For virtual private servers (one server shared with multiple virtual servers) Linode and DigitalOcean. And there are lots of good choices but those two are widely appreciated for excellent service at a good price.

AWS ec2 (the Amazon elastic cloud) is not great for minimal hosting in my opinion – it adds extra complexity and is likely more expensive. But it is a great solution when you have the resources to manage it and you have significantly variable demand. Because of the ability to add capacity on the fly as you need it you can maintain a low baseline and add capacity only as needed and drop that extra capacity as soon as it isn’t needed.

Rackspace is another good option for hosting. Rackspace and AWS are often used for very large applications and sites but Linode and Digital Ocean also can serve those needs and provide similar options to add capacity on the fly.

All of these options require you to manage your server (which may well be a virtual server – that is just a portion of a actual physical server that you control).

Rackspace also offers co-location where your physical server is put in their network operation center with electricity; cooling; network and internet connections; and physical security managed by them and the server managed by you.

As colocation has evolved what is included and to what level things like physical security and redundancy are dealt with have evolved. It has become quite complex to understand all the options for those organizations that need more than a simple virtual private server. As often happens when their is a business need, people offer solutions. And there are companies that specialize in helping you find the best colocation options for your needs.

Today the cloud options have led many organizations to eliminate (or greatly reduce) there own network operations centers and colocation needs. But cloud options are not always the right choice. And for some needs cloud options are not appropriate yet (mainly due to security or legal issues stemming from security concerns).

Managing your own servers with a colocation arrangement can be significantly cheaper than cloud hosting options (especially if you don’t need to massively increase capacity to deal with short term bursts of demand). Of course, technology continues to change so quickly it is hard to predict what the future will bring.

Service quality is absolutely critical for colocation. While saving money is important, the reason colocation was selected (over virtual private servers or the cloud) is normally how critical the function was. Using experts to help sort through the options and assure the quality of service of provides is wise.

Related: Keeping Your Hosted Ubuntu Web Server Software Up to Date – Checklist for Setting Up a New Domain on VPS – System Monitoring Tools for VPS

Turn It Off and On Again

Posted on by

I had a WordPress blog lose the database connection. As I tried to troubleshoot it I rapidly got to the point of thinking that maybe just rebooting the server would fix things – since no changes had been made that should cause the database connection to be lost. But also I figured I couldn’t be so lucky that such a simple thing would work.

I tried to log into the database using the cli and it was failing. MySQL was still running – according to top. I have automated security updates setup for the VPS server running Ubuntu. My thought now is those updates somehow messed things up to the extent the server database connections somehow wasn’t working. I sure hoped that was the case.

I rebooted and hoped.

After a bit I was able to see that it was working.

When I went to reboot I noticed I hadn’t rebooted in a long time. My guess is I might have the automated security updates setup wrong on this server (maybe it is installing more than just the security updates), I’ll check. I have not had this problem before with other servers, obviously breaking a database in this way would cause lots of problems on production machines so I tend to think it is more likely I messed something up than this is a likely outcome when using automated security updates (but such things are possible which is why I think places with full time system admins and important servers rely on manual updates with professionals watching everything to be sure nothing obvious breaks).

My feeling is the longer you go with not rebooting the server the more likely some issues are to crop up (but also every reboot is more like to result in some broken thing, right then, than doing nothing so it is a tradeoff). And I could be wrong in that feeling, it is just what I guess without much evidence to support my guess.

Anyway I was very happy turning it off and on again worked. Honestly I went ahead and updated the server before I rebooted, but my guess is just turning it off and on again would have worked.

Related: Making Sure You Donโ€™t Run Out of Space on Your VPS – Basic MySQL Performance Monitoring – WordPress error: Image could not be processed. Please go back and try again.

Top with Better Display Options

Posted on by

Scout Realtime is a Ruby Gem that allows you to view top in the browser. One huge advantage is to view charts of activity over time.

Scout Realtime is open source and free.

Related: System Monitoring Tools for VPS – Basic MySQL Performance Monitoring – Keeping Your Hosted Ubuntu Web Server Software Up to Date

Checklist: Setting Up a New Domain on VPS

Posted on by

Two great hosts for Ruby on Rails are Slicehost and Linode. With these hosts you fully manager your virtual private server, installing the operating system, modifying apache (on Ning…), etc.. I use Ubuntu as the operating system and Apache as the web server.

If you are moving a domain from elsewhere it can be wise to reduce the TTL time to say 5 minutes a few days before you make the switch. This is make the change propagate across the internet more quickly.

  1. And DNS entry on your profile (login to your, for example, Linode account)
  2. Add a new file for /etc/apache2/sites-available/your_site_name.com

    3. [bash]sudo nano /etc/apache2/sites-available/example.com[/bash]

    The file would look something like this: with your ip in place of 8.8.8.8

    ServerAdmin is the address Apache will use to send error messages to. Using gmail and the + option lets you use one gmail account and just use rules to filter all your sites.

  3. create the directories needed on your server
  4. enable the site (for apache)

    [bash]sudo a2ensite example.com[/bash]

    5. You should see the file you created /etc/apache2/sites-available/example.com now also at /etc/apache2/sites-enabled/example.com

  5. Test the site out to make sure the setup is working properly. Create a index.html page and just verify the page is displayed. Change your local hosts file to point to your server IP address for the new domain you created. If not, take steps to get this to work, before continuing with the rest of the checklist.
  6. copy over the site – if you are moving the site from elsewhere
  7. remember to move the database over, if the site relies on a database
  8. restart apache
    [bash]sudo /etc/init.d/apache2 restart[/bash]

    9. You can test the site out, before updating the DNS, by changing your local hosts file to point to your server IP address for the new domain you created.

    Resources: Install the Apache 2 Web Server on Ubuntu 10.04 LTS (Lucid)

Replacing a Host Key

Posted on by

Host keys are used to security log into remote servers (such as Virtual Private Servers – VPS). With Ubuntu if you are using host keys to sign into servers securely and have asked for strict checking, if you make a change (such as rebuilding your VPS) the host key will change and you cannot login and will get a message like:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
…
Please contact your system administrator.
…
RSA host key for 128.0.0.128 has changed and you have requested strict checking.
Host key verification failed.

if that happens you need to remove your local host key. Then you can sign back in and you will be able to save a new copy of the host key. If you don’t know why the key has changed you should figure that out first as it maybe be an indication of an important security problem. To remove you local key, you can use ssh-keygen -R [ip address of server with the bad key] for example: ssh-keygen -R 128.0.0.128

Then when you try to sign in you will get

The authenticity of host '128.0.0.128 (128.0.0.128)' can't be established.

RSA key fingerprint is ed:...:ea.
Are you sure you want to continue connecting (yes/no)?

And if you know why (such as you made changes to the server) you can say yes and connect and save the new known host key.

Basic MySQL Performance Monitoring

Posted on by

Basic MySQL Performance Monitoring

regular Ubuntu cli tools

  • mysqladmin status – mysqladmin status -uroot -p

MySQL command line interface tools

[bash]mysql -uroot -p[/bash] to open the command line.

[sql]SHOW GLOBAL STATUS;
SHOW ENGINE INNODB STATUS;
SHOW PROCESSLIST;
SHOW GLOBAL STATUS LIKE ‘Qcache_%’;[/sql]

cli tools

  • mytop – top for MySQL. Install using: sudo apt-get install mytop (assuming Ubuntu operating system). There is a very useful setting file that can be used to set parameters instead of having to include them in each command. Save the file as ~/.mytop.
  • MySQLTuner – provides suggestions on performance improvements and my.cnf settings by analyzing data on your mysql database server.

Setting considerations

  • If Open_tables (SHOW GLOBAL STATUS will show this) is equal to your [bash]table_cache size[/bash] (set in /etc/mysql/my.cnf) that means it is being capped by your setting. The more MySQL has to read the table from disk the more IO and slower response, so if you have available RAM increasing the table_cache size may well make a big difference.
  • Key_reads/Key_read_request ratio should normally be < 0.01 (per MySQL manual, this means that nearly all key requests are taken from RAM). You can get both values using SHOW GLOBAL STATUS and then calculate the ratio. If the ratio is too high, consider increasing the key_buffer (in /etc/mysql/my.sql).
  • key_writes/key_writes_request should normally be near 1 (per MySQL manual)

Phusion Passenger Tips and Troubleshooting Ideas

Posted on by

Some tips and troubleshooting ideas for Phusion Passenger

Phusion Passenger manages resources for rails applications – spawning new instances as needed, etc..

  • [bash]passenger-status[/bash] – provide the status of passenger rails processes

Configuring Phusion Passenger

Add lines to /etc/apache2/apache2.conf to change the default settings

  • [bash]PassengerMaxPoolSize 10[/bash] – maximum number of total rails application instances, the default is 6
  • PassengerMaxInstancesPerApp 5 – sets the maximum pool size for any 1 rails application to 10 instances (default is no limit).
  • [bash]PassengerUseGlobalQueue ON[/bash] – sets globaly queing on, it is off by default. You want globaly queuing on if your requests have large differences in response times (slow and fast responses).

Related: Passenger documentation

Troubleshooting

If you try [bash]sudo passenger-status[/bash] and get something like
*** ERROR: Cannot query status for Passenger instance 2280:
Connection refused – /tmp/passenger.2280/info/status.socket
Restarting (not reloading) apache [bash]sudo /etc/init.d/apache2 restart[/bash] may fix the problem.

System Monitoring Tools for VPS

Posted on by

Tools for monitoring performance and troubleshooting Ubuntu VPS web servers

  • Munin – graphs of system resources over time. Very nice. Can be a bit difficult to setup.
  • top – system stats
  • iotop – like top, but for io stats. Install [bash]sudo apt-get install iotop[/bash] Useful setup [bash]iotop -b -o -d 30 -t[/bash] -b (batch – so you can keep a running tally of results) -o (only those processes with io) -d (delay and seconds – how often to print out stats) -t (include time in printout)
  • vmstat – stats on memory, io, swap, cpu and system. Example: [bash]vmstat 10[/bash] (prints out stats every ten seconds.
  • iostat

Error logs

  • [bash]sudo nano /var/log/apache2/error.log[/bash]

Apache web server access log statistics

  • Webalizer – [bash]sudo apt-get install webalizer[/bash] GeoIP is required for webalizer [bash]sudo apt-get install geoip-bin[/bash] detailed instructions

Checklist: Moving WordPress site to a New Host

Posted on by

Checklist for moving an existing WordPress site to a new web host

A checklist for moving an existing WordPress site to a new VPS web host, when you have full admin rights over the server.

  1. Set DNS TTL’s down to 5 minutes (a few days prior to the move). This will allow the nameserver update to propagate more quickly.
  2. Set up new domain on the new host (Checklist for setting up a new domain on your VPS)
  3. Install WordPress on new host
  4. Copy old content directory to new WordPress host
  5. Copy old database to new host (how to use secure copy to copy database to new server)
  6. Update wp-config on new host
  7. Enable mod_rewrite in Apache on the new server. From the command line:

    8. a2enmod rewrite

  8. Restart apache
    /etc/init.d/apache2 restart
  9. Test that everything works (you can change your host file to test things out easily)
  10. Update registrar to point domain to the nameservers for the new host

Related: create a new database and run .sql file – Display text based on if it is the WordPress blog home page

If WordPress is up to date you could also just copy over everything for steps 1 and 2. I am using this for several sites I have had for years, I figure starting with a clean install of WordPress is a good idea, but it is not necessary.