Don’t Assume All Web Users Have a Fiber-like Connection

I have pointed out for more than a decade how poorly many web pages are coded. They often assume a very low latency connection and without it the user experience is poor.

Most of the web really sucks if you have a slow connection

The flaw in the “page weight doesn’t matter because average speed is fast” is that if you average the connection of someone in my apartment building (which is wired for 1Gb internet) and someone on 56k dialup, you get an average speed of 500 Mb/s. That doesn’t mean the person on dialup is actually going to be able to load a 5MB website.

This is so true. While living in Asia I had a perfectly good connection to watch high resolution video but just the latency would cause many websites to be unusable (or so frustrating you wouldn’t want to use it). The huge number of files that must be downloaded as well as the large size of that content is something that most sites don’t care about. I can only assume they just test the pages on their fiber connection and if it works they are ok. This is a very bad idea for nearly every website.

When I was at Google, someone told me a story about a time that “they” completed a big optimization push only to find that measured page load times increased. When they dug into the data, they found that the reason load times had increased was that they got a lot more traffic from Africa after doing the optimizations. The team’s product went from being unusable for people with slow connections to usable, which caused so many users with slow connections to start using the product that load times actually increased.

This illustrates some additional interesting tidbits: learning from data requires thought, potential customers are avoiding you for reasons you are likely unaware of. Learn to get a deep understanding of your customers and potential customers focus on the customer’s “Jobs to be Done.” Also learn to be thoughtful about the use of data: Understanding Data, Simpson’s Paradox, Managing to Test Result Instead of Customer Value, All Data is Wrong, Some is Useful, Data is only a Proxy – it isn’t reality.

The quoted post is good. But it doesn’t display a date 🙁 This is a very bad oversight for such an article (where the date of the article can greatly impact what you take from of it). By looking on the RSS feed I was able to see it was published in 2017.

Related: Functional Websites are Normally Far Superior to AppsThe Edge-case Excuse for Poor Software Coding PracticesDelighting CustomersFocus on End UsersUse Urls: Don’t Use Click x, Then Click y, Then Click z Instructions

Some Quick cli Syntax for Postgres

How to Dump SQL result to a text file using cli

> psql -U postgres -d [database_name] -c ‘SELECT * FROM users’

How to exit postgres command-line utility psql
Type \q and then press ENTER to quit psql

Reset Root Password on MySQL Database
Notice that in PostgreSQL superuser is called postres (not root). If you forgot superuser password, you can reset it this way:

edit file pg_hba.conf

> sudo vim /etc/postgresql/9.3/main/pg_hba.conf

and find there a line similar to:

# TYPE  DATABASE        USER            ADDRESS                 METHOD
local   all             postgres                                md5

md5 here means that postgres asks for password. If there is this line then replace
md5 with peer, if there is no this line then add it

local   all             postgres                                peer

save the file, exit and reload postgresql service to pick up the updated configuration:

> sudo service postgresql reload

Then you should be able to get access to psql without providing a password this way:

> sudo -u postgres psql

In psql you can change user password using this command:

ALTER USER postgres PASSWORD 'new password';

PostgreSQL performance and monitoring

switch current database

\connect database_name or \c database_name

you can user autocomplete by pressing [tab]

list tables in current database


create a new database and import tables, data from a dump
in psql

CREATE DATABASE some_database

create a dump (in terminal)

> pg_dump dbname > outfile

You may notice that no password is used. That is because with Postgres you can setup a password file that is used to verify your access as such:

> vim ~/.pgpass

Then (format is hostname:port:database:username:password):

[ctrl-x] then confirm with y, then hit [enter]

save and exit

> chmod 600 ~/.pgpass

restore a dump (in terminal)

> psql dbname < infile more details on dump-restore

create a user (it sql)

CREATE USER tester WITH PASSWORD 'test_password';
GRANT ALL PRIVILEGES ON DATABASE "test_database" to tester;

notice difference in quotation marks – in first case it must be single quotation mark ‘ and in the second case – double quotation mark “, otherwise PostgreSQL raises a syntax error

restart PostgreSQL

> sudo service postgresql restart

other regular service actions available too – [force-reload, reload, start, status, stop]

Also for Nginx (unrelated to this post really), Restart Nginx

sudo service nginx restart
other regular service actions available too – [force-reload, reload, start, status, stop]
reload only reloads configuration files, without stopping a service (which restart does)

Related: How to Dump SQL Result in MySQL to a Text File Using cliReset Root Password on MySQL DatabaseSome MySQL cli Syntax

The Edge-case Excuse

I find the excuse that the bug is just for your small “edge case” as an explanation for why it won’t be fixed annoying.

I have found “edge cases” to actually mean we don’t want to fix it. Often the issue isn’t needing some special code to deal with an “edge case” it is the coding was done poorly and breaks in many different “edge cases.” It isn’t that those edge cases need to be coded for. It is that the code should have been written in a robust way that didn’t break for lots of “edge cases” but the excuse given for not fixing the fundamental coding fragility is the bugs found are just “edge cases.”

There are real instances where “edge cases” is a justifiable excuse. For example, adding in special code to deal with some odd category of users that just isn’t worth the cost.

But I just am so tired of fragile coding being excused as if breaking in lots of “edge cases” is perfectly acceptable when the only reason it fails is because the code is fragile instead of being built in a robust way to begin with. The issue isn’t that you have some special edge case that you want special coding for the issue is the code was written in an unnecessarily fragile way that makes it not work unless you follow a list of acceptable use cases.

Code should avoid adding in requirements that are not necessary. The edge case excuse I see used far more due to requirements that the code added which never should have existed instead of actually being an edge case that would require special code. For example, most web pages don’t require javascript (or IE, or flash, or downloading 5 mb of code to view simple text…) to do what should be done (display text, display images…) but some sites code their page to break if javascript… isn’t used by the user. Seeing this as an “edge case” issue missing the point of creating code that has superfluous requirements for the user that create “edge case” failures where they shouldn’t exist but for poor coding practices. In some cases jasvascript is required to do fancy things that are useful, in which case gracefully degrading and potentially not working fully is acceptable.

Related: W. Edwards Deming and Software DevelopmentSoftware Supporting Processes Not the Other Way AroundComplicating SimplicityWhich Prime Minister Said “The last programme I wrote was a Sudoku solver in C++”?

Locking and Unlocking a User in Ubuntu

To lock a user in Ubuntu sudo usermod –expiredate 1 [username] Substitute the user’s username where username is shown below.

sudo usermod --expiredate 1 username

This also can be shortened to

sudo usermod -e 1 username

To unlock a user

sudo usermod -e -1 username

This will disable the user both from accessing via password and from accessing via a private key.

You can also expire a user at a future date using

sudo usermod -e YYYY-MM-DD username

To expire a password and force a user to enter a new password

sudo passwd -e YYYY-MM-DD username

Related: Managing Users in UbuntuReset Root Password on MySQL DatabaseDon’t Copy-Paste Directly from Website to Terminal

How to Manually Run Cron Tasks

To manually run cron tasks you can use the run-parts command in Linux.

So to run your cron-weekly, for example, to test that a fix you just made runs without error (this is what I just did, in fact)

run-parts /etc/cron-weekly

run-parts will run all the executables in a directory (you must point at the directory). So if you have several files in cron-weekly to run, you can’t just point to one of the files.

You may run into environmental differences running the script as a different user than the cron test runs at, so you can run as that user if needed. You need to be aware this is a quick and simple way of testing part of the process but it doesn’t do a perfect job of testing if it works as a cron task. But it will let you catch some failures quickly and fix them in time for the actual cron task to run. So do check that the everything works after the real cron job runs.

This is just the kind of thing I said I would put in this blog. Simple stuff but things I forget – so I put it here to remember and maybe help out others, like me, that need really basic tips.

If you have a cron task item (or have setup the whole task this way) that is just a script and you just want to test that 1 item you may run the script directly. For example (for a Linux shell script):

sh /etc/cron.weekly/

Related: Updates Needed When Upgrading from Apache 2.2 to 2.4Rsync to copy Files Between Servers and ComputersBash Profile Adjustments for Scrolling HistoryChecklist: Setting Up a New Domain on VPS

Which PM Said “The last programme I wrote was a Sudoku solver in C++”?

The last programme I wrote was a Sudoku solver in C++ several years ago, so I’m out of date. My children are in IT, two of them – both graduated from MIT. One of them browsed a book and said, “Here, read this”. It said “Haskell – learn you a Haskell for great good”, and one day that will be my retirement reading.

This quote was by Prime Minister Lee Hsien Loong of Singapore in April 2015. I must say I think the Western governments could be more effective with more scientists, engineers and coders in positions of power

His father was the first and long time Prime Minister of Singapore – Lee Kuan Yew

Another quote from the speech

40 years ago, after doing a math degree, I went on to study computer science, on my father’s advice. He said there is a future in that, and he was right. So for the Smart Nation Programme Office, I have put Minister Vivian Balakrishnan in charge, reporting to me. Vivian is both a hacker and a dabbler – He used to be an eye surgeon but since he does not get to operate on eyes nowadays, he dabbles in building simple robots, assembling watches, wireless devices and programming apps. His day job is to be the Minister for the Environment and Water Resources, and so when he builds apps, he uses the real time APIs generated by the Ministry.

It is useful to have governments around the world with different priorities. While the USA has turned against science and engineering in many ways others can pick up the slack. The USA had for decades been firmly in the position of promoting science and engineering. And the results of that are still blessing the USA with economic benefits including the wonderful results of silicon valley and far flung software development throughout the country.

photo of the Marina Bay Sands resort that resembles 3 sky-scapers connected on top by a large ship

The Marina Bay Sands casino and resort hotel has 3 towers connected on top by a what looks like a large ship. Singapore has a huge, and economically important port.

Singapore can improve but they sure do many things well. And the sense to continue supporting science, engineering and emerging technology will benefit them economically as we move into a world where those fields only grow in importance.

Prime Minister Lee Hsien Loong made the source code, the exe file, and a sample printout available after asked being asked about it online.

Related: If Tech Companies Made SudokuChina’s Technology Savvy Leadership (2008)Scientific Illiteracy (USA lead Singapore in this undesirable trait)Math Education Results Show China, Singapore, Korea and Japan Leading

Hacking Saved Apollo 13

Apollo 13 is a great movie on hacking. Hacking is applying intelligence to systems (including computer systems) to achieve a goal.

That can be done by criminals or devious people but it doesn’t have to be. It is a bit annoying that some people equating hacking only with criminal behavior.

The hacking culture is much more about figuring out ways to make technology work for people than about criminals. We shouldn’t let a small sub-set of hackers defile the term.

photo of the Apollo 13 Command Module

The Apollo 13 command module in which the astronauts splashed down into the Pacific Ocean. by HrAtsuo, via Wikimedia Commons.

When the oxygen tank exploded, Commander Jim Lovell made the famous statement: “Houston, we’ve had a problem.” The engineers on the ground and astronauts had to devise solutions to several very difficult problems and execute them quickly in order to return the damaged spacecraft to earth.

The amazing hacking done by the engineers (including the astronauts) at NASA to create a solution to the serious problems faced by Apollo 13 allowed the astronauts to return home safely. Without the amazing hacking done by those government employees the astronauts would have died.

It is also good to remind people, government workers do amazing things. Sure government workers can also harm society with bad work or by implementing bad policy. But it isn’t the fact that they work for the government that defines the value of the work they do.

Related: Hacking the Standard Bike WheelBuild Your Own Tabletop Interactive Multi-touch Computer (2009)ModSecurity, Adjustments for WordPressPhusion Passenger Tips and Troubleshooting Ideas

Chrome Remote Desktop

I am more often frustrated by Google the last few years that pleased with them. But they do still provide some pretty awesome tools. For example, Chrome Remote Desktop lets you access a computer over the internet (and lets you to allow another user to access your computer securely over the internet).

Chrome Remote Desktop allows users to remotely access another computer through Chrome browser or a Chromebook. Computers can be made available on an short-term basis for scenarios such as ad hoc remote support, or on a more long-term basis for remote access to your applications and files.

screen shot of chrome remote desktop

Chrome Remote Desktop is fully cross-platform. Provide remote assistance to Windows, Mac and Linux users, or access your Windows (XP and above) and Mac (OS X 10.6 and above) desktops at any time, all from the Chrome browser on virtually any device, including Chromebooks (including Android phones and iPhones). The iPhone app is new.

Some users worry about installing such an app given all the spying and hacking scandals. That is not a completely crazy worry. Google, and others, have been taking advantage of weak user control (and even bugs and work arounds to avoid stated user preferences) to track users and use that information to make money selling ads. With many cool and useful tools there are risk of them being misused. And practices of governments and huge corporations have been so egregious to give a sensible person pause. Still in the right situations this is a pretty cool looking tool (similar things exist but the combination of price [this being free] and simplicity make this interesting).

Related: Chrome Remote Desktop support forumUsing scp (secure copy) to Copy Files Between ComputersUsing Rsync to copy Files Between Servers and ComputersLinux/Ubuntu File and Directory PermissionsGovernments Shouldn’t Prevent Citizens from Having Secure Software Solutions

Compare WordPress Files on Server to Proper WordPress Version

Sadly one of the hassles in managing your own WordPress blog is dealing with people that use your blog to serve spam content. These hacks can insert spam links into your pages and posts or create spam directories that are completely their own content on your domain.

There are many issues to deal with in re-establishing control of your server; but that isn’t the scope of this post.

This is just a tips if you are troubleshooting to try and determine what is going on. Often your server has been hacked to allow uploaded php pages to be added or for WordPress php files to be edited.

One way to track down if the files have been changed or new ones added is to compare the WordPress files on your server to the current files for a fresh WordPress install. This assumes your blog is using the current version, which hopefully it is because on the big improvement WordPress made is to make those updates automatic. That greatly reduces the chance to have WordPress be the vector to infecting your server. If you were using a older version then just compare to the field for that version from the WordPress server.

If you don’t have a current backup I would make a backup before I tried this. Obviously, don’t make any deletions or changes to your server unless you understand what you are doing. You can create big problems for yourself.

You can use the diff command to view the difference between WordPress on your sever and the fresh install from WordPress. I install the new WordPress in a new directory outside public_html. At the cli on a Ubuntu/Linux server:

sudo wget
diff -rq wordpress ../public_html/ – replace with whatever the version is you are using.
../public_html/blog/ – replace with the path to your blog

Continue reading