Posted on

Ubuntu – Copy all files from another server by FTP

Moving servers from one infrastructure to another. In our case, from Webfusion UK to AWS.

Problem we face is that there are over 82Gb of files to move from one server to another. Traditionally we would have downloaded them all locally, then uploaded them, but what if there was a way to transfer them directly from one server to another.

We turned on FTP on the source server, and updated the firewall so that only the destination IP could connect.

Then on the destination server we can simply type:

wget -r ftp://sourceip/folderinftproot/* –ftp-user=username –ftp-password=password -P /var/www/html/ -q

This copies all folders from the FTP root on the source server in to the web root of the new server.

To transfer 82Gb of data between data-centres took 14 minutes, compared to the older download-upload method we used to use that took several overnights of downloading locally!

And of course, remembering to turn off FTP on the source server once completed!

Posted on

Ubuntu – count all files in a directory recursively

Ubuntu – count all files in a folder recursively.

Took a while to figure out how to do this on a single command, but so very useful to check if all files have copied successfully.

find . -type f | wc -l

How this works:
find . -type f finds all files ( -type f ) in this ( . ) directory and shows everything as one file on each line in a list.

The second command comes after the pipe | into wc (word count) the -l option tells wc to only count lines of its input.

Together they count all files in the folder you are in and all sub-folders.

Posted on

Ubuntu – Backup Amazon EC2 to S3 (Pre-requisites)

Pre-requisites on the server are Amazon S3 tools, and zip

sudo apt-get install s3cmd

sudo apt-get install zip unzip

You then need to configure the s3cmd to use the Access Code and Secret Key from the IAM within your AWS console. Its recommended that you set up a new user with programmatic access only for each server / project, and give the user the AmazonS3FullAccess permission.

Back on the server, run
sudo s3cmd --configure
Enter your access key, secret key

If, like me you are using EU-West-1 (Dublin) as your datacenter, then type in “eu-west-1” for the Default Region.

Enter a password to encrypt traffic between the EC2 instance and S3 (DO NOT USE YOUR MAIN ACCOUNT PASSWORD, MAKE A NEW ONE)

Path to GPG program – just press Enter

Use HTTPS – Yes

HTTP Proxy – leave blank, just press Enter

Test – Yes

 

Posted on

Adding free SSL to your server using LetsEncrypt

So you have a lovely new website running on your Ubuntu server, but wouldn’t it be nice to have that small padlock to give visitors peace of mind? To have all traffic between your website and the browser encrypted…..but you don’t want to spend days waiting, and a small fortune on an SSL Certificate, then wait for the certification authority to email you, with lengthy confirmations etc. Thankfully, there is a lovely easy way!

All you need to do is to ensure that the domain name (the only part you need to change in the script below, indicated in bold) has a DNS entry pointing to the IP address of the server you are running this on, and hey presto, a free SSL Certificate that automatically renews using a cron task that it adds automatically for you! No more renewals ever again!

NB. When running this, there will be a 10-15 second interruption in the Apache2 web-server as it stops and re-starts, meaning live site visitors at that moment may see an error.
/* Lets Encrypt */
sudo add-apt-repository ppa:certbot/certbot
sudo apt-get update
sudo apt-get install python-certbot-apache

sudo certbot --authenticator standalone --installer apache -d www.yourdomainnamehere.com --pre-hook "systemctl stop apache2" --post-hook "systemctl start apache2"

Follow the on-screen prompts (usually only 2 or 3, and it works seamlessly)!

Posted on

Setting up the LAMP Stack from scratch on Ubuntu

When migrating servers to AWS, its great to be able to set them up easily and quickly. This is effectively a lovely cheat-sheet. By running the scripts below you can have a full LAMP Stack running on an EC2 instance running Ubuntu in less than 5 minutes.

/* Set locale to UK - V IMPORTANT!! */
sudo locale-gen en_GB.UTF-8

/* Run update and upgrades */
sudo apt-get update
sudo apt-get upgrade

/* Install Apache 2 */
sudo apt-get install apache2

/* Server internal firewall config */
sudo ufw allow OpenSSH
sudo ufw allow in "Apache Full"
sudo ufw enable

/* Install MySQL */
sudo apt-get install mysql-server

mysql_secure_installation
(No,No,Yes,Yes,Yes,Yes)

/* Install php and associated plugins as needed */
sudo apt-get install php libapache2-mod-php php-mcrypt php-mysql
sudo apt-get install php-curl php-gd php-mbstring php-mcrypt php-xml php-xmlrpc

/* Restart services */
sudo systemctl restart apache2

/* Update Apache2 config to allow .htaccess files and modified file permissions */
sudo nano /etc/apache2/apache2.conf

<Directory /var/www/html/>
AllowOverride (Change None to All)
</Directory>

/* Turn on rewrite rules for Apache2 */
sudo a2enmod rewrite
sudo systemctl restart apache2

/* Update permissions so that FTP can write to the web root folder */
sudo chown -R ubuntu:www-data /var/www/html

/* Reboot server */
sudo reboot