Right, forewarning - this post is going to be pretty large. In fact, I'm going to include a contents list to break it down, for your convenience and also so I remember to include everything I've discovered over the last couple of weeks :)


  1. Your Server
    • Linux.
      1. Why?
      2. What you'll need.
        • A spare computer
        • Ubuntu 12.04 LTS (for now)
        • An Internet connection
      3. How to deploy this beast?
    • Creating your capability; building your toolkit...
      1. SSH
      2. Web Server
      3. Owncloud (Your own unlimited Dropbox)
        • What is it. and why do we need it?
        • Apache2 Web Server
        • MySQL
      4. Plex (Your own Spotify, Netflix and much more)
        • Again - what, and why?
        • Finding all your media, even over the network
        • Configurations for use over the Internet, and usage considerations.
    • The Dynamic IP problem.
      1. What is the problem?
      2. Solution #1: NoIP.com
        • noip2, the handy IP updating program that runs on your server
      3. Notifying others of your current IP
        • Running a script upon IP address change detected
        • Setting permissions with sudoers
        • Updating a Google Site, for instance.
  2. Your Domain Name
    • What is a Domain Name?
    • What you need to know about DNS.
      1. What is a Hostname?
      2. A Records
      3. CNAME Records
      4. MX Records
        • Using Windows Live to manage your domain's email
    • How do I get a free domain name?
      1. Why you may consider buying one.
        • Where from? (And where not to venture).
      2. When is free OK, and how much functionality can I squeeze out of it?
  3. Security
    • Transport Layer Security (TLS) formerly known as SSL.
      1. Why?
      2. Why is it so involved? (A brief explanation of the problem and the playing field)
    • Certification.
      1. Self-signed certificates
        • When are they OK to use?
        • How to set them up on host machines (it's quite fiddly) 
      2. CA Certs
        • Tiered system
        • Cost (and free trials)
    • Getting a free Certificate.
      1. Comodo
      2. Proving you control the domain in question
      3. Getting this to work with Owncloud
        • Configuring Apache2 Web Server
      4. Multiple certificates
  4. Drawing it all together; your empire...
    • No-IP redirect services for pesky URLS (Plex)
    • Linking Google's Blogger (Blogspot - see this page!) to your domain
    • One homepage to link them all :)
  5. Future projects and miscellanea
    • What is the best way to spend money if improvements are desired for this system?
    • Tips for making it easier, and helpful ISPs.
    • Maintenance
    • Troubleshooting

Your Server

Firstly, we are going to require a server of our very own. This may fly in the face of the current Cloud enamoured Zeitgeist, but there are a few decent reasons why it's nice to have a bit of your own space on the web. The first reasons are ideological, and the second lot are more pragmatic...

So, as we should all know, no right to privacy is really assumed in the security circus that is the Internet right now. I do feel strongly though - about always taking comfort in the knowledge that it could be worse: 

Dr. Panoctagon
Dr. Panoctagon is an extraterrestrial surveillance surgeon from Jupiter who uses space technology and primitive tools to perform medical procedures on his patients and dragnet surveillance of the world's digital communications, some of whom die as he conducts his rounds, while others are murdered by his careless, barbaric acts. Panoctagon also practices as an orthopedic gynecologist and seduces and engages in sexual intercourse with his female patients, nurses, and occasional multi-billion dollar telecommunication giants.

Panoctagon, who dubs himself the "paramedic indiscretus fetus of the east," is from the church of the operating room and was born on the planet Jupiter. His physical features include having five eyes, green and silver skin which also changes to blue and brown, a pink-and-white Afro,[1] and a brain that glows red, white and blue. Panoctagon also states that he can change his face with the press of a button, disappear, and wears a 7XL which has not yet been invented, X Ray sunglasses, hard shoes with razor blades, an airgapped laptop, two encrypted mobile phones, a .Reverse Engineering for Dummies. book, a white suit and a stethoscope.

Whether it be governmental surveillance, spyware, ad-nets, tradings in personal information, the whole system is in flux (which is not a bad thing in itself). The bad part is if you have stuff you want to keep private, and that you will find no government willing to grant you that privacy - nor will you truly find it in the vestiges of large corporate 'free' web services (Dropbox, par exemple) - whose servers sit within a foreign nation. Now, this is probably a case of needless paranoia, but if it were down to me, I'd rather it wasn't the case. GCHQ may be up to their necks in this dragnet surveillance as much as the NSA, but at least they're domestic, and data collected thus subject to parliamentary discretion. The things about secret domestic surveillance programs that people always seem to conveniently forget are:
- only the "good guys" do it
- No one ever abuses it.
- Even if someone abuses it they would never just get a slap on the wrist (maybe just fired, and security clearance revoked) rather than criminal charges.
- The disincentive for abuse is "patriotism" and no one has ever decided not to be a "patriot"... ever.
- Absolutely zero people in government seek out power, or have weird ideas about controlling what other people can do or think.
- No one in the intelligence "community" would ever abuse surveillance to (e.g.) blackmail politicians for funding/votes/more power. Because every single one of them always make 100% perfect moral choices.
- It's only used for "threats to national security"... except when it isn't, but the winning the "War on Drugs"/spying on animal rights activists/making sure domestic companies win bids for foreign contracts/making sure that the correct politicians are re-elected really all fall under the umbrella of "National Security" if you think about it... but seriously, this wasn't supposed to be a massive rant - and at least they handled the publication of medical records in a righteous way. 

Anyway, I think of it as my little part in not wilfully funding and entering into contractual agreement with these foreign data-centres, certainly before the Facebook effect occurs when people suddenly realise they cannot actually function properly in life without their 50GB Dropbox account. As Mikko Hypponen says in his TED Talk late last year - it's all about emergent behaviour. Many small waves. Anyway, the point is that we don't have to rely on these Cloudy beasts if we don't want to (if you do - and you're aware of the facts, then that's perfectly reasonable).

Now onto my second round of points,- the pragmatic reasons to get involved and do this: 1. Everything becomes unlimited (or at least is not charged at a premium) - for instance storage space (Dropbox: 100GB for $99 per User per Year, pretty much the exact price of a brand new 2TB (2000GB) SATA hard drive for your server), and is eminently more configurable and customizable. 
2. If you're worried about redundancy, RAID arrays within the server can protect against localised corruption - or geographical redundancy can be achieved by simply synchronising with a machine elsewhere (do a swap with your friend if you like). Or, and this brings me onto a strangely counterintuitive thought to run with for a bit - enter the Cloud at a lower-level. AWS has just about every type of service available you could ask for, and plenty that you couldn't. Admittedly this slightly defeats the 'self sustainable' element, but it is effectively still your own realisation of a solution - not part of a centralised system and undoubtedly far cheaper.

That's enough opening thoughts though, lets get down to business. I spent the last 2 weeks exploring some of the open source platforms that have the potential to be very powerful in the hands of the individual, to secure their privacy and to make things work a bit better for their own needs. In this blog I will describe what I did, just as a sort of rough guide to the neighbourhood of these things, and I will make it very simple as a lot of these blogged-guides are so bloody vacuous or assume random knowledge. There are undoubtedly better ways to do some of these things, and I would appreciate the comments below - but as I say, think of this as a taster of some of the things that are possible (for approximately zero £coin) by simply linking a few clever tools and platforms together.          

So, what can we use for this server? Do we need to go out any buy a load of components or a shiny new box? Absolutely not. In fact, please don't. One of the remarkable things about this Brave New digital World we inhabit is that there is alot of spare hardware to be had. In alot of households, there are spare (usually considered broken or obsolete) laptops or even rather ancient PC's knocking around. The beauty is that for the basic stuff, this Server of yours requires very little in terms of hardware spec. A 10 year old machine that still powers up can suddenly allow you to explore the workings of the space and concepts you'll be using, and if it the invokes your imagination - it's very cheap to upgrade by orders of magnitude. 

If you really have nothing - the one thing I will say is perhaps resist the urge to use your actual computer as the server, for many reasons. Although you can be clever and deploy Virtual Machines on it to run the server concurrently to your everyday stuff - it's going to create a huge collection of possible failure modes. Servers are supposed to be the stoic, reliable, quiet beasts of the computing realm - with minimal software overhead loaded onto them so there's less to go wrong, and problems much easier to diagnose and solve. There's also redundancy in the separation of the hardware systems, and security too. So if you are desperate - buy a Raspberry Pi (RPi) Raspberry Pi 2 (Model B) for around £30. This is a computer. It does everything you need. It's not incredibly actually pretty well spec'd out; the RPi 2 has 1GB RAM and a 900MHz quad-core ARM Cortex-A7 CPU - which can chug through most simple server tasks without breaking a sweat. Provided you can then hook up some storage you're away. This could be over the network, or via a powered External Hard Drive - but be warned this has the potential to be a rubbish idea if you're thinking it'd work as a cheap DIY NAS, think instead about remarkably slow transfer and disk I/O speeds, and a premature death for your drive, when you could've just plugged it straight into your main computer if you weren't trying to be all clever about it. But more importantly, it's tiny, look at it.


Raspberry Pi Model B (Rev 2.0)

Now we have a suitable hardware platform for the software, let's begin with the software - i.e. the Operating System. It's going to be Linux. 

Linux

Why?

A fairly reasonable question actually, the first answer I'll give is because it's free (and free in the good sense - it's well understood, constantly maintained [providing you choose a sensible distribution, which we will] and free from the ideological shackles of the raison d'être shared by most software in existence: to make money. In short, it's well loved and looked after). The second is that it's lightweight and relatively undemanding in terms of power and performance (i.e. it's efficient, even on older machines that would not support any modern incarnation of Windows).

The distribution of Linux (as there are many different kinds, and of those kinds there are many flavours, as these crazy kids call them) that we will be using is Ubuntu Server 12.04.4 LTS.(It's idiosyncratic named handle is 'Lucid Lynx', if you ever need it). If your hardware supports it, get the 64-bit version, otherwise get the 32-bit one. Most hardware these days will take 64-bit but check your motherboard and chipset to be sure.
Note: If using the RPi, you'll need to install a special ARM version of Linux, as the ARM CPU is not designed to handle x86 or x86-64 instructions natively (what 32 and 64 bit OS's use respectively). The default RPi distribution; a variant on Debian called Raspbian (Wheezy) is decent, and the image of the OS you download (usually a .iso, although for the RPi sometimes comes as a .deb package, and sometimes even more confusingly has no filetype extension), should be suffixed with armhf not i386 or amd_64.  

Installation of Ubuntu Server
  1. Either burn the Ubuntu 12.04 distribution to some optical media (e.g. DVD-R) or create a bootable flash drive with it by doing one of the following:
    • If using Windows; get something like PenDriveLinux and follow instructions here
    • If using Mac OS X, follow instructions here
    • If using Linux already, follow instructions here.   
  2. Insert the optical media or the bootable flash drive into the computer you are about to wipe and turn into an awesome Linux Server, and reboot it/turn it on. Make sure the BIOS is set to boot from either of these devices before the primary hard drive partition with the current OS. BIOS can usually be entered by hammering the DEL button on the keyboard from the moment the power is on (if not, try and read the instructions on screen, as fleeting as they may be).
  3. Follow these instructions up to step 6, then:
  4. When offered whether to install optional modules, select:
    • OpenSSH Server
    • LAMP Server
    • Samba File Sever
    • Mail Server
  5. Say yes to MySQL and note down the password you give.

Initial Setup

Add some users
    $ adduser mynameisjonas <press Return>
    Adding user `mynameisjonas' ...
    Adding new group `mynameisjonas' (1006) ...
    Adding new user `mynameisjonas' (1003) with group `mynameisjonas' ...
    Creating home directory `/home/mynameisjonas' ...
    Copying files from `/etc/skel' ...
    Enter new UNIX password:$ Str0ng_79163-paSs_woRd <press Return>
    Retype new UNIX password:$ Str0ng_79163-paSs_woRd <press Return>
    passwd: password updated successfully
    Changing the user information for mynameisjonas
    Enter the new value, or press ENTER for the default
            Full Name []:$ <press Return>
            Room Number []:$ <press Return>
            Work Phone []:$ <press Return>
            Home Phone []:$ <press Return>
            Other []:$ <press Return>
    Is the information correct? [Y/n]$ Y <press Return>

    Note that there should be no graphical interface installed, we will be running a headless installation to minimise overheads and potential complications or problems 

    SSH

    Make it a bit more secure by changing SSH port (preferably to a very high value ~20,000+). Also, use fail2ban to block brute force attempts

    Installation of OwnCloud

    Install OwnCloud initially from the Ubuntu repository, even though this will be an old version; it will ensure the preinstallation of a lot of requisite packages automatically. The latest version of OwnCloud will be installed from the OwnCloud repository itself, later. Run the following in terminal, either logged into the Server through SSH, or at the physical terminal itself on the machine. 
      $ sudo apt-get update 
      $ sudo apt-get dist-upgrade
      $ sudo apt-get install owncloud

      $ wget http://download.opensuse.org/repositories/isv:ownCloud:community/xUbuntu_12.04/Release.key

      $ sudo apt-key add - < Release.key

      $ sudo -i

      $ echo 'deb http://download.opensuse.org/repositories/isv:ownCloud:community/xUbuntu_12.04/ /' >> /etc/apt/sources.list.d/owncloud.list

      Exit from root: 

      $ exit

      And now get the latest version of OwnCloud:

      $ sudo apt-get update  
      $ sudo apt-get install owncloud  

      $ sudo chown -R www-data:www-data /var/www/owncloud/

      Configuring Apache2: your own Web Server 

      What's Going On?

      Apache (specifically, Apache2) is the Web Server chosen by OwnCloud to deal with all web based (HTTP(S)) transactions encountered by the server.
      What you'll need.
      ...

      Ok this looks immense, and may well take epic time to write up (certainly at least 2^kn time ^_^). Will add to it in parts :-D.

      -D