Jekyll and Swift Object Storage

Static Site generation and deployment.

Edouard Ravel, in 12 March 2015

Jekyll and Swift Object Storage.

Generally when it comes to maintaining an online presence your options are: Fast, Good, and Cheap. The caveat being that you can only pick any two.

By moving away from dynamically generated content one can benefit from blazing fast delivery, highly available replicated storage, for pennies.

Services and Tools.

Prerequisites.

Things you will need:

Local Setup.

To install the packages and the dependencies you will need:

sudo apt-get install ruby-dev golang nodejs git mercurial python-swiftclient

Finally install Jekyll:

sudo gem install jekyll

If you fancy using Sass CSS preprocessing (optional):

sudo gem install sass

Runabove.

Built with DevOps in mind, RunAbove is an IaaS solution that combines the power of bare-metal with the flexibility and high-availability of the public cloud. Runabove offers excellent value for money.

Runabove Object Storage.

The easiest way to communicate with the Swift Object Storage on Runabove (GUI asside) is with the swiftclient. To export the information you need to authenticate with the Swift API you can run tenantid-openrc.sh.

To get this file *1:

Log in on runabove, select OpenStack Horizon, go into Access & Security panel, then into API Access tab. Once there you can click on: Download OpenStack RC File.

It should look something like this:
Download the Runabove OpenStack RC File
Download the Runabove OpenStack RC File
source *-openrc.sh
export OS_REGION_NAME="SBG-1"

Runabove Object Storage Container Setup.

First we want to check if swiftclient can communicate with the Object Storage.

swift stat

It should return something along the lines of:

$ swift stat
       Account: AUTH_11111111111111111111111111111111
    Containers: 0
       Objects: 0
         Bytes: 0
 Accept-Ranges: bytes
    Connection: close
   X-Timestamp:
    X-Trans-Id:
  Content-Type: text/plain; charset=utf-8

Please make a note of the Account: AUTH_, you will need it later.

Creating Object Storage Containers.

Why create one container when you can have two? You will want to have one container to redirect your (www.example.com) subdomain to your (example.com) main domain.

swift post "redirect"
swift post "example_com"

Lets put the container ACL to public this being the internet after all.

swift post --header "X-Container-Read: .r:*" redirect
swift post --header "X-Container-Read: .r:*" example_com
Setting up a redirect.

The merit of using Meta Refresh for redirections with regards to SEO have long been debated, it is by all means not the most optimal or elegant of solutions, however since we can't return HTTP 301 using static pages the below will have to do.

<!DOCTYPE HTML>
<html lang="en-US">
    <head>
        <meta charset="UTF-8">
        <meta http-equiv="refresh" content="1;url=https://example.com">
        <script type="text/javascript">
            window.location.href = "https://example.com"
        </script>
        <title>Page Redirection</title>
    </head>
    <body>
        <!-- Note: don't tell people to `click` the link, just tell them that it is a link. -->
        If you are not redirected automatically, follow the link to: <a href='https://example.com'>Example.com</a>.
    </body>
</html>

Now that this is saved as index.html, lets upload it to Object Storage.

swift upload redirect index.html

Cloudflare.

CloudFlare is a US-based company that provides a content delivery network and distributed domain name server system, sitting between the visitor and the CloudFlare user's hosting provider, acting as a reverse proxy for websites. Its network protects, speeds up, and improves availability for a website or mobile application with a change in DNS.

Cloudflare CDN, CNAME Flattening and SSL.

We want to put our static website on a domain (example.com), now we're going to assume that your domain DNS has already been delegated to Cloudflare and that all that is left to be done is getting it to use the Runabove Object Storage as origin.

Configuring your Cloudflare DNS Zone
Configuring your Cloudflare DNS Zone

You will want to have the following records:

CNAME: @ storage.sbg-1.runabove.io.
CNAME: www storage.sbg-1.runabove.io.
TXT: _swift-remap example_com.AUTH-111111111111.storage.sbg-1.runabove.io.
TXT: _swift-remap.www redirect.AUTH-111111111111.storage.sbg-1.runabove.io.

Activate Cloudflare for both example.com and www.example.com and Cloudflare Cname Flattening will take care of resolving the Runabove Object Storage IPs. *3

Jekyll.

Jekyll is a simple, blog-aware, static site generator. It takes a template directory containing raw text files in various formats, runs it through a converter (like Markdown) and its Liquid renderer, and spits out a complete, ready-to-publish static website suitable for serving with your favourite web server.

Setting up a Jekyll Site.

We want to create a simple site with Jekyll, all it takes is the following:

jekyll new example_com
cd example_com
jekyll serve

Using your favourite browser and navigating to http://localhost:4000 will now display a stock Jekyll Site.

We're going to skip customizing the Jekyll pages itself, if you're curious on how to go about this the Jekyll Documentation is extensive and contains all you will need to know.

Rclone.

Rclone is a command line program to sync files and directories to and from cloudstorage solutions.

We want to use Rclone as it offers an easy way to sync our Jekyll _site to Runabove Object Storage. Whilst the same could be achieved with swiftclient, Rclone shines in being able to list the files on the Object Storage and only upload the files that have either been modified or newly created. To top it off, it will take care of removing files that are no longer there, saving us from the headache of manually removing these files with the swiftclient.

Setting up Rclone.

Rclone is written in Golang, which means we need to tell GO where to download and build packages.

export GOPATH="$HOME/dev/go/"

We can now build Rclone

go get github.com/ncw/rclone

Lastly we copy Rclone to our /usr/bin/ so that we can easily launch it in a terminal.

cd $GOPATH/bin
sudo cp rclone /usr/bin/

Rclone Configuration.

Rclone will do us no good if not configured for use with Runabove Object Storage.

You will need the following information:

Lets run:

rclone config
Failed to load config file /home/example/.rclone.conf - using defaults
No remotes found - make a new one
n) New remote
q) Quit config
n/q> n
name> runabove
What type of source is it?
Choose a number from below
 1) swift
 2) s3
 3) local
 4) google cloud storage
 5) dropbox
 6) drive
type> 1
User name to log in.
user> [email protected]
API key or password.
key> ****
Authentication URL for server.
Choose a number from below, or type in your own value
 * Rackspace US
 1) https://auth.api.rackspacecloud.com/v1.0
 * Rackspace UK
 2) https://lon.auth.api.rackspacecloud.com/v1.0
 * Rackspace v2
 3) https://identity.api.rackspacecloud.com/v2.0
 * Memset Memstore UK
 4) https://auth.storage.memset.com/v1.0
 * Memset Memstore UK v2
 5) https://auth.storage.memset.com/v2.0
auth> https://auth.runabove.io/v2.0
Tenant name - optional
tenant> 12345678
Region name - optional
region> SBG-1
Remote config
--------------------
[runabove]
user = [email protected]
key = ****
auth = https://auth.runabove.io/v2.0
tenant = 12345678
region = SBG-1
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y
Current remotes:

Name                 Type
====                 ====
runabove             swift

e) Edit existing remote
n) New remote
d) Delete remote
q) Quit config
e/n/d/q>

Lets double check the Rclone configuration for good measure.

cat ~/.rclone.conf
[runabove]
type = swift
user = [email protected]
key = ****
auth = https://auth.runabove.io/v2.0
tenant = 12345678
region = SBG-1

Sync a Jekyll Site to Runabove Object Storage.

You have entered a bunch of commands and so far you have little to show for exempt a blank page on your domain. It is time to build the Jekyll Site and get it online.

You could of course run jekyll build in your example_com folder, or you could write a bash script to do it for you.

#! /bin/bash

# _sync.sh

echo "Building site."
jekyll build --source example_com --destination example_com/_site
echo "Build complete!"

echo "Deploying to Object Storage."
rclone -v sync example_com/_site/ runabove:example_com
echo "Deploy complete!"

Place sync.sh below the example_com folder and make it executable chmod +x sync.sh.

You're good to go!

Launch ./sync.sh and Jekyll will build your site and place the static documents in _site, Rclone will then sync _site with your Object Storage and place your files in the example_com Container.


Sources:

*1 Upload your first object inside Openstack Swift - Runabove

*2 How to put Object Storage behind your domain name? - Runabove

*3 Introducing CNAME Flattening: RFC-Compliant CNAMEs at a Domain's Root - Cloudflare