The Right Code

Home of Greg Bergé. Let's speak about JavaScript.

Forget Capistrano and deploy your node.js application using Shipit

Shipit logo

Why Shipit?

When I worked at Le Monde, our need was simple: deploy an application on multiple servers. For our PHP backend we used a self-made tool written in PHP to deploy them, but it had some flaws (no atomic deployment, no rollback, centralized configuration).

For our fresh new node project I wanted to use a solid tool. I had heard of Capistrano to deploy everything an anything, so I started to use it. Capistrano is probably a great tool if you code in ruby, but if you don't know ruby at all, this is horrible! I persist to use it and when I tried to upgrade to the version 3 that has no documentation I abandoned and decided to code my own deployment tool in node.js. Shipit was born!

Shipit was built to be a Capistrano alternative for people who want to write tasks in JavaScript and don't have a piece of ruby in their beautiful codebase.

Grunt power

Grunt logo

First I wanted to do a project from scratch but then I realised that deploying is just running some tasks in a specified order. And for that in node, there is Grunt, it's the standard (or it was).

So in fact Shipit is just a grunt extension with some predefined tasks.

Your first deployment

I will try to explain how to set up your first deployment using shipit. For that, we will just deploy an "hello world" app. You will be able to follow the steps along commits in this tutorial repository.

1. Set up the hello world project

The first thing to do is to set up an hello world project, it can be any project written in any language but for this tutorial we will just write a project that echo "hello world".

So we create an index.js file with a single line:

console.log('hello world!');  

Next we create a package.json:

{
  "name": "shipit-tutorial",
  "version": "0.0.0",
  "description": "Shipit tutorial.",
  "main": "index.js",
  "private": true,
  "scripts": {
    "start": "node index"
  },
  "repository": {
    "type": "git",
    "url": "https://github.com/neoziro/shipit-tutorial.git"
  },
  "license": "MIT"
}

Be careful of remplacing the repository by yours.

We also create a .gitignore:

node_modules/  

And a README.md:

# hello-world

Demo project for Shipit tutorial.  

OK that's it, run npm start and you will see a beautiful "hello world!" in your terminal.

GitHub commit: https://github.com/neoziro/shipit-tutorial/commit/772e45fbfa74a1285cf2b21f3106db3824496bf7

2. Install and configure Shipit

Shipit is based on grunt, so you need grunt and grunt-shipit.

npm install --save-dev grunt grunt-shipit  

We need now a Gruntfile.js to set up Shipit config:

var pkg = require('./package.json');

module.exports = function (grunt) {

  /**
   * Initialize config.
   */

  grunt.initConfig({
    shipit: {
      options: {
        // Project will be build in this directory.
        workspace: '/tmp/hello-world-workspace',

        // Project will be deployed in this directory.
        deployTo: '/usr/src/hello-world',

        // Repository url.
        repositoryUrl: pkg.repository.url,

        // This files will not be transfered.
        ignores: ['.git', 'node_modules'],

        // Number of release to keep (for rollback).
        keepReleases: 3
      },

    // Staging environment.
      staging: {
        servers: ['my-remote-server.com']
      }
    }
  });

  /**
   * Load shipit task.
   */

  grunt.loadNpmTasks('grunt-shipit');
};

Shipit is now installed and configured, the next step will be preparing the remote server before deploying.

GitHub commit: https://github.com/neoziro/shipit-tutorial/commit/345661fac4faee030a6508ec7ef3649973893727

3. Prepare the remote server

By default, Shipit use a deploy user if you don't specify it.

The first step is to create this user:

useradd deploy -m  

Then this user must have your ssh key in its authorized keys:

mkdir /home/deploy/.ssh  
cat my_id_rsa.pub >> /home/deploy/.ssh/authorized_keys  
chown -R deploy:deploy /home/deploy/.ssh  
chmod 700 /home/deploy/.ssh  
chmod 600 /home/deploy/.ssh/authorized_keys  

We need to be sure that the deploy directory is created and have the good rights:

mkdir -p /usr/src/hello-world  
chown deploy:deploy /usr/src/hello-world  

Our remote server is ready for deployment.

GitHub commit: https://github.com/neoziro/shipit-tutorial/commit/3a0417ee326af1ea583750623af45789b70b98c4

4. Deploy on the remote server

We can now deploy on the remote server, to deploy the command is very simple:

grunt shipit:staging deploy  

The result is:

Running "shipit:staging" (shipit) task

Running "deploy:init" task

Running "deploy:fetch" task  
Create workspace "/tmp/hello-world-workspace"  
>> Workspace created.
Initialize local repository in "/tmp/hello-world-workspace"  
@ Reinitialized existing Git repository in /private/tmp/hello-world-workspace/.git/
>> Repository initialized.
List local remotes.  
@ shipit
Update remote "https://github.com/neoziro/shipit-tutorial.git" to local repository "/tmp/hello-world-workspace"  
>> Remote updated.
Fetching repository "https://github.com/neoziro/shipit-tutorial.git"  
>> Repository fetched.
Checking out commit-ish "master"  
@ Already on 'master'
>> Checked out.
Testing if commit-ish is a branch.  
@ * master
Commit-ish is a branch, merging...  
@ Already up-to-date.
>> Branch merged.

Running "deploy:update" task  
Create release path "/usr/src/hello-world/releases/20140730152424"  
Running "mkdir -p /usr/src/hello-world/releases/20140730152424" on host "my-remote-server.com".  
>> Release path created.
Copy project to remote servers.  
Remote copy "/tmp/hello-world-workspace/" to "deploy@my-remote-server.com:/usr/src/hello-world/releases/20140730152424"  
>> Finished copy.

Running "deploy:publish" task  
Publishing release "/usr/src/hello-world/releases/20140730152424"  
Running "rm -rf /usr/src/hello-world/current && ln -s /usr/src/hello-world/releases/20140730152424 /usr/src/hello-world/current" on host "my-remote-server.com".  
>> Release published.

Running "deploy:clean" task  
Keeping "3" last releases, cleaning others  
Running "(ls -rd /usr/src/hello-world/releases/*|head -n 3;ls -d /usr/src/hello-world/releases/*)|sort|uniq -u|xargs rm -rf" on host "my-remote-server.com".

Done, without errors.  

Your project is now deployed and avalaible in the repertory "/usr/src/hello-world/current".

5. Write a custom task

Now that the deployment is setted up, you probably want to make a task to run the project.

If you have already made some grunt task it will be very easy, if you didn't it will just be easy.

Let's modify our Gruntfile.js:

  /**
   * Start project on the remote server.
   */

  grunt.registerTask('start', function () {
    var done = this.async();
    var current = grunt.config('shipit.options.deployTo') + '/current';
    grunt.shipit.remote('cd ' + current + ' && npm start', done);
  });

And now run our fresh new task:

grunt shipit:staging start  
Running "shipit:staging" (shipit) task

Running "start" task  
Running "cd /usr/src/hello-world/current && npm start" on host "my-remote-server.com".  
@my-remote-server.com
@my-remote-server.com > hello-world@0.0.0 start /usr/src/hello-world/releases/20140730152424
@my-remote-server.com > node index
@my-remote-server.com
@my-remote-server.com hello world!

Done, without errors.  

GitHub commit: https://github.com/neoziro/shipit-tutorial/commit/0c733c37e031fedbda32b9c74c7578f8f5fc9808

6. Launch tasks automatically after deployment

Shipit emit some events during each phase of the deployment. You can easily run tasks after an event is emitted.

In our case we need to start the server after the "published" event. So we add to our Gruntfile.js, these lines:

  /**
   * Run start task after deployment.
   */

  grunt.shipit.on('published', function () {
    grunt.task.run(['start']);
  });

Now deploy again:

grunt shipit:staging deploy  

You will see this output:

Running "shipit:staging" (shipit) task

Running "deploy:init" task

Running "deploy:fetch" task  
Create workspace "/tmp/hello-world-workspace"  
>> Workspace created.
Initialize local repository in "/tmp/hello-world-workspace"  
@ Initialized empty Git repository in /private/tmp/hello-world-workspace/.git/
>> Repository initialized.
List local remotes.  
Update remote "https://github.com/neoziro/shipit-tutorial.git" to local repository "/tmp/hello-world-workspace"  
>> Remote updated.
Fetching repository "https://github.com/neoziro/shipit-tutorial.git"  
@ From https://github.com/neoziro/shipit-tutorial
@  * [new branch]      master     -> shipit/master
>> Repository fetched.
Checking out commit-ish "master"  
@ Already on 'master'
@ Branch master set up to track remote branch master from shipit.
>> Checked out.
Testing if commit-ish is a branch.  
@ * master
Commit-ish is a branch, merging...  
@ Already up-to-date.
>> Branch merged.

Running "deploy:update" task  
Create release path "/usr/src/hello-world/releases/20140730154508"  
Running "mkdir -p /usr/src/hello-world/releases/20140730154508" on host "my-remote-server.com".  
>> Release path created.
Copy project to remote servers.  
Remote copy "/tmp/hello-world-workspace/" to "deploy@my-remote-server.com:/usr/src/hello-world/releases/20140730154508"  
>> Finished copy.

Running "deploy:publish" task  
Publishing release "/usr/src/hello-world/releases/20140730154508"  
Running "rm -rf /usr/src/hello-world/current && ln -s /usr/src/hello-world/releases/20140730154508 /usr/src/hello-world/current" on host "my-remote-server.com".  
>> Release published.

Running "start" task  
Running "cd /usr/src/hello-world/current && npm start" on host "my-remote-server.com".  
@my-remote-server.com
@my-remote-server.com > shipit-tutorial@0.0.0 start /usr/src/hello-world/releases/20140730154508
@my-remote-server.com > node index
@my-remote-server.com
@my-remote-server.com hello world!

Running "deploy:clean" task  
Keeping "3" last releases, cleaning others  
Running "(ls -rd /usr/src/hello-world/releases/*|head -n 3;ls -d /usr/src/hello-world/releases/*)|sort|uniq -u|xargs rm -rf" on host "my-remote-server.com".

Done, without errors.  

The server is correctly started after the publishing.

GitHub commit: https://github.com/neoziro/shipit-tutorial/commit/8310344ebe0d77cb10dfc155e614db6a69b50b0e

7. Rollbacking

Imagine that you want to make a rollback, Shipit has already a built-in process for that (you have also some dedicated events to rollbacking database for an example).

To rollback, just run the command:

grunt shipit:staging rollback  

You will see this output:

Running "shipit:staging" (shipit) task

Running "rollback:init" task  
Get current release dirname.  
Running "readlink /usr/src/hello-world/current" on host "my-remote-server.com".  
@my-remote-server.com /usr/src/hello-world/releases/20140730154508
Current release dirname : 20140730154508.  
Getting dist releases.  
Running "ls -r1 /usr/src/hello-world/releases" on host "my-remote-server.com".  
@my-remote-server.com 20140730154508
@my-remote-server.com 20140730152424
@my-remote-server.com 20140730152229
Dist releases : ["20140730154508","20140730152424","20140730152229"].  
Will rollback to 20140730152424.

Running "deploy:publish" task  
Publishing release "/usr/src/hello-world/releases/20140730152424"  
Running "rm -rf /usr/src/hello-world/current && ln -s /usr/src/hello-world/releases/20140730152424 /usr/src/hello-world/current" on host "my-remote-server.com".  
>> Release published.

Running "start" task  
Running "cd /usr/src/hello-world/current && npm start" on host "my-remote-server.com".  
@my-remote-server.com
@my-remote-server.com > hello-world@0.0.0 start /usr/src/hello-world/releases/20140730152424
@my-remote-server.com > node index
@my-remote-server.com
@my-remote-server.com hello world!

Running "deploy:clean" task  
Keeping "3" last releases, cleaning others  
Running "(ls -rd /usr/src/hello-world/releases/*|head -n 3;ls -d /usr/src/hello-world/releases/*)|sort|uniq -u|xargs rm -rf" on host "my-remote-server.com".

Done, without errors.  

The server is rollbacked to the previous version deployed.

You have probably noticed that the server has started, it's because the task "deploy:publish" is used in the rollback too.

Conclusion

Congratulation, your are now a Shipit master!

Shipit is ready for production, we use it for 1 year at Le Monde at we never had a problem with it.

I hope that this tutorial will help you to deploy some projects. You can find the documentation of Shipit in the readme of the repository. I will be happy to accept your pull-requests if you want to contribute to it.

Shipit repository: https://github.com/neoziro/grunt-shipit

comments powered by Disqus