Showing posts with label tutorial. Show all posts
Showing posts with label tutorial. Show all posts

Monday, September 24, 2012

Create a Simple Node.js and Couchbase application... on OS X

NOTE: The Couchbase Node.js Client Library is currently changing. I will update this article and source code once the API is stable.

I am currently playing a little bit with Node.js . It is quite fun! In this article I won't go in a a very complex application but just give you the basic steps to create your first Node.js+Couchbase application... on Mac OS X.

Installation

Couchbase 2.0 Beta:
You can take a look the first steps of my previous article to install Couchbase. The basics steps are:

  • Download Couchbase 2. 0 Beta 
  • Start the server (Run the "Couchbase Server" application)
  • Configure the server

Other Components :
  • Node.js
  • Couchbase Client Library (C version)
To install these two components I am using homebrew (aka brew).

Hmmm, what is homebrew?

Homebrew is a package manager for OS X that allows you to install, update and uninstall unix tools using very simple commands. You can find more information on the homebrew site. So let's start by installing homebrew itself.

From a terminal window:
ruby -e "$(curl -fsSkL raw.github.com/mxcl/homebrew/go)"

Then let's install node
brew install node

and finally install Couchbase Client Library
brew install https://github.com/couchbase/homebrew/raw/preview/Library/Formula/libcouchbase.rb

Coding

You are now ready to start the development of your application. 

Create a very simple application

1. Create a folder for your application
mkdir sample-app
cd sample-app
2. Create an app.js file and copy the following code:
var http = require("http");

function onRequest(request, response) {
  response.writeHead(200, {"Content-Type": "text/plain"});
  response.write("Hello World");
  response.end();
}

var server = http.createServer(onRequest);
server.listen(8080);

console.log("> SERVER STARTED");
I won't go in all the details about a node application; for this I invite you to read The Node Beginnger Book.

3. Start your server
node app.js
You should be able to access your application using http://localhost:8080
To stop your server just use ctrl+c.

Install and use Couchbase client for node.js

npm is the node package manager, that allows you to easily install node.js modules and manage dependencies for your application (I will present the dependency management in another post)

1. Install the Couchbase Client for node.js, using the following command
npm install couchbase
Couldn't be simpler! You can find more information about Couchbase Client for node on its npm page.

2. Lets now insert some data into Couchbase

Create a simple function that inserts some documents, and call it after server startup:
var http = require("http");

// load the Couchbase driver and connect to the cluster
var driver = require('couchbase');
var cb = new driver.Couchbase("localhost:8091", null, null, "default");

...
...

function insertData() {
 // insert employees in Couchbase
 var emps = [{
  "type": "employee",
  "id": 100,
  "name": "Thomas",
  "dept": "Sales",
  "salary": 5000
 }, {
  "type": "employee",
  "id": 200,
  "name": "John",
  "dept": "Development",
  "salary": 4500
 }, {
  "type": "employee",
  "id": 300,
  "name": "Jane",
  "dept": "Marketing",
  "salary": 5000
 }]

 // Insert the data in Couchbase using the add method ()
 for (index = 0; index < emps.length; index++) {
  cb.add(JSON.stringify(emps[index].id), JSON.stringify(emps[index]), 0, undefined, function(data, err, key, cas) {
   if (err && err != 12) { // 12 : LCB_KEY_EEXISTS  
    console.log("Failed to store object:\n" + err);
   }
  });
 }
}

server.listen(8080, insertData());

  • Lines 4-5 : load the Couchbase driver and connect to the server. I am using the complete list of parameter connect("server:port", "username", "password", "bucket"). But you can use the short version connect("server:port")
  • Lines 12-30 : just create JSON object that will be pushed in Couchbase.
  • Line 33 : the application just read each element of the array and insert them into Couchbase using the couchbase.add() function.
  • Lines 34-38 : the couchbase.add() function set the value only if it does not exist. If the value exists the error code LCB_KEY_EEXISTS (12) is return by the callback function

3. Start your server - node app.js - and check using the Admin Console that employees are inserted into your Couchbase instance. Note that node.js applications do not support hot deployment, so you need to bounce your application when changing the code.


Create and use Couchbase View

Let's now create and use a view to return the employee list.

1. All Couchbase views are accessible using a simple REST API, but you can also use the node.js plugin : baseview; so let's install this module:

npm install baseview


2. Create a new view from your application

You can use the Admin Console to create the view, but it is also possible to do it from your node.js code. So let's add the view programmatically in the insertData function.
var http = require("http");

// load the Couchbase driver and connect to the cluster
var driver = require('couchbase');
var cb = new driver.Couchbase("localhost:8091", null, null, "default");
// load the baseview module
var baseview = require('baseview')({url: 'http://localhost:8092', bucket: 'default'});


...
...

function insertData() {

 //create a new view
  baseview.setDesign('design_employees', {
     'basic_list': {
        'map': "function (doc, meta) { if(doc.type == 'employee') {emit(meta.id, doc.name);}}"
      }
    },
    function(err, res){
  if (err != null) console.log(err);
    }
  ); 

...

}
...
 
This new code, create a new view in Couchbase server:
  • line 7 : load the module and set the properties to call the view services
  • line 16-17 : call the basevie.setDesign() method, that create a view.
  • line 18 : set the map function that list all the employees

3. Let's now call the view in the onRequest function.

function onRequest(request, response) {
 response.writeHead(200, {
  "Content-Type": "text/plain"
 });
 response.write("See list of employees in the console");
 var params = 
  { 'descending'  : 'true'
  , 'include_docs' : 'true'
  };
 baseview.view('design_employees', 'basic_list', params, function(error, data) {
    for( var i = data.rows.length-1,length = data.rows.length ; i >= 0; i-- ) {
    var employee = data.rows[i].doc.json;
    console.log(employee);   
   } 
  });   
 response.end();
}


Calling the view is quite simple :

  • lines 6-8 : create an object to send view parameters. In this example I am just using descending, and include_docs to get the full document as part of the response. You can find the list of all the parameters you can use in the Couchbase documentation : Querying Using the REST API (The baseview module is using REST API to call the views).
  • line 10-14 : just loop on the result content, returned in the data variable, and print the employee information in the console.

Note: Because of the asynchronous nature of node.js, and my lack of experience with node, I was not able to send the list of employee to the HTTP response.

In another article I will explain how to integrate Couchbase with an node.js application based on Express and Socket.io, where I list the Employee in my the Web page.

Below, the complete code of this simple node.js application:
var http = require("http");
var driver = require('couchbase');
var cb = new driver.Couchbase("localhost:8091", null, null, "default");
var baseview = require('baseview')({url: 'http://127.0.0.1:8092',bucket: 'default'});

function onRequest(request, response) {
 response.writeHead(200, {
  "Content-Type": "text/plain"
 });
 response.write("See list of employees in the console");
 var params = {
  'descending': 'true',
  'include_docs': 'true'
 };
 baseview.view('design_employees', 'basic_list', params, function(error, data) {
  for (var i = data.rows.length - 1, length = data.rows.length; i >= 0; i--) {
   var employee = data.rows[i].doc.json;
   console.log(employee);
  }
 });
 response.end();
}

function insertData() {
 //create a new view
 baseview.setDesign('design_employees', {
  'basic_list': {
   'map': "function (doc, meta) { if(doc.type == 'employee') {emit(meta.id, doc.name);}}"
  }
 }, function(err, res) {
  if (err != null) console.log(err);
 });

 // insert employees in Couchbase
 var emps = [{
  "type": "employee",
  "id": 100,
  "name": "Thomas",
  "dept": "Sales",
  "salary": 5000
 }, {
  "type": "employee",
  "id": 200,
  "name": "John",
  "dept": "Development",
  "salary": 4500
 }, {
  "type": "employee",
  "id": 300,
  "name": "Jane",
  "dept": "Marketing",
  "salary": 5000
 }]


 // Insert the data in Couchbase using the add method ()
 for (index = 0; index < emps.length; index++) {
  cb.add(JSON.stringify(emps[index].id), JSON.stringify(emps[index]), 0, undefined, function(data, err, key, cas) {
   if (err && err != 12) { // 12 : LCB_KEY_EEXISTS  
    console.log("Failed to store object:\n" + err);
   }
  });
 }
}

var server = http.createServer(onRequest);

server.listen(8080, insertData());

console.log("> SERVER STARTED");

Friday, July 6, 2012

Couchbase 101 : install, store and query data

Introduction

In this post I just want to show how easily is to get started with Couchbase, and also explain how to “query” the data. The basic steps of this tutorial are:
  1. Install Couchbase 
  2. Create Data 
  3. Query Data 

I will try to post more articles, if I have time to show how to use Couchbase from your applications (starting with Java).

Prerequisites :
  •  Could not be simpler : Couchbase 2.0 available here. (Currently in Developer Preview)

Couchbase 101 : Insert and Query data

Installation

I am using Couchbase on Mac OS X, so let me describe the installation in this environment. If you are using other operating system just take a look to the Couchbase documentation.

Couchbase installation is very (very!) fast:
  1. Download the Mac OS X Zip file.
  2. Double-click the downloaded Zip installation file to extract the contents. This will create a single file, the Couchbase.app application.
  3. Drag and Drop the Couchbase.app to your chosen installation folder, such as the system Applications folder.

Start and Configure Couchbase Server


To start Couchbase Server, just double click on the Couchbase Server. Once the server is started, a new icon is added in the OS X Menu to indicate that the Server is up and running.

You can now configure your Couchbase instance, for this you just need to access the Admin Console, available at the following location http://127.0.0.1:8091 (change the IP address if needed) or simply by going in the Couchbase menu and click on Open Admin Console entry.



  1. Welcome Screen : Click Setup
  2. Set the disk and cluster configuration. On my instance I keep the default location for the on disk storage. Just configure the size of the memory usage for your instance, for example 800Mb. So far, we have a single instance, so no need to join a cluster.
  3. Choose to generate sample data. This will be interesting to learn more about data and views.
  4. Create the default bucket (use for testing only). A bucket is used by Couchbase to store data. It could be compared to a “database” in RDBMS world.
  5. Configure update notifications to be alerted when new version of Couchbase is released
  6. Configure the server with a final step with the administrator username and password
  7. When this is done you are automatically redirected to the Admin Console.
This is it! You are ready to use your Couchbase server.

Couchbase has many interesting features, especially around scalability and elasticity but for not in this article let's focus on the basics :
  • Insert some data and query them

Insert Data

Couchbase has many ways to manipulate data from you favorite programming language using the different client libraries : Java, Python, PHP, Ruby, .Net, C. For now let's use the Admin Console to create and query data.

Couchbase can store any type of data, but when you need to manipulate some data with a structure the best way is to use JSON Documents. So let's use the console and create documents.

To create new documents in your database, click on the "Data Buckets" tab. If you have installed the sample you see 2 buckets: default and gamesim-sample.

Let's create a new documents in the default bucket:
  1. Click on Documents button
  2. Click on Create Document
  3. Since each document must have an id for example 100.
  4. Couchbase save the document and add some metadata such as _rev, $flags, expiration
  5. Add new attributes to the document that describe an employee : Name, Departement and Salary, then save it. You just need to update the JSON object with values
    {
      "_id": "100",
      "name": "Thomas",
      "dept": "Sales",
      "salary": 5000
    }

Repeat the operation with some other employees :
200,Jason,Technology,5500
300,Mayla,Technology,7000
400,Nisha,Marketing,9500
500,Randy,Technology,6000
501,Ritu,Accounting,5400


You have now a list of employees in your database. That was easy isn't? Let's now query them.

Query Data

Access document directly from its ID

First of all you can quickly access a document using a simple HTTP request using its id. For example to access the Mayla with the id 300 just enter the following URL:
  • http://127.0.0.1:8092/default/300  
In this URL you have :
  • 8092 is the Couch API REST port used to access data (where 8091 is the port for the Admin console)
  • default is the bucket in which the document is stored
  • 300 is the id of the document

Search your data with queries

So we have seen how you can access one document. But what if my need is :
  •  "Give me all the employee of the Technology department"
To achieve such query it is necessary to create views. The views are used by Couchbase server to index and search your data. A view is a Map function written in JavaScript, that will generate a key value list that is compliant with logic you put in the Map function. Note that this key,value is now indexed and sorted by key. This is important when you query your data.
So let's create a new view from the Admin Console:
  1. Click on the Views tab (be sure you are on the default bucket)
  2. Click on the "Create Development View"
  3. Enter the Document and View name:
    • Document Name : _design/dev_dept
    • View Name : dept
  4. Cick Save
  5. Click on your View to edit it
Since we need to provide the list of employees that are part of a the Technology department, we need to create a view that use the department as key, so the map function looks like :

function (doc) {
  emit(doc.dept, null);
}

Save the view

This function takes the document and create a list that contains the "dept" as key and null as value. The value itself is not that important in our case. A simple rule will be : do not put too much data in the value since at the end Couchbase server creates an index with this map. Will see that Couchbase allows developer to easily get the document information when accessing a view.

Click on the "Show Results" button, the result will look like:
{"total_rows":6,"rows":[
  {"id":"501","key":"Accounting","value":null},
  {"id":"400","key":"Marketing","value":null},
  {"id":"100","key":"Sales","value":null},
  {"id":"200","key":"Technology","value":null},
  {"id":"300","key":"Technology","value":null},
  {"id":"500","key":"Technology","value":null}
]
}

As we have seen in earlier it is possible to access the document using a single URL, it is the same for views. You can for example access the view we have just created using the following URL:

Now it is possible to use query parameter to filter the results using the key parameter with the value entered using a JSON String :
The result of this query is now :

{"total_rows":6,"rows":[
  {"id":"200","key":"Technology","value":null},
  {"id":"300","key":"Technology","value":null},
  {"id":"500","key":"Technology","value":null}
]
}

You have many other parameters you can use when accessing a view to control the size, the time out, .... One of them is quite interesting is include_docs that ask Couchbase to include the full content of the document in the result. So if you call :

The result is :

{"total_rows":6,"rows":[
  {"id":"200","key":"Technology","value":null,"doc":  {"_id":"200","_rev":"1-1de6e6751608eada0000003200000000","$flags":0,"$expiration":0,"name":"Jason","dept":"Technology","salary":5500}},
  {"id":"300","key":"Technology","value":null,"doc":{"_id":"300","_rev":"1-f3e44cee742bfae10000003200000000","$flags":0,"$expiration":0,"name":"Mayla","dept":"Technology","salary":7000}},
  {"id":"500","key":"Technology","value":null,"doc":  {"_id":"500","_rev":"1-05780359aac8f3790000003200000000","$flags":0,"$expiration":0,"name":"Randy","dept":"Technology","salary":6000}}
]
}



Let's now create a little more complicated view to answer the following business requirement: "Give me all the employee with a salary between 5000 and 6000"
So now you know that you need to create a new view with the salary as key let's with the following Map function:

function (doc) {
  emit(doc.salary, null);
}

Couchbase is automatically sorting the key when creating/updating the index so, let's use the startkey and endkey parameter when calling the view. So let's call the view with from the following URL:
The result is :
{"total_rows":6,"rows":[
  {"id":"100","key":5000,"value":null,"doc":{"_id":"100","_rev":"1-0f33580d780014060000002e00000000","$flags":0,"$expiration":0,"name":"Thomas","dept":"Sales","salary":5000}},
  {"id":"501","key":5400,"value":null,"doc":{"_id":"501","_rev":"1-b1fe5bc79637720e0000003100000000","$flags":0,"$expiration":0,"name":"Ritu","dept":"Accounting","salary":5400}},
  {"id":"200","key":5500,"value":null,"doc":{"_id":"200","_rev":"1-1de6e6751608eada0000003200000000","$flags":0,"$expiration":0,"name":"Jason","dept":"Technology","salary":5500}},
  {"id":"500","key":6000,"value":null,"doc":{"_id":"500","_rev":"1-05780359aac8f3790000003200000000","$flags":0,"$expiration":0,"name":"Randy","dept":"Technology","salary":6000}}
]
}


Conclusion

In this short article you have learn how to:
  • Install Couchbase
  • Create data using the Admin Console
  • Query data with views
When I get more time I will write another article that do the same from Java, and other languages.