MongoDB – cuteprogramming

WebMatrix + MongoLab + Windows Azure

This post is to continue the story of my MongoDB self-learning back in January. Also, the theme for March self-learning is about Windows Azure, thus I guess it’s a good opportunity to combine these two knowledge together. So, let’s continue the story now.

Basically, after the one-month MongoDB learning in January, I have successfully built a simple web application allowing users to add pinpoints on Google Map and store those info on MongoDB. However, all those are happening in local machine. So, how to do that if we would like to deploy it on, for example, Azure for the public to access?

Fortunately, with the help of Microsoft WebMatrix, the whole process is rather simple and straight-forward.

Deploy The Website in 3 Simple Steps

Firstly, there is a Publish feature available on WebMatrix. After adding your Windows account on WebMatrix, there is a simple Publish interface which allows you to publish our current website to either a new site or existing site on Azure.

Create a new website on Windows Azure with WebMatrix.

Secondly, we need to create new MongoDB database on cloud. Windows Azure Store offers a web-based cloud MongoDB management tool, called MongoLab. Currently, MongoLab provides a free sandbox plan for the developers to try out MongoDB on Windows Azure. It also provides some other plans with Replica Set cluster on shared or dedicated Azure VMs. Normally those are for large and heavy traffic sites. For learning purpose, a free plan with 0.5 GB of storage is enough.

MongoLab: One of the MongoDB hosting platforms available on Windows Azure.

Thirdly, once the MongoLab service is added, we can now happily get the Connection Info of the database and then paste it to our code in WebMatrix.

var dbc = monk('mongodb://GCLMongoDB:.../GCLMongoDB');

Connection Info can be found on Windows Azure Portal.

Finally, we can just hit the Publish button on WebMatrix to launch the website on Windows Azure with MongoDB. Ta-da!

MongoLab Helpful Features

In MongoLab, we get to see the documents either in list view or even the table view. List view allows us to read all the documents stored in the collection in JSON format. We can scroll through a consecutive set of documents each in its entirely. By clicking on a document, we then can edit and delete the selected document.

Documents can be presented as JSON List View in MongoLab.

In table view, we get to choose the format of the displayed table by defining how to translate JSON documents in the collection into a 2D table. This is especially useful for those who are familiar with relational database but are still new to document database.

Documents can be shown as table in MongoLab as well.

In addition, there is an editor provided to do query. A friendly quick reference of query displayed at the side of the page to guide new developers along on how to do query also.

For the database backup, there is a charge of $0.50 per run + $0.02 per run per GB if we store our backups in MongoLab-owned cloud container. Hence, even for a small-sized database that I have above (2.49 KB), I will already be charged for around $15 monthly for 30 backups.

Conclusion

My friend once said that I used too much Microsoft developer tools and products without knowing what have really been handled by them in the background secretly. I think it’s kind of true. As we can see, to deploy both the website and MongoDB on Windows Azure, it took only a few simple steps as shown above. Thus, I’d encourage to learn in this way only if you are totally new to MongoDB and you would just like to have an overview of how a Node.JS website can work together with MongoDB on the cloud.

If you want to learn more about MongoDB, you can also checkout the following slides from the presentation in Singapore MongoDB User Group Meetup #2. The first half of the slides basically cover some fundamental knowledge about MongoDB which is quite useful for those who are new to this document database.

The first meetup organized by local MongoDB user group was held at Hackerspace Singapore on last Tuesday. During the meetup, Matias Cascallares, consulting engineer from MongoDB, shared with us the importance of schema design in a “schemaless” MongoDB, the use cases of different schema design approach, and ended his talk with the new features that will be released in MongoDB 2.6.

Documents and Collections

MongoDB is an open-source document oriented database, an example of the NoSQL database, which is non-relational and horizontally scalable. Many of the databases currently in use are based on relational database model where we have tables and each table has columns and keys defined in database schema.

In my job, I deal with relational database. Sometimes there is a need to store two types of records in the same table. Even though they share some fields with one another, each of them also has its own unique elements. So, I simply combine all the fields and defined them as the columns in the table, leaving unused fields empty or null.

Not all records have discount info in the table

MongoDB stores the data in documents. Documents are stored on disk in BSON, a binary representation of JSON objects. A grouping of documents is called a collection, which is equivalent of a table in relational database model. So documents in MongoDB are equivalent of rows in a table. Because of “schemaless”, collections do not enforce document structure. Documents within a collection can have different fields. Thus, it’s allowed to write the two records shown in the image above into two documents as follows.

{
    ID: 1,
    ItemCode: "T934",
    CreateUser: "User01",
    CreateDate: "2014-01-01"
}
{
    ID: 2,
    ItemCode: "T987",
    DiscountID: 4,
    DiscountAmount: 0.32,
    CreateUser: "User02",
    CreateDate: "2014-02-01"
}

Schema in "schemaless" MongoDB? — Schema design in “schemaless” MongoDB?

Different Approach of Schema Design in MongoDB

I use relational database in my work. I realize that when the tables are growing bigger and bigger, it’s getting harder and harder to adjust the schema. I always hope to have the columns to be dynamic. Hence, the concept of schemaless database in MongoDB interests me a lot.

In relational database, we can use stuff like joins to provide results to a query. However, in MongoDB, joins are not supported. The data is either denormalized or stored together with related data in the same document. In addition, we can also use the _id field of one document in another document as a reference. The application just needs to run a second query to return the referenced data. Hence, it’s like doing a reverse engineering when we are designing schema in MongoDB because now we have to ask “What question will I have”.

Also, due to fact that documents can have different fields, we cannot describe the collection. Instead, we have to look into the codes to find out the schema information of the collections. Even though when we query with a key not existing in any of the document in the collection, it will still not raise an error. It just won’t return anything. So, it is impossible to describe the collection as how we do in relational database.

Shards and Shard Keys

Normally, we use the strongest servers we have to host our databases. We do vertical scaling by adding more CPU, RAM and storage resources to increase capacity. This gets expensive quickly. There is where horizontal scaling can come into play. Horizontal scaling, aka sharding, does a horizontal partition in a database. Instead of just increase the capacity of the one server, we add more servers. The data is then distributed to multiple database servers, aka shards.

MongoDB is built with horizontal scaling in mind. Sharding is implemented in MongoDB with the help of sharded cluster which consits of three components: shards, config server, and router. Router is in charge of routing the reads and writes from applications to the shards by refering to the metadata stored in the config server.

To shard a collection, shard key is needed to divide a collection’s data across the cluster’s shards. During the talk, Matias analyzed three different approaches of schema design for a social network application which has to support a great amount of read and write operations. By just changing the shard key used in the shard and the schema, we manage to get 3 different solutions. One of them is good at writing performance. One is good at reading performance with lots of random IO. Final one is good at reading performance with no random IO but with more work in write. For more details about these three schema design approaches, please read the blog post “Schema Design for Social Inboxes in MongoDB”.

Data Expiration

Data expiration is mentioned in the second half of the talk about schema design.

Data expiration is useful and important because some data stored in database will not be used anymore. However, data won’t expire itself.

In MS SQL Server, we create scheduled jobs with stored procedures to erase the data which has been there for more than 2 months, for example.

In Matias’ talk, I came to know about the feature that was introduced back in MongoDB 2.2, the Time To Live (TTL) Collection. By just specifying a value in the expireAfterSeconds index option, the documents in the TTL Collection will automatically removed after specific period of time.

That’s All

Important slide, the summary. Photo Credit: Singapore MongoDB User Group

The talk is very interesting. I agree that schema design in MongoDB is not trivial. As mentioned during the talk, it does require a lot of practices to get really, really good at schema design in MongoDB and know how to balance between the read performance and write performance. Thus, the talk is actually just a starting point for a beginner like me. It’s now up to me to find out more about MongoDB and other database technologies myself. Hopefully one day I’ll be so imba that I have the opportunity to give a talk about MongoDB as well.

For those who were there will now realize that I do not cover everything of the talk in this post. Firstly, I don’t want my post to be too lengthy. Most of the readers will just tl;dr. Secondly, I just started to learn MongoDB last month, so I try not to “act smart” here. Thirdly, it’s to encourage people to join our Singapore MongoDB User Group to find out more (such as the upcoming talks and other activities) instead of just reading my post. =P

I’d also like to take this opportunity to thank my friend, Laurence, for inviting me to attend this amazing talk. I’m now already looking forward to the next meetup in April.

Tag: MongoDB

Deploy MongoDB to Azure: It’s Never Been Easier