In this article by Fernando Monteiro, author of the book Learning Single-page Web Application Development, we will see how to build a solid foundation for our API. Our main aim is to discuss the techniques to build rich web applications, with the SPA approach.
We will be covering the following topics in this article:
The working of an API
Boilerplates and generators
The speakers API concept
Creating the package.json file
The Node server with server.js
The model with the Mongoose schema
Defining the API routes
Using MongoDB in the cloud
Inserting data with the Postman Chrome extension
(For more resources related to this topic, see here.)
The working of an API
An API works through communication between different codes, thus defining specific behavior of certain objects on an interface. That is, the API will connect several functions on one website (such as search, images, news, authentications, and so on) to enable it to be used in other applications.
Operating systems also have APIs, and they still have the same function. Windows, for example, has APIs such as the Win16 API, Win32 API, or Telephony API, in all its versions. When you run a program that involves some process of the operating system, it is likely that we make a connection with one or more Windows APIs. To clarify the concept of an API, we will give go through some examples of how it works.
On Windows, it works on an application that uses the system clock to display the same function within the program. It then associates a behavior to a given clock time in another application, for example, using the Time/Clock API from Windows to use the clock functionality on your own application.
Another example, is when you use the Android SDK to build mobile applications. When you use the device GPS, you are interacting with the API (android.location) to display the user location on the map through another API, in this case, Google Maps API.
The following is the API example:
When it comes to web APIs, the functionality can be even greater. There are many services that provide their code, so that they can be used on other websites. Perhaps, the best example is the Facebook API. Several other websites use this service within their pages, for instance a like button, share, or even authentication.
An API is a set of programming patterns and instructions to access a software application based on the Web.
So, when you access a page of a beer store in your town, you can log in with your Facebook account. This is accomplished through the API. Using it, software developers and web programmers can create beautiful programs and pages filled with content for their users.
Boilerplates and generators
On a MEAN stack environment, our ecosystem is infinitely diverse, and we can find excellent alternatives to start the construction of our API. At hand, we have simple boilerplates to complex code generators that can be used with other tools in an integrated way, or even alone.
Boilerplates are usually a group of tested code that provides the basic structure to the main goal, that is to create a foundation of a web project. Besides saving us from common tasks such as assembling the basic structure of the code and organizing the files, boilerplates already have a number of scripts to make life easier for the frontend.
Let's describe some alternatives that we consider as good starting points for the development of APIs with the Express framework, MongoDB database, Node server, and AngularJS for the frontend.
Some more accentuated knowledge of JavaScript might be necessary for the complete understanding of the concepts covered here; so we will try to make them as clearly as possible.
It is important to note that everything is still very new when we talk about Node and all its ecosystems, and factors such as scalability, performance, and maintenance are still major risk factors. Bearing in mind also that languages such as Ruby on Rails, Scala, and the Play framework have a higher reputation in building large and maintainable web applications, but without a doubt, Node and JavaScript will conquer your space very soon.
That being said, we present some alternatives for the initial kickoff with MEAN, but remember that our main focus is on SPA and not directly on MEAN stack.
Hackathon starter
Hackathon is highly recommended for a quick start to develop with Node. This is because the boilerplate has the main necessary characteristics to develop applications with the Express framework to build RESTful APIs, as it has no MVC/MVVM frontend framework as a standard but just the Bootstrap UI framework. Thus, you are free to choose the framework of your choice, as you will not need to refactor it to meet your needs.
Other important characteristics are the use of the latest version of the Express framework, heavy use of Jade templates and some middleware such as Passport - a Node module to manage authentication with various social network sites such as Twitter, Facebook, APIs for LinkedIn, Github, Last.fm, Foursquare, and many more.
They provide the necessary boilerplate code to start your projects very fast, and as we said before, it is very simple to install; just clone the Git open source repository:
git clone --depth=1 https://github.com/sahat/hackathon-starter.git myproject
Run the NPM install command inside the project folder:
npm install
Then, start the Node server:
node app.js
Remember, it is very important to have your local database up and running, in this case MongoDB, otherwise the command node app.js will return the error: Error connecting to database: failed to connect to [localhost: 27017]
MEAN.io or MEAN.JS
This is perhaps the most popular and currently available boilerplate. MEAN.JS is a fork of the original project MEAN.io; both are open source, with a very peculiar similarity, both have the same author. You can check for more details at http://meanjs.org/.
However, there are some differences. We consider MEAN.JS to be a more complete and robust environment. It has a structure of directories, better organized, subdivided modules, and better scalability by adopting a vertical modules development.
To install it, follow the same steps as previously:
Clone the repository to your machine:
git clone https://github.com/meanjs/mean.git
Go to the installation directory and type on your terminal:
npm install
Finally, execute the application; this time with the Grunt.js command:
grunt
If you are on Windows, type the following command:
grunt.cmd
Now, you have your app up and running on your localhost.
The most common problem when we need to scale a SPA is undoubtedly the structure of directories and how we manage all of the frontend JavaScript files and HTML templates using MVC/MVVM. Later, we will see an alternative to deal with this on a large-scale application; for now, let's see the module structure adopted by MEAN.JS:
Note that MEAN.JS leaves more flexibility to the AngularJS framework to deal with the MVC approach for the frontend application, as we can see inside the public folder. Also, note the modules approach; each module has its own structure, keeping some conventions for controllers, services, views, config, and tests. This is very useful for team development, so keep all the structure well organized. It is a complete solution that makes use of additional modules such as passport, swig, mongoose, karma, among others.
The Passport module
Some things about the Passport module must be said; it can be defined as a simple, unobtrusive authentication module. It is a powerful middleware to use with Node; it is very flexible and also modular. It can also adapt easily within applications that use the Express.
It has more than 140 alternative authentications and support session persistence; it is very lightweight and extremely simple to be implemented. It provides us with all the necessary structure for authentication, redirects, and validations, and hence it is possible to use the username and password of social networks such as Facebook, Twitter, and others.
The following is a simple example of how to use local authentication:
var passport = require('passport'),
LocalStrategy = require('passport-local').Strategy,
User = require('mongoose').model('User');
module.exports = function() {
// Use local strategy
passport.use(new LocalStrategy({
usernameField: 'username',
passwordField: 'password'
},
function(username, password, done) {
User.findOne({
username: username
},
function(err, user) {
if (err) {
return done(err);
}
if (!user) {
return done(null, false, {
message: 'Unknown user'
});
}
if (!user.authenticate(password)) {
return done(null, false, {
message: 'Invalid password'
});
}
return done(null, user);
});
}
));
};
Here's a sample screenshot of the login page using the MEAN.JS boilerplate with the Passport module:
Back to the boilerplates topic; most boilerplates and generators already have the Passport module installed and ready to be configured. Moreover, it has a code generator so that it can be used with Yeoman, which is another essential frontend tool to be added to your tool belt.
Yeoman is the most popular code generator for scaffold for modern web applications; it's easy to use and it has a lot of generators such as Backbone, Angular, Karma, and Ember to mention a few. More information can be found at http://yeoman.io/.
Generators
Generators are for the frontend as gem is for Ruby on Rails. We can create the foundation for any type of application, using available generators.
Here's a console output from a Yeoman generator:
It is important to bear in mind that we can solve almost all our problems using existing generators in our community. However, if you cannot find the generator you need, you can create your own and make it available to the entire community, such as what has been done with RubyGems by the Rails community.
RubyGem, or simply gem, is a library of reusable Ruby files, labeled with a name and a version (a file called gemspec).
Keep in mind the Don't Repeat Yourself (DRY) concept; always try to reuse an existing block of code. Don't reinvent the wheel.
One of the great advantages of using a code generator structure is that many of the generators that we have currently, have plenty of options for the installation process. With them, you can choose whether or not to use many alternatives/frameworks that usually accompany the generator.
The Express generator
Another good option is the Express generator, which can be found at https://github.com/expressjs/generator.
In all versions up to Express Version 4, the generator was already pre-installed and served as a scaffold to begin development. However, in the current version, it was removed and now must be installed as a supplement.
They provide us with the express command directly in terminal and are quite useful to start the basic settings for utilization of the framework, as we can see in the following commands:
create : .
create : ./package.json
create : ./app.js
create : ./public
create : ./public/javascripts
create : ./public/images
create : ./public/stylesheets
create : ./public/stylesheets/style.css
create : ./routes
create : ./routes/index.js
create : ./routes/users.js
create : ./views
create : ./views/index.jade
create : ./views/layout.jade
create : ./views/error.jade
create : ./bin
create : ./bin/www
install dependencies:
$ cd . && npm install
run the app:
$ DEBUG=express-generator ./bin/www
Very similar to the Rails scaffold, we can observe the creation of the directory and files, including the public, routes, and views folders that are the basis of any application using Express.
Note the npm install command; it installs all dependencies provided with the package.json file, created as follows:
{
"name": "express-generator",
"version": "0.0.1",
"private": true,
"scripts": {
"start": "node ./bin/www"
},
"dependencies": {
"express": "~4.2.0",
"static-favicon": "~1.0.0",
"morgan": "~1.0.0",
"cookie-parser": "~1.0.1",
"body-parser": "~1.0.0",
"debug": "~0.7.4",
"jade": "~1.3.0"
}
}
This has a simple and effective package.json file to build web applications with the Express framework.
The speakers API concept
Let's go directly to build the example API. To be more realistic, let's write a user story similar to a backlog list in agile methodologies.
Let's understand what problem we need to solve by the API.
The user history
We need a web application to manage speakers on a conference event. The main task is to store the following speaker information on an API:
Name
Company
Track title
Description
A speaker picture
Schedule presentation
For now, we need to add, edit, and delete speakers. It is a simple CRUD function using exclusively the API with JSON format files.
Creating the package.json file
Although not necessarily required at this time, we recommend that you install the Webstorm IDE, as we'll use it throughout the article.
Note that we are using the Webstorm IDE with an integrated environment with terminal, Github version control, and Grunt to ease our development. However, you are absolutely free to choose your own environment.
From now on, when we mention terminal, we are referring to terminal Integrated WebStorm, but you can access it directly by the chosen independent editor, terminal for Mac and Linux and Command Prompt for Windows.
Webstorm is very useful when you are using a Windows environment, because Windows Command Prompt does not have the facility to copy and paste like Mac OS X on the terminal window.
Initiating the JSON file
Follow the steps to initiate the JSON file:
Create a blank folder and name it as conference-api, open your terminal, and place the command:
npm init
This command will walk you through creating a package.json file with the baseline configuration for our application. Also, this file is the heart of our application; we can control all the dependencies' versions and other important things like author, Github repositories, development dependencies, type of license, testing commands, and much more.
Almost all commands are questions that guide you to the final process, so when we are done, we'll have a package.json file very similar to this:
{
"name": "conference-api",
"version": "0.0.1",
"description": "Sample Conference Web Application",
"main": "server.js",
"scripts": {
"test": "test"
},
"keywords": [
"api"
],
"author": "your name here",
"license": "MIT"
}
Now, we need to add the necessary dependencies, such as Node modules, which we will use in our process. You can do this in two ways, either directly via terminal as we did here, or by editing the package.json file. Let's see how it works on the terminal first; let's start with the Express framework. Open your terminal in the api folder and type the following command:
npm install
[email protected] –-save
This command installs the Express module, in this case, Express Version 4, and updates the package.json file and also creates dependencies automatically, as we can see:
{
"name": "conference-api",
"version": "0.0.1",
"description": "Sample Conference Web Application",
"main": "server.js",
"scripts": {
"test": "test"
},
"keywords": [
"api"
],
"author": "your name here",
"license": "MIT",
"dependencies": {
"express": "^4.0.0"
}
}
Now, let's add more dependencies directly in the package.json file. Open the file in your editor and add the following lines:
{
"name": "conference-api",
"version": "0.0.1",
"description": "Sample Conference Web Application",
"main": "server.js",
"scripts": {
"test": "test"
},
"keywords": [
"api"
],
"author": "your name here",
"license": "MIT",
"engines": {
"node": "0.8.4",
"npm": "1.1.49"
},
"dependencies": {
"body-parser": "^1.0.1",
"express": "^4.0.0",
"method-override": "^1.0.0",
"mongoose": "^3.6.13",
"morgan": "^1.0.0",
"nodemon": "^1.2.0"
},
}
It's very important when you deploy your application using some services such as Travis Cl or Heroku hosting company. It's always good to set up the Node environment.
Open the terminal again and type the command:
npm install
You can actually install the dependencies in two different ways, either directly into the directory of your application or globally with the -g command. This way, you will have the modules installed to use them in any application.
When using this option, make sure that you are the administrator of the user machine, as this command requires special permissions to write to the root directory of the user.
At the end of the process, we'll have all Node modules that we need for this project; we just need one more action.
Let's place our code over a version control, in our case Git.
More information about the Git can be found at http://git-scm.com however, you can use any version control as subversion or another.
We recommend using Git, as we will need it later to deploy our application in the cloud, more specificly, on Heroku cloud hosting.
At this time, our project folder must have the same structure as that of the example shown here:
We must point out the utilization of an important module called the Nodemon module. Whenever a file changes it restarts the server automatically; otherwise, you will have to restart the server manually every time you make a change to a file, especially in a development environment that is extremely useful, as it constantly updates our files.
Node server with server.js
With this structure formed, we will start the creation of the server itself, which is the creation of a main JavaScript file.
The most common name used is server.js, but it is also very common to use the app.js name, especially in older versions.
Let's add this file to the root folder of the project and we will start with the basic server settings.
There are many ways to configure our server, and probably you'll find the best one for yourself. As we are still in the initial process, we keep only the basics.
Open your editor and type in the following code:
// Import the Modules installed to our server
var express = require('express');
var bodyParser = require('body-parser');
// Start the Express web framework
var app = express();
// configure app
app.use(bodyParser());
// where the application will run
var port = process.env.PORT || 8080;
// Import Mongoose
var mongoose = require('mongoose');
// connect to our database
// you can use your own MongoDB installation at: mongodb://127.0.0.1/databasename
mongoose.connect('mongodb://username:
[email protected]:10073/node-api');
// Start the Node Server
app.listen(port);
console.log('Magic happens on port ' + port);
Realize that the line-making connection with MongoDB on our localhost is commented, because we are using an instance of MongoDB in the cloud. In our case, we use MongoHQ, a MongoDB-hosting service. Later on, will see how to connect with MongoHQ.
Model with the Mongoose schema
Now, let's create our model, using the Mongoose schema to map our speakers on MongoDB.
// Import the Mongoose module.
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
// Set the data types, properties and default values to our Schema.
var SpeakerSchema = new Schema({
name: { type: String, default: '' },
company: { type: String, default: '' },
title: { type: String, default: '' },
description: { type: String, default: '' },
picture: { type: String, default: '' },
schedule: { type: String, default: '' },
createdOn: { type: Date, default: Date.now}
});
module.exports = mongoose.model('Speaker', SpeakerSchema);
Note that on the first line, we added the Mongoose module using the require() function.
Our schema is pretty simple; on the left-hand side, we have the property name and on the right-hand side, the data type. We also we set the default value to nothing, but if you want, you can set a different value.
The next step is to save this file to our project folder. For this, let's create a new directory named server; then inside this, create another folder called models and save the file as speaker.js. At this point, our folder looks like this:
The README.md file is used for Github; as we are using the Git version control, we host our files on Github.
Defining the API routes
One of the most important aspects of our API are routes that we take to create, read, update, and delete our speakers.
Our routes are based on the HTTP verb used to access our API, as shown in the following examples:
To create record, use the POST verb
To read record, use the GET verb
To update record, use the PUT verb
To delete records, use the DELETE verb
So, our routes will be as follows:
Routes
Verb and Action
/api/speakers
GET retrieves speaker's records
/api/speakers/
POST inserts speakers' record
/api/speakers/:speaker_id
GET retrieves a single record
/api/speakers/:speaker_id
PUT updates a single record
/api/speakers/:speaker_id
DELETE deletes a single record
Configuring the API routes:
Let's start defining the route and a common message for all requests:
var Speaker = require('./server/models/speaker');
// Defining the Routes for our API
// Start the Router
var router = express.Router();
// A simple middleware to use for all Routes and Requests
router.use(function(req, res, next) {
// Give some message on the console
console.log('An action was performed by the server.');
// Is very important using the next() function, without this the Route stops here.
next();
});
// Default message when access the API folder through the browser
router.get('/', function(req, res) {
// Give some Hello there message
res.json({ message: 'Hello SPA, the API is working!' });
});
Now, let's add the route to insert the speakers when the HTTP verb is POST:
// When accessing the speakers Routes
router.route('/speakers')
// create a speaker when the method passed is POST
.post(function(req, res) {
// create a new instance of the Speaker model
var speaker = new Speaker();
// set the speakers properties (comes from the request)
speaker.name = req.body.name;
speaker.company = req.body.company;
speaker.title = req.body.title;
speaker.description = req.body.description;
speaker.picture = req.body.picture;
speaker.schedule = req.body.schedule;
// save the data received
speaker.save(function(err) {
if (err)
res.send(err);
// give some success message
res.json({ message: 'speaker successfully created!' });
});
})
For the HTTP GET method, we need this:
// get all the speakers when a method passed is GET
.get(function(req, res) {
Speaker.find(function(err, speakers) {
if (err)
res.send(err);
res.json(speakers);
});
});
Note that in the res.json() function, we send all the object speakers as an answer. Now, we will see the use of different routes in the following steps:
To retrieve a single record, we need to pass speaker_id, as shown in our previous table, so let's build this function:
// on accessing speaker Route by id
router.route('/speakers/:speaker_id')
// get the speaker by id
.get(function(req, res) {
Speaker.findById(req.params.speaker_id, function(err,
speaker) {
if (err)
res.send(err);
res.json(speaker);
});
})
To update a specific record, we use the PUT HTTP verb and then insert the function:
// update the speaker by id
.put(function(req, res) {
Speaker.findById(req.params.speaker_id, function(err, speaker) {
if (err)
res.send(err);
// set the speakers properties (comes from the request)
speaker.name = req.body.name;
speaker.company = req.body.company;
speaker.title = req.body.title;
speaker.description = req.body.description;
speaker.picture = req.body.picture;
speaker.schedule = req.body.schedule;
// save the data received
speaker.save(function(err) {
if (err)
res.send(err);
// give some success message
res.json({ message: 'speaker successfully updated!'});
});
});
})
To delete a specific record by its id:
// delete the speaker by id
.delete(function(req, res) {
Speaker.remove({
_id: req.params.speaker_id
}, function(err, speaker) {
if (err)
res.send(err);
// give some success message
res.json({ message: 'speaker successfully deleted!' });
});
});
Finally, register the Routes on our server.js file:
// register the route
app.use('/api', router);
All necessary work to configure the basic CRUD routes has been done, and we are ready to run our server and begin creating and updating our database.
Open a small parenthesis here, for a quick step-by-step process to introduce another tool to create a database using MongoDB in the cloud.
There are many companies that provide this type of service but we will not go into individual merits here; you can choose your preference. We chose Compose (formerly MongoHQ) that has a free sandbox for development, which is sufficient for our examples.
Using MongoDB in the cloud
Today, we have many options to work with MongoDB, from in-house services to hosting companies that provide Platform as a Service (PaaS) and Software as a Service (SaaS).
We will present a solution called Database as a Service (DbaaS) that provides database services for highly scalable web applications.
Here's a simple step-by-step process to start using a MongoDB instance with a cloud service:
Go to https://www.compose.io/.
Create your free account.
On your dashboard panel, click on add Database.
On the right-hand side, choose Sandbox Database.
Name your database as node-api.
Add a user to your database.
Go back to your database title, click on admin.
Copy the connection string.
The string connection looks like this:
mongodb://<user>:<password>@kahana.mongohq.com:10073/node-api.
Let's edit the server.js file using the following steps:
Place your own connection string to the Mongoose.connect() function.
Open your terminal and input the command:
nodemon server.js
Open your browser and place http://localhost:8080/api.
You will see a message like this in the browser:
{
Hello SPA, the API is working!
}
Remember the api folder was defined on the server.js file when we registered the routes:
app.use('/api', router);
But, if you try to access http://localhost:8080/api/speakers, you must have something like this: []
This is an empty array, because we haven't input any data into MongoDB.
We use an extension for the Chrome browser called JSONView. This way, we can view the formatted and readable JSON files. You can install this for free from the Chrome Web Store.
Inserting data with Postman
To solve our empty database and before we create our frontend interface, let's add some data with the Chrome extension Postman. By the way, it's a very useful browser interface to work with RESTful APIs.
As we already know that our database is empty, our first task is to insert a record. To do so, perform the following steps:
Open Postman and enter http://localhost:8080/api/speakers. Select the x-www-form-urlencoded option and add the properties of our model:
var SpeakerSchema = new Schema({
name: { type: String, default: '' },
company: { type: String, default: '' },
title: { type: String, default: '' },
description: { type: String, default: '' },
picture: { type: String, default: '' },
schedule: { type: String, default: '' },
createdOn: { type: Date, default: Date.now}
});
Now, click on the blue button at the end to send the request.
With everything going as expected, you should see message: speaker successfully created! at the bottom of the screen, as shown in the following screenshot:
Now, let's try http://localhost:8080/api/speakers in the browser again.
Now, we have a JSON file like this, instead of an empty array:
{
"_id": "53a38ffd2cd34a7904000007",
"__v": 0,
"createdOn": "2014-06-20T02:20:31.384Z",
"schedule": "10:20",
"picture": "fernando.jpg",
"description": "Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod...",
"title": "MongoDB",
"company": "Newaeonweb",
"name": "Fernando Monteiro"
}
When performing the same action on Postman, we see the same result, as shown in the following screenshot:
Go back to Postman, copy _id from the preceding JSON file and add to the end of the http://localhost:8080/api/speakers/53a38ffd2cd34a7904000005 URL and click on Send. You will see the same object on the screen.
Now, let's test the method to update the object. In this case, change the method to PUT on Postman and click on Send. The output is shown in the following screenshot:
Note that on the left-hand side, we have three methods under History; now, let's perform the last operation and delete the record. This is very simple to perform; just keep the same URL, change the method on Postman to DELETE, and click on Send. Finally, we have the last method executed successfully, as shown in the following screenshot:
Take a look at your terminal, you can see four messages that are the same: An action was performed by the server. We configured this message in the server.js file when we were dealing with all routes of our API.
router.use(function(req, res, next) {
// Give some message on the console
console.log('An action was performed by the server.');
// Is very important using the next() function, without this the Route stops here.
next();
});
This way, we can monitor all interactions that take place at our API.
Now that we have our API properly tested and working, we can start the development of the interface that will handle all this data.
Summary
In this article, we have covered almost all modules of the Node ecosystem to develop the RESTful API.
Resources for Article:
Further resources on this subject:
Web Application Testing [article]
A look into responsive design frameworks [article]
Top Features You Need to Know About – Responsive Web Design [article]
United States
Great Britain
India
Germany
France
Canada
Russia
Spain
Brazil
Australia
South Africa
Thailand
Ukraine
Switzerland
Slovakia
Luxembourg
Hungary
Romania
Denmark
Ireland
Estonia
Belgium
Italy
Finland
Cyprus
Lithuania
Latvia
Malta
Netherlands
Portugal
Slovenia
Sweden
Argentina
Colombia
Ecuador
Indonesia
Mexico
New Zealand
Norway
South Korea
Taiwan
Turkey
Czechia
Austria
Greece
Isle of Man
Bulgaria
Japan
Philippines
Poland
Singapore
Egypt
Chile
Malaysia
