This article is penned by Nathan Kozyra, the author of the book, Mastering Go Web Services. This quickly introduces—or reintroduces—some core concepts related to Go setup and usage as well as the http package.
(For more resources related to this topic, see here.)
If you spend any time developing applications on the Web (or off it, for that matter), it won't be long before you find yourself facing the prospect of interacting with a web service or an API.
Whether it's a library that you need or another application's sandbox with which you have to interact, the world of development relies in no small part on the cooperation among dissonant applications, languages, and formats.
That, after all, is why we have APIs to begin with—to allow standardized communication between any two given platforms.
If you spend a long amount of time working on the Web, you'll encounter bad APIs. By bad we mean APIs that are not all-inclusive, do not adhere to best practices and standards, are confusing semantically, or lack consistency. You'll encounter APIs that haphazardly use OAuth or simple HTTP authentication in some places and the opposite in others, or more commonly, APIs that ignore the stated purposes of HTTP verbs.
Google's Go language is particularly well suited to servers. With its built-in HTTP serving, a simple method for XML and JSON encoding of data, high availability, and concurrency, it is the ideal platform for your API.
We will cover the following topics in this article:
- Understanding requirements and dependencies
- Introducing the HTTP package
Understanding requirements and dependencies
Before we get too deep into the weeds in this article, it would be a good idea for us to examine the things that you will need to have installed.
Installing Go
It should go without saying that we will need to have the Go language installed. However, there are a few associated items that you will also need to install in order to do everything we do in this book.
Go is available for Mac OS X, Windows, and most common Linux variants. You can download the binaries at http://golang.org/doc/install.
On Linux, you can generally grab Go through your distribution's package manager. For example, you can grab it on Ubuntu with a simple apt-get install golang command. Something similar exists for most distributions.
In addition to the core language, we'll also work a bit with the Google App Engine, and the best way to test with the App Engine is to install the Software Development Kit (SDK). This will allow us to test our applications locally prior to deploying them and simulate a lot of the functionality that is provided only on the App Engine.
The App Engine SDK can be downloaded from https://developers.google.com/appengine/downloads.
While we're obviously most interested in the Go SDK, you should also grab the Python SDK as there are some minor dependencies that may not be available solely in the Go SDK.
Installing and using MySQL
We'll be using quite a few different databases and datastores to manage our test and real data, and MySQL will be one of the primary ones.
We will use MySQL as a storage system for our users; their messages and their relationships will be stored in our larger application (we will discuss more about
this in a bit).
MySQL can be downloaded from http://dev.mysql.com/downloads/.
You can also grab it easily from a package manager on Linux/OS X as follows:
Redis
Redis is the first of the two NoSQL datastores that we'll be using for a couple of different demonstrations, including caching data from our databases as well as the API output.
If you're unfamiliar with NoSQL, we'll do some pretty simple introductions to results gathering using both Redis and Couchbase in our examples. If you know MySQL, Redis will at least feel similar, and you won't need the full knowledge base to be able to use the application in the fashion in which we'll use it for our purposes.
Redis can be downloaded from http://redis.io/download.
Redis can be downloaded on Linux/OS X using the following:
Couchbase
As mentioned earlier, Couchbase will be our second NoSQL solution that we'll use in various products, primarily to set short-lived or ephemeral key store lookups to avoid bottlenecks and as an experiment with in-memory caching.
Unlike Redis, Couchbase uses simple REST commands to set and receive data, and everything exists in the JSON format.
Couchbase can be downloaded from http://www.couchbase.com/download.
-
For Ubuntu (deb), use the following command to download Couchbase:
dpkg -i couchbase-server version.deb
-
For OS X with Homebrew use the following command to download Couchbase:
brew install
https://github.com/couchbase/homebrew/raw/
stable/Library/Formula/libcouchbase.rb
Nginx
Although Go comes with everything you need to run a highly concurrent, performant web server, we're going to experiment with wrapping a reverse proxy around our results. We'll do this primarily as a response to the real-world issues regarding availability and speed. Nginx is not available natively for Windows.
-
For Ubuntu, use the following command to download Nginx:
apt-get install nginx
-
For OS X with Homebrew, use the following command to download Nginx:
brew install nginx
Apache JMeter
We'll utilize JMeter for benchmarking and tuning our API for performance. You have a bit of a choice here, as there are several stress-testing applications for simulating traffic. The two we'll touch on are JMeter and Apache's built-in Apache Benchmark (AB) platform. The latter is a stalwart in benchmarking but is a bit limited in what you can throw at your API, so JMeter is preferred.
One of the things that we'll need to consider when building an API is its ability to stand up to heavy traffic (and introduce some mitigating actions when it cannot), so we'll need to know what our limits are.
Apache JMeter can be downloaded from http://jmeter.apache.org/download_jmeter.cgi.
Using predefined datasets
While it's not entirely necessary to have our dummy dataset, you can save a lot of time as we build our social network by bringing it in because it is full of users, posts, and images.
By using this dataset, you can skip creating this data to test certain aspects of the API and API creation.
Our dummy dataset can be downloaded at https://github.com/nkozyra/masteringwebservices.
Choosing an IDE
A choice of Integrated Development Environment (IDE) is one of the most personal choices a developer can make, and it's rare to find a developer who is not steadfastly passionate about their favorite.
Nothing in this article will require one IDE over another; indeed, most of Go's strength in terms of compiling, formatting, and testing lies at the command-line level. That said, we'd like to at least explore some of the more popular choices for editors and IDEs that exist for Go.
Eclipse
As one of the most popular and expansive IDEs available for any language, Eclipse is an obvious first mention. Most languages get their support in the form of an Eclipse plugin and Go is no exception.
There are some downsides to this monolithic piece of software; it is occasionally buggy on some languages, notoriously slow for some autocompletion functions, and is a bit heavier than most of the other available options.
However, the pluses are myriad. Eclipse is very mature and has a gigantic community from which you can seek support when issues arise. Also, it's free to use.
Sublime Text
Sublime Text is our particular favorite, but it comes with a large caveat—it is the only one listed here that is not free.
This one feels more like a complete code/text editor than a heavy IDE, but it includes code completion options and the ability to integrate the Go compilers (or other languages' compilers) directly into the interface.
Although Sublime Text's license costs $70, many developers find its elegance and speed to be well worth it. You can try out the software indefinitely to see if it's right for you; it operates as nagware unless and until you purchase a license.
Sublime Text can be downloaded from http://www.sublimetext.com/2.
LiteIDE
LiteIDE is a much younger IDE than the others mentioned here, but it is noteworthy because it has a focus on the Go language.
It's cross-platform and does a lot of Go's command-line magic in the background, making it truly integrated. LiteIDE also handles code autocompletion, go fmt, build, run, and test directly in the IDE and a robust package browser.
It's free and totally worth a shot if you want something lean and targeted directly for the Go language.
LiteIDE can be downloaded from https://code.google.com/p/golangide/.
IntelliJ IDEA
Right up there with Eclipse is the JetBrains family of IDE, which has spanned approximately the same number of languages as Eclipse. Ultimately, both are primarily built with Java in mind, which means that sometimes other language support can feel secondary.
The Go integration here, however, seems fairly robust and complete, so it's worth a shot if you have a license. If you do not have a license, you can try the Community Edition, which is free.
Some client-side tools
Although the vast majority of what we'll be covering will focus on Go and API services, we will be doing some visualization of client-side interactions with our API.
In doing so, we'll primarily focus on straight HTML and JavaScript, but for our more interactive points, we'll also rope in jQuery and AngularJS.
Most of what we do for client-side demonstrations will be available at this book's GitHub repository at https://github.com/nkozyra/goweb under client.
Both jQuery and AngularJS can be loaded dynamically from Google's CDN, which will prevent you from having to download and store them locally. The examples hosted on GitHub call these dynamically.
To load AngularJS dynamically, use the following code:
<script src="//ajax.googleapis.com/ajax/libs/
angularjs/1.2.18/angular.min.js"></script>
To load jQuery dynamically, use the following code:
<script src="//ajax.googleapis.com/ajax/
libs/jquery/1.11.1/jquery.min.js"></script>
Looking at our application
Well in the book, we'll be building myriad small applications to demonstrate points, functions, libraries, and other techniques. However, we'll also focus on a larger project that mimics a social network wherein we create and return to users, statuses, and so on, via the API. For that you'll need to have a copy of it.
Setting up our database
As mentioned earlier, we'll be designing a social network that operates almost entirely at the API level (at least at first) as our master project in the book. Time and space wouldn't allow us to cover this here in the article.
When we think of the major social networks (from the past and in the present), there are a few omnipresent concepts endemic among them, which are as follows:
- The ability to create a user and maintain a user profile
- The ability to share messages or statuses and have conversations based on them
- The ability to express pleasure or displeasure on the said statuses/messages to dictate the worthiness of any given message
There are a few other features that we'll be building here, but let's start with the basics. Let's create our database in MySQL as follows:
create database social_network;
This will be the basis of our social network product in the book. For now, we'll just need a users table to store our individual users and their most basic information. We'll amend this to include more features as we go along:
CREATE TABLE users (
user_id INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
user_nickname VARCHAR(32) NOT NULL,
user_first VARCHAR(32) NOT NULL,
user_last VARCHAR(32) NOT NULL,
user_email VARCHAR(128) NOT NULL,
PRIMARY KEY (user_id),
UNIQUE INDEX user_nickname (user_nickname)
)
We won't need to do too much in this article, so this should suffice. We'll have a user's most basic information—name, nickname, and e-mail, and not much else.
Introducing the HTTP package
The vast majority of our API work will be handled through REST, so you should become pretty familiar with Go's http package.
In addition to serving via HTTP, the http package comprises of a number of other very useful utilities that we'll look at in detail. These include cookie jars, setting up clients, reverse proxies, and more.
The primary entity about which we're interested right now, though, is the http.Server struct, which provides the very basis of all of our server's actions and parameters. Within the server, we can set our TCP address, HTTP multiplexing for routing specific requests, timeouts, and header information.
Go also provides some shortcuts for invoking a server without directly initializing the struct. For example, if you have a lot of default properties, you could use the following code:
Server := Server {
Addr: ":8080",
Handler: urlHandler,
ReadTimeout: 1000 * time.MicroSecond,
WriteTimeout: 1000 * time.MicroSecond,
MaxHeaderBytes: 0,
TLSConfig: nil
}
You can simply execute using the following code:
http.ListenAndServe(":8080", nil)
This will invoke a server struct for you and set only the Addr and Handler properties within.
There will be times, of course, when we'll want more granular control over our server, but for the time being, this will do just fine. Let's take this concept and output some JSON data via HTTP for the first time.
Quick hitter – saying Hello, World via API
As mentioned earlier in this article, we'll go off course and do some work that we'll preface with quick hitter to denote that it's unrelated to our larger project.
In this case, we just want to rev up our http package and deliver some JSON to the browser. Unsurprisingly, we'll be merely outputting the uninspiring Hello, world message to, well, the world.
Let's set this up with our required package and imports:
package main
import
(
"net/http"
"encoding/json"
"fmt"
)
This is the bare minimum that we need to output a simple string in JSON via HTTP. Marshalling JSON data can be a bit more complex than what we'll look at here, so if the struct for our message doesn't immediately make sense, don't worry.
This is our response struct, which contains all of the data that we wish to send to the client after grabbing it from our API:
type API struct {
Message string "json:message"
}
There is not a lot here yet, obviously. All we're setting is a single message string in the obviously-named Message variable.
Finally, we need to set up our main function (as follows) to respond to a route and deliver a marshaled JSON response:
func main() {
http.HandleFunc("/api", func(w http.ResponseWriter, r
*http.Request) {
message := API{"Hello, world!"}
output, err := json.Marshal(message)
if err != nil {
fmt.Println("Something went wrong!")
}
fmt.Fprintf(w, string(output))
})
http.ListenAndServe(":8080", nil)
}
Upon entering main(), we set a route handling function to respond to requests at /api that initializes an API struct with Hello, world! We then marshal this to a JSON byte array, output, and after sending this message to our iowriter class (in this case, an http.ResponseWriter value), we cast that to a string.
The last step is a kind of quick-and-dirty approach for sending our byte array through a function that expects a string, but there's not much that could go
wrong in doing so.
Go handles typecasting pretty simply by applying the type as a function that flanks the target variable. In other words, we can cast an int64 value to an integer by simply surrounding it with the int(OurInt64) function. There are some exceptions to this—types that cannot be directly cast and some other pitfalls, but that's the general idea. Among the possible exceptions, some types cannot be directly cast to others and some require a package like strconv to manage typecasting.
If we head over to our browser and call localhost:8080/api (as shown in the following screenshot), you should get exactly what we expect, assuming everything went correctly:
Summary
We've touched on the very basics of developing a simple web service interface in Go. Admittedly, this particular version is extremely limited and vulnerable to attack, but it shows the basic mechanisms that we can employ to produce usable, formalized output that can be ingested by other services.
At this point, you should have the basic tools at your disposal that are necessary to start refining this process and our application as a whole.
Resources for Article:
Further resources on this subject:
United States
Great Britain
India
Germany
France
Canada
Russia
Spain
Brazil
Australia
South Africa
Thailand
Ukraine
Switzerland
Slovakia
Luxembourg
Hungary
Romania
Denmark
Ireland
Estonia
Belgium
Italy
Finland
Cyprus
Lithuania
Latvia
Malta
Netherlands
Portugal
Slovenia
Sweden
Argentina
Colombia
Ecuador
Indonesia
Mexico
New Zealand
Norway
South Korea
Taiwan
Turkey
Czechia
Austria
Greece
Isle of Man
Bulgaria
Japan
Philippines
Poland
Singapore
Egypt
Chile
Malaysia
