Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletter Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials - Programming

1081 Articles
Packt
16 Feb 2016
3 min read
Save for later

Machine learning and Python – the Dream Team

Packt
16 Feb 2016
3 min read
In this article we will be learning more about machine learning and Python. Machine learning (ML) teaches machines how to carry out tasks by themselves. It is that simple. The complexity comes with the details, and that is most likely the reason you are reading this article. (For more resources related to this topic, see here.) Machine learning and Python – the dream team The goal of machine learning is to teach machines (software) to carry out tasks by providing them a couple of examples (how to do or not do a task). Let us assume that each morning when you turn on your computer, you perform the same task of moving e-mails around so that only those e-mails belonging to a particular topic end up in the same folder. After some time, you feel bored and think of automating this chore. One way would be to start analyzing your brain and writing down all the rules your brain processes while you are shuffling your e-mails. However, this will be quite cumbersome and always imperfect. While you will miss some rules, you will over-specify others. A better and more future-proof way would be to automate this process by choosing a set of e-mail meta information and body/folder name pairs and let an algorithm come up with the best rule set. The pairs would be your training data, and the resulting rule set (also called model) could then be applied to future e-mails, which we have not yet seen. This is machine learning in its simplest form. Of course, machine learning (often also referred to as data mining or predictive analysis) is not a brand new field in itself. Quite the contrary, its success over the recent years can be attributed to the pragmatic way of using rock-solid techniques and insights from other successful fields; for example, statistics. There, the purpose is for us humans to get insights into the data by learning more about the underlying patterns and relationships. As you read more and more about successful applications of machine learning (you have checked out kaggle.com already, haven't you?), you will see that applied statistics is a common field among machine learning experts. As you will see later, the process of coming up with a decent ML approach is never a waterfall-like process. Instead, you will see yourself going back and forth in your analysis, trying out different versions of your input data on diverse sets of ML algorithms. It is this explorative nature that lends itself perfectly to Python. Being an interpreted high-level programming language, it may seem that Python was designed specifically for the process of trying out different things. What is more, it does this very fast. Sure enough, it is slower than C or similar statically typed programming languages; nevertheless, with a myriad of easy-to-use libraries that are often written in C, you don't have to sacrifice speed for agility. Summary In this is article we learned about machine learning and its goals. To learn more please refer to the following books: Building Machine Learning Systems with Python - Second Edition (https://www.packtpub.com/big-data-and-business-intelligence/building-machine-learning-systems-python-second-edition) Expert Python Programming (https://www.packtpub.com/application-development/expert-python-programming) Resources for Article:   Further resources on this subject: Python Design Patterns in Depth – The Observer Pattern [article] Python Design Patterns in Depth: The Factory Pattern [article] Customizing IPython [article]
Read more
  • 0
  • 0
  • 1164

article-image-unlocking-javascript-core
Packt
16 Feb 2016
19 min read
Save for later

Unlocking the JavaScript Core

Packt
16 Feb 2016
19 min read
You may have owned an iPhone for years and regard yourself as an experienced user. At the same time, you keep removing unwanted characters one at a time while typing by pressing delete. However, one day you find out that a quick shake allows you to delete the whole message in one tap. Then you wonder why on earth you didn't know this earlier. The same thing happens with programming. We can be quite satisfied with our coding until, all of sudden, we run into a trick or a lesser-known language feature that makes us reconsider the entire work done over the years. It turns out that we could do this in a cleaner, more readable, more testable, and more maintainable way. So it's presumed that you already have experience with JavaScript; however, this article equips you with the best practices to improve your code. (For more resources related to this topic, see here.) We will cover the following topics: Making your code readable and expressive Mastering multiline strings in JavaScript Manipulating arrays in the ES5 way Traversing an object in an elegant, reliable, safe, and fast way The most effective way of declaring objects How to magic methods in JavaScript Make your code readable and expressive There are numerous practices and heuristics to make a code more readable, expressive, and clean. We will cover this topic later on, but here we will talk about syntactic sugar. The term means an alternative syntax that makes the code more expressive and readable. In fact, we already had some of this in JavaScript from the very beginning. For instance, the increment/decrement and addition/subtraction assignment operators inherited from C. foo++ is syntactic sugar for foo = foo + 1, and foo += bar is a shorter form for foo = foo + bar. Besides, we have a few tricks that serve the same purpose. JavaScript applies logical expressions to so-called short-circuit evaluation. This means that an expression is read left to right, but as soon as the condition result is determined at an early stage, the expression tail is not evaluated. If we have true || false || false, the interpreter will know from the first test that the result is true regardless of other tests. So the false || false part is not evaluated, and this opens a way for creativity. Function argument default value When we need to specify default values for parameters we can do like that: function stub( foo ) { return foo || "Default value"; } console.log( stub( "My value" ) ); // My value console.log( stub() ); // Default value What is going on here? When foo is true (not undefined, NaN, null, false, 0, or ""), the result of the logical expression is foo otherwise the expression is evaluated until Default value and this is the final result. Starting with 6th edition of EcmaScript (specification of JavaScript language) we can use nicer syntax: function stub( foo = "Default value" ) { return foo; } Conditional invocation While composing our code we shorten it on conditions:" var age = 20; age >= 18 && console.log( "You are allowed to play this game" ); age >= 18 || console.log( "The game is restricted to 18 and over" ); In the preceding example, we used the AND (&&) operator to invoke console.log if the left-hand condition is Truthy. The OR (||) operator does the opposite, it calls console.log if the condition is Falsy. I think the most common case in practice is the shorthand condition where the function is called only when it is provided: /** * @param {Function} [cb] - callback */ function fn( cb ) { cb && cb(); }; The following is one more example on this: /** * @class AbstractFoo */ AbstractFoo = function(){ // call this.init if the subclass has init method this.init && this.init(); }; Syntactic sugar was introduced to its full extent to the JavaScript world only with the advance in CoffeeScript, a subset of the language that trans-compiles (compiles source-to-source) into JavaScript. Actually CoffeeScript, inspired by Ruby, Python, and Haskell, has unlocked arrow-functions, spreads, and other syntax to JavaScript developers. In 2011, Brendan Eich (the author of JavaScript) admitted that CoffeeScript influenced him in his work on EcmaScript Harmony, which was finalized this summer in ECMA-262 6th edition specification. From a marketing perspective, the specification writers agreed on using a new name convention that calls the 6th edition as EcmaScript 2015 and the 7th edition as EcmaScript 2016. Yet the community is used to abbreviations such as ES6 and ES7. To avoid confusion further in the book, we will refer to the specifications by these names. Now we can look at how this affects the new JavaScript. Arrow functions Traditional function expression may look like this: function( param1, param2 ){ /* function body */ } When declaring an expression using the arrow function (aka fat arrow function) syntax, we will have this in a less verbose form, as shown in the following: ( param1, param2 ) => { /* function body */ } In my opinion, we don't gain much with this. But if we need, let's say, an array method callback, the traditional form would be as follows: function( param1, param2 ){ return expression; } Now the equivalent arrow function becomes shorter, as shown here: ( param1, param2 ) => expression We may do filtering in an array this way: // filter all the array elements greater than 2 var res = [ 1, 2, 3, 4 ].filter(function( v ){ return v > 2; }) console.log( res ); // [3,4] Using an array function, we can do filtering in a cleaner form: var res = [ 1, 2, 3, 4 ].filter( v => v > 2 ); console.log( res ); // [3,4] Besides shorter function declaration syntax, the arrow functions bring the so called lexical this. Instead of creating its own context, it uses the context of the surrounding object as shown here: "use strict"; /** * @class View */ let View = function(){ let button = document.querySelector( "[data-bind="btn"]" ); /** * Handle button clicked event * @private */ this.onClick = function(){ console.log( "Button clicked" ); }; button.addEventListener( "click", () => { // we can safely refer surrounding object members this.onClick(); }, false ); } In the preceding example, we subscribed a handler function to a DOM event (click). Within the scope of the handler, we still have access to the view context (this), so we don't need to bind the handler to the outer scope or pass it as a variable through the closure: var that = this; button.addEventListener( "click", function(){ // cross-cutting concerns that.onClick(); }, false ); Method definitions As mentioned in the preceding section, arrow functions can be quite handy when declaring small inline callbacks, but always applying it for a shorter syntax is controversial. However, ES6 provides new alternative method definition syntax besides the arrow functions. The old-school method declaration may look as follows: var foo = { bar: function( param1, param2 ) { } } In ES6 we can get rid of the function keyword and the colon. So the preceding code can be put this way: let foo = { bar ( param1, param2 ) { } } The rest operator Another syntax structure that was borrowed from CoffeeScript came to JavaScript as the rest operator (albeit, the approach is called splats in CoffeeScript). When we had a few mandatory function parameters and an unknown number of rest parameters, we used to do something like this: "use strict"; var cb = function() { // all available parameters into an array var args = [].slice.call( arguments ), // the first array element to foo and shift foo = args.shift(), // the new first array element to bar and shift bar = args.shift(); console.log( foo, bar, args ); }; cb( "foo", "bar", 1, 2, 3 ); // foo bar [1, 2, 3] Now check out how expressive this code becomes in ES6: let cb = function( foo, bar, ...args ) { console.log( foo, bar, args ); } cb( "foo", "bar", 1, 2, 3 ); // foo bar [1, 2, 3] Function parameters aren't the only application of the rest operator. For example, we can use it in destructions as well, as follows: let [ bar, ...others ] = [ "bar", "foo", "baz", "qux" ]; console.log([ bar, others ]); // ["bar",["foo","baz","qux"]] The spread operator Similarly, we can spread array elements into arguments: let args = [ 2015, 6, 17 ], relDate = new Date( ...args ); console.log( relDate.toString() ); // Fri Jul 17 2015 00:00:00 GMT+0200 (CEST) Mastering multiline strings in JavaScript Multi-line strings aren't a good part of JavaScript. While they are easy to declare in other languages (for instance, NOWDOC), you cannot just keep single-quoted or double-quoted strings in multiple lines. This will lead to syntax error as every line in JavaScript is considered as a possible command. You can set backslashes to show your intention: var str = "Lorem ipsum dolor sit amet, n consectetur adipiscing elit. Nunc ornare, n diam ultricies vehicula aliquam, mauris n ipsum dapibus dolor, quis fringilla leo ligula non neque"; This kind of works. However, as soon as you miss a trailing space, you get a syntax error, which is not easy to spot. While most script agents support this syntax, it's, however, not a part of the EcmaScript specification. In the times of EcmaScript for XML (E4X), we could assign a pure XML to a string, which opened a way for declarations such as these: var str = <>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc ornare </>.toString(); Nowadays E4X is deprecated, it's not supported anymore. Concatenation versus array join We can also use string concatenation. It may feel clumsy, but it's safe: var str = "Lorem ipsum dolor sit amet, n" + "consectetur adipiscing elit. Nunc ornare,n" + "diam ultricies vehicula aliquam, mauris n" + "ipsum dapibus dolor, quis fringilla leo ligula non neque"; You may be surprised, but concatenation is slower than array joining. So the following technique will work faster: var str = [ "Lorem ipsum dolor sit amet, n", "consectetur adipiscing elit. Nunc ornare,n", "diam ultricies vehicula aliquam, mauris n", "ipsum dapibus dolor, quis fringilla leo ligula non neque"].join( "" ); Template literal What about ES6? The latest EcmaScript specification introduces a new sort of string literal, template literal: var str = `Lorem ipsum dolor sit amet, n consectetur adipiscing elit. Nunc ornare, n diam ultricies vehicula aliquam, mauris n ipsum dapibus dolor, quis fringilla leo ligula non neque`; Now the syntax looks elegant. But there is more. Template literals really remind us of NOWDOC. You can refer any variable declared in the scope within the string: "use strict"; var title = "Some title", text = "Some text", str = `<div class="message"> <h2>${title}</h2> <article>${text}</article> </div>`; console.log( str ); The output is as follows: <div class="message"> <h2>Some title</h2> <article>Some text</article> </div> If you wonder when can you safely use this syntax, I have a good news for you—this feature is already supported by (almost) all the major script agents (http://kangax.github.io/compat-table/es6/). Multi-line strings via transpilers With the advance of ReactJS, Facebook's EcmaScript language extension named JSX (https://facebook.github.io/jsx/) is now really gaining momentum. Apparently influenced by previously mentioned E4X, they proposed a kind of string literal for XML-like content without any screening at all. This type supports template interpolation similar to ES6 templates: "use strict"; var Hello = React.createClass({ render: function() { return <div class="message"> <h2>{this.props.title}</h2> <article>{this.props.text}</article> </div>; } }); React.render(<Hello title="Some title" text="Some text" />, node); Another way to declare multiline strings is by using CommonJS Compiler (http://dsheiko.github.io/cjsc/). While resolving the 'require' dependencies, the compiler transforms any content that is not .js/.json content into a single-line string: foo.txt Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc ornare, diam ultricies vehicula aliquam, mauris ipsum dapibus dolor, quis fringilla leo ligula non neque consumer.js var str = require( "./foo.txt" ); console.log( str ); Manipulating arrays in the ES5 way Some years ago when the support of ES5 features was poor (EcmaScript 5th edition was finalized in 2009), libraries such as Underscore and Lo-Dash got highly popular as they provided a comprehensive set of utilities to deal with arrays/collections. Today, many developers still use third-party libraries (including jQuery/Zepro) for methods such as map, filter, every, some, reduce, and indexOf, while these are available in the native form of JavaScript. It still depends on how you use such libraries, but it may likely happen that you don't need them anymore. Let's see what we have now in JavaScript. Array methods in ES5 Array.prototype.forEach is probably the most used method of the arrays. That is, it is the native implementation of _.each, or for example, of the $.each utilities. As parameters, forEach expects an iteratee callback function and optionally a context in which you want to execute the callback. It passes to the callback function an element value, an index, and the entire array. The same parameter syntax is used for most array manipulation methods. Note that jQuery's $.each has the inverted callback parameters order: "use strict"; var data = [ "bar", "foo", "baz", "qux" ]; data.forEach(function( val, inx ){ console.log( val, inx ); }); Array.prototype.map produces a new array by transforming the elements of a given array: "use strict"; var data = { bar: "bar bar", foo: "foo foo" }, // convert key-value array into url-encoded string urlEncStr = Object.keys( data ).map(function( key ){ return key + "=" + window.encodeURIComponent( data[ key ] ); }).join( "&" ); console.log( urlEncStr ); // bar=bar%20bar&foo=foo%20foo Array.prototype.filter returns an array, which consists of given array values that meet the callback's condition: "use strict"; var data = [ "bar", "foo", "", 0 ], // remove all falsy elements filtered = data.filter(function( item ){ return !!item; }); console.log( filtered ); // ["bar", "foo"] Array.prototype.reduce/Array.prototype.reduceRight retrieves the product of values in an array. The method expects a callback function and optionally the initial value as arguments. The callback function receive four parameters: the accumulative value, current one, index and original array. So we can, for an instance, increment the accumulative value by the current one (return acc += cur;) and, thus, we will get the sum of array values. Besides calculating with these methods, we can concatenate string values or arrays: "use strict"; var data = [[ 0, 1 ], [ 2, 3 ], [ 4, 5 ]], arr = data.reduce(function( prev, cur ) { return prev.concat( cur ); }), arrReverse = data.reduceRight(function( prev, cur ) { return prev.concat( cur ); }); console.log( arr ); // [0, 1, 2, 3, 4, 5] console.log( arrReverse ); // [4, 5, 2, 3, 0, 1] Array.prototype.some tests whether any (or some) values of a given array meet the callback condition: "use strict"; var bar = [ "bar", "baz", "qux" ], foo = [ "foo", "baz", "qux" ], /** * Check if a given context (this) contains the value * @param {*} val * @return {Boolean} */ compare = function( val ){ return this.indexOf( val ) !== -1; }; console.log( bar.some( compare, foo ) ); // true In this example, we checked whether any of the bar array values are available in the foo array. For testability, we need to pass a reference of the foo array into the callback. Here we inject it as context. If we need to pass more references, we would push them in a key-value object. As you probably noticed, we used in this example Array.prototype.indexOf. The method works the same as String.prototype.indexOf. This returns an index of the match found or -1. Array.prototype.every tests whether every value of a given array meets the callback condition: "use strict"; var bar = [ "bar", "baz" ], foo = [ "bar", "baz", "qux" ], /** * Check if a given context (this) contains the value * @param {*} val * @return {Boolean} */ compare = function( val ){ return this.indexOf( val ) !== -1; }; console.log( bar.every( compare, foo ) ); // true If you are still concerned about support for these methods in a legacy browser as old as IE6-7, you can simply shim them with https://github.com/es-shims/es5-shim. Array methods in ES6 In ES6, we get just a few new methods that look rather like shortcuts over the existing functionality. Array.prototype.fill populates an array with a given value, as follows: "use strict"; var data = Array( 5 ); console.log( data.fill( "bar" ) ); // ["bar", "bar", "bar", "bar", "bar"] Array.prototype.includes explicitly checks whether a given value exists in the array. Well, it is the same as arr.indexOf( val ) !== -1, as shown here: "use strict"; var data = [ "bar", "foo", "baz", "qux" ]; console.log( data.includes( "foo" ) ); Array.prototype.find filters out a single value matching the callback condition. Again, it's what we can get with Array.prototype.filter. The only difference is that the filter method returns either an array or a null value. In this case, this returns a single element array, as follows: "use strict"; var data = [ "bar", "fo", "baz", "qux" ], match = function( val ){ return val.length < 3; }; console.log( data.find( match ) ); // fo Traversing an object in an elegant, reliable, safe, and fast way It is a common case when we have a key-value object (let's say options) and need to iterate it. There is an academic way to do this, as shown in the following code: "use strict"; var options = { bar: "bar", foo: "foo" }, key; for( key in options ) { console.log( key, options[ key] ); } The preceding code outputs the following: bar bar foo foo Now let's imagine that any of the third-party libraries that you load in the document augments the built-in Object: Object.prototype.baz = "baz"; Now when we run our example code, we will get an extra undesired entry: bar bar foo foo baz baz The solution to this problem is well known, we have to test the keys with the Object.prototype.hasOwnProperty method: //… for( key in options ) { if ( options.hasOwnProperty( key ) ) { console.log( key, options[ key] ); } } Iterating the key-value object safely and fast Let's face the truth—the structure is clumsy and requires optimization (we have to perform the hasOwnProperty test on every given key). Luckily, JavaScript has the Object.keys method that retrieves all string-valued keys of all enumerable own (non-inherited) properties. This gives us the desired keys as an array that we can iterate, for instance, with Array.prototype.forEach: "use strict"; var options = { bar: "bar", foo: "foo" }; Object.keys( options ).forEach(function( key ){ console.log( key, options[ key] ); }); Besides the elegance, we get a better performance this way. In order to see how much we gain, you can run this online test in distinct browsers such as: http://codepen.io/dsheiko/pen/JdrqXa. Enumerating an array-like object Objects such as arguments and nodeList (node.querySelectorAll, document.forms) look like arrays, in fact they are not. Similar to arrays, they have the length property and can be iterated in the for loop. In the form of objects, they can be traversed in the same way that we previously examined. But they do not have any of the array manipulation methods (forEach, map, filter, some and so on). The thing is we can easily convert them into arrays as shown here: "use strict"; var nodes = document.querySelectorAll( "div" ), arr = Array.prototype.slice.call( nodes ); arr.forEach(function(i){ console.log(i); }); The preceding code can be even shorter: arr = [].slice.call( nodes ) It's a pretty convenient solution, but looks like a trick. In ES6, we can do the same conversion with a dedicated method: arr = Array.from( nodes ); The collections of ES6 ES6 introduces a new type of objects—iterable objects. These are the objects whose elements can be retrieved one at a time. They are quite the same as iterators in other languages. Beside arrays, JavaScript received two new iterable data structures, Set and Map. Set which are a collection of unique values: "use strict"; let foo = new Set(); foo.add( 1 ); foo.add( 1 ); foo.add( 2 ); console.log( Array.from( foo ) ); // [ 1, 2 ] let foo = new Set(), bar = function(){ return "bar"; }; foo.add( bar ); console.log( foo.has( bar ) ); // true The map is similar to a key-value object, but may have arbitrary values for the keys. And this makes a difference. Imagine that we need to write an element wrapper that provides jQuery-like events API. By using the on method, we can pass not only a handler callback function but also a context (this). We bind the given callback to the cb.bind( context ) context. This means addEventListener receives a function reference different from the callback. How do we unsubscribe the handler then? We can store the new reference in Map by a key composed from an event name and a callback function reference: "use strict"; /** * @class * @param {Node} el */ let El = function( el ){ this.el = el; this.map = new Map(); }; /** * Subscribe a handler on event * @param {String} event * @param {Function} cb * @param {Object} context */ El.prototype.on = function( event, cb, context ){ let handler = cb.bind( context || this ); this.map.set( [ event, cb ], handler ); this.el.addEventListener( event, handler, false ); }; /** * Unsubscribe a handler on event * @param {String} event * @param {Function} cb */ El.prototype.off = function( event, cb ){ let handler = cb.bind( context ), key = [ event, handler ]; if ( this.map.has( key ) ) { this.el.removeEventListener( event, this.map.get( key ) ); this.map.delete( key ); } }; Any iterable object has methods, keys, values, and entries, where the keys work the same as Object.keys and the others return array values and an array of key-value pairs respectively. Now let's see how we can traverse the iterable objects: "use strict"; let map = new Map() .set( "bar", "bar" ) .set( "foo", "foo" ), pair; for ( pair of map ) { console.log( pair ); } // OR let map = new Map([ [ "bar", "bar" ], [ "foo", "foo" ], ]); map.forEach(function( value, key ){ console.log( key, value ); }); Iterable objects have manipulation methods such as arrays. So we can use forEach. Besides, they can be iterated by for...in and for...of loops. The first one retrieves indexes and the second, the values. Summary This article gives practices and tricks on how to use the JavaScript core features for the maximum effect. This article also discusses the techniques to improve the expressiveness of the code, to master multi-line strings and templating, and to manipulate arrays and array-like objects. Further, we are introduced to the "magic methods" of JavaScript and gives a practical example of their use. JavaScript was born as a scripting language at the most inappropriate time—the time of browser wars. It was neglected and misunderstood for a decade and endured six editions. And look at it now! JavaScript has become a mainstream programming language. You can learn more about JavaScript with the help of the following books: https://www.packtpub.com/application-development/mastering-javascript-design-patterns https://www.packtpub.com/application-development/javascript-promises-essentials https://www.packtpub.com/application-development/learning-javascript-data-structures-and-algorithms Resources for Article: Further resources on this subject: Using JavaScript with HTML[article] Dart with JavaScript[article] Creating a basic JavaScript plugin[article]
Read more
  • 0
  • 0
  • 1244

Packt
16 Feb 2016
11 min read
Save for later

Python Design Patterns in Depth – The Observer Pattern

Packt
16 Feb 2016
11 min read
In this atricle you will see a group of objects when the state of another object changes. A very popular example lies in the Model-View-Controller (MVC) pattern. Assume that we are using the data of the same model in two views, for instance in a pie chart and in a spreadsheet. Whenever the model is modified, both the views need to be updated. That's the role of the Observer design pattern [Eckel08, page 213]. The Observer pattern describes a publish-subscribe relationship between a single object, : the publisher, which is also known as the subject or observable, and one or more objects, : the subscribers, also known as observers. In the MVC example, the publisher is the model and the subscribers are the views. However, MVC is not the only publish-subscribe example. Subscribing to a news feed such as RSS or Atom is another example. Many readers can subscribe to the feed typically using a feed reader, and every time a new item is added, they receive the update automatically. The ideas behind Observer are the same as the ideas behind MVC and the separation of concerns principle, that is, to increase decoupling between the publisher and subscribers, and to make it easy to add/remove subscribers at runtime. Additionally, the publisher is not concerned about who its observers are. It just sends notifications to all the subscribers [GOF95, page 327]. (For more resources related to this topic, see here.) A real-life example In reality, an auction resembles Observer. Every auction bidder has a number paddle that is raised whenever they want to place a bid. Whenever the paddle is raised by a bidder, the auctioneer acts as the subject by updating the price of the bid and broadcasting the new price to all bidders (subscribers). The following figure, courtesy of www.sourcemaking.com, [j.mp/observerpat], shows how the Observer pattern relates to an auction: A software example The django-observer package [j.mp/django-obs] is a third-party Django package that can be used to register callback functions that are executed when there are changes in several Django fields. Many different types of fields are supported (CharField, IntegerField, and so forth). RabbitMQ is a library that can be used to add asynchronous messaging support to an application. Several messaging protocols are supported, such as HTTP and AMQP. RabbitMQ can be used in a Python application to implement a publish-subscribe pattern, which is nothing more than the Observer design pattern [j.mp/rabbitmqobs]. Use cases We generally use the Observer pattern when we want to inform/update one or more objects (observers/subscribers) about a change that happened to another object (subject/publisher/observable). The number of observers as well as who the observers are may vary and can be changed dynamically (at runtime). We can think of many cases where Observer can be useful. Whether it is RSS, Atom, or another format, the idea is the same; you follow a feed, and every time it is updated, you receive a notification about the update [Zlobin13, page 60]. The same concept exists in social networking. If you are connected to another person using a social networking service, and your connection updates something, you are notified about it. It doesn't matter if the connection is a Twitter user that you follow, a real friend on Facebook, or a business colleague on LinkedIn. Event-driven systems are another example where Observer can be (and usually is) used. In such systems, listeners are used to "listen" for specific events. The listeners are triggered when an event they are listening to is created. This can be typing a specific key (of the keyboard), moving the mouse, and more. The event plays the role of the publisher and the listeners play the role of the observers. The key point in this case is that multiple listeners (observers) can be attached to a single event (publisher) [j.mp/magobs]. Implementation We will implement a data formatter. The ideas described here are based on the ActiveState Python Observer code recipe [j.mp/pythonobs]. There is a default formatter that shows a value in the decimal format. However, we can add/register more formatters. In this example, we will add a hex and binary formatter. Every time the value of the default formatter is updated, the registered formatters are notified and take action. In this case, the action is to show the new value in the relevant format. Observer is actually one of the patterns where inheritance makes sense. We can have a base Publisher class that contains the common functionality of adding, removing, and notifying observers. Our DefaultFormatter class derives from Publisher and adds the formatter-specific functionality. We can dynamically add and remove observers on demand. The following class diagram shows an instance of the example using two observers: HexFormatter and BinaryFormatter. Note that, because class diagrams are static, they cannot show the whole lifetime of a system, only the state of it at a specific point in time. We begin with the Publisher class. The observers are kept in the observers list. The add() method registers a new observer, or throws an error if it already exists. The remove() method unregisters an existing observer, or throws an exception if it does not exist. Finally, the notify() method informs all observers about a change: class Publisher: def __init__(self): self.observers = [] def add(self, observer): if observer not in self.observers: self.observers.append(observer) else: print('Failed to add: {}'.format(observer)) def remove(self, observer): try: self.observers.remove(observer) except ValueError: print('Failed to remove: {}'.format(observer)) def notify(self): [o.notify(self) for o in self.observers] Let's continue with the DefaultFormatter class. The first thing that __init__() does is call __init__() method of the base class, since this is not done automatically in Python. A DefaultFormatter instance has name to make it easier for us to track its status. We use name mangling in the _data variable to state that it should not be accessed directly. Note that this is always possible in Python [Lott14, page 54] but fellow developers have no excuse for doing so, since the code already states that they shouldn't. There is a serious reason for using name mangling in this case. Stay tuned. DefaultFormatter treats the _data variable as an integer, and the default value is zero: class DefaultFormatter(Publisher): def __init__(self, name): Publisher.__init__(self) self.name = name self._data = 0 The __str__() method returns information about the name of the publisher and the value of _data. type(self).__name__ is a handy trick to get the name of a class without hardcoding it. It is one of those things that make the code less readable but easier to maintain. It is up to you to decide if you like it or not: def __str__(self): return "{}: '{}' has data = {}".format(type(self).__name__, self.name, self._data) There are two data() methods. The first one uses the @property decorator to give read access to the _data variable. Using this, we can just execute object.data instead of object.data(): @property def data(self): return self._data The second data() method is more interesting. It uses the @setter decorator, which is called every time the assignment (=) operator is used to assign a new value to the _data variable. This method also tries to cast a new value to an integer, and does exception handling in case this operation fails: @data.setter def data(self, new_value): try: self._data = int(new_value) except ValueError as e: print('Error: {}'.format(e)) else: self.notify() The next step is to add the observers. The functionality of HexFormatter and BinaryFormatter is very similar. The only difference between them is how they format the value of data received by the publisher, that is, in hexadecimal and binary, respectively: class HexFormatter: def notify(self, publisher): print("{}: '{}' has now hex data = {}".format(type(self).__name__, publisher.name, hex(publisher.data))) class BinaryFormatter: def notify(self, publisher): print("{}: '{}' has now bin data = {}".format(type(self).__name__, publisher.name, bin(publisher.data))) No example is fun without some test data. The main() function initially creates a DefaultFormatter instance named test1 and afterwards attaches (and detaches) the two available observers. Exception handling is also exercised to make sure that the application does not crash when erroneous data is passed by the user. Moreover, things such as trying to add the same observer twice or removing an observer that does not exist should cause no crashes: def main(): df = DefaultFormatter('test1') print(df) print() hf = HexFormatter() df.add(hf) df.data = 3 print(df) print() bf = BinaryFormatter() df.add(bf) df.data = 21 print(df) print() df.remove(hf) df.data = 40 print(df) print() df.remove(hf) df.add(bf) df.data = 'hello' print(df) print() df.data = 15.8 print(df) Here's how the full code of the example (observer.py) looks: class Publisher: def __init__(self): self.observers = [] def add(self, observer): if observer not in self.observers: self.observers.append(observer) else: print('Failed to add: {}'.format(observer)) def remove(self, observer): try: self.observers.remove(observer) except ValueError: print('Failed to remove: {}'.format(observer)) def notify(self): [o.notify(self) for o in self.observers] class DefaultFormatter(Publisher): def __init__(self, name): Publisher.__init__(self) self.name = name self._data = 0 def __str__(self): return "{}: '{}' has data = {}".format(type(self).__name__, self.name, self._data) @property def data(self): return self._data @data.setter def data(self, new_value): try: self._data = int(new_value) except ValueError as e: print('Error: {}'.format(e)) else: self.notify() class HexFormatter: def notify(self, publisher): print("{}: '{}' has now hex data = {}".format(type(self).__name__, publisher.name, hex(publisher.data))) class BinaryFormatter: def notify(self, publisher): print("{}: '{}' has now bin data = {}".format(type(self).__name__, publisher.name, bin(publisher.data))) def main(): df = DefaultFormatter('test1') print(df) print() hf = HexFormatter() df.add(hf) df.data = 3 print(df) print() bf = BinaryFormatter() df.add(bf) df.data = 21 print(df) print() df.remove(hf) df.data = 40 print(df) print() df.remove(hf) df.add(bf) df.data = 'hello' print(df) print() df.data = 15.8 print(df) if __name__ == '__main__': main() Executing observer.py gives the following output: >>> python3 observer.py DefaultFormatter: 'test1' has data = 0 HexFormatter: 'test1' has now hex data = 0x3 DefaultFormatter: 'test1' has data = 3 HexFormatter: 'test1' has now hex data = 0x15 BinaryFormatter: 'test1' has now bin data = 0b10101 DefaultFormatter: 'test1' has data = 21 BinaryFormatter: 'test1' has now bin data = 0b101000 DefaultFormatter: 'test1' has data = 40 Failed to remove: <__main__.HexFormatter object at 0x7f30a2fb82e8> Failed to add: <__main__.BinaryFormatter object at 0x7f30a2fb8320> Error: invalid literal for int() with base 10: 'hello' BinaryFormatter: 'test1' has now bin data = 0b101000 DefaultFormatter: 'test1' has data = 40 BinaryFormatter: 'test1' has now bin data = 0b1111 DefaultFormatter: 'test1' has data = 15 What we see in the output is that as the extra observers are added, more (and relevant) output is shown, and when an observer is removed, it is not notified any longer. That's exactly what we want: runtime notifications that we are able to enable/disable on demand. The defensive programming part of the application also seems to work fine. Trying to do funny things such as removing an observer that does not exist or adding the same observer twice is not allowed. The messages shown are not very user-friendly but I leave that up to you as an exercise. Runtime failures of trying to pass a string when the API expects a number are also properly handled without causing the application to crash/terminate. This example would be much more interesting if it were interactive. Even a simple menu that allows the user to attach/detach observers at runtime and modify the value of DefaultFormatter would be nice because the runtime aspect becomes much more visible. Feel free to do it. Another nice exercise is to add more observers. For example, you can add an octal formatter, a roman numeral formatter, or any other observer that uses your favorite representation. Be creative and have fun! Summary In this article, we covered the Observer design pattern. We use Observer when we want to be able to inform/notify all stakeholders (an object or a group of objects) when the state of an object changes. An important feature of observer is that the number of subscribers/observers as well as who the subscribers are may vary and can be changed at runtime. You can refer more books on this topic mentioned as follows: Expert Python Programming: https://www.packtpub.com/application-development/expert-python-programming Learning Python Design Patterns: https://www.packtpub.com/application-development/learning-python-design-patterns Resources for Article: Further resources on this subject: Python Design Patterns in Depth: The Singleton Pattern[article] Python Design Patterns in Depth: The Factory Pattern[article] Automating Processes with ModelBuilder and Python[article]
Read more
  • 0
  • 0
  • 8127
Banner background image

article-image-getting-started-rstudio
Packt
16 Feb 2016
5 min read
Save for later

Getting Started with RStudio

Packt
16 Feb 2016
5 min read
The number of users adopting the R programming language has been increasing faster and faster in the last few years. The functions of the R console are limited when it comes to managing a lot of files, or when we want to work with version control systems. This is the reason, in combination with the increasing adoption rate, why a need for a better development environment arose. To serve this need, a team of R fans began to develop an integrated development environment (IDE) to make it easier to work on bigger projects and to collaborate with others. This IDE has the name, RStudio. In this article, we will see how to work with RStudio and projects (For more resources related to this topic, see here.) Working with RStudio and projects In the times before RStudio, it was very hard to manage bigger projects with R in the R console, as you had to create all the folder structures on your own. When you work with projects or open a project, RStudio will instantly take several actions. For example, it will start a new and clean R session, it will source the .Rprofile file in the project's main directory, and it will set the current working directory to the project directory. So, you have a complete working environment individually for every project. RStudio will even adjust its own settings, such as active tabs, splitter positions, and so on, to where they were when the project was closed. But just because you can create projects with RStudio easily, it does not mean that you should create a project for every single time that you write R code. For example, if you just want to do a small analysis, we would recommend that you create a project where you save all your smaller scripts. Creating a project with RStudio RStudio offers you an easy way to create projects. Just navigate to File | New Project and you will see a popup window as with the options shown in the following screenshot: These options let you decide from where you want to create your project. So, if you want to start it from scratch and create a new directory, associate your new project to an existing one, or if you want to create a project from a version control repository, you can avail of the respective options. For now, we will focus on creating a new directory. The following screenshot shows you the next options available: Locating your project A very important question you have to ask yourself when creating a new project is where you want to save it? There are several options and details you have to pay attention to especially when it comes to collaboration and different people working on the same project. You can save your project locally, on a cloud storage or with the help of a revision control system such as Git. Creating your first project To begin your first project, choose the New Directory option we described before and create an empty project. Then, choose a name for the directory and the location that you want to save it in. You should create a projects folder on your Dropbox. The first project will be a small data analysis based on a dataset that was extracted from the 1974 issue of the Motor Trend US magazine. It comprises fuel consumption and ten aspects of automobile design and performance, such as the weight or number of cylinders for 32 automobiles, and is included in the base R package. So, we do not have to install a separate package to work with this dataset, as it is automatically loaded when you start R: As you can see, we left the Use packrat with this project option unchecked. Packrat is a dependency management tool that makes your R code more isolated, portable, and reproducible by giving your project its own privately managed package library. This is especially important when you want to create projects in an organizational context where the code has to run on various computer systems, and has to be usable for a lot of different users. This first project will just run locally and will not focus on a specific combination of package versions. Organizing your folders RStudio creates an empty directory for you that includes just the file, Motor-Car-Trend-Analysis.Rproj. This file will store all the information on your project that RStudio will need for loading. But to stay organized, we have to create some folders in the directory. Create the following folders: data: This includes all the data that we need for our analysis code: This includes all the code files for cleaning up data, generating plots, and so on plots: This includes all graphical outputs reports: This comprises all the reports that we create from our dataset Saving the data The Motor Trend Car Road Tests dataset is part of the dataset package, which is one of the preinstalled packages in R. But, we will save the data in a CSV file in our data folder, after extracting the data from the mtcars variable, to make sure our analysis is reproducible. Put the following line of code in a new R script and save it as data.R in the code folder: #write data into csv file write.csv(mtcars, file = "data/cars.csv", row.names=FALSE) Analyzing the data The analysis script will first have to load the data from the CSV file with the following line: cars_data <- read.csv(file = "data/cars.csv", header = TRUE, sep = ",") Summary To learn more about RStudio, the following books published by Packt Publishing (https://www.packtpub.com/) are recommended: Mastering Machine Learning with R (https://www.packtpub.com/big-data-and-business-intelligence/mastering-machine-learning-r) R Data Analysis Cookbook (https://www.packtpub.com/big-data-and-business-intelligence/r-data-analysis-cookbook) Mastering Data Analysis with R (https://www.packtpub.com/big-data-and-business-intelligence/mastering-data-analysis-r) Resources for Article: Further resources on this subject: RefresheR [article] Deep learning in R [article] Aspects of Data Manipulation in R [article]
Read more
  • 0
  • 0
  • 2474

article-image-python-design-patterns-depth-singleton-pattern
Packt
15 Feb 2016
14 min read
Save for later

Python Design Patterns in Depth: The Singleton Pattern

Packt
15 Feb 2016
14 min read
There are situations where you need to create only one instance of data throughout the lifetime of a program. This can be a class instance, a list, or a dictionary, for example. The creation of a second instance is undesirable. This can result in logical errors or malfunctioning of the program. The design pattern that allows you to create only one instance of data is called singleton. In this article, you will learn about module-level, classic, and borg singletons; you'll also learn about how they work, when to use them, and build a two-threaded web crawler that uses a singleton to access the shared resource. (For more resources related to this topic, see here.) Singleton is the best candidate when the requirements are as follows: Controlling concurrent access to a shared resource If you need a global point of access for the resource from multiple or different parts of the system When you need to have only one object Some typical use cases of using a singleton are: The logging class and its subclasses (global point of access for the logging class to send messages to the log) Printer spooler (your application should only have a single instance of the spooler in order to avoid having a conflicting request for the same resource) Managing a connection to a database File manager Retrieving and storing information on external configuration files Read-only singletons storing some global states (user language, time, time zone, application path, and so on) There are several ways to implement singletons. We will look at module-level singleton, classic singletons, and borg singleton. Module-level singleton All modules are singletons by nature because of Python's module importing steps: Check whether a module is already imported. If yes, return it. If not, find a module, initialize it, and return it. Initializing a module means executing a code, including all module-level assignments. When you import the module for the first time, all of the initializations will be done; however, if you try to import the module for the second time, Python will return the initialized module. Thus, the initialization will not be done, and you get a previously imported module with all of its data. So, if you want to quickly make a singleton, use the following steps and keep the shared data as the module attribute. singletone.py: only_one_var = "I'm only one var" module1.py: import single tone print singleton.only_one_var singletone.only_one_var += " after modification" import module2 module2.py: import singletone print singleton.only_one_var Here, if you try to import a global variable in a singleton module and change its value in the module1 module, module2 will get a changed variable. This function is quick and sometimes is all that you need; however, we need to consider the following points: It's pretty error-prone. For example, if you happen to forget the global statements, variables local to the function will be created and, the module's variables won't be changed, which is not what you want. It's ugly, especially if you have a lot of objects that should remain as singletons. They pollute the module namespace with unnecessary variables. They don't permit lazy allocation and initialization; all global variables will be loaded during the module import process. It's not possible to re-use the code because you can not use the inheritance. No special methods and no object-oriented programming benefits at all. Classic singleton In classic singleton in Python, we check whether an instance is already created. If it is created, we return it; otherwise, we create a new instance, assign it to a class attribute, and return it. Let's try to create a dedicated singleton class: class Singleton(object): def __new__(cls): if not hasattr(cls, 'instance'): cls.instance = super(Singleton, cls).__new__(cls) return cls.instance Here, before creating the instance, we check for the special __new__ method, which is called right before __init__ if we had created an instance earlier. If not, we create a new instance; otherwise, we return the already created instance. Let's check how it works: >>> singleton = Singleton() >>> another_singleton = Singleton() >>> singleton is another_singleton True >>> singleton.only_one_var = "I'm only one var" >>> another_singleton.only_one_var I'm only one var Try to subclass the Singleton class with another one. class Child(Singleton): pass If it's a successor of Singleton, all of its instances should also be the instances of Singleton, thus sharing its states. But this doesn't work as illustrated in the following code: >>> child = Child() >>> child is singleton >>> False >>> child.only_one_var AttributeError: Child instance has no attribute 'only_one_var' To avoid this situation, the borg singleton is used. Borg singleton Borg is also known as monostate. In the borg pattern, all of the instances are different, but they share the same state. In the following code , the shared state is maintained in the _shared_state attribute. And all new instances of the Borg class will have this state as defined in the __new__ class method. class Borg(object):    _shared_state = {}    def __new__(cls, *args, **kwargs):        obj = super(Borg, cls).__new__(cls, *args, **kwargs)        obj.__dict__ = cls._shared_state        return obj Generally, Python stores the instance state in the __dict__ dictionary and when instantiated normally, every instance will have its own __dict__. But, here we deliberately assign the class variable _shared_state to all of the created instances. Here is how it works with subclassing: class Child(Borg):    pass>>> borg = Borg()>>> another_borg = Borg()>>> borg is another_borgFalse>>> child = Child()>>> borg.only_one_var = "I'm the only one var">>> child.only_one_varI'm the only one var So, despite the fact that you can't compare objects by their identity, using the is statement, all child objects share the parents' state. If you want to have a class, which is a descendant of the Borg class but has a different state, you can reset shared_state as follows: class AnotherChild(Borg):    _shared_state = {}>>> another_child = AnotherChild()>>> another_child.only_one_varAttributeError: AnotherChild instance has no attribute 'shared_state' Which type of singleton should be used is up to you. If you expect that your singleton will not be inherited, you can choose the classic singleton; otherwise, it's better to stick with borg. Implementation in Python As a practical example, we'll create a simple web crawler that scans a website you open on it, follows all the links that lead to the same website but to other pages, and downloads all of the images it'll find. To do this, we'll need two functions: a function that scans a website for links, which leads to other pages to build a set of pages to visit, and a function that scans a page for images and downloads them. To make it quicker, we'll download images in two threads. These two threads should not interfere with each other, so don't scan pages if another thread has already scanned them, and don't download images that are already downloaded. So, a set with downloaded images and scanned web pages will be a shared resource for our application, and we'll keep it in a singleton instance. In this example, you will need a library for parsing and screen scraping websites named BeautifulSoup and an HTTP client library httplib2. It should be sufficient to install both with either of the following commands: $ sudo pip install BeautifulSoup httplib2 $ sudo easy_install BeautifulSoup httplib2 First of all, we'll create a Singleton class. Let's use the classic singleton in this example: import httplib2import osimport reimport threadingimport urllibfrom urlparse import urlparse, urljoinfrom BeautifulSoup import BeautifulSoupclass Singleton(object):    def __new__(cls):        if not hasattr(cls, 'instance'):             cls.instance = super(Singleton, cls).__new__(cls)        return cls.instance It will return the singleton objects to all parts of the code that request it. Next, we'll create a class for creating a thread. In this thread, we'll download images from the website: class ImageDownloaderThread(threading.Thread):    """A thread for downloading images in parallel."""    def __init__(self, thread_id, name, counter):        threading.Thread.__init__(self)        self.name = name    def run(self):        print 'Starting thread ', self.name        download_images(self.name)        print 'Finished thread ', self.name The following function traverses the website using BFS algorithms, finds links, and adds them to a set for further downloading. We are able to specify the maximum links to follow if the website is too large. def traverse_site(max_links=10):    link_parser_singleton = Singleton()    # While we have pages to parse in queue    while link_parser_singleton.queue_to_parse:        # If collected enough links to download images, return        if len(link_parser_singleton.to_visit) == max_links:            return        url = link_parser_singleton.queue_to_parse.pop()        http = httplib2.Http()        try:            status, response = http.request(url)        except Exception:            continue        # Skip if not a web page        if status.get('content-type') != 'text/html':            continue        # Add the link to queue for downloading images        link_parser_singleton.to_visit.add(url)        print 'Added', url, 'to queue'        bs = BeautifulSoup(response)        for link in BeautifulSoup.findAll(bs, 'a'):            link_url = link.get('href')            # <img> tag may not contain href attribute            if not link_url:                continue            parsed = urlparse(link_url)            # If link follows to external webpage, skip it            if parsed.netloc and parsed.netloc != parsed_root.netloc:                continue            # Construct a full url from a link which can be relative            link_url = (parsed.scheme or parsed_root.scheme) + '://' + (parsed.netloc or parsed_root.netloc) + parsed.path or ''            # If link was added previously, skip it            if link_url in link_parser_singleton.to_visit:                continue            # Add a link for further parsing            link_parser_singleton.queue_to_parse = [link_url] + link_parser_singleton.queue_to_parse The following function downloads images from the last web resource page in the singleton.to_visit queue and saves it to the img directory. Here, we use a singleton for synchronizing shared data, which is a set of pages to visit between two threads: def download_images(thread_name):    singleton = Singleton()    # While we have pages where we have not download images    while singleton.to_visit:        url = singleton.to_visit.pop()        http = httplib2.Http()        print thread_name, 'Starting downloading images from', url        try:            status, response = http.request(url)        except Exception:            continue        bs = BeautifulSoup(response)       # Find all <img> tags        images = BeautifulSoup.findAll(bs, 'img')        for image in images:            # Get image source url which can be absolute or relative            src = image.get('src')            # Construct a full url. If the image url is relative,            # it will be prepended with webpage domain.            # If the image url is absolute, it will remain as is            src = urljoin(url, src)            # Get a base name, for example 'image.png' to name file locally            basename = os.path.basename(src)            if src not in singleton.downloaded:                singleton.downloaded.add(src)                print 'Downloading', src                # Download image to local filesystem                urllib.urlretrieve(src, os.path.join('images', basename))        print thread_name, 'finished downloading images from', url Our client code is as follows: if __name__ == '__main__':    root = 'http://python.org'    parsed_root = urlparse(root)    singleton = Singleton()    singleton.queue_to_parse = [root]    # A set of urls to download images from    singleton.to_visit = set()    # Downloaded images    singleton.downloaded = set()    traverse_site()    # Create images directory if not exists    if not os.path.exists('images'):        os.makedirs('images')    # Create new threads    thread1 = ImageDownloaderThread(1, "Thread-1", 1)    thread2 = ImageDownloaderThread(2, "Thread-2", 2)    # Start new Threads    thread1.start()    thread2.start() Run a crawler using the following command: $ python crawler.py You should get the following output (your output may vary because the order in which the threads access resources is not predictable): If you go to the images directory, you will find the downloaded images there. Summary To learn more about design patterns in depth, the following books published by Packt Publishing (https://www.packtpub.com/) are recommended: Learning Python Design Patterns – Second Edition (https://www.packtpub.com/application-development/learning-python-design-patterns-second-edition) Mastering Python Design Patterns (https://www.packtpub.com/application-development/mastering-python-design-patterns) Resources for Article: Further resources on this subject: Python Design Patterns in Depth: The Factory Pattern [Article] Recommending Movies at Scale (Python) [Article] Customizing IPython [Article]
Read more
  • 0
  • 0
  • 17384

article-image-python-design-patterns-depth-factory-pattern
Packt
15 Feb 2016
17 min read
Save for later

Python Design Patterns in Depth: The Factory Pattern

Packt
15 Feb 2016
17 min read
Creational design patterns deal with an object creation [j.mp/wikicrea]. The aim of a creational design pattern is to provide better alternatives for situations where a direct object creation (which in Python happens by the __init__() function [j.mp/divefunc], [Lott14, page 26]) is not convenient. In the Factory design pattern, a client asks for an object without knowing where the object is coming from (that is, which class is used to generate it). The idea behind a factory is to simplify an object creation. It is easier to track which objects are created if this is done through a central function, in contrast to letting a client create objects using a direct class instantiation [Eckel08, page 187]. A factory reduces the complexity of maintaining an application by decoupling the code that creates an object from the code that uses it [Zlobin13, page 30]. Factories typically come in two forms: the Factory Method, which is a method (or in Pythonic terms, a function) that returns a different object per input parameter [j.mp/factorympat]; the Abstract Factory, which is a group of Factory Methods used to create a family of related products [GOF95, page 100], [j.mp/absfpat] (For more resources related to this topic, see here.) Factory Method In the Factory Method, we execute a single function, passing a parameter that provides information about what we want. We are not required to know any details about how the object is implemented and where it is coming from. A real-life example An example of the Factory Method pattern used in reality is in plastic toy construction. The molding powder used to construct plastic toys is the same, but different figures can be produced using different plastic molds. This is like having a Factory Method in which the input is the name of the figure that we want (soldier and dinosaur) and the output is the plastic figure that we requested. The toy construction case is shown in the following figure, which is provided by www.sourcemaking.com [j.mp/factorympat]. A software example The Django framework uses the Factory Method pattern for creating the fields of a form. The forms module of Django supports the creation of different kinds of fields (CharField, EmailField) and customizations (max_length, required) [j.mp/djangofacm]. Use cases If you realize that you cannot track the objects created by your application because the code that creates them is in many different places instead of a single function/method, you should consider using the Factory Method pattern [Eckel08, page 187]. The Factory Method centralizes an object creation and tracking your objects becomes much more easier. Note that it is absolutely fine to create more than one Factory Method, and this is how it is typically done in practice. Each Factory Method logically groups the creation of objects that have similarities. For example, one Factory Method might be responsible for connecting you to different databases (MySQL, SQLite), another Factory Method might be responsible for creating the geometrical object that you request (circle, triangle), and so on. The Factory Method is also useful when you want to decouple an object creation from an object usage. We are not coupled/bound to a specific class when creating an object, we just provide partial information about what we want by calling a function. This means that introducing changes to the function is easy without requiring any changes to the code that uses it [Zlobin13, page 30]. Another use case worth mentioning is related with improving the performance and memory usage of an application. A Factory Method can improve the performance and memory usage by creating new objects only if it is absolutely necessary [Zlobin13, page 28]. When we create objects using a direct class instantiation, extra memory is allocated every time a new object is created (unless the class uses caching internally, which is usually not the case). We can see that in practice in the following code (file id.py), it creates two instances of the same class A and uses the id() function to compare their memory addresses. The addresses are also printed in the output so that we can inspect them. The fact that the memory addresses are different means that two distinct objects are created as follows: class A(object):     pass if __name__ == '__main__':     a = A()     b = A()     print(id(a) == id(b))     print(a, b) Executing id.py on my computer gives the following output:>> python3 id.pyFalse<__main__.A object at 0x7f5771de8f60> <__main__.A object at 0x7f5771df2208> Note that the addresses that you see if you execute the file are not the same as I see because they depend on the current memory layout and allocation. But the result must be the same: the two addresses should be different. There's one exception that happens if you write and execute the code in the Python Read-Eval-Print Loop (REPL) (interactive prompt), but that's a REPL-specific optimization which is not happening normally. Implementation Data comes in many forms. There are two main file categories for storing/retrieving data: human-readable files and binary files. Examples of human-readable files are XML, Atom, YAML, and JSON. Examples of binary files are the .sq3 file format used by SQLite and the .mp3 file format used to listen to music. In this example, we will focus on two popular human-readable formats: XML and JSON. Although human-readable files are generally slower to parse than binary files, they make data exchange, inspection, and modification much more easier. For this reason, it is advised to prefer working with human-readable files, unless there are other restrictions that do not allow it (mainly unacceptable performance and proprietary binary formats). In this problem, we have some input data stored in an XML and a JSON file, and we want to parse them and retrieve some information. At the same time, we want to centralize the client's connection to those (and all future) external services. We will use the Factory Method to solve this problem. The example focuses only on XML and JSON, but adding support for more services should be straightforward. First, let's take a look at the data files. The XML file, person.xml, is based on the Wikipedia example [j.mp/wikijson] and contains information about individuals (firstName, lastName, gender, and so on) as follows: <persons>   <person>     <firstName>John</firstName>     <lastName>Smith</lastName>     <age>25</age>     <address>       <streetAddress>21 2nd Street</streetAddress>       <city>New York</city>       <state>NY</state>       <postalCode>10021</postalCode>     </address>     <phoneNumbers>       <phoneNumber type="home">212 555-1234</phoneNumber>       <phoneNumber type="fax">646 555-4567</phoneNumber>     </phoneNumbers>     <gender>       <type>male</type>     </gender>   </person>   <person>     <firstName>Jimy</firstName>     <lastName>Liar</lastName>     <age>19</age>     <address>       <streetAddress>18 2nd Street</streetAddress>       <city>New York</city>       <state>NY</state>       <postalCode>10021</postalCode>     </address>     <phoneNumbers>       <phoneNumber type="home">212 555-1234</phoneNumber>     </phoneNumbers>     <gender>       <type>male</type>     </gender>   </person>   <person>     <firstName>Patty</firstName>     <lastName>Liar</lastName>     <age>20</age>     <address>       <streetAddress>18 2nd Street</streetAddress>       <city>New York</city>       <state>NY</state>       <postalCode>10021</postalCode>     </address>     <phoneNumbers>       <phoneNumber type="home">212 555-1234</phoneNumber>       <phoneNumber type="mobile">001 452-8819</phoneNumber>     </phoneNumbers>     <gender>       <type>female</type>     </gender>   </person> </persons> The JSON file, donut.json, comes from the GitHub account of Adobe [j.mp/adobejson] and contains donut information (type, price/unit i.e. ppu, topping, and so on) as follows: [   {     "id": "0001",     "type": "donut",     "name": "Cake",     "ppu": 0.55,     "batters": {       "batter": [         { "id": "1001", "type": "Regular" },         { "id": "1002", "type": "Chocolate" },         { "id": "1003", "type": "Blueberry" },         { "id": "1004", "type": "Devil's Food" }       ]     },     "topping": [       { "id": "5001", "type": "None" },       { "id": "5002", "type": "Glazed" },       { "id": "5005", "type": "Sugar" },       { "id": "5007", "type": "Powdered Sugar" },       { "id": "5006", "type": "Chocolate with Sprinkles" },       { "id": "5003", "type": "Chocolate" },       { "id": "5004", "type": "Maple" }     ]   },   {     "id": "0002",     "type": "donut",     "name": "Raised",     "ppu": 0.55,     "batters": {       "batter": [         { "id": "1001", "type": "Regular" }       ]     },     "topping": [       { "id": "5001", "type": "None" },       { "id": "5002", "type": "Glazed" },       { "id": "5005", "type": "Sugar" },       { "id": "5003", "type": "Chocolate" },       { "id": "5004", "type": "Maple" }     ]   },   {     "id": "0003",     "type": "donut",     "name": "Old Fashioned",     "ppu": 0.55,     "batters": {       "batter": [         { "id": "1001", "type": "Regular" },         { "id": "1002", "type": "Chocolate" }       ]     },     "topping": [       { "id": "5001", "type": "None" },       { "id": "5002", "type": "Glazed" },       { "id": "5003", "type": "Chocolate" },       { "id": "5004", "type": "Maple" }     ]   } ] We will use two libraries that are part of the Python distribution for working with XML and JSON: xml.etree.ElementTree and json as follows: import xml.etree.ElementTree as etree import json The JSONConnector class parses the JSON file and has a parsed_data() method that returns all data as a dictionary (dict). The property decorator is used to make parsed_data() appear as a normal variable instead of a method as follows: class JSONConnector:     def __init__(self, filepath):         self.data = dict()         with open(filepath, mode='r', encoding='utf-8') as f:             self.data = json.load(f)       @property     def parsed_data(self):         return self.data The XMLConnector class parses the XML file and has a parsed_data() method that returns all data as a list of xml.etree.Element as follows: class XMLConnector:     def __init__(self, filepath):         self.tree = etree.parse(filepath)     @property    def parsed_data(self):         return self.tree The connection_factory() function is a Factory Method. It returns an instance of JSONConnector or XMLConnector depending on the extension of the input file path as follows: def connection_factory(filepath):     if filepath.endswith('json'):         connector = JSONConnector     elif filepath.endswith('xml'):         connector = XMLConnector     else:         raise ValueError('Cannot connect to {}'.format(filepath))     return connector(filepath) The connect_to() function is a wrapper of connection_factory(). It adds exception handling as follows: def connect_to(filepath):     factory = None     try:         factory = connection_factory(filepath)     except ValueError as ve:         print(ve)     return factory The main() function demonstrates how the Factory Method design pattern can be used. The first part makes sure that exception handling is effective as follows: def main():     sqlite_factory = connect_to('data/person.sq3') The next part shows how to work with the XML files using the Factory Method. XPath is used to find all person elements that have the last name Liar. For each matched person, the basic name and phone number information are shown as follows: xml_factory = connect_to('data/person.xml')     xml_data = xml_factory.parsed_data()     liars = xml_data.findall     (".//{person}[{lastName}='{}']".format('Liar'))     print('found: {} persons'.format(len(liars)))     for liar in liars:         print('first name:         {}'.format(liar.find('firstName').text))         print('last name: {}'.format(liar.find('lastName').text))         [print('phone number ({}):'.format(p.attrib['type']),         p.text) for p in liar.find('phoneNumbers')] The final part shows how to work with the JSON files using the Factory Method. Here, there's no pattern matching, and therefore the name, price, and topping of all donuts are shown as follows: json_factory = connect_to('data/donut.json')     json_data = json_factory.parsed_data     print('found: {} donuts'.format(len(json_data)))     for donut in json_data:         print('name: {}'.format(donut['name']))         print('price: ${}'.format(donut['ppu']))         [print('topping: {} {}'.format(t['id'], t['type'])) for t         in donut['topping']] For completeness, here is the complete code of the Factory Method implementation (factory_method.py) as follows: import xml.etree.ElementTree as etree import json class JSONConnector:     def __init__(self, filepath):         self.data = dict()         with open(filepath, mode='r', encoding='utf-8') as f:             self.data = json.load(f)     @property     def parsed_data(self):         return self.data class XMLConnector:     def __init__(self, filepath):         self.tree = etree.parse(filepath)     @property     def parsed_data(self):         return self.tree def connection_factory(filepath):     if filepath.endswith('json'):         connector = JSONConnector     elif filepath.endswith('xml'):         connector = XMLConnector     else:         raise ValueError('Cannot connect to {}'.format(filepath))     return connector(filepath) def connect_to(filepath):     factory = None     try:        factory = connection_factory(filepath)     except ValueError as ve:         print(ve)     return factory def main():     sqlite_factory = connect_to('data/person.sq3')     print()     xml_factory = connect_to('data/person.xml')     xml_data = xml_factory.parsed_data     liars = xml_data.findall(".//{}[{}='{}']".format('person',     'lastName', 'Liar'))     print('found: {} persons'.format(len(liars)))     for liar in liars:         print('first name:         {}'.format(liar.find('firstName').text))         print('last name: {}'.format(liar.find('lastName').text))         [print('phone number ({}):'.format(p.attrib['type']),         p.text) for p in liar.find('phoneNumbers')]     print()     json_factory = connect_to('data/donut.json')     json_data = json_factory.parsed_data     print('found: {} donuts'.format(len(json_data)))     for donut in json_data:     print('name: {}'.format(donut['name']))     print('price: ${}'.format(donut['ppu']))     [print('topping: {} {}'.format(t['id'], t['type'])) for t     in donut['topping']] if __name__ == '__main__':     main() Here is the output of this program as follows: >>> python3 factory_method.pyCannot connect to data/person.sq3found: 2 personsfirst name: Jimylast name: Liarphone number (home): 212 555-1234first name: Pattylast name: Liarphone number (home): 212 555-1234phone number (mobile): 001 452-8819found: 3 donutsname: Cakeprice: $0.55topping: 5001 Nonetopping: 5002 Glazedtopping: 5005 Sugartopping: 5007 Powdered Sugartopping: 5006 Chocolate with Sprinklestopping: 5003 Chocolatetopping: 5004 Maplename: Raisedprice: $0.55topping: 5001 Nonetopping: 5002 Glazedtopping: 5005 Sugartopping: 5003 Chocolatetopping: 5004 Maplename: Old Fashionedprice: $0.55topping: 5001 Nonetopping: 5002 Glazedtopping: 5003 Chocolatetopping: 5004 Maple Notice that although JSONConnector and XMLConnector have the same interfaces, what is returned by parsed_data() is not handled in a uniform way. Different codes must be used to work with each connector. Although it would be nice to be able to use the same code for all connectors, this is at most times not realistic unless we use some kind of common mapping for the data which is very often provided by external data providers. Assuming that you can use exactly the same code for handling the XML and JSON files, what changes are required to support a third format, for example, SQLite? Find an SQLite file or create your own and try it. As is now, the code does not forbid a direct instantiation of a connector. Is it possible to do this? Try doing it (hint: functions in Python can have nested classes). Summary To learn more about design patterns in depth, the following books published by Packt Publishing (https://www.packtpub.com/) are recommended: Learning Python Design Patterns (https://www.packtpub.com/application-development/learning-python-design-patterns) Learning Python Design Patterns – Second Edition (https://www.packtpub.com/application-development/learning-python-design-patterns-second-edition) Resources for Article:   Further resources on this subject: Recommending Movies at Scale (Python) [article] An In-depth Look at Ansible Plugins [article] Elucidating the Game-changing Phenomenon of the Docker-inspired Containerization Paradigm [article]
Read more
  • 0
  • 0
  • 10174
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at $15.99/month. Cancel anytime
article-image-modular-programming-ecmascript-6
Packt
15 Feb 2016
18 min read
Save for later

Modular Programming in ECMAScript 6

Packt
15 Feb 2016
18 min read
Modular programming is one of the most important and frequently used software design techniques. Unfortunately, JavaScript didn't support modules natively that lead JavaScript programmers to use alternative techniques to achieve modular programming in JavaScript. But now, ES6 brings modules into JavaScript officially. This article is all about how to create and import JavaScript modules. In this article, we will first learn how the modules were created earlier, and then we will jump to the new built-in module system that was introduced in ES6, known as the ES6 modules. In this article, we'll cover: What is modular programming? The benefits of modular programming The basics of IIFE modules, AMD, UMD, and CommonJS Creating and importing the ES6 modules The basics of the Modular Loader Creating a basic JavaScript library using modules (For more resources related to this topic, see here.) The JavaScript modules in a nutshell The practice of breaking down programs and libraries into modules is called modular programming. In JavaScript, a module is a collection of related objects, functions, and other components of a program or library that are wrapped together and isolated from the scope of the rest of the program or library. A module exports some variables to the outside program to let it access the components wrapped by the module. To use a module, a program needs to import the module and the variables exported by the module. A module can also be split into further modules called as its submodules, thus creating a module hierarchy. Modular programming has many benefits. Some benefits are: It keeps our code both cleanly separated and organized by splitting into multiple modules Modular programming leads to fewer global variables, that is, it eliminates the problem of global variables, because modules don't interface via the global scope, and each module has its own scope Makes code reusability easier as importing and using the same modules in different projects is easier It allows many programmers to collaborate on the same program or library, by making each programmer to work on a particular module with a particular functionality Bugs in an application can easily be easily identified as they are localized to a particular module Implementing modules – the old way Before ES6, JavaScript had never supported modules natively. Developers used other techniques and third-party libraries to implement modules in JavaScript. Using Immediately-invoked function expression (IIFE), Asynchronous Module Definition (AMD), CommonJS, and Universal Module Definition (UMD) are various popular ways of implementing modules in ES5. As these ways were not native to JavaScript, they had several problems. Let's see an overview of each of these old ways of implementing modules. The Immediately-Invoked Function Expression The IIFE is used to create an anonymous function that invokes itself. Creating modules using IIFE is the most popular way of creating modules. Let's see an example of how to create a module using IIFE: //Module Starts (function(window){   var sum = function(x, y){     return x + y;   }     var sub = function(x, y){     return x - y;   }   var math = {     findSum: function(a, b){       return sum(a,b);     },     findSub: function(a, b){       return sub(a, b);     }   }   window.math = math; })(window) //Module Ends console.log(math.findSum(1, 2)); //Output "3" console.log(math.findSub(1, 2)); //Output "-1" Here, we created a module using IIFE. The sum and sub variables are global to the module, but not visible outside of the module. The math variable is exported by the module to the main program to expose the functionalities that it provides. This module works completely independent of the program, and can be imported by any other program by simply copying it into the source code, or importing it as a separate file. A library using IIFE, such as jQuery, wraps its all of its APIs in a single IIFE module. When a program uses a jQuery library, it automatically imports the module. Asynchronous Module Definition AMD is a specification for implementing modules in browser. AMD is designed by keeping the browser limitations in mind, that is, it imports modules asynchronously to prevent blocking the loading of a webpage. As AMD is not a native browser specification, we need to use an AMD library. RequireJS is the most popular AMD library. Let's see an example on how to create and import modules using RequireJS. According to the AMD specification, every module needs to be represented by a separate file. So first, create a file named math.js that represents a module. Here is the sample code that will be inside the module: define(function(){   var sum = function(x, y){     return x + y;   }   var sub = function(x, y){     return x - y;   }   var math = {     findSum: function(a, b){       return sum(a,b);     },     findSub: function(a, b){       return sub(a, b);     }   }   return math; }); Here, the module exports the math variable to expose its functionality. Now, let's create a file named index.js, which acts like the main program that imports the module and the exported variables. Here is the code that will be inside the index.js file: require(["math"], function(math){   console.log(math.findSum(1, 2)); //Output "3"   console.log(math.findSub(1, 2)); //Output "-1" }) Here, math variable in the first parameter is the name of the file that is treated as the AMD module. The .js extension to the file name is added automatically by RequireJS. The math variable, which is in the second parameter, references the exported variable. Here, the module is imported asynchronously, and the callback is also executed asynchronously. CommonJS CommonJS is a specification for implementing modules in Node.js. According to the CommonJS specification, every module needs to be represented by a separate file. The CommonJS modules are imported synchronously. Let's see an example on how to create and import modules using CommonJS. First, we will create a file named math.js that represents a module. Here is a sample code that will be inside the module: var sum = function(x, y){   return x + y; } var sub = function(x, y){   return x - y; } var math = {   findSum: function(a, b){     return sum(a,b);   },   findSub: function(a, b){     return sub(a, b);   } } exports.math = math; Here, the module exports the math variable to expose its functionality. Now, let's create a file named index.js, which acts like the main program that imports the module. Here is the code that will be inside the index.js file: var math = require("./math").math; console.log(math.findSum(1, 2)); //Output "3" console.log(math.findSub(1, 2)); //Output "-1" Here, the math variable is the name of the file that is treated as module. The .js extension to the file name is added automatically by CommonJS. Universal Module Definition We saw three different specifications of implementing modules. These three specifications have their own respective ways of creating and importing modules. Wouldn't it have been great if we can create modules that can be imported as an IIFE, AMD, or CommonJS module? UMD is a set of techniques that is used to create modules that can be imported as an IIFE, CommonJS, or AMD module. Therefore now, a program can import third-party modules, irrespective of what module specification it is using. The most popular UMD technique is returnExports. According to the returnExports technique, every module needs to be represented by a separate file. So, let's create a file named math.js that represents a module. Here is the sample code that will be inside the module: (function (root, factory) {   //Environment Detection   if (typeof define === 'function' && define.amd) {     define([], factory);   } else if (typeof exports === 'object') {     module.exports = factory();   } else {     root.returnExports = factory();   } }(this, function () {   //Module Definition   var sum = function(x, y){     return x + y;   }   var sub = function(x, y){     return x - y;   }   var math = {     findSum: function(a, b){       return sum(a,b);     },     findSub: function(a, b){       return sub(a, b);     }   }   return math; })); Now, you can successfully import the math.js module any way that you wish, for instance, by using CommonJS, RequireJS, or IIFE. Implementing modules – the new way ES6 introduced a new module system called ES6 modules. The ES6 modules are supported natively and therefore, they can be referred as the standard JavaScript modules. You should consider using ES6 modules instead of the old ways, because they have neater syntax, better performance, and many new APIs that are likely to be packed as the ES6 modules. Let's have a look at the ES6 modules in detail. Creating the ES6 modules Every ES6 module needs to be represented by a separate .js file. An ES6 module can contain any JavaScript code, and it can export any number of variables. A module can export a variable, function, class, or any other entity. We need to use the export statement in a module to export variables. The export statement comes in many different formats. Here are the formats: export {variableName}; export {variableName1, variableName2, variableName3}; export {variableName as myVariableName}; export {variableName1 as myVariableName1, variableName2 as myVariableName2}; export {variableName as default}; export {variableName as default, variableName1 as myVariableName1, variableName2}; export default function(){}; export {variableName1, variableName2} from "myAnotherModule"; export * from "myAnotherModule"; Here are the differences in these formats: The first format exports a variable. The second format is used to export multiple variables. The third format is used to export a variable with another name, that is, an alias. The fourth format is used to export multiple variables with different names. The fifth format uses default as the alias. We will find out the use of this later in this article. The sixth format is similar to fourth format, but it also has the default alias. The seventh format works similar to fifth format, but here you can place an expression instead of a variable name. The eighth format is used to export the exported variables of a submodule. The ninth format is used to export all the exported variables of a submodule. Here are some important things that you need to know about the export statement: An export statement can be used anywhere in a module. It's not compulsory to use it at the end of the module. There can be any number of export statements in a module. You cannot export variables on demand. For example, placing the export statement in the if…else condition throws an error. Therefore, we can say that the module structure needs to be static, that is, exports can be determined on compile time. You cannot export the same variable name or alias multiple times. But you can export a variable multiple times with a different alias. All the code inside a module is executed in the strict mode by default. The values of the exported variables can be changed inside the module that exported them. Importing the ES6 modules To import a module, we need to use the import statement. The import statement comes in many different formats. Here are the formats: import x from "module-relative-path"; import {x} from "module-relative-path"; import {x1 as x2} from "module-relative-path"; import {x1, x2} from "module-relative-path"; import {x1, x2 as x3} from "module-relative-path"; import x, {x1, x2} from "module-relative-path"; import "module-relative-path"; import * as x from "module-relative-path"; import x1, * as x2 from "module-relative-path"; An import statement consists of two parts: the variable names we want to import and the relative path of the module. Here are the differences in these formats: In the first format, the default alias is imported. The x is alias of the default alias. In the second format, the x variable is imported. The third format is the same as the second format. It's just that x2 is an alias of x1. In the fourth format, we import the x1 and x2 variables. In the fifth format, we import the x1 and x2 variables. The x3 is an alias of the x2 variable. In the sixth format, we import the x1 and x2 variable, and the default alias. The x is an alias of the default alias. In the seventh format, we just import the module. We do not import any of the variables exported by the module. In the eighth format, we import all the variables, and wrap them in an object called x. Even the default alias is imported. The ninth format is the same as the eighth format. Here, we give another alias to the default alias.[RR1]  Here are some important things that you need to know about the import statement: While importing a variable, if we import it with an alias, then to refer to that variable, we have to use the alias and not the actual variable name, that is, the actual variable name will not be visible, only the alias will be visible. The import statement doesn't import a copy of the exported variables; rather, it makes the variables available in the scope of the program that imports it. Therefore, if you make a change to an exported variable inside the module, then the change is visible to the program that imports it. The imported variables are read-only, that is, you cannot reassign them to something else outside of the scope of the module that exports them. A module can only be imported once in a single instance of a JavaScript engine. If we try to import it again, then the already imported instance of the module will be used. We cannot import modules on demand. For example, placing the import statement in the if…else condition throws an error. Therefore, we can say that the imports should be able to be determined on compile time. The ES6 imports are faster than the AMD and CommonJS imports, because the ES6 imports are supported natively and also as importing modules and exporting variables are not decided on demand. Therefore, it makes JavaScript engine easier to optimize performance. The module loader A module loader is a component of a JavaScript engine that is responsible for importing modules. The import statement uses the build-in module loader to import modules. The built-in module loaders of the different JavaScript environments use different module loading mechanisms. For example, when we import a module in JavaScript running in the browsers, then the module is loaded from the server. On the other hand, when we import a module in Node.js, then the module is loaded from filesystem. The module loader loads modules in a different manner, in different environments, to optimize the performance. For example, in the browsers, the module loader loads and executes modules asynchronously in order to prevent the importing of the modules that block the loading of a webpage. You can programmatically interact with the built-in module loader using the module loader API to customize its behavior, intercept module loading, and fetch the modules on demand. We can also use this API to create our own custom module loaders. The specifications of the module loader are not specified in ES6. It is a separate standard, controlled by the WHATWG browser standard group. You can find the specifications of the module loader at http://whatwg.github.io/loader/. The ES6 specifications only specify the import and export statements. Using modules in browsers The code inside the <script> tag doesn't support the import statement, because the tag's synchronous nature is incompatible with the asynchronicity of the modules in browsers. Instead, you need to use the new <module> tag to import modules. Using the new <module> tag, we can define a script as a module. Now, this module can import other modules using the import statement. If you want to import a module using the <script> tag, then you have to use the Module Loader API. The specifications of the <module> tag are not specified in ES6. Using modules in the eval() function You cannot use the import and export statements in the eval() function. To import modules in the eval() function, you need to use the Module Loader API. The default exports vs. the named exports When we export a variable with the default alias, then it's called as a default export. Obviously, there can only be one default export in a module, as an alias can be used only once. All the other exports except the default export are called as named exports. It's recommended that a module should either use default export or named exports. It's not a good practice to use both together. The default export is used when we want to export only one variable. On the other hand, the named exports are used when we want to export the multiple variables. Diving into an example Let's create a basic JavaScript library using the ES6 modules. This will help us understand how to use the import and export statements. We will also learn how a module can import other modules. The library that we will create is going to be a math library, which provides basic logarithmic and trigonometric functions. Let's get started with creating our library: Create a file named math.js, and a directory named math_modules. Inside the math_modules directory, create two files named logarithm.js and trigonometry.js, respectively. Here, the math.js file is the root module, whereas the logarithm.js and the trigonometry.js files are its submodules. Place this code inside the logarithm.js file: var LN2 = Math.LN2; var N10 = Math.LN10;   function getLN2() {   return LN2; }   function getLN10() {   return LN10; }   export {getLN2, getLN10}; Here, the module is exporting the functions named as exports. It's preferred that the low-level modules in a module hierarchy should export all the variables separately, because it may be possible that a program may need just one exported variable of a library. In this case, a program can import this module and a particular function directly. Loading all the modules when you need just one module is a bad idea in terms of performance. Similarly, place this code in the trigonometry.js file: var cos = Math.cos; var sin = Math.sin; function getSin(value) {   return sin(value); } function getCos(value) {   return cos(value); } export {getCos, getSin}; Here we do something similar. Place this code inside the math.js file, which acts as the root module: import * as logarithm from "math_modules/logarithm"; import * as trigonometry from "math_modules/trigonometry"; export default {   logarithm: logarithm,   trigonometry: trigonometry } It doesn't contain any library functions. Instead, it makes easy for a program to import the complete library. It imports its submodules, and then exports their exported variables to the main program. Here, in case the logarithm.js and trigonometry.js scripts depends on other submodules, then the math.js module shouldn't import those submodules, because logarithm.js and trigonometry.js are already importing them. Here is the code using which a program can import the complete library: import math from "math"; console.log(math.trigonometry.getSin(3)); console.log(math.logarithm.getLN2(3)); Summary In this article, we saw what modular programming is and learned different modular programming specifications. We also saw different ways to create modules using JavaScript. Technologies such as the IIFE, CommonJS, AMD, UMD, and ES6 modules are covered. Finally, we created a basic library using the modular programming design technique. Now, you should be confident enough to build the JavaScript apps using the ES6 modules. To learn more about ECMAScript and JavaScript, the following books published by Packt Publishing (https://www.packtpub.com/) are recommended: JavaScript at Scale (https://www.packtpub.com/web-development/javascript-scale) Google Apps Script for Beginners (https://www.packtpub.com/web-development/google-apps-script-beginners) Learning TypeScript (https://www.packtpub.com/web-development/learning-typescript) JavaScript Concurrency (https://www.packtpub.com/web-development/javascript-concurrency) You can also watch out for an upcoming title, Mastering JavaScript Object-Oriented Programming, on this technology on Packt Publishing's website at https://www.packtpub.com/web-development/mastering-javascript-object-oriented-programming. Resources for Article:   Further resources on this subject: Concurrency Principles [article] Using Client Methods [article] HTML5 APIs [article]
Read more
  • 0
  • 0
  • 3023

article-image-team-project-setup-0
Packt
10 Feb 2016
12 min read
Save for later

Building Your Application

Packt
10 Feb 2016
12 min read
"Measuring programming progress by lines of code is like measuring aircraft building progress by weight."                                                                --Bill Gates In this article, by Tarun Arora, the author of the book Microsoft Team Foundation Server 2015 Cookbook, provides you information about: Configuring TFBuild Agent, Pool, and Queues Setting up a TFBuild Agent using an unattended installation (For more resources related to this topic, see here.) As a developer, compiling code and running unit tests gives you an assurance that your code changes haven't had an impact on the existing codebase. Integrating your code changes into the source control repository enables other users to validate their changes with yours. As a best practice, Teams integrate changes into the shared repository several times a day to reduce the risk of introducing breaking changes or worse, overwriting each other's. Continuous integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is verified by an automated build, allowing Teams to detect problems early. The automated build that runs as part of the CI process is often referred to as the CI build. There isn't a clear definition of what the CI build should do, but at the very minimum, it is expected to compile code and run unit tests. Running the CI build on a non-developer remote workspace helps identify the dependencies that may otherwise go unnoticed into the release process. We can talk endlessly about the benefits of CI; the key here is that it enables you to have potentially deployable software at all times. Deployable software is the most tangible asset to customers. Moving from concept to application, in this article, you'll learn how to leverage the build tooling in TFS to set up a quality-focused CI process. But first, let's have a little introduction to the build system in TFS. The following image illustrates the three generations of build systems in TFS: TFS has gone through three generations of build systems. The very first was MSBuild using XML for configuration; the next one was XAML using Windows Workflow Foundation for configuration, and now, there's TFBuild using JSON for configuration. The XAML-based build system will continue to be supported in TFS 2015. No automated migration path is available from XAML build to TFBuild. This is generally because of the difference in the architecture between the two build systems. The new build system in TFS is called Team Foundation Build (TFBuild). It is an extensible task-based execution system with a rich web interface that allows authoring, queuing, and monitoring builds. TFBuild is fully cross platform with the underlying build agents that are capable of running natively on both Windows and non-Windows platforms. TFBuild provides out-of-the-box integration with Centralized Version Control such as TFVC and Distributed Version Controls such as Git and GitHub. TFBuild supports building .NET, Java, Android, and iOS applications. All the recipes in this article are based on TFBuild. TFBuild is a task orchestrator that allows you to run any build engine, such as Ant, CMake, Gradle, Gulp, Grunt, Maven, MSBuild, Visual Studio, Xamarin, XCode, and so on. TFBuild supports work item integration, publishing drops, and publishing test execution results into the TFS that is independent of the build engine that you choose. The build agents are xCopyable and do not require any installation. The agents are auto-updating in nature; there's no need to update every agent in your infrastructure: TFBuild offers a rich web-based interface. It does not require Visual Studio to author or modify a build definition. From simple to complex, all build definitions can easily be created in the web portal. The web interface is accessible from any device and any platform: The build definition can be authored from the web portal directly A build definition is a collection of tasks. A task is simply a build step. Build definition can be composed by dragging and dropping tasks. Each task supports Enabled, Continue on error, and Always run flags making it easier to manage build definitions as the task list grows: The build system supports invoking PowerShell, batch, command line, and shell scripts. All out-of-the-box tasks are open source. If a task does not satisfy your requirements, you can download the task from GitHub at https://github.com/Microsoft/vso-agent-tasks and customize it. If you can't find a task, you can easily create one. You'll learn more about custom tasks in this article. Changes to build definitions can be saved as drafts. Build definitions maintain a history of all changes in the History tab. A side-by-side comparison of the changes is also possible. Comments entered when changing the build definition show up in the change history: Build definitions can be saved as templates. This helps standardize the use of certain tasks across new build definitions: An existing build definition can be saved as a template Multiple triggers can be set for the same build, including CI triggers and multiple scheduled triggers: Rule-based retention policies support the setting up of multiple rules. Retention can be specified by "days" or "number" of the builds: The build output logs are displayed in web portal in real time. The build log can be accessed from the console even after the build gets completed: The build reports have been revamped to offer more visibility into the build execution, and among other things, the test results can now directly be accessed from the web interface. The .trx file does not need to be downloaded into Visual Studio to view the test results: The old build system had restrictions on one Team Project Collection per build controller and one controller per build machine. TFBuild removes this restriction and supports the reuse of queues across multiple Team Project Collections. The following image illustrates the architecture of the new build system: In the preceding diagram, we observe the following: Multiple agents can be configured on one machine Agents from across different machines can be grouped into a pool Each pool can have only one queue One queue can be used across multiple Team Project Collections To demonstrate the capabilities of TFBuild, we'll use the FabrikamTFVC and FabrikamGit Team Projects. Configuring TFBuild Agent, Pool, and Queues In this recipe, you'll learn how to configure agents and create pools and queues. You'll also learn how a queue can be used across multiple Team Project Collections. Getting ready Scenario: At Fabrikam, the FabrikamTFVC and FabrikamGit Team Projects need their own build queues. The FabrikamTFVC Teams build process can be executed on a Windows Server. The FabrikamGit Team build process needs both Windows and OS X. The Teams want to set up three build agents on a Windows Server; one build agent on an OS X machine. The Teams want to group two Windows Agents into a Windows Pool for FabrikamTFVC Team and group one Windows and one Mac Agent into another pool for the FabrikamGit Team: Permission: To configure a build agent, you should be in the Build Administrators Group. The prerequisites for setting up the build agent on a Windows-based machine are as follows: The build agent should have a supporting version of Windows. The list of supported versions is listed at https://msdn.microsoft.com/en-us/Library/vs/alm/TFS/administer/requirements#Operatingsystems. The build agent should have Visual Studio 2013 or 2015. The build agent should have PowerShell 3 or a newer version. A build agent is configured for your TFS as part of the server installation process if you leave the Configure the build service to start automatically option selected: For the purposes of this recipe, we'll configure the agents from scratch. Delete the default pool or any other pool you have by navigating to the Agent pools option in the TFS Administration Console http://tfs2015:8080/tfs/_admin/_AgentPool: How to do it Log into the Windows machine that you desire to set the agents upon. Navigate to the Agent pools in the TFS Administration Console by browsing to http://tfs2015:8080/tfs/_admin/_AgentPool. Click on New Pool, enter the pool name as Pool 1, and uncheck Auto-Provision Queue in Project Collections: Click on the Download agent icon. Copy the downloaded folder into E: and unzip it into E:Win-A1. You can use any drive; however, it is recommended to use the non-operating system drive: Run the PowerShell console as an administrator and change the current path in PowerShell to the location of the agent in this case E:Win-A1. Call the ConfigureAgent.ps1 script in the PowerShell console and click on Enter. This will launch the Build Agent Configuration utility: Enter the configuration details as illustrated in the following screenshot: It is recommended to install the build agent as a service; however, you have an option to run the agent as an interactive process. This is great when you want to debug a build or want to temporarily use a machine as a build agent. The configuration process creates a JSON settings file; it creates the working and diagnostics folders: Refresh the Agent pools page in the TFS Administration Console. The newly configured agent shows up under Pool 1: Repeat steps 2 to 5 to configure Win-A2 in Pool 1. Repeat steps 1 to 5 to configure Win-A3 in Pool 2. It is worth highlighting that each agent runs from its individual folder: Now, log into the Mac machine and launch terminal: Install the agent installer globally by running the commands illustrated here. You will be required to enter the machine password to authorize the install: This will download the agent in the user profile, shown as follows: The summary of actions performed when the agent is downloaded Run the following command to install the agent installer globally for the user profile: Running the following command will create a new directory called osx-A1 for the agent; create the agent in the directory: The agent installer has been copied from the user profile into the agent directory, shown as follows: Pass the following illustrated parameters to configure the agent: This completes the configuration of the xPlatform agent on the Mac. Refresh the Agent pools page in the TFS Administration Console to see the agent appear in Pool 2: The build agent has been configured at the Team Foundation Server level. In order to use the build agent for a Team Project Collection, a mapping between the build agent and Team Project Collection needs to be established. This is done by creating queues. To configure queues, navigate to the Collection Administration Console by browsing to http://tfs2015:8080/tfs/DefaultCollection/_admin/_BuildQueue. From the Build tab, click on New queue; this dialog allows you to reference the pool as a queue: Map Pool 1 as Queue 1 and Pool 2 as Queue 2 as shown here: The TFBuild Agent, Pools, and Queues are now ready to use. The green bar before the agent name and queue in the administration console indicates that the agent and queues are online. How it works... To test the setup, create a new build definition by navigating to the FabrikamTFVC Team Project Build hub by browsing to http://tfs2015:8080/tfs/DefaultCollection/FabrikamTFVC/_build. Click on the Add a new build definition icon. In the General tab, you'll see that the queues show up under the Queue dropdown menu. This confirms that the queues have been correctly configured and are available for selection in the build definition: Pools can be used across multiple Team Project Collections. As illustrated in the following screenshot, in Team Project Collection 2, clicking on the New queue... shows that the existing pools are already mapped in the default collection: Setting up a TFBuild Agent using an unattended installation The new build framework allows the unattended setup of build agents by injecting a set of parameter values via script. This technique can be used to spin up new agents to be attached into an existing agent pool. In this recipe, you'll learn how to configure and unconfigure a build agent via script. Getting ready Scenario: The FabrikamTFVC Team wants the ability to install, configure, and unconfigure a build agent directly via script without having to perform this operation using the Team Portal. Permission: To configure a build agent, you should be in the Build Administrators Group. Download the build agent as discussed in the earlier recipe Configuring TFBuild Agent, Pool, and Queues. Copy the folder to E:Agent. The script refers to this Agent folder. How to do it... Launch PowerShell in the elevated mode and execute the following command: .AgentVsoAgent.exe /Configure /RunningAsService /ServerUrl:"http://tfs2015:8080/tfs" /WindowsServiceLogonAccount:svc_build /WindowsServiceLogonPassword:xxxxx /Name:WinA-10 /PoolName:"Pool 1" /WorkFolder:"E:Agent_work" /StartMode:Automatic Replace the value of the username and password accordingly. Executing the script will result in the following output: The script installs an agent by the name WinA-10 as Windows Service running as svc_build. The agent is added to Pool 1: To unconfigure WinA-10, run the following command in an elevated PowerShell prompt: .AgentVsoAgent.exe /Unconfigure "vsoagent.tfs2015.WinA-10" To unconfigure, script needs to be executed from outside the scope of the Agent folder. Running the script from within the Agent folder scope will result in an error message. How it works... The new build agent natively allows configuration via script. A new capability called Personal Access Token (PAT) is due for release in the future updates of TFS 2015. PAT allows you to generate a personal OAuth token for a specific scope; it replaces the need to key in passwords into configuration files. Summary In this article, we have looked at configuring TFBuild Agent, Pool, and Queues and setting up a TFBuild Agent using an unattended installation. Resources for Article: Further resources on this subject: Overview of Process Management in Microsoft Visio 2013 [article] Introduction to the Raspberry Pi's Architecture and Setup [article] Implementing Microsoft Dynamics AX [article]
Read more
  • 0
  • 0
  • 1490

article-image-factory-method-pattern
Packt
10 Feb 2016
10 min read
Save for later

The Factory Method Pattern

Packt
10 Feb 2016
10 min read
In this article by Anshul Verma and Jitendra Zaa, author of the book Apex Design Patterns, we will discuss some problems that can occur mainly during the creation of class instances and how we can write the code for the creation of objects in a more simple, easy to maintain, and scalable way. (For more resources related to this topic, see here.) In this article, we will discuss the the factory method creational design pattern. Often, we find that some classes have common features (behavior) and can be considered classes of the same family. For example, multiple payment classes represent a family of payment services. Credit card, debit card, and net banking are some of the examples of payment classes that have common methods, such as makePayment, authorizePayment, and so on. Using the factory method pattern, we can develop controller classes, which can use these payment services, without knowing the actual payment type at design time. The factory method pattern is a creational design pattern used to create objects of classes from the same family without knowing the exact class name at design time. Using the factory method pattern, classes can be instantiated from the common factory method. The advantage of using this pattern is that it delegates the creation of an object to another class and provides a good level of abstraction. Let's learn this pattern using the following example: The Universal Call Center company is new in business and provides free admin support to customers to resolve issues related to their products. A call center agent can provide some information about the product support; for example, to get the Service Level Agreement (SLA) or information about the total number of tickets allowed to open per month. A developer came up with the following class: public class AdminBasicSupport{ /** * return SLA in hours */ public Integer getSLA() { return 40; } /** * Total allowed support tickets allowed every month */ public Integer allowedTickets() { // As this is basic support return 9999; } } Now, to get the SLA of AdminBasicSupport, we need to use the following code every time: AdminBasicSupport support = new AdminBasicSupport(); System.debug('Support SLA is - '+support.getSLA()); Output - Support SLA is – 40 The "Universal Call Centre" company was doing very well, and in order to grow the business and increase the profit, they started the premium support for customers who were willing to pay for cases and get a quick support. To make them special from the basic support, they changed the SLA to 12 hours and maximum 50 cases could be opened in one month. A developer had many choices to make this happen in the existing code. However, instead of changing the existing code, they created a new class that would handle only the premium support-related functionalities. This was a good decision because of the single responsibility principle. public class AdminPremiumSupport{ /** * return SLA in hours */ public Integer getSLA() { return 12; } /** * Total allowed support tickets allowed every month is 50 */ public Integer allowedTickets() { return 50; } } Now, every time any information regarding the SLA or allowed tickets per month is needed, the following Apex code can be used: if(Account.supportType__c == 'AdminBasic') { AdminBasicSupport support = new AdminBasicSupport(); System.debug('Support SLA is - '+support.getSLA()); }else{ AdminPremiumSupport support = new AdminPremiumSupport(); System.debug('Support SLA is - '+support.getSLA()); } As we can see in the preceding example, instead of adding some conditions to the existing class, the developer decided to go with a new class. Each class has its own responsibility, and they need to be changed for only one reason. If any change is needed in the basic support, then only one class needs to be changed. As we all know that this design principle is known as the Single Responsibility Principle. Business was doing exceptionally well in the call center, and they planned to start the golden and platinum support as well. Developers started facing issues with the current approach. Currently, they have two classes for the basic and premium support and requests for two more classes were in the pipeline. There was no guarantee that the support type will not remain the same in future. Because of every new support type, a new class is needed; and therefore, the previous code needs to be updated to instantiate these classes. The following code will be needed to instantiate these classes: if(Account.supportType__c == 'AdminBasic') { AdminBasicSupport support = new AdminBasicSupport(); System.debug('Support SLA is - '+support.getSLA()); }else if(Account.supportType__c == 'AdminPremier') { AdminPremiumSupport support = new AdminPremiumSupport(); System.debug('Support SLA is - '+support.getSLA()); }else if(Account.supportType__c == 'AdminGold') { AdminGoldSupport support = new AdminGoldSupport(); System.debug('Support SLA is - '+support.getSLA()); }else{ AdminPlatinumSupport support = new AdminPlatinumSupport(); System.debug('Support SLA is - '+support.getSLA()); } We are only considering the getSLA() method, but in a real application, there can be other methods and scenarios as well. The preceding code snippet clearly depicts the code duplicity and maintenance nightmare. The following image shows the overall complexity of the example that we are discussing: Although they are using a separate class for each support type, an introduction to a new support class will lead to changes in the code in all existing code locations where these classes are being used. The development team started brainstorming to make sure that the code is capable to extend easily in future with the least impact on the existing code. One of the developers came up with a suggestion to use an interface for all support classes so that every class can have the same methods and they can be referred to using an interface. The following interface was finalized to reduce the code duplicity: public Interface IAdminSupport{ Integer getSLA() ; Integer allowedTickets(); } Methods defined within an interface have no access modifiers and just contain their signatures. Once an interface was created, it was time to update existing classes. In our case, only one line needed to be changed and the remaining part of the code was the same because both the classes already have the getSLA() and allowedTickets() methods. Let's take a look at the following line of code: public class AdminPremiumSupport{ This will be changed to the following code: public class AdminBasicSupportImpl implements IAdminSupport{ The following line of code is as follows: public class AdminPremiumSupport{ This will be changed to the following code: public class AdminPremiumSupportImpl implements IAdminSupport{ In the same way, the AdminGoldSupportImpl and AdminPlatinumSupportImpl classes are written. A class diagram is a type of Unified Modeling Language (UML), which describes classes, methods, attributes, and their relationships, among other objects in a system. You can read more about class diagrams at https://en.wikipedia.org/wiki/Class_diagram. The following image shows a class diagram of the code written by developers using an interface: Now, the code to instantiate different classes of the support type can be rewritten as follows: IAdminSupport support = null; if(Account.supportType__c == 'AdminBasic') { support = new AdminBasicSupportImpl(); }else if(Account.supportType__c == 'AdminPremier') { support = new AdminPremiumSupportImpl(); }else if(Account.supportType__c == 'AdminGold') { support = new AdminGoldSupportImpl(); }else{ support = new AdminPlatinumSupportImpl(); } System.debug('Support SLA is - '+support.getSLA()); There is no switch case statement in Apex, and that's why multiple if and else statements are written. As per the product team, a new compiler may be released in 2016 and it will be supported. You can vote for this idea at https://success.salesforce.com/ideaView?id=08730000000BrSIAA0. As we can see, the preceding code is minimized to create a required instance of a concrete class, and then uses an interface to access methods. This concept is known as program to interface. This is one of the most recommended OOP principles suggested to be followed. As interfaces are kinds of contracts, we already know which methods will be implemented by concrete classes, and we can completely rely on the interface to call them, which hides their complex implementation and logic. It has a lot of advantages and a few of them are loose coupling and dependency injection. A concrete class is a complete class that can be used to instantiate objects. Any class that is not abstract or an interface can be considered a concrete class. We still have one problem in the previous approach. The code to instantiate concrete classes is still present at many locations and will still require changes if a new support type is added. If we can delegate the creation of concrete classes to some other class, then our code will be completely independent of the existing code and new support types. This concept of delegating decisions and creation of similar types of classes is known as the factory method pattern. The following class can be used to create concrete classes and will act as a factory: /** * This factory class is used to instantiate concrete class * of respective support type * */ public class AdminSupportFactory { public static IAdminSupport getInstance(String supporttype){ IAdminSupport support = null; if(supporttype == 'AdminBasic') { support = new AdminBasicSupportImpl(); }else if(supporttype == 'AdminPremier') { support = new AdminPremiumSupportImpl(); }else if(supporttype == 'AdminGold') { support = new AdminGoldSupportImpl(); }else if(supporttype == 'AdminPlatinum') { support = new AdminPlatinumSupportImpl(); } return support ; } } In the preceding code, we only need to call the getInstance(string) method, and this method will take a decision and return the actual implementation. As a return type is an interface, we already know the methods that are defined, and we can use the method without actually knowing its implementation. This is a very good example of abstraction. The final class diagram of the factory method pattern that we discussed will look like this: The following code snippet can be used repeatedly by any client code to instantiate a class of any support type: IAdminSupport support = AdminSupportFactory.getInstance ('AdminBasic'); System.debug('Support SLA is - '+support.getSLA()); Output : Support SLA is – 40 Reflection in Apex The problem with the preceding design is that whenever a new support needs to be added, we need to add a condition to AdminSupportFactory. We can store the mapping between a support type and its concrete class name in Custom setting. This way, whenever a new concrete class is added, we don't even need to change the factory class and a new entry needs to be added to custom setting. Consider custom setting created by the Support_Type__c name with the Class_Name__c field name of the text type with the following records: Name Class name AdminBasic AdminBasicSupportImpl AdminGolden AdminGoldSupportImpl AdminPlatinum AdminPlatinumSupportImpl AdminPremier AdminPremiumSupportImpl However, using reflection, the AdminSupportFactory class can also be rewritten to instantiate service types at runtime as follows: /** * This factory class is used to instantiate concrete class * of respective support type * */ public class AdminSupportFactory { public static IAdminSupport getInstance(String supporttype) { //Read Custom setting to get actual class name on basis of Support type Support_Type__c supportTypeInfo = Support_Type__c.getValues(supporttype); //from custom setting get appropriate class name Type t = Type.forName(supportTypeInfo.Class_Name__c); IAdminSupport retVal = (IAdminSupport)t.newInstance(); return retVal; } } In the preceding code, we are using the Type system class. This is a very powerful class used to instantiate a new class at runtime. It has the following two important methods: forName: This returns a type that is equivalent to a string passed newInstance: This creates a new object for a specified type Inspecting classes, methods, and variables at runtime without knowing a class name, or instantiating a new object and invoking methods at runtime is known as Reflection in computer science. One more advantage of using the factory method, custom setting, and reflection together is that if in future one of the support types need to be replaced by another service type permanently, then we need to simply change the appropriate mapping in custom setting without any changes in the code. Summary In this article, we discussed how to deal with various situations while instantiating objects using design patterns, using the factory method. Resources for Article: Further resources on this subject: Getting Your APEX Components Logic Right[article] AJAX Implementation in APEX[article] Custom Coding with Apex[article]
Read more
  • 0
  • 23
  • 5079

article-image-introduction-akka
Packt
04 Feb 2016
15 min read
Save for later

Introduction to Akka

Packt
04 Feb 2016
15 min read
In this article written by Prasanna Kumar Sathyanarayanan and Suraj Atreya (the authors of this book, Reactive Programming with Scala and Akka), we will see what the Akka framework is all about in detail. Akka is a framework to write distributed, asynchronous, concurrent, and scalable applications. Akka actors react to the messages that are sent to them. Actors can be viewed as a passive component unless an external message is triggered. They are a higher level abstraction and provide a neat way to handle concurrency instead of a traditional multithreaded application. (For more resources related to this topic, see here.) One of the examples of Akka actor is handling request response in a web server. A web server typically handles millions of requests per second. These requests must be handled concurrently to cater to the user requests. One way is to have a pool of threads and let these threads accept the requests and hand it off to the actual worker threads. In this case, the thread pool has to be managed by the application developer including error handling, thread locking, and synchronization. Most often, the application logic is intertwined with the business logic. In case of the web server, the thread pool has to be manually handled. Whereas, using actors, the thread pool is managed by the Akka engine and actors receive messages asynchronously. Each request can be thought of as a message to the actor, and the actor reacts to the message. Actors are very light weight event-driven processes. Several million actors can exist within a GB of heap memory. Actor mailbox Every actor has a mailbox. Since actors communicate exclusively using messages, every actor maintains a queue of messages called mailbox. Therefore, an actor will read the messages in the order that it was sent. Actor systems An ActorSystem is a heavyweight process which is the first step before creating actors. During initialization, it allocates 1 to N threads per ActorSystem. Before creating an actor, an actor system must be created and this process involves the creation of a hierarchical structure of actors. Since an actor can create other actors, the handling of failure is also a vital part of the Akka engine, which handles it gracefully. This design helps to take action if an actor dies or has an unexpected exception for some reason. When an actor system is first created, three actors are created as shown in the figure: Root guardian (/): The root guardian is the grand parent of all the actors. This actor supervises all actors under the user actors. Since the root guardian is the supervisor of all the actors underneath it, there is no supervisor for the root guardian itself. Root guardian actor is not a real actor and it will terminate the actor system if it finds any throwables from its children. User ( /user): The user guardian actor supervises all the actors that are created by the users. If the user guardian actor terminates, all its children will also terminate. System guardian ( /system): This is a special actor that oversees the orderly shutdown of actors while logging remains active. This is achieved by having the system guardian watch the user guardian and initialize a shut-down sequence when the Terminated message is received. Message passing The figure at the end of this section shows the different steps that are involved when a message is passed to an actor. For example, let's assume there is a pizza website and a customer wants to order some pizzas. For simplicity, let's remove the non-essential details, such as billing and other information, and instead focus on just the order of a pizza. If a customer is some kind of an application (pizza customer) and the one who receives orders is a chef (pizza chef), then each request for a pizza can be illustrated as an asynchronous request to the chef. The figure shows how when a message is passed to an actor, all the different components such as mailbox and dispatcher does its job. Broadly these are explained in the following six steps when a message is passed to the actor: A PizzaCustomer creates and uses the ActorSystem. This is the first step before sending a message to an actor. The PizzaCustomer acquires a PizzaChef. In Akka, an actor is created using the actorOf(...) function call. Akka doesn't return the actual actor but instead returns a reference to the actor reference PizzaChef for safety. The PizzaRequest is sent to this PizzaChef. The PizzaChef sends this message to the Dispatcher. Dispatcher then enqueues the message into the PizzaChef's actor mailbox. Dispatcher then puts the mailbox on the thread. Finally, the mailbox dequeues and sends the message to the PizzaChef receive method. Creating an actor system An actor system is the first thing that should be created before creating actors. The actor system is created using the following API: val system = ActorSystem("Pizza") The string "Pizza" is just a name given to the actor system. Creating an ActorRef The following snippet shows the creation of an actor inside the previously created actor system: val pizzaChef: ActorRef = system.actorOf(Props[PizzaChef]) An actor is created using the actorOf(...) function call. The actorOf() call doesn't return the actor itself, instead returns a reference to the actor. Once an actor's reference is obtained, clients can send messages using the ActorRef. This is a safe way of communicating between actors since the state of the actor itself is not manipulated in any way. Sending a PizzaRequest to the ActorRef Now that we have an actor system and a reference to the actor, we would like to send requests to the pizzaChef actor reference. We send the message to an actor using the ! also called Tell. Here in the code snippet, we Tell the message MarinaraRequest to the pizza ActorRef: pizzaChef ! MarinaraRequest The Tell is also called as fire-forget. There is no acknowledgement returned from a Tell. When the message is sent to an actor, the actor's receive method will receive the message and processes it further. receive is a partial function and has the following signature: def receive: PartialFunction[Any, Unit] The return type receive suggests it is Unit and therefore this function is side effecting. The following code is what we discussed: import akka.actor.{Actor, ActorRef, Props, ActorSystem} sealed trait PizzaRequest case object MarinaraRequest extends PizzaRequest case object MargheritaRequest extends PizzaRequest class PizzaChef extends Actor {   def receive = {     case MarinaraRequest => println("I have a Marinara request!")     case MargheritaRequest => println("I have a Margherita request!")   } } object PizzaCustomer{     def main(args: Array[String]) : Unit = {     val system = ActorSystem("Pizza")     val pizzaChef: ActorRef = system.actorOf(Props[PizzaChef])     pizzaChef ! MarinaraRequest     pizzaChef ! MargheritaRequest   } } The preceding code shows the receive block that handles two kinds of requests; one for MarinaraRequest, and the other for MargheritaRequest. These two requests are defined as case objects. Actor message We saw when a message needs to be sent, ! (Tell) was used. But, we didn't discuss how exactly this message is processed. We will explore how the ideas of dispatcher and execution context are used to carry out the message passing techniques between actors. In the pizza example, we used two kinds of messages: MarinaraRequest and MargheritaRequest. For simplicity, all that these messages did was to print on the console. When PizzaCustomer sent PizzaRequest to PizzaChef ActorRef, the messages are sent to the dispatcher. The dispatcher then sends this message to the corresponding actor's mailbox. Mailbox Every time a new actor is created, a corresponding mailbox is also created. There are exceptions to this rule when multiple actors share a same mailbox. PizzaChef will have a mailbox. This mailbox stores the messages that appear asynchronously in a FIFO manner. Therefore, when a new message is sent to an actor, Akka guarantees that the messages are enqueued and dequeued in a FIFO manner. Here is the signature of the mailbox from the Akka source: It can be found at https://github.com/akka/akka/blob/master/akka-actor/src/main/scala/akka/dispatch/Mailbox.scala private[akka] abstract class Mailbox(val messageQueue: MessageQueue)     extends ForkJoinTask[Unit]       with SystemMessageQueue with Runnable From this signature, we can see that Mailbox takes a MessageQueue as an input. Also, we can see that it extends Runnable and suggests that Mailbox is a thread. We will see why the Mailbox is a thread, in a bit. Dispatcher Dispatchers dispatch actors to threads. There is no one-to-one mapping between actors and threads. If that was the case, then the whole system would crumble under its own weight. Also, the amount of context switching would be much more than the actual work. Therefore, it is important to understand that creating a number of actors is not equal to creating the same number of threads. The main objective of dispatcher is to coordinate between the actors and its messages to the underlying threads. A dispatcher picks the actor next in the queue based on the dispatcher policy and the actor's message in the queue. These two are then passed on to one of the available threads in the execution context. To illustrate this point, let's see the code snippet: protected[akka] override   def registerForExecution(mbox: Mailbox, ...): Boolean = {   ...      if (mbox.setAsScheduled()) {      try {          executorService execute mbox          true       }   } This code snippet shows us that the dispatcher accepts a Mailbox as a parameter and has ExecutorService wrapped around to execute the mailbox. We saw that Mailbox is a thread and the dispatcher executes this Mailbox against this ExecutorService. When the mailbox's run method is triggered, it dequeues a message from Mailbox and passes it to the actor for processing. This is the code snippet of run from Mailbox.scala from the Akka source code: The source code can be found at https://github.com/akka/akka/blob/master/akka-actor/src/main/scala/akka/dispatch/Mailbox.scala override final def run(): Unit = {     try {       if (!isClosed) { //Volatile read, needed here         processAllSystemMessages() //First, deal with any system messages         processMailbox() //Then deal with messages       }     } finally {       setAsIdle() //Volatile write, needed here       dispatcher.registerForExecution(this, false, false)     }   } Actor Path The interesting thing about actor system is that actors are created in a hierarchical manner. All the user created actors are created under the /user actor. The actor path looks very similar to the UNIX file system hierarchy, for example, /home/akka/akka_book. We will see how this is similar to the Akka path in the following code example. Let's take our pizza example, and let's add a few toppings on the pizza. So, whenever a customer gives MarinaraRequest, he will get extra cheese too: class PizzaToppings extends Actor{   def receive = {     case ExtraCheese => println("Aye! Extra cheese it is")     case Jalapeno => println("More Jalapenos!")   } } class PizzaSupervisor extends Actor {   val pizzaToppings =         context.actorOf(Props[PizzaToppings], "PizzaToppings")   def receive = {     case MarinaraRequest   =>       println("I have a Marinara request with extra cheese!")       println(pizzaToppings.path)       pizzaToppings ! ExtraCheese     case MargheritaRequest =>       println("I have a Margherita request!")     case PizzaException    =>      throw new Exception("Pizza fried!")   } } The PizzaSupervisor class is very similar to our earlier example of pizza actor. However, if you observe carefully, there is another actor created within this PizzaSupervisor called the PizzaToppings actor. This PizzaToppings is created using context.actorOf(...) instead of using system.actorOf(...). Therefore, PizzaToppings will become the child of PizzaSupervisor. The actor path of PizzaSupervisor will look like this: akka://Pizza/user/PizzaSupervisor The actor path for PizzaToppings will look like this: akka://Pizza/user/PizzaSupervisor/PizzaToppings When this main program is run, the actor system is created using system.actorOf(...) and prints the path of the pizza actor system and its corresponding child as shown previously: object TestActorPath {   def main(args: Array[String]): Unit = {     val system = ActorSystem("Pizza")     val pizza: ActorRef =  system.actorOf(Props[PizzaSupervisor],                    "PizzaSupervisor")     println(pizza.path)     pizza ! MarinaraRequest     system.shutdown()   } } The following is the output:akka://Pizza/user/PizzaSupervisorI have a Marinara request with extra cheese!akka://Pizza/user/PizzaSupervisor/PizzaToppingsAye! Extra cheese it is The name akka in the actor path is fixed and the actors created are shown under the user. If you remember from earlier discussions, all the user-created actors are created under the user guardian. Therefore, the actor path shows that it is a user-created actor. The name pizza is the name we gave to the actor system while it was being created. Therefore, the hierarchy explains that pizza is the actor system and all the actors are children below it. In the following figure, we can clearly see the actor hierarchy and its actor path: Actor lifecycle Akka actors have a life cycle that is very useful for writing a bug free concurrent code. Akka follows a philosophy of Let it crash and it is assumed that actors too can crash. But if an actor crashes, several actions can be taken including restarting it. As usual let's look at our pizza baking process. As before, we will have an actor to accept the pizza requests. But, this time we will see the workflow of the pizza baking process! Using this example, we will see how the actor life cycle works: class PizzaLifeCycle extends Actor with ActorLogging {   override def preStart() = {     log.info("Pizza request received!")   }   def receive = {     case MarinaraRequest   => log.info("I have a Marinara request!")     case MargheritaRequest => log.info("I have a Margherita request!")     case PizzaException    => throw new Exception("Pizza fried!")   }  //Old actor instance   override def preRestart(reason: Throwable, message: Option[Any]) = {     log.info("Pizza baking restarted because " + reason.getMessage)     postStop()   }   //New actor instance   override def postRestart(reason: Throwable) = {     log.info("New Pizza process started because earlier " + reason.getMessage)     preStart()   }   override def postStop() = {     log.info("Pizza request finished")   } } The PizzaLifeCycle actor takes pizza requests but with additional states. An actor can go through many different states during its lifetime. Let's send some messages to find out what happens with our PizzaLifeCycle actor and how it behaves:     pizza ! MarinaraRequest         pizza ! PizzaException         pizza ! MargheritaRequest Here is the output for the preceding requests sent:Pizza request received!I have a Marinara request!Pizza fried!java.lang.Exception: Pizza fried!at PizzaLifeCycle$$anonfun$receive$1.applyOrElse(PizzaLifeCycle.scala:12)at akka.actor.Actor$class.aroundReceive(Actor.scala:467)at PizzaLifeCycle.aroundReceive(PizzaLifeCycle.scala:3)at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)Pizza baking restarted because Pizza fried!Pizza request finishedNew Pizza process started because Pizza fried!Pizza request received!I have a Margherita request! When we sent our first MarinaraRequest request, we see the following in the log we received: Pizza request received! I have a Marinara request! Akka called the preStart() method and then entered the receive block. Then, we simulated an exception by sending PizzaException and as expected, we got an exception: Pizza fried!java.lang.Exception: Pizza fried!  at PizzaLifeCycle$$anonfun$receive$1.applyOrElse(PizzaLifeCycle.scala:12)  at akka.actor.Actor$class.aroundReceive(Actor.scala:467)  at PizzaLifeCycle.aroundReceive(PizzaLifeCycle.scala:3)  at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)Pizza baking restarted because Pizza fried!Pizza request finished There are some interesting things to note here. Although we got an exception Pizza fried!, we also got two other log messages. The reason for this is quite simple. When we have an exception, Akka called preRestart(). During preRestart(), is called on the old instance of the actor and have a chance to clean-up some of the resources here. But in our example, we just called postStop(). During preRestart(), the old instance prepares to handoff to the new actor instance. Finally, we sent another request called MargheritaRequest. Here, we get these log messages: New Pizza process started because Pizza fried!Pizza request received!I have a Margherita request! We saw that the old instance actor was stopped. Here, the requests are handled by a new actor instance. The postRestart()is now called on the new actor instance, which calls preStart() to resume normal operations of our pizza baking process. During the preRestart() and postRestart() methods, we got a reason as to why the old actor died. Summary In this article, you learned about the details of the Akka framework, the actor mailbox, actor systems, how to create an actor system and ActorRef, how to send PizzaRequest to ActorRef, and so on. Resources for Article:   Further resources on this subject: The Design Patterns Out There and Setting Up Your Environment [article] Creating First Akka Application [article] Content-based recommendation [article]
Read more
  • 0
  • 0
  • 2101
article-image-protocol-extensions
Packt
02 Feb 2016
7 min read
Save for later

Protocol Extensions

Packt
02 Feb 2016
7 min read
In this article by John Hoffman, the author of Protocol Oriented Programming with Swift, you will study the types of protocols that can be extended. Protocol extensions can be used to provide common functionality to all the types that conform to a particular protocol. This gives us the ability to add functionality to any type that conforms to a protocol, rather than adding the functionality to each individual type or though a global function. Protocol extensions, like regular extensions, also give us the ability to add functionality to types that we do not have the source code for. (For more resources related to this topic, see here.) Protocol-Oriented programming would not be possible without protocol extensions. Without protocol extensions, if we wanted to add specific functionality to a group of types that conformed to a protocol, we would have to add the functionality to each of the types. If we were using reference types (classes), we could create a class hierarchy, but this is not possible for value types. Apple has stated that we should prefer value types to reference types, and with protocol-extensions, we have the ability to add common functionality to a group of value and/or reference types that conform to a specific protocol without having to implement that functionality in all the types. Let's take a look at what protocol extensions can do for us. The Swift standard library provides a protocol named CollectionType (documentation here). This protocol inherits from the Indexable and SequenceType protocols and is adopted by all of Swift's standard collection types such as Dictionary and Array. Let's say that we want to add the functionality to types that conform to CollectionType. These would shuffle the items in a collection or return only the items whose index number is an even number. We could very easily add this functionality by extending the CollectionType protocol, as shown in the following code: extension CollectionType { func evenElements() -> [Generator.Element] { var index = self.startIndex var result: [Generator.Element] = [] var i = 0 repeat { if i % 2 == 0 { result.append(self[index]) } index = index.successor() i++ } while (index != self.endIndex) return result } func shuffle() -> [Self.Generator.Element] { return sort(){ left, right in return arc4random() < arc4random() } } } Notice that when we extend a protocol, we use the same syntax and format that we use when we extend other types. We use the extension keyword, followed by the name of the protocol that we extend. We then put the functionality that we add to the protocol between curly brackets. Now, every type that conforms to the CollectionType protocol will receive both the evenElements() and shuffle() functions. The following code shows how we can use these functions with an array: var origArray = [1,2,3,4,5,6,7,8,9,10] var newArray = origArray.evenElements() var ranArray = origArray.shuffle() In the previous code, the newArray array would contain the elements 1, 3, 5, 7, and 9 because these elements have even index numbers (we are looking at the index number, not the value of the element). The ranArray array would contain the same elements as origArray, but the order will be shuffled. Protocol extensions are great to add functionality to a group of types without the need to add the code to each of the individual types; however, it is important to know what types conform to the protocol we extend. In the previous example, we extended the CollectionType protocol by adding the evenElements() and shuffle() methods to all the types that conform to the protocol. One of the types that conform to this protocol is the Dictionary type; however, the Dictionary type is an unordered collection. Therefore, the evenElements() method will not work as expected. The following example illustrates this: var origDict = [1:"One",2:"Two",3:"Three",4:"Four"] var returnElements = origDict.evenElements() for item in returnElements { print(item) } Since the Dictionary type does not promise to store the items in the dictionary in any particular order, any of the two items could be printed to the screen in this example. The following shows one possible output from this code: (2, "two") (1, "One") Another problem is that anyone who is not familiar with how the evenElements() method is implemented may expect the returnElements instance to be a dictionary. This is because the original collection is a dictionary type; however, it is actually an instance of the Array type. This can cause some confusion; therefore, we need to be carful when we extend a protocol to make sure that the functionality we add works as expected for the types that conform to the protocol. In the case of the shuffle() and evenElements() methods, we may have been better served if we added the functionality as an extension directly to the Array type, rather than the CollectionType protocol; however, there is another way. We can add constraints to our extension that will limit the types that receive the functionality defined in an extension. In order for a type to receive the functionality defined in a protocol extension, it must satisfy all the constraints defined within the protocol extension. A constraint is added after the name of the protocol that we extend using the where keyword. The following code shows how we could add a constraint to our CollectionType extension: extension CollectionType where Self: ArrayLiteralConvertible { //Extension code here } In the CollectionType protocol extensions, as shown in the previous example, only types that also conform to the ArrayLiteralConvertible protocol will receive the functionality defined in the extension. Since the Dictionary type does not conform to the ArrayLiteralConvertible protocol, it will not receive the functionality defined within the protocol. We could also use constraints to define that our CollectionType protocol extensions only apply to a collection whose elements conform to a specific protocol. In the next example, we use constraints to make sure that the elements in the collection conform to the Comparable protocol. This may be necessary if the functionality that we add relies on the ability to compare two or more elements in the collection. We could add the constraint like this: extension CollectionType where Generator.Element: Comparable { // Add functionality here } Constraints give us the ability to limit which types receive the functionality defined in the extension. One thing that we need to be careful of is using protocol extensions when we should actually be extending an individual type. Protocol extensions should be used when we want to add a functionality to a group of types. If we try to add the functionality to a single type, we should look at extending this individual type. We created a series of protocols that defined the Tae Kwon Do testing areas. Let's take a look at how we can extend the TKDRank protocol from this example to add the ability to store which testing areas the student passed and the areas in which they failed. The following code is for the original TKDRank protocol: protocol TKDRank { var color: TKDBeltColors {get} var rank: TKDColorRank {get} } We will begin by adding an instance of the Dictionary type to our protocol. This dictionary type will store the results of our tests. The following example shows what the new TKDRank protocol will look like: protocol TKDRank { var color: TKDBeltColors {get} var rank: TKDColorRank {get} var passFailTests: [String:Bool] {get set} } We can now extend the TKDRank protocol to add a method that we can use to set in instances where the student passes or fails individual tests. The following code shows how we can do this: extension TKDRank { mutating func setPassFail(testName: String, pass: Bool) { passFailTests[testName] = pass } Now, any type that conforms to the TKDRank protocol will have the setPassFail() method automatically. Since we have seen how to use extensions and protocol extensions, let's take a look at a real-world example. In this example, we will explore ways in which we can create a text validation framework. Summary In this article, we looked at an extension. In the original version of Swift, we were able to use extensions to extend structures, classes, and enumerations, but starting with Swift 2, we are able to use extensions extend protocols as well. Without protocol extensions, protocol-oriented programming would not be possible, but we need to make sure that we use protocol extensions where appropriate, and do not try to use them in place of regular extensions. Resources for Article: Further resources on this subject: The Swift Programming Language [article] Your First Swift App [article] Using Protocols and Protocol Extensions [article]
Read more
  • 0
  • 0
  • 1245

article-image-cluster-basics-and-installation-centos-7
Packt
01 Feb 2016
8 min read
Save for later

Cluster Basics and Installation On CentOS 7

Packt
01 Feb 2016
8 min read
In this article by Gabriel A. Canepa, author of the book CentOS High Performance, we will review the basic principles of clustering and show you, step by step, how to set up two CentOS 7 servers as nodes to later use them as members of a cluster. (For more resources related to this topic, see here.) As part of this process, we will install CentOS 7 from scratch in a brand new server as our first cluster member, along with the necessary packages, and finally, configure key-based authentication for SSH access from one node to the other. Clustering fundamentals In computing, a cluster consists of a group of computers (which are referred to as nodes or members) that work together so that the set is seen as a single system from the outside. One typical cluster setup involves assigning a different task to each node, thus achieving a higher performance than if several tasks were performed by a single member on its own. Another classic use of clustering is helping to ensure high availability by providing failover capabilities to the set, where one node may automatically replace a failed member to minimize the downtime of one or several critical services. In either case, the concept of clustering implies not only taking advantage of the computing functionality of each member alone, but also maximizing it by complementing it with the others. As we just mentioned, HA (High-availability) clusters aim to eliminate system downtime by failing services from one node to another in case one of them experiences an issue that renders it inoperative. As opposed to switchover, which requires human intervention, a failover procedure is performed automatically by the cluster without any downtime. In other words, this operation is transparent to end users and clients from outside the cluster. On the other hand, HP (High-performance) clusters use their nodes to perform operations in parallel in order to enhance the performance of one or more applications. High-performance clusters are typically seen in scenarios involving applications that use large collections of data. Why CentOS? Just as the saying goes, Every journey begins with a small step, we will begin our own journey toward clustering by setting up the separate nodes that will make up our system. Our choice of operating system is Linux and CentOS, version 7, as the distribution, that being the latest available release of CentOS as of today. The binary compatibility with Red Hat Enterprise Linux © (which is one of the most well-used distributions in enterprise and scientific environments) along with its well-proven stability are the reasons behind this decision. CentOS 7 along with its previous versions of the distribution are available for download, free of charge, from the project's website at http://www.centos.org/. In addition, specific details about the release can always be consulted in the CentOS wiki, http://wiki.centos.org/Manuals/ReleaseNotes/CentOS7. Among the distinguishing features of CentOS 7, I would like to name the following: It includes systemd as the central system management and configuration utility It uses XFS as the default filesystem It only supports the x86_64 architecture Downloading CentOS To download CentOS, go to http://www.centos.org/download/ and click on one of the three options outlined in the following figure: Download options for CentOS 7 These options are detailed as follows: DVD ISO (~4 GB) is an .iso file that can be burned into regular DVD optical media and includes the common tools. Download this file if you have immediate access to a reliable Internet connection that you can use to download other packages and utilities. Everything ISO (~7 GB) is an .iso file with the complete set of packages that are made available in the base repository of CentOS 7. Download this file if you do not have access to a reliable Internet connection or if your plan contemplates the possibility of installing or populating a local or network mirror. The alternative downloads link will take you to a public directory within an official nearby CentOS mirror, where the previous options are available as well as others, including different choices of desktop versions (GNOME or KDE) and the minimal .iso file (~570 MB), which contains the bare bone packages of the distribution. As the minimal install is sufficient for our purpose at hand, we can install other needed packages using yum later, that is, the recommended .iso file to download. CentOS-7.X-YYMM-x86_64-Minimal.iso Here, X indicates the current update number of CentOS 7 and YYMM represent the year and month, both in two-digit notation, when the source code this version is based on was released. CentOS-7.0-1406-x86_64-Minimal.iso This tells us the source code this release is based on dates from the month of June, 2014. Independently of our preferred download method, we will need this .iso file in order to begin with the installation. In addition, feel free to burn it to optical media or a USB drive. Setting up CentOS 7 nodes If you do not have dedicated hardware that you can use to set up the nodes of your cluster, you can still create one using virtual machines over some virtualization software, such as Oracle Virtualbox © or VMware ©, for example. The following setup is going to be performed on a Virtualbox VM with 1 GB of RAM and 30 GB of disk space. We will use the default partitioning schema over LVM as suggested by the installation process. Installing CentOS 7 The splash screen shown in the following screenshot is the first step in the installation process. Highlight Install CentOS 7 using the up and down arrows and press Enter: Splash screen before starting the installation of CentOS 7 Select English (or your preferred installation language) and click on Continue, as shown in the following screenshot: Selecting the language for the installation of CentOS 7 In the following screenshot, you can choose a keyboard layout, set the current date and time, choose a partitioning method, connect the main network interface, and assign a unique hostname for the node. We will name the current node node01 and leave the rest of the settings as default (we will configure the extra network card later). Then, click on Begin installation: Configure keyboard layout, date and time, network and hostname, and partitioning schema While the installation continues in the background, we will be prompted to set the password for the root account and create an administrative user for the node. Once these steps have been confirmed, the corresponding warnings no longer appear, as shown in the following screenshot: Setting the password for root and creating an administrative user account When the process is completed, click on Finish configuration and the installation will finish configuring the system and devices. When the system is ready to boot on its own, you will be prompted to do so. Remove the installation media and click on Reboot. Now, we can proceed with setting up our network interfaces. Setting up the network infrastructure Our rather basic network infrastructure consists of 2 CentOS 7 boxes, with the node01 [192.168.0.2] and node02 [192.168.0.3] host names, respectively, and a gateway router called simply gateway [192.168.0.1]. In CentOS, network cards are configured using scripts in the /etc/sysconfig/network-scripts directory. This is the minimum content that is needed in /etc/sysconfig/network-scripts/ifcfg-enp0s3 for our purposes: HWADDR="08:00:27:C8:C2:BE" TYPE="Ethernet" BOOTPROTO="static" NAME="enp0s3" ONBOOT="yes" IPADDR="192.168.0.2" NETMASK="255.255.255.0" GATEWAY="192.168.0.1" PEERDNS="yes" DNS1="8.8.8.8" DNS2="8.8.4.4" Note that the UUID and HWADDR values will be different in your case. In addition, be aware that cluster machines need to be assigned a static IP address—never leave that up to DHCP! In the preceding configuration file, we used Google's DNS, but if you wish, feel free to use another DNS. When you're done making changes, save the file and restart the network service in order to apply them: systemctl restart network.service # Restart the network service You can verify that the previous changes have taken effect (shown in the Restarting the network service and verifying settings figure) with the following two commands: systemctl status network.service # Display the status of the network service And the changes have also taken effect due to this command: ip addr | grep 'inet addr' # Display the IP addresse Restarting the network service and verifying settings You can disregard all error messages related to the loopback interface, as shown in preceding screenshot. However, you will need to examine carefully any error messages related to the enp0s3 interface, if any, and get them resolved in order to proceed further. The second interface will be called enp0sX, where X is typically 8. You can verify with the following command (shown in the following figure): ip link show Displaying NIC information As for the configuration file of enp0s8, you can safely create it, copying the contents of ifcfg-enp0s3. Do not forget, however, to change the hardware (MAC) address as returned by the information on the NIC and leave the IP address field blank for now. ip link show enp0s8 cp /etc/sysconfig/network-scripts/ifcfg-enp0s3 /etc/sysconfig/network-scripts/ifcfg-enp0s8 Then, restart the network service. Note that you will also need to set up at least a basic DNS resolution method. Considering that we will set up a cluster with 2 nodes only, we will use /etc/hosts for this purpose. Edit /etc/hosts with the following content: 192.168.0.2 node01 192.168.0.3 node02 192.168.0.1 gateway Summary In this article, we reviewed how to install the operating system and listed the necessary software components to implement the basic cluster functionality. Resources for Article: Further resources on this subject: CentOS 7's new firewalld service[article] Mastering CentOS 7 Linux Server[article] Resource Manager on CentOS 6[article]
Read more
  • 0
  • 0
  • 9786

article-image-creating-simple-maps-openlayers-3
Packt
22 Jan 2016
14 min read
Save for later

Creating Simple Maps with OpenLayers 3

Packt
22 Jan 2016
14 min read
In this article by Gábor Farkas, the author of the book Mastering OpenLayers 3, you will learn about OpenLayers 3 which is the most robust open source web mapping library out there, highly capable of handling the client side of a WebGIS environment. Whether you know how to use OpenLayers 3 or you are new to it, this article will help you to create a simple map and either refresh some concepts or get introduced to them. As this is a mastering book, we will mainly discuss the library's structure and capabilities in greater depth. In this article we will create a simple map with the library, and revise the basic terms related to it. In this article we will cover the following topics: Structure of OpenLayers 3 Architectural considerations Creating a simple map Using the API documentation effectively Debugging the code (For more resources related to this topic, see here.) Before getting started Take a look at the code provided with the book. You should see a js folder in which the required libraries are stored. For this article, ol.js, and ol.css in the ol3-3.11.0 folder will be sufficient. The code is also available on GitHub. You can download a copy from the following URL: https://github.com/GaborFarkas/mastering_openlayers3/releases. You can download the latest release of OpenLayers 3 from its GitHub repository at https://github.com/openlayers/ol3/releases. For now, grabbing the distribution version (v3.11.0-dist.zip) should be enough. Creating a working environment There is a security restriction in front end development, called CORS (Cross Origin Resource Sharing). By default, this restriction prevents the application from grabbing content from a different domain. On top of that, some browsers disallow reaching content from the hard drive when a web page is opened from the file system. To prevent this behavior, please make sure you possess one of the following: A running web server (highly recommended) Firefox web browser with security.fileuri.strict_origin_policy set to false (you can reach flags in Firefox by opening about:config from the address bar) Google Chrome web browser started with the --disable-web-security parameter (make sure you have closed every other instance of Chrome before) Safari web browser with Disable Local File Restrictions (in the Develop menu, which can be enabled in the Advanced tab of Preferences) You can easily create a web server if you have Python 2 with SimpleHTTPServer, or if you have Python 3 with http.server. For basic tutorials, you can consult the appropriate Python documentation pages. Structure of OpenLayers 3 OpenLayers 3 is a well structured, modular, and complex library, where flexibility, and consistency take a higher priority than performance. However, this does not mean OpenLayers 3 is slow. On the contrary, the library highly outperforms its predecessor; therefore its comfortable and logical design does not really adversely affect its performance. The relationship of some of the most essential parts of the library can be described with a radial UML (Universal Modeling Language) diagram, such as the following : Reading an UML scheme can seem difficult, and can be difficult if it is a proper one. However, this simplified scheme is quite easy to understand. With regard to the arrows, a single 1 represents a one-to-one relation, while the 0..n and 1 symbols denote a one-to-many relationship. You will probably never get into direct contact with the two superclasses at the top of the OpenLayers 3 hierarchy: ol.Observable, and ol.Object. However, most of the classes you actively use are children of these classes. You can always count with their methods, when you design a web mapping or WebGIS application. In the diagram we can see, that the parent of the most essential objects is the ol.Observable class. This superclass ensures all of its children have consistent listener methods. For example, every descendant of this superclass bears the on, once, and un functions, making registering event listeners to them as easy as possible. The next superclass, ol.Object, extends its parent with methods capable of easy property management. Every inner property managed by its methods (get, set, and unset) are observable. There are also convenience methods for bulk setting and getting properties, called getProperties, and setProperties. Most of the other frequently used classes are direct, or indirect, descendants of this superclass. Building the layout Now, that we covered some of the most essential structural aspects of the library, let's consider the architecture of an application deployed in a production environment. Take another look at the code. There is a chapters folder, in which you can access the examples within the appropriate subfolder. If you open ch01, you can see three file types in it. As you have noticed, the different parts of the web page (HTML, CSS, and JavaScript) are separated. There is one main reason behind this: the code remains as clean as possible. With a clean and rational design, you will always know where to look when you would like to make a modification. Moreover, if you're working for a company there is a good chance someone else will also work with your code. This kind of design will make sure your colleague can easily handle your code. On top of that, if you have to develop a wrapper API around OpenLayers 3, this is the only way your code can be integrated into future projects. Creating the appeal As the different parts of the application are separated, we will create a minimalistic HTML document. It will expand with time, as the application becomes more complicated and needs more container elements. For now, let's write a simple HTML document: <!DOCTYPE html> <html lang="en"> <head> <title>chapter 1 - Creating a simple map</title> <link href="../../js/ol3-3.11.0/ol.css" rel="stylesheet"> <link href="ch01.css" rel="stylesheet"> <script type="text/javascript" src="../../js/ol3- 3.11.0/ol.js"></script> <script type="text/javascript" src="ch01_simple_map.js"></script> </head> <body> <div id="map" class="map"></div> </body> </html> In this simple document, we defined the connection points between the external resources, and our web page. In the body, we created a simple div element with the required properties. We don't really need anything else; the magic will happen entirely in our code. Now we can go on with our CSS file and define one simple class, called map: .map { width: 100%; height: 100%; } Save this simple rule to a file named ch01.css, in the same folder you just saved the HTML file. If you are using a different file layout, don't forget to change the relative paths in the link, and script tags appropriately. Writing the code Now that we have a nice container for our map, let's concentrate on the code. In this book, most of the action will take place in the code; therefore this will be the most important part. First, we write the main function for our code. function init() { document.removeEventListener('DOMContentLoaded', init); } document.addEventListener('DOMContentLoaded', init); By using an event listener, we can make sure the code only runs when the structure of the web page has been initialized. This design enables us to use relative values for sizing, which is important for making adaptable applications. Also, we make sure the map variable is wrapped into a function (therefore we do not expose it) and seal a potential security breach. In the init function, we detach the event listener from the document, because it will not be needed once the DOM structure has been created. The DOMContentLoaded event waits for the DOM structure to build up. It does not wait for images, frames, and dynamically added content; therefore the application will load faster. Only IE 8, and prior versions, do not support this event type, but if you have to fall back you can always use the window object's load event. To check a feature's support in major browsers, you can consult the following site: http://www.caniuse.com/. Next, we extend the init function, by creating a vector layer and assigning it to a variable. Note that, in OpenLayers 3.5.0, creating vector layers has been simplified. Now, a vector layer has only a single source class, and the parser can be defined as a format in the source. var vectorLayer = new ol.layer.Vector({ source: new ol.source.Vector({ format: new ol.format.GeoJSON({ defaultDataProjection: 'EPSG:4326' }), url: '../../res/world_capitals.geojson', attributions: [ new ol.Attribution({ html: 'World Capitals © Natural Earth' }) ] }) }); We are using a GeoJSON data source with a WGS84 projection. As the map will use a Web Mercator projection, we provide a defaultDataProjection value to the parser, so the data will be transformed automatically into the view's projection. We also give attribution to the creators of the vector dataset. You can only give attribution with an array of ol.Attribution instances passed to the layer's source. Remember: giving attribution is not a matter of choice. Always give proper attribution to every piece of data used. This is the only way to avoid copyright infringement. Finally, construct the map object, with some extra controls and one extra interaction. var map = new ol.Map({ target: 'map', layers: [ new ol.layer.Tile({ source: new ol.source.OSM() }), vectorLayer ], controls: [ //Define the default controls new ol.control.Zoom(), new ol.control.Rotate(), new ol.control.Attribution(), //Define some new controls new ol.control.ZoomSlider(), new ol.control.MousePosition(), new ol.control.ScaleLine(), new ol.control.OverviewMap() ], interactions: ol.interaction.defaults().extend([ new ol.interaction.Select({ layers: [vectorLayer] }) ]), view: new ol.View({ center: [0, 0], zoom: 2 }) }); In this example, we provide two layers: a simple OpenStreetMap tile layer and the custom vector layer saved into a separate variable. For the controls, we define the default ones, then provide a zoom slider, a scale bar, a mouse position notifier, and an overview map. There are too many default interactions, therefore we extend the default set of interactions with ol.interaction.Select. This is the point where saving the vector layer into a variable becomes necessary. The view object is a simple view that defaults to projection EPSG:3857 (Web Mercator). OpenLayers 3 also has a default set of controls that can be accessed similarly to the interactions, under ol.control.defaults(). Default controls and interactions are instances of ol.Collection, therefore both of them can be extended and modified like any other collection object. Note that the extend method requires an array of features. Save the code to a file named ch01_simple_map.js in the same folder as your HTML file. If you open the HTML file, you should see the following map: You have different, or no results? Do not worry, not even a bit! Open up your browser's developer console (F12 in modern ones, or CTRL + J if F12 does not work), and resolve the error(s) noted there. If there is no result, double-check the HTML and CSS files; if you have a different result, check the code or the CORS requirements based on the error message. If you use Internet Explorer, make sure you have version 9, or better. Using the API documentation The API documentation for OpenLayers 3.11.0, the version we are using, can be found at http://www.openlayers.org/en/v3.11.0/apidoc/. The API docs, like the library itself, are versioned, thus you can browse the appropriate documentation for your OpenLayers 3 version by changing v3.11.0 in the URL to the version you are currently using. The development version of the API is also documented; you can always reach it at http://www.openlayers.org/en/master/apidoc/. Be careful when you use it, though. It contains all of the newly implemented methods, which probably won't work with the latest stable version. Check the API documentation by typing one of the preceding links in your browser. You should see the home page with the most frequently used classes. There is also a handy search box, with all of the classes listed on the left side. We have talked about default interactions, and their lengthy nature before. On the home page you can see a link to the default interactions. If you click on it, you will be directed to the following page: Now you can also see that nine interactions are added to the map by default. It would be quite verbose to add them one by one just to keep them when we define only one extra interaction, wouldn't it? You can see some features marked as experimental while you browse the API documentation with the Stable Only checkbox unchecked. Do not consider those features to be unreliable. They are stable, but experimental, and therefore they can be modified or removed in future versions. If the developer team considers a feature is useful and does not need further optimization or refactoring, it will be marked as stable. Understanding type definitions For every constructor and function in the API, the input and expected output types are well documented. To see a good example, let's search for a function with inputs and outputs as well. If you search for ol.proj.fromLonLat, you will see the following function: The function takes two arguments as input, one named coordinate and one named projection; projection is an optional one. coordinate is an ol.Coordinate type (an array with two numbers), while projection is an ol.proj.ProjectionLike type (a string representing the projection). The returned value, as we can see next to the white arrow, is also an ol.Coordinate type, with the transformed values. A good developer always keeps track of future changes in the library. This is especially important with OpenLayers 3, as it lacks backward-compatibility, when a major change occurs. You can see all of the major changes in the library in the OpenLayers 3 GitHub repository: https://github.com/openlayers/ol3/blob/master/changelog/upgrade-notes.md. Debugging the code As you will have noticed, there was a third file in the OpenLayers 3 folder discussed at the beginning of the article (js/ol3-3.11.0). This file, named ol-debug.js, is the uncompressed source file, in which the library is concatenated with all of its dependencies. We will use this file for two purpose in this book. Now, we will use it for debugging. First, open up ch01_simple_map.js. Next, extend the init function with an obvious mistake: var geometry = new ol.geom.Point([0, 0]); vectorLayer.getSource().addFeature(geometry); Don't worry if you can't spot the error immediately. That's what is debugging for. Save this extended JavaScript file with the name ch01_error.js. Next, replace the old script with the new one in the HTML file, like this: <script type="text/javascript" src="ch01_error.js"></script> If you open the updated HTML, and open your browser's developer console, you will see the following error message: Now that we have an error, let's check it in the source file by clicking on the error link on the right side of the error message: Quite meaningless, isn't it? The compiled library is created with Google's Closure Library, which obfuscates everything by default in order to compress the code. We have to tell it which precise part of the code should be exported. We will learn how to do that in the last article. For now, let's use the debug file. Change the ol.js in the HTML to ol-debug.js, load up the map, and check for the error again. Finally, we can see, in a well-documented form, the part that caused the error. This is a validating method, which makes sure the added feature is compatible with the library. It requires an ol.Feature as an input, which is how we caught our error. We passed a simple geometry to the function, instead of wrapping it in an ol.Feature first. Summary In this article, you were introduced to the basics of OpenLayers 3 with a more advanced approach. We also discussed some architectural considerations, and some of the structural specialties of the library. Hopefully, along with the general revision, we acquired some insight in using the API documentation and debugging practices. Congratulations! You are now on your way to mastering OpenLayers 3. Resources for Article: Further resources on this subject: What is OpenLayers? [article] OpenLayers' Key Components [article] OpenLayers: Overview of Vector Layer [article]
Read more
  • 0
  • 0
  • 5470
article-image-design-patterns-out-there-and-setting-your-environment
Packt
14 Jan 2016
27 min read
Save for later

The Design Patterns Out There and Setting Up Your Environment

Packt
14 Jan 2016
27 min read
In this article by Ivan Nikolov, author of the book Scala Design Patterns, explains in the world of computer programming, there are multiple different ways to create a solution that does something. However, some might contemplate whether there is a correct way of achieving a specific task. The answer is yes; there is always a right way, but in software development, there are usually multiple ways to do achieve a task. Some factors exist, which guide the programmer to the right solution, and depending on them, people tend to get the expected result. These factors could define many things—the actual language being used, algorithm, type of executable produced, output format, and the code structure. The language is already chosen for us—Scala. There are, however, a number of ways to use Scala, and we will be focusing on them—the design patterns. In this article, we will explain what design patterns are and why they exist. We will go through the different types of design patterns that are out there. This aims to provide useful examples to aid you in the learning process, and being able to run them easily is key. Hence, some points on how to set up a development environment properly will be given here. The top-level topics we will go through are as follows: What is a design pattern and why do they exist? The main types of design patterns and their features Choosing the right design pattern Setting up a development environment in real life The last point doesn't have to do much with design patterns. However, it is always a good idea to build projects properly, as this makes it much easier to work in future. (For more resources related to this topic, see here.) Design patterns Before delving into the Scala design patterns, we have to explain what they actually are, why they exist, and why it is worth being familiar with them. Software is a broad subject, and there are innumerable examples of things people can do with it. At first glance, most of the things are completely different—games, websites, mobile phone applications, and specialized systems for different industries. There are, however, many similarities in how software is built. Many times, people have to deal with similar issues no matter the type of software they create. For example, computer games as well as websites might need to access a database. And throughout time, by experience, developers learn how structuring their code differs for various tasks that they perform. A formal definition for design patterns would help you understand where we are actually trying to get using good practices in building software. The formal definition for design patterns A design pattern is a reusable solution to a recurring problem in a software design. It is not a finished piece of code, but a template, which helps solving the particular problem or family of problems. Design patterns are best practices to which the software community has arrived over a period of time. They are supposed to help write efficient, readable, testable, and easily extendable code. In some cases, they can be a result of a programming language not being expressive enough to elegantly achieve a goal. This means that more feature-rich languages might not even need a design pattern, while others still do. Scala is one of those rich languages, and in some cases, it makes use of a design pattern that is obsolete or simpler. The lack or existence of a certain functionality within a programming language also makes it able to implement additional design patterns that others cannot. The opposite is also valid—it might not be able to implement things that others can. Scala and design patterns Scala is a hybrid language, which combines features from object-oriented and functional languages. This not only allows it to keep some of the well-known object-oriented design patterns relevant, but it also provides various other ways of exploiting its features to write code, which is clean, efficient, testable, and extendable all at the same time. The hybrid nature of the language also makes some of the traditional object-oriented design patterns obsolete, or possible, using other cleaner techniques. The need for design patterns and their benefits Everybody needs design patterns and should look into some before writing code. As we mentioned earlier, they help writing efficient, readable, extendable, and testable code. All these features are really important to companies in the industry. Even though in some cases it is preferred to quickly write a prototype and get it out, usually the case is that a piece of software is supposed to evolve. Maybe you will have experience of extending some badly written code, but regardless, it is a challenging task and takes really long, and sometimes it feels that rewriting it would be easier. Moreover, this makes introducing bugs into the system much more likely. Code readability is also something that should be appreciated. Of course, one could use a design pattern and still have their code hard to read, but generally, design patterns help. Big systems are usually worked on by many people, and everyone should be able to understand what exactly is going on. Also, people who join a team are able to integrate much easily and quickly if they work on some well-written piece of software. Testability is something that prevents developers from introducing bugs when writing or extending code. In some cases, code could be created so badly that it is not even testable. Design patterns are supposed to eliminate these problems as well. While efficiency is many times connected to algorithms, design patterns could also affect it. A simple example could be an object, which takes a long time to instantiate and instances are used in many places in an application, but could be made singleton instead. Design pattern categories The fact that software development is an extremely broad topic leads to a number of things that can be done with programming. Everything is different and this leads to various requirements about the qualities of programs. All these facts have caused many different design patterns to be invented. This is further contributed to by the existence of various programming languages with different features and levels of expressiveness. This article focuses on the design patterns from the point of view of Scala. As we already mentioned previously, Scala is a hybrid language. This leads us to a few famous design patterns that are not needed anymore—one example is the null object design pattern, which can simply be replaced by Scala's Option. Other design patterns become possible using different approaches—the decorator design pattern can be implemented using stackable traits. Finally, some new design patterns become available, which are applicable specifically to the Scala programming language —the cake design pattern, pimp my library, and so on. We will focus on all of these and make it clear where the richness of Scala helps us to make our code even cleaner and simpler. Even if there are many different design patterns, they can all be grouped in a few main groups: Creational Structural Behavioral Functional Scala-specific design patterns Some of the design patterns that are specific to Scala can be assigned to the previous groups. They can either be additional or replacements of the already existing ones. They are typical to Scala and take advantage of some advanced language features or simply features not available in other languages. The first three groups contain the famous Gang of Four design patterns. Every design pattern article covers them and so will we. The rest, even if they can be assigned to one of the first three groups, will be specific to Scala and functional programming languages. In the next few subsections, we will explain the main characteristics of these groups and briefly present the actual design patterns that fall under them. Creational design patterns The creational design patterns deal with object creation mechanisms. Their purpose is to create objects in a way that is suitable to the current situation, which could lead to unnecessary complexity and the need of extra knowledge if they were not there. The main ideas behind the creational design patterns are as follows: Knowledge encapsulation about the concrete classes Hiding details about the actual creation and how objects are combined We will be focusing on the following creational design patterns in this article: The abstract factory pattern The factory method pattern The lazy initialization pattern The singleton pattern The object pool pattern The builder pattern The prototype pattern The following few sections give a brief definition of what these patterns are. The abstract factory design pattern This is used to encapsulate a group of individual factories that have a common theme. When used, the developer creates a specific implementation of the abstract factory and uses its methods in the same way as in the factory design pattern to create objects. It can be thought of as another layer of abstraction that helps instantiating classes. The factory method design pattern This design pattern deals with creation of objects without explicitly specifying the actual class that the instance will have—it could be something that is decided at runtime based on many factors. Some of these factors can include operating systems, different data types, or input parameters. It gives developers the peace of mind of just calling a method rather than invoking a concrete constructor. The lazy initialization design pattern This pattern is an approach to delay the creation of an object or the evaluation of a value until the first time it is needed. It is much more simplified in Scala than it is in an object-oriented language as Java. The singleton design pattern This design pattern restricts the creation of a specific class just to one object. If more than one class in the application tries to use such an instance, then this same instance is returned for everyone. This is another design pattern that with the use of basic Scala features can be easily achieved. The object pool design pattern This pattern uses a pool of objects that are already instantiated and ready for use. Whenever someone requires an object from the pool, it is returned, and after the user is finished with it, it puts it back into the pool manually or automatically. A common use for pools are database connections, which generally are expensive to create; hence, they are created once and then served to the application on request. The builder design pattern The builder design pattern is extremely useful for objects with many possible constructor parameters, which would otherwise require developers to create many overrides for the different scenarios an object could be created in. This is different to the factory design pattern, which aims to enable polymorphism. Many of the modern libraries today employ this design pattern. As we will see later, Scala can achieve this pattern really easily. The prototype design pattern This design pattern allows object creation using a clone() method from an already created instance. It can be used in cases when a specific resource is expensive to create or when the abstract factory pattern is not desired. Structural design patterns Structural design patterns exist in order to help establish the relationships between different entities in order to form larger structures. They define how each component should be structured so that it has very flexible interconnecting modules that can work together in a larger system. The main features of structural design patterns include the following: The use of composition to combine the implementations of multiple objects Help build a large system made of various components by maintaining a high level of flexibility In this article, we will focus on the following structural design patterns: Adapter ge Composite FacDecorator Bridade Flyweight Proxy The next subsections will put some light on what these patterns are about. The adapter design pattern The adapter design pattern allows the interface of an existing class to be used from another interface. Imagine that there is a client who expects your class to expose a doWork() method. You might have the implementation ready in another class but the method is called differently and is incompatible. It might require extra parameters too. This could also be a library that the developer doesn't have access to for modifications. This is where the adapter can help by wrapping the functionality and exposing the required methods. The adapter is useful for integrating the existing components. In Scala, the adapter design pattern can be easily achieved using implicit classes. The decorator design pattern Decorators are a flexible alternative to sub classing. They allow developers to extend the functionality of an object without affecting other instances of the same class. This is achieved by wrapping an object of the extended class into one that extends the same class and overrides the methods whose functionality is supposed to be changed. Decorators in Scala can be built much easily using another design pattern called stackable traits. The bridge design pattern The purpose of the bridge design pattern is to decouple an abstraction from its implementation so that the two can vary independently. It is useful when the class and its functionality vary a lot. The bridge reminds us of the adapter pattern, but the difference is that the adapter pattern is used when something is already there and you cannot change it, while the bridge design pattern is used when things are being built. It helps us to avoid ending up with multiple concrete classes that will be exposed to the client. You will get a clearer understanding when we delve deeper in the topic, but for now, let's imagine that we want to have a FileReader class that supports multiple different platforms. The bridge will help us end up with FileReader, which will use a different implementation, depending on the platform. In Scala, we can use self-types in order to implement a bridge design pattern. The composite design pattern The composite is a partitioning design pattern that represents a group of objects that are to be treated as only one object. It allows developers to treat individual objects and compositions uniformly and to build complex hierarchies without complicating the source code. An example of composite could be a tree structure where a node can contain other nodes, and so on. The facade design pattern The purpose of the facade design pattern is to hide the complexity of a system and its implementation details by providing a simpler interface to use to the client. This also helps to make the code more readable and to reduce the dependencies of the outside code. It works as a wrapper around the system that is being simplified, and of course, it can be used in conjunction with some of the other design patterns we mentioned previously. The flyweight design pattern The flyweight design pattern provides an object that is used to minimize memory usage by sharing it throughout the application. This object should contain as much data as possible. A common example given is a word processor, where each character's graphical representation is shared with the other same characters. The local information then is only the position of the character, which is stored internally. The proxy design pattern The proxy design pattern allows developers to provide an interface to other objects by wrapping them. They can also provide additional functionality, for example, security or thread-safety. Proxies can be used together with the flyweight pattern, where the references to shared objects are wrapped inside proxy objects. Behavioral design patterns Behavioral design patterns increase communication flexibility between objects based on the specific ways they interact with each other. Here, creational patterns mostly describe a moment in time during creation, structural patterns describe a more or less static structure, and behavioral patterns describe a process or flow. They simplify this flow and make it more understandable. The main features of behavioral design patterns are as follows: What is being described is a process or flow The flows are simplified and made understandable They accomplish tasks that would be difficult or impossible to achieve with objects In this article, we will focus our attention on the following behavioral design patterns: Value object Null object Strategy Command Chain of responsibility Interpreter Iterator Mediator Memento Observer State Template method Visitor The following subsections will give brief definitions of the aforementioned behavioral design patterns. The value object design pattern Value objects are immutable and their equality is based not on their identity, but on their fields being equal. They can be used as data transfer objects, and they can represent dates, colors, money amounts, numbers, and so on. Their immutability makes them really useful in multithreaded programming. The Scala programming language promotes immutability, and value objects are something that naturally occur there. The null object design pattern Null objects represent the absence of a value and they define a neutral behavior. This approach removes the need to check for null references and makes the code much more concise. Scala adds the concepts of optional values, which can replace this pattern completely. The strategy design pattern The strategy design pattern allows algorithms to be selected at runtime. It defines a family of interchangeable encapsulated algorithms and exposes a common interface to the client. Which algorithm is chosen could depend on various factors that are determined while the application runs. In Scala, we can simply pass a function as a parameter to a method, and depending on the function, a different action will be performed. The command design pattern This design pattern represents an object that is used to store information about an action that needs to be triggered at a later time. The information includes the following: The method name The owner of the method Parameter values The client then decides which commands to be executed and when by the invoker. This design pattern can easily be implemented in Scala using the by-name parameters feature of the language. The chain of responsibility design pattern The chain of responsibility is a design pattern where the sender of a request is decoupled from its receiver. This way, it makes it possible for multiple objects to handle the request and to keep logic nicely separate. The receivers form a chain where they pass the request, and if possible, they process it, and if not, they pass it to the next receiver. There are variations where a handler might dispatch the request to multiple other handlers at the same time. This somehow reminds of function composition, which in Scala can be achieved using the stackable traits design pattern. The interpreter design pattern The interpreter design pattern is based on the possibility to characterize a well-known domain with a language with strict grammar. It defines classes for each grammar rule in order to interpret sentences in the given language. These classes are likely to represent hierarchies as grammar is usually hierarchical as well. Interpreters can be used in different parsers, for example, SQL or other languages. The iterator design pattern The iterator design pattern is when an iterator is used to traverse a container and access its elements. It helps to decouple containers from the algorithms performed on them. What an iterator should provide is sequential access to the elements of an aggregate object without exposing the internal representation of the iterated collection. The mediator design pattern This pattern encapsulates the communication between different classes in an application. Instead of interacting directly with each other, objects communicate through the mediator, which reduces the dependencies between them, lowers the coupling, and makes the overall application easier to read and maintain. The memento design pattern This pattern provides the ability to rollback an object to its previous state. It is implemented with three objects: originator, caretaker, and memento. The originator is the object with the internal state; the caretaker will modify the originator, and a memento is an object that contains the state that the originator returns. The originator knows how to handle a memento in order to restore its previous state. The observer design pattern This pattern allows the creation of publish/subscribe systems. There is a special object called subject that automatically notifies all the observers when there are any changes in the state. This design pattern is popular in various GUI toolkits and generally where event handling is needed. It is also related to reactive programming, which is enabled by libraries such as Akka. The state design pattern This pattern is similar to the strategy design pattern, and it uses a state object to encapsulate different behavior for the same object. It improves the code's readability and maintainability by avoiding the use of large conditional statements. The template method design pattern This pattern defines the skeleton of an algorithm in a method and then passes some of the actual steps to the subclasses. It allows developers to alter some of the steps of an algorithm without having to modify its structure. An example of this could be a method in an abstract class that calls other abstract methods, which will be defined in the children. The visitor design pattern The visitor design pattern represents an operation to be performed on the elements of an object structure. It allows developers to define a new operation without changing the original classes. Scala can minimize the verbosity of this pattern compared to the pure object-oriented way of implementing it by passing functions to methods. Functional design patterns We will be looking into all of the preceding design patterns from the point of view of Scala. This means that they will look different than in other languages, but they've still been designed not specifically for functional programming. Functional programming is much more expressive than object-oriented programming. It has its own design patterns that help making the life of a programmer easier. We will focus on: Monoids Monads Functors After we've looked at some Scala functional programming concepts, and we've been through these, we will mention some interesting design patterns from the Scala world. A brief explanation of the preceding listed patterns will follow in the next few subsections. Monoids Monoid is a concept that comes from mathematics. For now, it will be enough to remember that a monoid is an algebraic structure with a single associative binary operation and an identity element. Here are the keywords you should remember: The associative binary operation. This means (a+b)+c = a+(b+c) The identity element. This means a+i = i+a = a. Here, the identity is i What is important about monoids is that they give us the possibility to work with many different types of values in a common way. They allow us to convert pairwise operations to work with sequences; the associativity gives us the possibility for parallelization, and the identity element allows us to know what to do with empty lists. Monoids are great to easily describe and implement aggregations. Monads In functional programming, monads are structures that represent computations as sequences of steps. Monads are useful for building pipelines, adding operations with side effects cleanly to a language where everything is immutable, and implementing compositions. This definition might sound vague and unclear, but explaining monads in a few sentences seems to be something hard to achieve. We will try to show why monads are useful and what they can help with as long as developers understand them well. Functors Functors come from a category theory, and as for monads, it takes time to explain them properly. For now, you could remember that functors are things that can allow us to lift a function of the type A => B to a function of the type F[A] => F[B]. Scala-specific design patterns The design patterns in this group could be assigned to some of the previous groups. However, they are specific to Scala and exploit some of the language features that we will focus on, and we've decided to place them in their own group. We will focus our attention on the following ones: The lens design pattern The cake design pattern Pimp my library Stackable traits The Type class design pattern Lazy evaluation Partial functions Implicit injection Duck typing Memoization The next subsections will give you some brief information about these patterns. The lens design pattern The Scala programming language promotes immutability. Having objects immutable makes it harder to make mistakes. However, sometimes mutability is required and the lens design pattern helps us achieve this nicely. The cake design pattern The cake design pattern is the Scala way to implement dependency injection. It is something that is used quite a lot in real-life applications, and there are numerous libraries that help developers achieve it. Scala has a way of doing this using language features, and this is what the cake design pattern is all about. Pimp my library Many times, engineers need to work with libraries, which are made to be as generic as possible. Sometimes, we need to do something more specific to our use case, though. The pimp my library design pattern provides a way to write extension methods for libraries, which we cannot modify. We can also use it for our own libraries as well. This design pattern also helps to achieve a better code readability. Stackable traits The stackable traits is the Scala way to implement the decorator design pattern. It can also be used to compose functions, and it's based on a few advanced Scala features. The type class design pattern This pattern allows us to write generic code by defining a behavior that must be supported by all members of a specific type class. For example, all numbers must support the addition and subtraction operations. Lazy evaluation Many times, engineers have to deal with operations that are slow and/or expensive. Sometimes, the result of these operations might not even be needed. Lazy evaluation is a technique that postpones the operation execution until it is actually needed. It could be used for application optimization. Partial functions Mathematics and functional programming are really close together. As a consequence, there are functions that exist that are only defined for a subset of all the possible input values they can get. A popular example is the square root function, which only works for non-negative numbers. In Scala, such functions can be used to efficiently perform multiple operations at the same time or to compose functions. Implicit injection Implicit injection is based on the implicit functionality of the Scala programming language. It automatically injects objects whenever they are needed, as long as they exist in a specific scope. It can be used for many things, including dependency injection. Duck typing This is a feature that is available in Scala and is similar to what some dynamic languages provide. It allows developers to write code, which requires the callers to have some specific methods (but not implement an interface). When someone uses a method with a duck type, it is actually checked during compile time whether the parameters are valid. Memoization This design pattern helps with optimization by remembering function results, based on the inputs. This means that as long as the function is stable and will return the same result when the same parameters are passed, one can remember its results and simply return them for every consecutive identical call. How to choose a design pattern As we already saw, there is a huge number of design patterns. In many cases, they are suitable to be used in combinations as well. Unfortunately, there is no definite answer about how to choose the concept of designing our code. There are many factors that could affect the final decision, and you should ask yourselves the following questions: Is this piece of code going to be fairly static or will it change in the future? Do we have to dynamically decide what algorithms to use? Is our code going to be used by others? Do we have an agreed interface? What libraries are we planning to use if any? Are there any special performance requirements or limitations? This is by no means an exhaustive list of questions. There are a lot of amount of factors that could dictate our decision in how we build our systems. It is, however, really important to have a clear specification, and if something seems missing, it should always be checked first. They should help you ask the right questions and take the right decision before going on and writing code. Setting up the development environment This will aim to give real code examples for you to run and experiment with. This means that it is important to be able to easily run any examples we have provided here and not to fight with the code. We will do our best to have the code tested and properly packaged, but you should also make sure that you have everything needed for the examples. Installing Scala Of course, you will need the Scala programming language. It evolves quickly, and the newest version could be found at http://www.scala-lang.org/download/. There are a few tips about how to install the language in your operating system at http://www.scala-lang.org/download/install.html. Tips about installing Scala You can always download multiple versions of Scala and experiment with them. I use Linux and my tips will be applicable to Mac OS users, too. Windows users can also do a similar setup. Here are the steps: Install Scala under /opt/scala-{version}/. Then, create a symlink using the following command: sudo ln -s /opt/scala-{version} scala-current. Finally, add the path to the Scala bin folder to my .bashrc (or equivalent) file using the following lines: export SCALA_HOME=/opt/scala-current and export PATH=$PATH:$SCALA_HOME/bin. This allows us to quickly change versions of Scala by just redefining the symlink. Another way to experiment with any Scala version is to install SBT (you can find more information on this). Then, simply run sbt in your console, type ++ 2.11.7 (or any version you want), and then issue the console command. Now you can test Scala features easily. Using SBT or Maven or any other build tool will automatically download Scala for you. If you don't need to experiment with the console, you can skip the preceding steps. Using the preceding tips, we can use the Scala interpreter by just typing scala in the terminal or follow the sbt installation process and experiment with different language features in the REPL. Scala IDEs There are multiple IDEs out there that support development in Scala. There is absolutely no preference about which one to use to work with the code. Some of the most popular ones are as follows: IntelliJ Eclipse NetBeans They contain plugins to work with Scala, and downloading and using them should be straightforward. Summary By now, we have a fair idea about what a design pattern means and how it can affect the way we write our code. We've iterated the most famous design patterns out there, and we have outlined the main differences between them. We saw that in many cases, we could use Scala's features in order to make a pattern obsolete, simpler, or different to implement compared to the classical case for pure object-oriented languages. Knowing what to look for when picking a design pattern is important, and you should already know what specific details to watch out for and how important specifications are. Resources for Article: Further resources on this subject: Content-based recommendation [article] Getting Started with Apache Spark DataFrames [article] RESTServices with Finagle and Finch [article]
Read more
  • 0
  • 0
  • 4955

article-image-unleashing-creativity-mit-app-inventor-2
Packt
14 Jan 2016
9 min read
Save for later

Unleashing Creativity with MIT App Inventor 2

Packt
14 Jan 2016
9 min read
This article by Felicia Kamriani and Dr. Krishnendu Roy, authors of the book App Inventor 2 Essentials, covers what is MIT App Inventor 2 and why you should learn to use it? There are apps for just about everything—entertainment, socializing, dining, traveling, philanthropy, shopping, education, navigation, and so on. And just about everyone with a smartphone or tablet is using them to make their lives easier or better. But you have decided to move from just using mobile apps to creating mobile apps. Congratulations! Thanks to MIT App Inventor 2, mobile app development is no longer exclusively the realm of experienced software programmers. It empowers anyone with an idea to create technology. This book offers people of all ages a step-by-step guide to creating mobile apps with App Inventor. While this visual programming language is an ideal tool for people who have little or no coding experience, don’t be fooled into thinking that the capabilities of MIT App Inventor 2 are basic! The simple drag-and-drop blocks format is actually a powerful software language capable of creating complex and sophisticated mobile apps. The purpose of this article is to provide an overview of MIT App Inventor 2, and of your new role as a mobile app developer. You are in for more skill building than you ever imagined! Of course, you will learn to code mobile apps, but there are countless other valuable skills weaved into the mobile app building process. Most significantly, you will learn to think differently, master the design-thinking process, become a problem solver, and be resourceful. This article also offers tips on brainstorming app ideas and design principles. Lastly, it reveals the potential of MIT App Inventor 2 and showcases an array of mobile apps so that you, a budding app designer, can begin thinking about the full spectrum of possibilities. (For more resources related to this topic, see here.) What is MIT App Inventor 2? MIT App Inventor 2 is a free, drag-and-drop, blocks-based visual programming language that enables people, regardless of coding experience, to create mobile apps for Android devices. In 2008, iPhones and Android phones had just hit the market and MIT professor Hal Abelson had the idea to create an easy-to-use programming language to make mobile apps that would harness the power of the emerging smartphone technology. Equipped with fast processors, large memory storage, and sensors, smartphones were enabling people to monitor and interact with their environment like never before. Abelson’s goal was to democratize the mobile app development process by making it easy for anyone to create mobile apps that were meaningful and important to them. While on sabbatical at Google in Mountain View, CA, Abelson worked with engineer Mark Friedman to create App Inventor (yes, it was originally called Google App Inventor!). In 2011, Abelson brought App Inventor to MIT and together with the Media Lab and CSAIL (the Computer Science and Artificial Intelligence Lab) created the Center for Mobile Learning. In December 2013, Abelson and his team of developers launched MIT App Inventor 2, (from here on referred to as MIT App Inventor) an even easier to use web-based application version with an Integrated Development Environment (IDE). This means that you can see your app come to life on your smartphone as you are building it. All you need is a computer (Mac or PC), an internet connection (or a USB connection), a Google Gmail account and an Android device (phone or tablet). But, if you don’t have an Android device, don’t worry! You can still create apps with the on-screen Emulator. The MIT App Inventor (http://appinventor.mit.edu/) browser includes a Designer Screen, a graphical user interface (GUI) where you create the look and feel of the app (choosing which components you want it to include) and the Blocks Editor, where you add behavior to the app by coding with colorful blocks. Users build apps by dragging components and blocks from menu bars onto a workspace and a connected Android device (or Emulator) displays progress in real time. All the apps are saved on the MIT server and once completed, they can be can be shared on the MIT App Inventor Gallery, submitted to app contests (such as MIT App of the Month) or uploaded to the Google Play store (or other app marketplaces) for sharing or selling. To date, MIT App Inventor has empowered millions of people to become creators of technology by learning to be mobile app developers. And now, you will become one of them! Understanding your role as a mobile app developer Since you are reading this book, it is safe to assume that not only do you regularly use mobile apps, but on occasion, you have also had the thought, “I wish there were an app for that!” Now, with the help of MIT App Inventor and this guidebook to mobile app development, you will soon be able to say, “I can create an app for that!” In embracing your new role as a mobile app developer, you will not just be learning how to code, but you will also learn an array of other valuable skills. You will learn to think differently. Every time you open an app, you will start looking at it from the developer’s perspective rather than just as a user. You will start noticing what functions are logical and smooth and which are choppy and unintuitive. You will learn to get inspiration from your environment. What type of app could make the attendance process at my club/class/meeting more streamlined or efficient? What app idea could help solve the problem of inaccurate inventory at the gym? You will learn to become a data gatherer without even realizing it. When people make comments about apps, your ears will perk up and you will take note. You will start asking questions like why do you prefer Waze to Google Maps? You will learn to think logically so that you can tell the computer in a step-by-step manner how to perform an operation. You will learn to become a problem solver. Any coder will confirm that programming is an iterative process. It’s a continual cycle of coding, troubleshooting and debugging. Trial and error will become second nature, as will taking a step back to figure out why something that just worked a minute ago now seems broken. And, you will learn to assume the role of a designer. It is no longer accurate to merely depict programmers holed up by themselves at a computer creating white text-based code on black screens. Coders of mobile apps are also designers who think about and create attractive and intuitive user interfaces (UIs). Much of the design work happens not at the computer—it includes conversations with potential users, involves pens paper, and post-it notes, and uses story-boards or sketches. Only once you have your app designed on paper do you sit down at the computer to begin coding. And then, you will not find the traditional black and white interface, as the MIT App Inventor platform is interactive and full of colorful blocks that snap together. Brainstorming app ideas Chances are you already have an idea for a mobile app. But if not, how can you think of one? Sometimes, you have so many ideas; it’s hard to narrow them down to just one. The best way to start brainstorming app ideas is by starting with what you know. What’s an app you wish existed? What’s an app you and your friends, co-workers, or family members would use, need, or like? What’s a problem in your community, network, or circle of friends that could be solved with a digital solution? Maybe, you loan out books to each friends, but don’t have a system to keep track of who borrowed what. Maybe, you want to do a clothing swap with people who are your size so you want to post pictures of the items that you have available for trade and you want to view listed items in your size. Maybe, you have a favorite app that you use all the time but wish it just had this one other feature. Maybe, when you meet your friends in a public place, it’s hard to know if they’re nearby without a lot of texting back and forth, so you want to create an app that shows everyone’s location on one screen. The possibilities are endless! The key to successful brainstorming is to write down all of your ideas no matter how wild they are and talk to people about them to get feedback. Input from others is an essential part of the research needed to ensure that your app idea becomes a successful app that people will want, use, and/or buy. On a recent business trip, we had an idea for a travel app because we always seemed to forget at least one essential item. Over breakfast at the hotel, we discussed the app idea with a couple of colleagues and received amazing insight that we hadn’t thought of like a reminder notification to fill any prescriptions well before the trip and a weather component so we could be sure to pack appropriate clothes for each destination. The more people you talk to, the more market research you will conduct and the more defined the overall app concept will be. Summary This article highlights the many other learning outcomes from engaging in the mobile app development process. Taking an app concept and building it out into an actual mobile app is both a concrete and a creative process. Attention to detail and iteration is vital for both code and design to work effectively and synergistically. Whether you’re creating a game to play with your friend, an app to promote philanthropy involvement on campus, or an app to kickstart a recycling program in your neighborhood, the design thinking process is as much a part of app development as coding. Skills such as brainstorming, research, interviewing, synthesizing, ideating, storyboarding, designing, troubleshooting, problem solving, and testing are not only integral to app building, they are also transferrable to other disciplines, helping to unlock creativity and flow in any endeavor. Resources for Article: Further resources on this subject: Google Apps: Surfing the Web [article] Introduction to IT Inventory and Resource Management [article] How to Expand your Knowledge [article]
Read more
  • 0
  • 0
  • 2220