How-To Tutorials - Programming

(For more resources related to this topic, see here.) In today's world, real-time information is continuously getting generated by applications (business, social, or any other type), and this information needs easy ways to be reliably and quickly routed to multiple types of receivers. Most of the time, applications that are producing information and applications that are consuming this information are well apart and inaccessible to each other. This, at times, leads to redevelopment of information of producers or consumers to provide an integration point between them. Therefore, a mechanism is required for seamless integration of information of producers and consumers to avoid any kind of rewriting of an application at either end. In the present era of big data, the first challenge is to collect the data and the second challenge is to analyze it. As it is a huge amount of data, the analysis typically includes the following and much more: User behavior data Application performance tracing Activity data in the form of logs Event messages Message publishing is a mechanism for connecting various applications with the help of messages that are routed between them, for example, by a message broker such as Kafka. Kafka is a solution to the real-time problems of any software solution, that is, to deal with real-time volumes of information and route it to multiple consumers quickly. Kafka provides seamless integration between information of producers and consumers without blocking the producers of the information, and without letting producers know who the final consumers are. Apache Kafka is an open source, distributed publish-subscribe messaging system, mainly designed with the following characteristics: Persistent messaging: To derive the real value from big data, any kind of information loss cannot be afforded. Apache Kafka is designed with O(1) disk structures that provide constant-time performance even with very large volumes of stored messages, which is in order of TB. High throughput: Keeping big data in mind, Kafka is designed to work on commodity hardware and to support millions of messages per second. Distributed: Apache Kafka explicitly supports messages partitioning over Kafka servers and distributing consumption over a cluster of consumer machines while maintaining per-partition ordering semantics. Multiple client support: Apache Kafka system supports easy integration of clients from different platforms such as Java, .NET, PHP, Ruby, and Python. Real time: Messages produced by the producer threads should be immediately visible to consumer threads; this feature is critical to event-based systems such as Complex Event Processing (CEP) systems. Kafka provides a real-time publish-subscribe solution, which overcomes the challenges of real-time data usage for consumption, for data volumes that may grow in order of magnitude, larger that the real data. Kafka also supports parallel data loading in the Hadoop systems. The following diagram shows a typical big data aggregation-and-analysis scenario supported by the Apache Kafka messaging system: At the production side, there are different kinds of producers, such as the following: Frontend web applications generating application logs Producer proxies generating web analytics logs Producer adapters generating transformation logs Producer services generating invocation trace logs At the consumption side, there are different kinds of consumers, such as the following: Offline consumers that are consuming messages and storing them in Hadoop or traditional data warehouse for offline analysis Near real-time consumers that are consuming messages and storing them in any NoSQL datastore such as HBase or Cassandra for near real-time analytics Real-time consumers that filter messages in the in-memory database and trigger alert events for related groups Need for Kafka A large amount of data is generated by companies having any form of web-based presence and activity. Data is one of the newer ingredients in these Internet-based systems. This data typically includes user-activity events corresponding to logins, page visits, clicks, social networking activities such as likes, sharing, and comments, and operational and system metrics. This data is typically handled by logging and traditional log aggregation solutions due to high throughput (millions of messages per second). These traditional solutions are the viable solutions for providing logging data to an offline analysis system such as Hadoop. However, the solutions are very limiting for building real-time processing systems. According to the new trends in Internet applications, activity data has become a part of production data and is used to run analytics at real time. These analytics can be: Search based on relevance Recommendations based on popularity, co-occurrence, or sentimental analysis Delivering advertisements to the masses Internet application security from spam or unauthorized data scraping Real-time usage of these multiple sets of data collected from production systems has become a challenge because of the volume of data collected and processed. Apache Kafka aims to unify offline and online processing by providing a mechanism for parallel load in Hadoop systems as well as the ability to partition real-time consumption over a cluster of machines. Kafka can be compared with Scribe or Flume as it is useful for processing activity stream data; but from the architecture perspective, it is closer to traditional messaging systems such as ActiveMQ or RabitMQ. Few Kafka usages Some of the companies that are using Apache Kafka in their respective use cases are as follows: LinkedIn (www.linkedin.com): Apache Kafka is used at LinkedIn for the streaming of activity data and operational metrics. This data powers various products such as LinkedIn news feed and LinkedIn Today in addition to offline analytics systems such as Hadoop. DataSift (www.datasift.com/): At DataSift, Kafka is used as a collector for monitoring events and as a tracker of users' consumption of data streams in real time. Twitter (www.twitter.com/): Twitter uses Kafka as a part of its Storm— a stream-processing infrastructure. Foursquare (www.foursquare.com/): Kafka powers online-to-online and online-to-offline messaging at Foursquare. It is used to integrate Foursquare monitoring and production systems with Foursquare, Hadoop-based offline infrastructures. Square (www.squareup.com/): Square uses Kafka as a bus to move all system events through Square's various datacenters. This includes metrics, logs, custom events, and so on. On the consumer side, it outputs into Splunk, Graphite, or Esper-like real-time alerting. The source of the above information is https: //cwiki. apache.org/confluence/display/KAFKA/Powered+By. Summary In this article, we have seen how companies are evolving the mechanism of collecting and processing application-generated data, and that of utilizing the real power of this data by running analytics over it. Resources for Article: Further resources on this subject: Apache Felix Gogo [Article] Hadoop and HDInsight in a Heartbeat [Article] Advanced Hadoop MapReduce Administration [Article]

Creating a torrent In order to share data, we first need to create and share a torrent. Go to File | Create New Torrent. We use this interface to create new torrents. Here we can select the source files we wish to share, the trackers to use, and configure other sharing behaviors. Click on the Add file / Add directory button. If you plan on sharing a file or sharing a directory, press the relevant button and find the data to share. If you wish to skip files in the directory you have chosen, add which files to skip in the Skip Files textbox. This textbox works by matching the filenames within a directory with a string, which can use wildcards (*) and pipes (|). Note If you want to match a specific file, enter the full filename (for example, filename.jpg). If you want to skip all .jpg files, enter *.jpg. If you want to skip all .jpg files and all .txt files, enter *.jpg|*.txt without spaces. Enter the trackers that this torrent will use. By default, uTorrent will enter trackers www.openbittorrent.com and www.publicbt.com, though you can add other trackers to this list if required. By adding more trackers to this list, the torrent will get more exposure to the potential participants. If you have the torrent contents accessible on a web server and want to list it as a backup location, enter the URL in the Web Seeds textbox. This option ensures the files inside the torrent are accessible even if no other participants are present to transfer data. Use the Comment textbox to enter any extra information that the users may require. Keep the Piece size as (auto detect) . This can be altered if desired, but it is better to leave it as it is, since the Piece size relates to how the files will be divided. If you manually choose 16 KB, the data will be divided into 16 KB chunks. Choosing the wrong piece size for the data size can make torrents inefficient, so it is better to let uTorrent decide. Enable the Start seeding checkbox. This option will make the torrent available to upload as soon as the torrent creation is complete. If you want this torrent to be private, enable the Private torrent checkbox. This will disable the DHT and PEX ( Peer Exchange ) protocols, making tracker the only method of interacting with this torrent.

(For more resources related to this topic, see here.) Drawing basics The screens of modern computers consist of a number of small squares, called pixels ( picture elements ). Each pixel can light in one color. You create pictures on the screen by changing the colors of the pixels. Graphics based on pixels is called raster graphics. Another kind of graphics is vector graphics, which is based on primitives such as lines and circles. Today, most computer screens are arrays of pixels and represent raster graphics. But images based on vector graphics (vector images) are still used in computer graphics. Vector images are drawn on raster screens using the rasterization procedure. The openFrameworks project can draw on the whole screen (when it is in fullscreen mode) or only in a window (when fullscreen mode is disabled). For simplicity, we will call the area where openFrameworks can draw, the screen . The current width and height of the screen in pixels may be obtained using the ofGetWidth() and ofGetHeight() functions. For pointing the pixels, openFrameworks uses the screen's coordinate system. This coordinate system has its origin on the top-left corner of the screen. The measurement unit is a pixel. So, each pixel on the screen with width w and height h pixels can be pointed by its coordinates (x, y), where x and y are integer values lying in the range 0 to w-1 and from 0 to h-1 respectively. In this article, we will deal with two-dimensional (2D) graphics, which is a number of methods and algorithms for drawing objects on the screen by specifying the two coordinates (x, y) in pixels. The other kind of graphics is three-dimensional (3D) graphics, which represents objects in 3D space using three coordinates (x, y, z) and performs rendering on the screen using some kind of projection of space (3D) to the screen (2D). The background color of the screen The drawing on the screen in openFrameworks should be performed in the testApp::draw() function. Before this function is called by openFrameworks, the entire screen is filled with a fixed color, which is set by the function ofSetBackground( r, g, b ). Here r, g, and b are integer values corresponding to red, green, and blue components of the background color in the range 0 to 255. Note that each of the ofSetBackground() function call fills the screen with the specified color immediately. You can make a gradient background using the ofBackgroundGradient() function. You can set the background color just once in the testApp::setup() function, but we often call ofSetBackground() in the beginning of the testApp::draw() function to not mix up the setup stage and the drawing stage. Pulsating background example You can think of ofSetBackground() as an opportunity to make the simplest drawings, as if the screen consists of one big pixel. Consider an example where the background color slowly changes from black to white and back using a sine wave. This is example 02-2D/01-PulsatingBackground. The project is based on the openFrameworks emptyExample example. Copy the folder with the example and rename it. Then fill the body of the testApp::draw() function with the following code: float time = ofGetElapsedTimef(); //Get time in seconds//Get periodic value in [-1,1],with wavelength equal to 1 second float value = sin( time * M_TWO_PI );//Map value from [-1,1] to [0,255] float v = ofMap( value, -1, 1, 0, 255 );ofBackground( v, v, v ); //Set background color This code gets the time lapsed from the start of the project using the ofGetElapsedTimef() function, and uses this value for computing value = sin( time * M_TWO_PI ). Here, M_TWO_PI is an openFrameworks constant equal to 2π; that is, approximately 6.283185. So, time * M_TWO_PI increases by 2π per second. The value 2π is equal to the period of the sine wave function, sin(). So, the argument of sin(...) will go through its wavelength in one second, hence value = sin(...) will run from -1 to 1 and back. Finally, we map the value to v, which changes in range from 0 to 255 using the ofMap() function, and set the background to a color with red, green, and blue components equal to v. Run the project; you will see how the screen color pulsates by smoothly changing its color from black to white and back. Replace the last line, which sets the background color to ofBackground( v, 0, 0 );, and the color will pulsate from black to red. Replace the argument of the sin(...) function to the formula time * M_TWO_PI * 2 and the speed of the pulsating increases by two times. We will return to background in the Drawing with an uncleared background section. Now we will consider how to draw geometric primitives. Geometric primitives In this article we will deal with 2D graphics. 2D graphics can be created in the following ways: Drawing geometric primitives such as lines, circles, and other curves and shapes like triangles and rectangles. This is the most natural way of creating graphics by programming. Generative art and creative coding projects are often based on this graphics method. We will consider this in the rest of the article. Drawing images lets you add more realism to the graphics. Setting the contents of the screen directly, pixel-by-pixel, is the most powerful way of generating graphics. But it is harder to use for simple things like drawing curves. So, such method is normally used together with both of the previous methods. A somewhat fast technique for drawing a screen pixel-by-pixel consists of filling an array with pixels colors, loading it in an image, and drawing the image on the screen. The fastest, but a little bit harder technique, is using fragment shaders. openFrameworks has the following functions for drawing primitives: ofLine( x1, y1, x2, y2 ): This function draws a line segment connecting points (x1, y1) and (x2, y2) ofRect( x, y, w, h ): This function draws a rectangle with the top-left corner (x, y), width w, and height h ofTriangle( x1, y1, x2, y2, x3, y3 ): This function draws a triangle with vertices (x1, y1), (x2, y2), and (x3, y3) ofCircle( x, y, r ): This function draws a circle with center (x, y) and radius r openFrameworks has no special function for changing the color of a separate pixel. To do so, you can draw the pixel (x, y) as a rectangle with width and height equal to 1 pixel; that is, ofRect( x, y, 1, 1 ). This is a very slow method, but we sometimes use it for educational and debugging purposes. All the coordinates in these functions are float type. Although the coordinates (x, y) of a particular pixel on the screen are integer values, openFrameworks uses float numbers for drawing geometric primitives. This is because a video card can draw objects with the float coordinates using modeling, as if the line goes between pixels. So the resultant picture of drawing with float coordinates is smoother than with integer coordinates. Using these functions, it is possible to create simple drawings. The simplest example of a flower Let's consider the example that draws a circle, line, and two triangles, which forms the simplest kind of flower. This is example 02-2D/02-FlowerSimplest. This example project is based on the openFrameworks emptyExample project. Fill the body of the testApp::draw() function with the following code: ofBackground( 255, 255, 255 ); //Set white background ofSetColor( 0, 0, 0 ); //Set black colorofCircle( 300, 100, 40 ); //Blossom ofLine( 300, 100, 300, 400 ); //Stem ofTriangle( 300, 270, 300, 300, 200, 220 ); //Left leaf ofTriangle( 300, 270, 300, 300, 400, 220 ); //Right leaf On running this code, you will see the following picture of the "flower": Controlling the drawing of primitives There are a number of functions for controlling the parameters for drawing primitives. ofSetColor( r, g, b ): This function sets the color of drawing primitives, where r, g, and b are integer values corresponding to red, green, and blue components of the color in the range 0 to 255. After calling ofSetColor(), all the primitives will be drawn using this color until another ofSetColor() calling. We will discuss colors in more detail in the Colors section. ofFill() and ofNoFill(): These functions enable and disable filling shapes like circles, rectangles, and triangles. After calling ofFill() or ofNoFill(), all the primitives will be drawn filled or unfilled until the next function is called. By default, the shapes are rendered filled with color. Add the line ofNoFill(); before ofCircle(...); in the previous example and you will see all the shapes unfilled, as follows: ofSetLineWidth( lineWidth ): This function sets the width of the rendered lines to the lineWidth value, which has type float. The default value is 1.0, and calling this function with larger values will result in thick lines. It only affects drawing unfilled shapes. The line thickness is changed up to some limit depending on the video card. Normally, this limit is not less than 8.0. Add the line ofSetLineWidth( 7 ); before the line drawing in the previous example, and you will see the flower with a thick vertical line, whereas all the filled shapes will remain unchanged. Note that we use the value 7; this is an odd number, so it gives symmetrical line thickening. Note that this method for obtaining thick lines is simple but not perfect, because adjacent lines are drawn quite crudely. For obtaining smooth thick lines, you should draw these as filled shapes. ofSetCircleResolution( res ): This function sets the circle resolution; that is, the number of line segments used for drawing circles to res. The default value is 20, but with such settings only small circles look good. For bigger circles, it is recommended to increase the circle resolution; for example, to 40 or 60. Add the line ofSetCircleResolution( 40 ); before ofCircle(...); in the previous example and you will see a smoother circle. Note that a large res value can decrease the performance of the project, so if you need to draw many small circles, consider using smaller res values. ofEnableSmoothing() and ofDisableSmoothing(): These functions enable and disable line smoothing. Such settings can be controlled by your video card. In our example, calling these functions will not have any effect. Performance considerations The functions discussed work well for drawings containing not more than a 1000 primitives. When you draw more primitives, the project's performance can decrease (it depends on your video card). The reason is that each command such as ofSetColor() or ofLine() is sent to drawing separately, which takes time. So, for drawing 10,000, 100,000, or even 1 million primitives, you should use advanced methods, which draw many primitives at once. In openFrameworks, you can use the ofMesh and ofVboMesh classes for this. Using ofPoint Maybe you noted a problem when considering the preceding flower example: drawing primitives by specifying the coordinates of all the vertices is a little cumbersome. There are too many numbers in the code, so it is hard to understand the relation between primitives. To solve this problem, we will learn about using the ofPoint class and then apply it for drawing primitives using control points. ofPoint is a class that represents the coordinates of a 2D point. It has two main fields: x and y, which are float type. Actually, ofPoint has the third field z, so ofPoint can be used for representing 3D points too. If you do not specify z, it sets to zero by default, so in this case you can think of ofPoint as a 2D point indeed. Operations with points To represent some point, just declare an object of the ofPoint class. ofPoint p; To initialize the point, set its coordinates. p.x = 100.0; p.y = 200.0; Or, alternatively, use the constructor. p = ofPoint( 100.0, 200.0 ); You can operate with points just as you do with numbers. If you have a point q, the following operations are valid: p + q or p - q provides points with coordinates (p.x + q.x, p.y + q.y) or (p.x - q.x, p.y - q.y) p * k or p / k, where k is the float value, provides the points (p.x * k, p.y * k) or (p.x / k, p.y / k) p += q or p -= q adds or subtracts q from p There are a number of useful functions for simplifying 2D vector mathematics, as follows: p.length(): This function returns the length of the vector p, which is equal to sqrt( p.x * p.x + p.y * p.y ). p.normalize(): This function normalizes the point so it has the unit length p = p / p.length(). Also, this function handles the case correctly when p.length() is equal to zero. See the full list of functions for ofPoint in the libs/openFrameworks/math/ofVec3f.h file. Actually, ofPoint is just another name for the ofVec3f class, representing 3D vectors and corresponding functions. All functions' drawing primitives have overloaded versions working with ofPoint: ofLine( p1, p2 ) draws a line segment connecting the points p1 and p2 ofRect( p, w, h ) draws a rectangle with top-left corner p, width w, and height h ofTriangle( p1, p2, p3 ) draws a triangle with the vertices p1, p2, and p3 ofCircle( p, r ) draws a circle with center p and radius r Using control points example We are ready to solve the problem stated in the beginning of the Using ofPoint section. To avoid using many numbers in drawing code, we can declare a number of points and use them as vertices for primitive drawing. In computer graphics, such points are called control points . Let's specify the following control points for the flower in our simplest flower example: Now we implement this in the code. This is example 02-2D/03-FlowerControlPoints. Add the following declaration of control points in the testApp class declaration in the testApp.h file: ofPoint stem0, stem1, stem2, stem3, leftLeaf, rightLeaf; Then set values for points in the testApp::update() function as follows: stem0 = ofPoint( 300, 100 ); stem1 = ofPoint( 300, 270 ); stem2 = ofPoint( 300, 300 ); stem3 = ofPoint( 300, 400 ); leftLeaf = ofPoint( 200, 220 ); rightLeaf = ofPoint( 400, 220 ); Finally, use these control points for drawing the flower in the testApp::draw() function: ofBackground( 255, 255, 255 ); //Set white background ofSetColor( 0, 0, 0 ); //Set black colorofCircle ( stem0, 40 ); //Blossom ofLine( stem0, stem3 ); //Stem ofTriangle( stem1, stem2, leftLeaf ); //Left leaf ofTriangle( stem1, stem2, rightLeaf ); //Right leaf You will observe that when drawing with control points the code is much easier to understand. Furthermore, there is one more advantage of using control points: we can easily change control points' positions and hence obtain animated drawings. See the full example code in 02-2D/03-FlowerControlPoints. In addition to the already explained code, it contains a code for shifting the leftLeaf and rightLeaf points depending on time. So, when you run the code, you will see the flower with moving leaves. Coordinate system transformations Sometimes we need to translate, rotate, and resize drawings. For example, arcade games are based on the characters moving across the screen. When we perform drawing using control points, the straightforward solution for translating, rotating, and resizing graphics is in applying desired transformations to control points using corresponding mathematical formulas. Such idea works, but sometimes leads to complicated formulas in the code (especially when we need to rotate graphics). The more elegant solution is in using coordinate system transformations. This is a method of temporarily changing the coordinate system during drawing, which lets you translate, rotate, and resize drawings without changing the drawing algorithm. The current coordinate system is represented in openFrameworks with a matrix. All coordinate system transformations are made by changing this matrix in some way. When openFrameworks draws something using the changed coordinate system, it performs exactly the same number of computations as with the original matrix. It means that you can apply as many coordinate system transformations as you want without any decrease in the performance of the drawing. Coordinate system transformations are managed in openFrameworks with the following functions: ofPushMatrix(): This function pushes the current coordinate system in a matrix stack. This stack is a special container that holds the coordinate system matrices. It gives you the ability to restore coordinate system transformations when you do not need them. ofPopMatrix(): This function pops the last added coordinate system from a matrix stack and uses it as the current coordinate system. You should take care to see that the number of ofPopMatrix() calls don't exceed the number of ofPushMatrix() calls. Though the coordinate system is restored before testApp::draw() is called, we recommend that the number of ofPushMatrix() and ofPopMatrix() callings in your project should be exactly the same. It will simplify the project's debugging and further development. ofTranslate( x, y ) or ofTranslate( p ): This function moves the current coordinate system at the vector (x, y) or, equivalently, at the vector p. If x and y are equal to zero, the coordinate system remains unchanged. ofScale( scaleX, scaleY ): This function scales the current coordinate system at scaleX in the x axis and at scaleY in the y axis. If both parameters are equal to 1.0, the coordinate system remains unchanged. The value -1.0 means inverting the coordinate axis in the opposite direction. ofRotate( angle ): This function rotates the current coordinate system around its origin at angle degrees clockwise. If the angle value is equal to 0, or k * 360 with k as an integer, the coordinate system remains unchanged. All transformations can be applied in any sequence; for example, translating, scaling, rotating, translating again, and so on. The typical usage of these functions is the following: Store the current transformation matrix using ofPushMatrix(). Change the coordinate system by calling any of these functions: ofTranslate(), ofScale(), or ofRotate(). Draw something. Restore the original transformation matrix using ofPopMatrix(). Step 3 can include steps 1 to 4 again. For example, for moving the origin of the coordinate system to the center of the screen, use the following code in testApp::draw(): ofPushMatrix(); ofTranslate( ofGetWidth() / 2, ofGetHeight() / 2 ); //Draw something ofPopMatrix(); If you replace the //Draw something comment to ofCircle( 0, 0, 100 );, you will see the circle in the center of the screen. This transformation significantly simplifies coding the drawings that should be located at the center of the screen. Now let's use coordinate system transformation for adding triangular petals to the flower. For further exploring coordinate system transformations.

If you were to create an API for Python, you should write it using Cython to create a more type-safe Python API. Or, you could take the C types from Cython to implement the same algorithms in your Python code, and they will be faster because you're specifying the types and you avoid a lot of the type conversion required. Consider you are implementing a fresh project in C. There are a few issues we always come across in starting fresh; for example, choosing the logging or configuration system we will use or implement. With Cython, we can reuse the Python logging system as well as the ConfigParser standard libraries from Python in our C code to get a head start. If this doesn't prove to be the correct solution, we can chop and change easily. We can even extend and get Python to handle all usage. Since the Python API is very powerful, we might as well make Python do as much as it can to get us off the ground. Another question is do we want Python be our "driver" (main entry function) or do we want to handle this from our C code? Cython cdef In the next two examples, I will demonstrate how we can reuse the Python logging and Python ConfigParser modules directly from C code. But there are a few formalities to get over first, namely the Python initialization API and the link load model for fully embedded Python applications for using the shared library method. It's very simple to embed Python within a C/C++ application; you will require the following boilerplate: #include <Python.h>int main (int argc, char ** argv){Py_SetProgramName (argv [0]);Py_Initialize ();/* Do all your stuff in side here...*/Py_Finalize ();return 0;} Make sure you always put the Python.h header at the very beginning of each C file, because Python contains a lot of headers defined for system headers to turn things on and off to make things behave correctly on your system. Later, I will introduce some important concepts about the GIL that you should know and the relevant Python API code you will need to use from time to time. But for now, these few calls will be enough for you to get off the ground. Linking models Linking models are extremely important when considering how we can extend or embed things in native applications. There are two main linking models for Cython: fully embedded Python and code, which looks like the following figure: This demonstrates a fully embedded Python application where the Python runtime is linked into the final binary. This means we already have the Python runtime, whereas before we had to run the Python interpreter to call into our Cython module. There is also a Python shared object module as shown in the following figure: We have now fully modularized Python. This would be a more Pythonic approach to Cython, and if your code base is mostly Python, this is the approach you should take if you simply want to have a native module to call into some native code, as this lends your code to be more dynamic and reusable. The public keyword Moving on from linking models, we should next look at the public keyword, which allows Cython to generate a C/C++ header file that we can include with the prototypes to call directly into Python code from C. The main caveat if you're going to call Python public declarations directly from C is if your link model is fully embedded and linked against libpython.so; you need to use the boilerplate code as shown in the previous section. And before calling anything with the function, you need to initialize the Python module example if you have a cythonfile.pyx file and compile it with public declarations such as the following: cdef public void cythonFunction ():print "inside cython function!!!" You will not only get a cythonfile.c file but also cythonfile.h; this declares a function called extern void initcythonfile (void). So, before calling anything to do with the Cython code, use the following: /* Boiler plate init Python */Py_SetProgramName (argv [0]);Py_Initialize ();/* Init our config module into Python memory */initpublicTest ();cythonFunction ();/* cleanup python before exit ... */Py_Finalize (); Calling initcythonfile can be considered as the following in Python: import cythonfile Just like the previous examples, this only affects you if you're generating a fully embedded Python binary. Logging into Python A good example of Cython's abilities in my opinion is reusing the Python logging module directly from C. So, for example, we want a few macros we can rely on, such as info (…) that can handle VA_ARGS and feels as if we are calling a simple printf method. I think that after this example, you should start to see how things might work when mixing C and Python now that the cdef and public keywords start to bring things to life: import loggingcdef public void initLogging (char * logfile):logging.basicConfig (filename = logfile,level = logging.DEBUG,format = '%(levelname)s %(asctime)s:%(message)s',datefmt = '%m/%d/%Y %I:%M:%S')cdef public void pyinfo (char * message):logging.info (message)cdef public void pydebug (char * message):logging.debug (message)cdef public void pyerror (char * message):logging.error (message) This could serve as a simple wrapper for calling directly into the Python logger, but we can make this even more awesome in our C code with C99 __VA_ARGS__ and an attribute that is similar to GCC printf. This will make it look and work just like any function that is similar to printf. We can define some headers to wrap our calls to this in C as follows: #ifndef __MAIN_H__#define __MAIN_H__#include <Python.h>#include <stdio.h>#include <stdarg.h>#define printflike __attribute__ ((format (printf, 3, 4)))extern void printflike cinfo (const char *, unsigned, const char *,...);extern void printflike cdebug (const char *, unsigned, const char *,...);extern void printflike cerror (const char *, unsigned, const char *,...);#define info(...) cinfo (__FILE__, __LINE__, __VA_ARGS__)#define error(...) cerror (__FILE__, __LINE__, __VA_ARGS__)#define debug(...) cdebug (__FILE__, __LINE__, __VA_ARGS__)#include "logger.h" // remember to import our cython public's#endif //__MAIN_H__ Now we have these macros calling cinfo and the rest, and we can see the file and line number where we call these logging functions: void cdebug (const char * file, unsigned line,const char * fmt, ...){char buffer [256];va_list args;va_start (args, fmt);vsprintf (buffer, fmt, args);va_end (args);char buf [512];snprintf (buf, sizeof (buf), "%s-%i -> %s",file, line, buffer);pydebug (buf);} On calling debug ("debug message"), we see the following output: Philips-MacBook:cpy-logging redbrain$ ./example log Philips-MacBook:cpy-logging redbrain$ cat log INFO 05/06/2013 12:28:24: main.c-62 -> info message DEBUG 05/06/2013 12:28:24: main.c-63 -> debug messageERROR 05/06/2013 12:28:24: main.c-64 -> error message Also, you should note that we import and do everything we would do in Python as we would in here, so don't be afraid to make lists or classes and use these to help out. Remember if you had a Cython module with public declarations calling into the logging module, this integrates your applications as if it were one. More importantly, you only need all of this boilerplate when you fully embed Python, not when you compile your module to a shared library. Python ConfigParser Another useful case is to make Python's ConfigParser accessible in some way from C; ideally, all we really want is to have a function to which we pass the path to a config file to receive a STATUS OK/FAIL message and a filled buffer of the configuration that we need: from ConfigParser import SafeConfigParser, NoSectionErrorcdef extern from "main.h":struct config:char * pathint numbercdef config myconfig Here, we've Cythoned our struct and declared an instance on the stack for easier management: cdef public config * parseConfig (char * cfg):# initialize the global stack variable for our config...myconfig.path = NULLmyconfig.number = 0# buffers for assigning python types into C typescdef char * path = NULLcdef number = 0parser = SafeConfigParser ()try:parser.readfp (open (cfg))pynumber = int (parser.get ("example", "number"))pypath = parser.get ("example", "path")except NoSectionError:print "No section named example"return NULLexcept IOError:print "no such file ", cfgreturn NULLfinally:myconfig.number = pynumbermyconfig.path = pypathreturn &myconfig This is a fairly trivial piece of Cython code that will return NULL on error as well as the pointer to the struct containing the configuration: Philips-MacBook:cpy-configparser redbrain$ ./example sample.cfgcfg->path = some/path/to/somethingcfg-number = 15 As you can see, we easily parsed a config file without using any C code. I always found figuring out how I was going to parse config files in C to be a nightmare. I usually ended up writing my own mini domain-specific language using Flex and Bison as a parser as well as my own middle-end, which is just too involved.

(For more resources related to this topic, see here.) In the early days of Java, when projects were small and didn't have many external dependencies, developers tended to manage dependencies manually by copying the required JAR files in the lib folder and checking it in their SCM/VCS with their code. This is still followed by a lot of developers, even today. But due to the aforementioned issues, this is not an option for larger projects. In many enterprises, there are central servers, FTP, shared drives, and so on, which store the approved libraries for use and also internally released libraries. But managing and tracking them manually is never easy. They end up relying on scripts and build files. Maven came and standardized this process. Maven defines standards for the project format to define its dependencies, formats for repositories to store libraries, the automated process to fetch transitive dependencies, and much more. Most of the systems today either back onto Maven's dependency management system or on Ivy's, which can function in the same way, and also provides its own standards, which is heavily inspired by Maven. SBT uses Ivy in the backend for dependency management, but uses a custom DSL to specify the dependency. Quick introduction to Maven or Ivy dependency management Apache Maven is not a dependency management tool. It is a project management and a comprehension tool. Maven is configured using a Project Object Model (POM), which is represented in an XML file. A POM has all the details related to the project right from the basic ones, such as groupId, artifactId, version, and so on, to environment settings such as prerequisites, and repositories. Apache Ivy is a dependency management tool and a subproject of Apache Ant. Ivy integrates publicly available artifact repositories automatically. The project dependencies are declared using XML in a file called ivy.xml. This is commonly known as the Ivy file. Ivy is configured using a settings file. The settings file (ivysettings.xml) defines a set of dependency resolvers. Each resolver points to an Ivy file and/or artifacts. So, the configuration essentially indicates which resource should be used to resolve a module. How Ivy works The following diagram depicts the usual cycle of Ivy modules between different locations: The tags along the arrows are the Ivy commands that need to be run for that task, which are explained in detail in the following sections. Resolve Resolve is the phase where Ivy resolves the dependencies of a module by accessing the Ivy file defined for that module. For each dependency in the Ivy file, Ivy finds the module using the configuration. A module could be an Ivy file or artifact. Once a module is found, its Ivy file is downloaded to the Ivy cache. Then, Ivy checks for the dependencies of that module. If the module has dependencies on other modules, Ivy recursively traverses the graph of dependencies, handling conflicts simultaneously. After traversing the whole graph, Ivy downloads all the dependencies that are not already in the cache and have not been evicted by conflict management. Ivy uses a filesystem-based cache to avoid loading dependencies already available in the cache. In the end, an XML report of the dependencies of the module is generated in the cache. Retrieve Retrieve is the act of copying artifacts from the cache to another directory structure. The destination for the files to be copied is specified using a pattern. Before copying, Ivy checks if the files are not already copied to maximize performance. After dependencies have been copied, the build becomes independent of Ivy. Publish Ivy can then be used to publish the module to a repository. This can be done by manually running a task or from a continuous integration server. Dependency management in SBT In SBT, library dependencies can be managed in the following two ways: By specifying the libraries in the build definition By manually adding the JAR files of the library Manual addition of JAR files may seem simple in the beginning of a project. But as the project grows, it may depend on a lot of other projects, or the projects it depends on may have newer versions. These situations make handling dependencies manually a cumbersome task. Hence, most developers prefer to automate dependency management. Automatic dependency management SBT uses Apache Ivy to handle automatic dependency management. When dependencies are configured in this manner, SBT handles the retrieval and update of the dependencies. An update does not happen every time there is a change, since that slows down all the processes. To update the dependencies, you need to execute the update task. Other tasks depend on the output generated through the update. Whenever dependencies are modified, an update should be run for these changes to get reflected. There are three ways in which project dependencies can be specified. They are as follows: Declarations within the build definition Maven dependency files, that is, POM files Configuration and settings files used for Ivy Adding JAR files manually Declaring dependencies in the build definition The Setting key libraryDependencies is used to configure the dependencies of a project. The following are some of the possible syntaxes for libraryDependencies: libraryDependencies += groupID % artifactID % revision libraryDependencies += groupID %% artifactID % revision libraryDependencies += groupID % artifactID % revision % configuration libraryDependencies ++= Seq( groupID %% artifactID % revision, groupID %% otherID % otherRevision ) Let's explain some of these examples in more detail: groupID: This is the organization/group's ID by whom it was published artifactID: This is the project's name on which there is a dependency revision: This is the Ivy revision of the project on which there is a dependency configuration: This is the Ivy configuration for which we want to specify the dependency Notice that the first and second syntax are not the same. The second one has a %% symbol after groupID. This tells SBT to append the project's Scala version to artifactID. So, in a project with Scala Version 2.9.1, libraryDependencies ++= Seq("mysql" %% "mysql-connector-java" % "5.1.18") is equivalent to libraryDependencies ++= Seq("mysql" % "mysql-connector-java_2.9.1" % "5.1.18"). The %% symbol is very helpful for cross-building a project. Cross-building is the process of building a project for multiple Scala versions. SBT uses the crossScalaVersion key's value to configure dependencies for multiple versions of Scala. Cross-building is possible only for Scala Version 2.8.0 or higher. The %% symbol simply appends the current Scala version, so it should not be used when you know that there is no dependency for a given Scala version, although it is compatible with an older version. In such cases, you have to hardcode the version using the first syntax. Using the third syntax, we could add a dependency only for a specific configuration. This is very useful as some dependencies are not required by all configurations. For example, the dependency on a testing library is only for the test configuration. We could declare this as follows: libraryDependencies ++= Seq("org.specs2" % "specs2_2.9.1" % "1.12.3" % "test") We could also specify dependency for the provided scope (where the JDK or container provides the dependency at runtime).This scope is only available on compilation and test classpath, and is not transitive. Generally, servlet-api dependencies are declared in this scope: libraryDependencies += "javax.servlet" % "javax.servlet-api" % "3.0.1" % "provided" The revision does not have to be a single-fixed version, that is, it can be set with some constraints, and Ivy will select the one that matches best. For example, it could be latest integration or 12.0 or higher, or even a range of versions. A URL for the dependency JAR If the dependency is not published to a repository, you can also specify a direct URL to the JAR file: libraryDependencies += groupID %% artifactID % revision from directURL directURL is used only if the dependency cannot be found in the specified repositories and is not included in published metadata. For example: libraryDependencies += "slinky" % "slinky" % "2.1" from "http://slinky2.googlecode.com/svn/artifacts/2.1/slinky.jar" Extra attributes SBT also supports Ivy's extra attributes. To specify extra attributes, one could use the extra method. Consider that the project has a dependency on the following Ivy module: <ivy-module version ="2.0" > <info organization="packt" module = "introduction" e:media = "screen" status = "integration" e:codeWord = "PP1872"</ivy-module> A dependency on this can be declared by using the following: libraryDependencies += "packt" % "introduction" % "latest.integration" extra( "media"->"screen", "codeWord"-> "PP1872") The extra method can also be used to specify extra attributes for the current project, so that when it is published to the repository its Ivy file will also have extra attributes. An example for this is as follows: projectID << projectID {id => id extra( "codeWord"-> "PP1952")} Classifiers Classifiers ensure that the dependency being loaded is compatible with the platform for which the project is written. For example, to fetch the dependency relevant to JDK 1.5, use the following: libraryDependencies += "org.testng" % "testng" % "5.7" classifier "jdk15" We could also have multiple classifiers, as follows: libraryDependencies += "org.lwjgl.lwjgl" % "lwjgl-platform" % lwjglVersion classifier "natives-windows" classifier "natives-linux" classifier "natives-osx" Transitivity In logic and mathematics, a relationship between three elements is said to be transitive. If the relationship holds between the first and second elements and between the second and third elements, it implies that it also holds a relationship between the first and third elements. Relating this to the dependencies of a project, imagine that you have a project that depends on the project Foo for some of its functionality. Now, Foo depends on another project, Bar, for some of its functionality. If a change in the project Bar affects your project's functionality, then this implies that your project indirectly depends on project Bar. This means that your project has a transitive dependency on the project Bar. But if in this case a change in the project Bar does not affect your project's functionality, then your project does not depend on the project Bar. This means that your project does not have a dependency on the project Bar. SBT cannot know whether your project has a transitive dependency or not, so to avoid dependency issues, it loads the library dependencies transitively by default. In situations where this is not required for your project, you can disable it using intransitive() or notTransitive(). A common case where artifact dependencies are not required is in projects using the Felix OSGI framework (only its main JAR is required). The dependency can be declared as follows: libraryDependencies += "org.apache.felix" % "org.apache.felix.framework" % "1.8.0" intransitive() Or, it can be declared as follows: libraryDependencies += "org.apache.felix" % "org.apache.felix.framework" % "1.8.0" notTransitive() If we need to exclude certain transitive dependencies of a dependency, we could use the excludeAll or exclude method. libraryDependencies += "log4j" % "log4j" % "1.2.15" exclude("javax.jms", "jms")libraryDependencies += "log4j" % "log4j" % "1.2.15" excludeAll( ExclusionRule(organization = "com.sun.jdmk"), ExclusionRule(organization = "com.sun.jmx"), ExclusionRule(organization = "javax.jms") ) Although excludeAll provides more flexibility, it should not be used in projects that will be published in the Maven style as it cannot be represented in a pom.xml file. The exclude method is more useful in projects that require pom.xml when being published, since it requires both organizationID and name to exclude a module. Download documentation Generally, an IDE plugin is used to download the source and API documentation JAR files. However, one can configure SBT to download the documentation without using an IDE plugin. To download the dependency's sources, add withSources() to the dependency definition. For example: libraryDependencies += "org.apache.felix" % "org.apache.felix.framework" % "1.8.0" withSources() To download API JAR files, add withJavaDoc() to the dependency definition. For example: libraryDependencies += "org.apache.felix" % "org.apache.felix.framework" % "1.8.0" withSources() withJavadoc() The documentation downloaded like this is not transitive. You must use the update-classifiers task to do so. Dependencies using Maven files SBT can be configured to use a Maven POM file to handle the project dependencies by using the externalPom method. The following statements can be used in the build definition: externalPom(): This will set pom.xml in the project's base directory as the source for project dependencies externalPom(baseDirectory{base=>base/"myProjectPom"}): This will set the custom-named POM file myProjectPom.xml in the project's base directory as the source for project dependencies There are a few restrictions with using a POM file, as follows: It can be used only for configuring dependencies. The repositories mentioned in POM will not be considered. They need to be specified explicitly in the build definition or in an Ivy settings file. There is no support for relativePath in the parent element of POM and its existence will result in an error. Dependencies using Ivy files or Ivy XML Both Ivy settings and dependencies can be used to configure project dependencies in SBT through the build definition. They can either be loaded from a file or can be given inline in the build definition. The Ivy XML can be declared as follows: ivyXML := <dependencies> <dependency org="org.specs2" name="specs2" rev="1.12.3"></dependency></ dependencies> The commands to load from a file are as follows: externalIvySettings(): This will set ivysettings.xml in the project's base directory as the source for dependency settings. externalIvySettings(baseDirectory{base=>base/"myIvySettings"}): This will set the custom-named settings file myIvySettings.xml in the project's base directory as the source for dependency settings. externalIvySettingsURL(url("settingsURL")): This will set the settings file at settingsURL as the source for dependency settings. externalIvyFile(): This will set ivy.xml in the project's base directory as the source for dependency. externalIvyFile(baseDirectory(_/"myIvy"): This will set the custom-named settings file myIvy.xml in the project's base directory as the source for project dependencies. When using Ivy settings and configuration files, the configurations need to be mapped, because Ivy files specify their own configurations. So, classpathConfiguration must be set for the three main configurations. For example: classpathConfiguration in Compile := Compile classpathConfiguration in Test := Test classpathConfiguration in Runtime := Runtime Adding JAR files manually To handle dependencies manually in SBT, you need to create a lib folder in the project and add the JAR files to it. That is the default location where SBT looks for unmanaged dependencies. If you have the JAR files located in some other folder, you could specify that in the build definition. The key used to specify the source for manually added JAR files is unmanagedBase. For example, if the JAR files your project depends on are in project/extras/dependencies instead of project/lib, modify the value of unmanagedBase as follows: unmanagedBase <<= baseDirectory {base => base/"extras/dependencies"} Here, baseDirectory is the project's root directory. unmanagedJars is a task which lists the JAR files from the unmanagedBase directory. To see the list of JAR files in the interactive shell type, type the following: > show unmanaged-jars [info] ArrayBuffer() Or in the project folder, type: $ sbt show unmanaged-jars If you add a Spring JAR (org.springframework.aop-3.0.1.jar) to the dependencies folder, then the result of the previous command would be: > show unmanaged-jars [info] ArrayBuffer(Attributed(/home/introduction/extras/dependencies/ org.springframework.aop-3.0.1.jar)) It is also possible to specify the path(s) of JAR files for different configurations using unmanagedJars. In the build definition, the unmanagedJars task may need to be replaced when the jars are in multiple directories and other complex cases. unmanagedJars in Compile += file("/home/downloads/ org.springframework.aop-3.0.1.jar") Resolvers Resolvers are alternate resources provided for the projects on which there is a dependency. If the specified project's JAR is not found in the default repository, these are tried. The default repository used by SBT is Maven2 and the local Ivy repository. The simplest ways of adding a repository are as follows: resolvers += name at location. For example: resolvers += "releases" at "http://oss.sonatype.org/content/ repositories/releases" resolvers ++= Seq (name1 at location1, name2 at location2). For example: resolvers ++= Seq("snapshots" at "http://oss.sonatype.org/ content/repositories/snapshots", "releases" at "http://oss.sonatype.org/content/repositories/releases") resolvers := Seq (name1 at location1, name2 at location2). For example: resolvers := Seq("sgodbillon" at "https://bitbucket.org/ sgodbillon/repository/raw/master/snapshots/", "Typesafe backup repo" at " http://repo.typesafe.com/typesafe/repo/", "Maven repo1" at "http://repo1.maven.org/") ) You can also add their own local Maven repository as a resource using the following syntax: resolvers += "Local Maven Repository" at "file://"+Path.userHome.absolutePath+"/.m2/repository" An Ivy repository of the file types URL, SSH, or SFTP can also be added as resources using sbt.Resolver. Note that sbt.Resolver is a class with factories for interfaces to Ivy repositories that require a hostname, port, and patterns. Let's see how to use the Resolver class. For filesystem repositories, the following line defines an atomic filesystem repository in the test directory of the current working directory: resolvers += Resolver.file ("my-test-repo", file("test")) transactional() For URL repositories, the following line defines a URL repository at http://example.org/repo-releases/: resolvers += Resolver.url(" my-test-repo", url("http://example.org/repo-releases/")) The following line defines an Ivy repository at http://joscha.github.com/play-easymail/repo/releases/: resolvers += Resolver.url("my-test-repo", url("http://joscha.github.com/play-easymail/repo/releases/")) (Resolver.ivyStylePatterns) For SFTP repositories, the following line defines a repository that is served by SFTP from the host example.org: resolvers += Resolver.sftp(" my-sftp-repo", "example.org") The following line defines a repository that is served by SFTP from the host example.org at port 22: resolvers += Resolver.sftp("my-sftp-repo", "example.org", 22) The following line defines a repository that is served by SFTP from the host example.org with maven2/repo-releases/ as the base path: resolvers += Resolver.sftp("my-sftp-repo", "example.org", "maven2/repo-releases/") For SSH repositories, the following line defines an SSH repository with user-password authentication: resolvers += Resolver.ssh("my-ssh-repo", "example.org") as("user", "password") The following line defines an SSH repository with an access request for the given user. The user will be prompted to enter the password to complete the download. resolvers += Resolver.ssh("my-ssh-repo", "example.org") as("user") The following line defines an SSH repository using key authentication: resolvers += { val keyFile: File = ... Resolver.ssh("my-ssh-repo", "example.org") as("user", keyFile, "keyFilePassword") } The next line defines an SSH repository using key authentication where no keyFile password is required to be prompted for before download: resolvers += Resolver.ssh("my-ssh-repo", "example.org") as("user", keyFile) The following line defines an SSH repository with the permissions. It is a mode specification such as chmod: resolvers += Resolver.ssh("my-ssh-repo", "example.org") withPermissions("0644") SFTP authentication can be handled in the same way as shown for SSH in the previous examples. Ivy patterns can also be given to the factory methods. Each factory method uses a Patterns instance which defines the patterns to be used. The default pattern passed to the factory methods gives the Maven-style layout. To use a different layout, provide a Patterns object describing it. The following are some examples that specify custom repository layouts using patterns: resolvers += Resolver.url("my-test-repo", url)( Patterns("[organisation]/[module]/ [revision]/[artifact].[ext]") ) You can specify multiple patterns or patterns for the metadata and artifacts separately. For filesystem and URL repositories, you can specify absolute patterns by omitting the base URL, passing an empty patterns instance, and using Ivy instances and artifacts. resolvers += Resolver.url("my-test-repo") artifacts "http://example.org/[organisation]/[module]/ [revision]/[artifact].[ext]" When you do not need the default repositories, you must override externalResolvers. It is the combination of resolvers and default repositories. To use the local Ivy repository without the Maven repository, define externalResolvers as follows: externalResolvers <<= resolvers map { rs => Resolver.withDefaultResolvers(rs, mavenCentral = false) } Summary In this article, we have seen how dependency management tools such as Maven and Ivy work and how SBT handles project dependencies. This article also talked about the different options that SBT provides to handle your project dependencies and configuring resolvers for the module on which your project has a dependency. Resources for Article : Further resources on this subject: So, what is Play? [Article] Play! Framework 2 – Dealing with Content [Article] Integrating Scala, Groovy, and Flex Development with Apache Maven [Article]

(For more resources related to this topic, see here.) The following screenshot (Images and colors) shows the final result of this article:   Images and colors The following is the corresponding drawing.kv code: 64. # File name: drawing.kv (Images and colors) 65. <DrawingSpace>: 66. canvas: 67. Ellipse: 68. pos: 10,10 69. size: 80,80 70. source: 'kivy.png' 71. Rectangle: 72. pos: 110,10 73. size: 80,80 74. source: 'kivy.png' 75. Color: 76. rgba: 0,0,1,.75 77. Line: 78. points: 10,10,390,10 79. width: 10 80. cap: 'square' 81. Color: 82. rgba: 0,1,0,1 83. Rectangle: 84. pos: 210,10 85. size: 80,80 86. source: 'kivy.png' 87. Rectangle: 88. pos: 310,10 89. size: 80,80 This code starts with an Ellipse (line 67) and a Rectangle (line 71). We use the source property, which inserts an image to decorate the polygon. The image kivy.png is 80 x 80 pixels with a white background (without any alpha/transparency channel). The result is shown in the first two columns of the previous screenshot (Images and colors). In line 75, we use the context instruction Color to change the color (with the rgba property: red, green, blue, and alpha) of the coordinate space context. This means that the next VertexInstructions will be drawn with the color changed by rgba. A ContextInstruction changes the current coordinate space context. In the previous screenshot, the blue bar at the bottom (line 77) has a transparent blue (line 76) instead of the default white (1,1,1,1) as seen in the previous examples. We set the ends shape of the line to a square with the cap property (line 80). We change the color again in line 81. After that, we draw two more rectangles, one with the kivy.png image and other without it. In the previous screenshot (Images and color) you can see that the white part of the image has become as green as the basic Rectangle on the left. Be very careful with this. The Color instruction acts as a light that is illuminating the kivy.png image. This is why you can still see the Kivy logo on the background instead of it being all covered by the color. There is another important detail to notice in the previous screenshot. There is a blue line that crosses the first two polygons in front and then crosses behind the last two. This illustrates the fact that the instructions are executed in order and this might bring some unwanted results. In this example we have full control of the order but for more complicated scenarios Kivy provides an alternative. We can specify three Canvas instances (canvas.before, canvas, and canvas.after) for each Widget. They are useful to organize the order of execution to guarantee that the background component remains in the background, or to bring some of the elements to the foreground. The following drawing.kv file shows an example of these three sets (lines 92, 98, and 104) of instructions: 90. # File name: drawing.kv (Before and After Canvas) 91. <DrawingSpace>: 92. canvas.before: 93. Color: 94. rgba: 1,0,0,1 95. Rectangle: 96. pos: 0,0 97. size: 100,100 98. canvas: 99. Color: 100. rgba: 0,1,0,1 101. Rectangle: 102. pos: 100,0 103. size: 100,100 104. canvas.after: 105. Color: 106. rgba: 0,0,1,1 107. Rectangle: 108. pos: 200,0 109. size: 100,100 110. Button: 111. text: 'A very very very long button' 112. pos_hint: {'center_x': .5, 'center_y': .5} 113. size_hint: .9,.1 In each set, a Rectangle of different color is drawn (lines 95, 101, and 107). The following diagram illustrates the execution order of the canvas. The number on the top-left margin of each code block indicates the order of execution: Execution order of the canvas Please note that we didn't define any canvas, canvas.before, or canvas.after for the Button, but Kivy does. The Button is a Widget and it displays graphics on the screen. For example, the gray background is just a Rectangle. That means that it has instructions in its internal Canvas instances. The following screenshot shows the result (executed with python drawing.py --size=300x100): Before and after canvas The graphics of the Button (the child) are covered up by the graphics of instructions in the canvas.after. But what is executed between canvas.before and canvas? It could be code of a base class when we are working with inheritance and we want to add instructions in the subclass that should be executed before the base class Canvas instances. A practical example of this will be covered when we apply them in the last section of this article in the comic creator project. The canvas.before will also be useful when we study how to dynamically add instruction to Canvas instances For now, it is sufficient to understand that there are three sets of instructions (Canvas instances) that provide some flexibility when we are displaying graphics on the screen. We will now explore some more context instructions related to three basic transformations. Summary In this article we learned how to add images and colors to shapes and how to position graphics at a front or back level. Resources for Article: Further resources on this subject: Easily Writing SQL Queries with Spring Python [Article] Python Testing: Installing the Robot Framework [Article] Advanced Output Formats in Python 2.6 Text Processing [Article]

(For more resources related to this topic, see here.) The spatial component of a real-world feature is the geometric representation of its shape in some coordinate space (either in 2D or 3D), and in vector space, this is referred to as its geometry. Oracle Spatial is designed to make spatial data management easier and more natural to users of location-enabled business applications and geographic information system (GIS) applications. Oracle allows the storage of spatial data in a table using the SDO_GEOMETRY data type that is just like any other data type in the database. Once the spatial data is stored in the Oracle database, it can be easily manipulated, retrieved, and related to all other data stored in the database. A spatial database should be designed just like any other database with a fully specified model. A fully specified model that is application independent should control the spatial data storage. A good data model supports and enhances application access without compromising the quality. In addition to these features, database features can be used to support applications that have limited functionality when it comes to table and column design. For example, some applications mandate a single spatial column per table or only a single homogeneous geometry type per spatial column. These limitations can be accommodated quite easily using database features such as views and triggers. In addition, there are a number of issues that arise when designing a data model that directly affects the data quality, performance, and access. The goal of this article is to give readers an understanding of how to model spatial data as SDO_GEOMETRY columns within tables, how to support spatial constraints for improved data quality, how to use synchronous and asynchronous triggers for implementing topological constraint checking, and to present methods for coping with multiple representations for faster web service access. All these issues, with solutions, are covered in this article: Defining a sample schema Using spatial metadata Using Oracle metadata views Using OGC metadata views Using different types of geometric representations Implementing tables with homogeneous and heterogeneous columns Implementing multiple representations for a single object Implementing multiple instances of a single column, for example, pre-thinned data for different scales and reprojection for faster web service access Restricting data access via views Using views to expose a single geometry type when multiple geometry types are present in the table Using views to expose tables with single geometry columns when multiple geometry columns are present in the table Implementing spatial constraints at the database level Restricting geometry types Spatial topological constraints Implementation of synchronous triggers Implementation of asynchronous triggers Defining a sample schema We will first define a sample schema that will be used for all the examples in this article. The schema is intended to model typical spatial assets maintained in a city-level GIS. Oracle Spatial provides all the functionality needed to model or describe the spatial properties of an asset (in modeling, it is often called an entity). This spatial description of an asset should not be treated differently from any other descriptive attribute. In addition, a data model should describe all assets/entities within it independently of any application. This should include, to the best of the ability of SQL, all business rules that define or control these assets/entities within the database, and these rules should be implemented using standard database practices. Defining the data model We use a schema with 12 tables to represent a spatial database for a city. This schema has tables to represent administrative areas managed at the city level, such as land parcels and neighborhoods, along with tables to manage natural features such as water boundaries. The LAND_PARCELS table has information about land at the lowest administrative level of the city. Buildings have to be fully contained in these land parcels. A table called BUILDING_FOOTPRINTS has information about all the buildings in the city. This table has the footprint of each building along with other information, such as name, height, and other attributes. Sets of neighborhoods are defined as a collection of land parcels to create more granular administrative areas. These neighborhoods are stored in the PLANNING_NEIGHBORHOODS table. There is a master table, BASE_ADDRESSES, to store information about all the valid street addresses in the city. Every record in the BUILDING_FOOTPRINTS table must have one parent record in this master address table. Note that the master address table does not list all the addresses of the apartments in a building. Rather, it stores one record for each street level address. So, each record in the BUILDING_FOOTPRINTS table has only one corresponding record in the master address table. There is also a master table, ROADS, that is used to store information about all the roads in the city. ROADS stores one record for each named road in the city so that all common information for the road can be stored together in one table. This is the only table in the schema without any geometry information. Each road in turn maps to a set of road segments that are stored in the ROAD_CLINES table. This table is used to store the geometric representation of center lines of road segments. This table also stores information about address ranges on these road segments. Road segments typically have different address ranges on the left side of the road and on the right side of the road. Each road segment also has a parent ROAD_ID associated with it from the ROADS table. A city usually manages sidewalks and other assets, such as street lamps, trashcans, and benches that are placed on these sidewalks. The SIDEWALKS table stores the information for all the sidewalks managed by the city. The CITY_FURNITURE table stores all the data corresponding to the assets, such as benches, streetlights, and trashcans. The ORTHO_PHOTOS table stores the collected information using aerial photography. The raster information stored in this table can be used to look for changes over time for the built-in features of the city. The water features of the city are stored in two different tables: the WATER_LINE table is used to store the water line features, such as creeks, rivers, and canals. The WATER_AREA table is used to store area features, such as lakes, rivers, and bays. The following figure shows the entity-relationship (E-R) diagram for this data model: The following figure shows the further E-R diagram for same data model: Creating tables in the schema Create a user called BOOK and assign it a password. Load the script <schema_load.sql> and it will create the tables required for running the examples described in this article. It will also create the Oracle Spatial metadata required for these tables. The following privileges are granted to the BOOK user: grant connect,resource to Book identified by <password>; grant connect, resource to book; grant create table to book; grant create view to book; grant create sequence to book; grant create synonym to book; grant create any directory to book; grant query rewrite to book; grant unlimited tablespace to book; Understanding spatial metadata Oracle Spatial requires certain metadata before the spatial data can be meaningfully used by applications. The database views that contain this metadata also act as a catalog for all the spatial data in the database. There are two basic views defined to store this metadata information: USER_SDO_GEOM_METADATA and ALL_SDO_GEOM_METADATA. The USER_ view is used to create a metadata entry for a single SDO_GEOMETRY column within a database table or view. An entry must be created for each SDO_GEOMETRY column within a table; entries for SDO_GEOMETRY columns in views are optional. If a table has more than one column of type SDO_GEOMETRY, then there is one metadata entry for each column of spatial data in that table. The ALL_ view shows all of the spatial layers that can be accessed by the current user. If a user has the Select grant on another user’s table with SDO_GEOMETRY columns, the first user can see the metadata entries for those tables in the ALL_ view. The views are set up so that owners of the spatial tables or views can create the metadata for them. And, the owner of a layer can grant read access to a layer to other users in the system. Granting a Select privilege on the table or view to other users will let them see the metadata for these tables and views. The ALL_ view displays all the spatial tables owned by the user along with other spatial tables for which the current user has read access. Spatial Reference System Each SDO_GEOMETRY object has a Spatial Reference System (SRS) associated with it, and all the SDO_GEOMETRY objects in a column should have the same SRS. In Oracle Spatial, a Spatial Reference ID (SRID) is used to associate an SRS with SDO_GEOMETRY objects. There are cases (for example, engineering drawings) where there is no SRS associated with an SDO_GEOMETRY object. In such cases, a NULL SRID is used to denote that the spatial data has no spatial reference information. An SRS can be geographic or non-geographic. A geographic SRS is used when the spatial data is used to represent features on the surface of the Earth. These types of SRS usually have a reference system that can relate the coordinates of the spatial data to locations on Earth. A unit of measurement is also associated with an SRS so that measurements can be done using a well-defined system. A non-geographic SRS is used when the spatial data is not directly related to locations on Earth. But these systems usually have a unit of measurement associated with them. Building floor plans is a good example of spatial data that is often not directly related to locations on Earth. A geographic system can be either geodetic or projected. Coordinates in a geodetic system are often described using longitude and latitude. In Oracle Spatial, the convention is to use longitude as the first coordinate and latitude as the second coordinate. A projected system is a Cartesian system that is defined as a planar projection based on the datum and projection parameters. Before an entry is created for a layer of data, the SRID associated with the data should be identified along with the tolerance to be used for the spatial layer. All the spatial data has an inherent accuracy associated with it. Hence, the tolerance value used for a spatial layer is very important and should be determined based on the accuracy of the data. Once these two values are identified, you are ready to create the metadata for the spatial layer. More on Spatial Reference Systems Oracle Spatial supports hundreds of SRSs, and it is very important to choose the right SRS for any given data set. The definition of an SRS can be easily obtained by looking at the well-known text (WKT) for that SRS. The WKTs for all the SRSs supplied as part of Oracle Spatial are available from the MDSYS.CS_SRS view. In addition to this view, there are several other metadata tables under MDSYS that contain more details on how these SRSs are defined. Oracle Spatial also supports the EPSG standard-based SRSs. SRS in Oracle Spatial is flexible and allows users to define new reference systems if they are not present in the supplied SRSs. Creating spatial metadata The tables used in our sample schema contain data that is geographically referenced. Spatial metadata can be created using Insert statements into the USER_SDO_GEOM_METADATA view. This view is defined as a public-writable view on top of MDSYS.SDO_GEOM_METADATA_TABLE that is used to store metadata for all the spatial columns in the database. Let us look at the metadata creation process for some of the spatial tables. Most of the tables used in the current schema have spatial data in the California State Plane Zone 3 Coordinate System. In Oracle Spatial, the corresponding SRID for this SRS is 2872. This coordinate system has foot as the unit of measurement and we will use 0.05 as our tolerance (that is, five-hundredths of a foot). The metadata is created using the Insert statement as shown in the following code: Insert Into USER_SDO_GEOM_METDATA Values (‘LAND_PARCELS’, ‘GEOM’, SDO_DIM_ARRAY(SDO_DIM_ARRAY(SDO_DIM_ELEMENT(‘X’, 5900000, 6100000, 0.05), SDO_DIM_ELEMENT(‘Y’,2000000, 2200000, 0.05)), 2872); The SDO_DIM_ELEMENT object is used to specify the lower and upper bounds for each dimension of the coordinate system along with the tolerance value. The metadata allows one entry for each dimension, even though it is very common to use the same tolerance value for the X and Y dimensions. When storing 3D data, it is very common to use a different tolerance value for the Z dimension. The BASE_ADDRESSES table has geometries stored in two columns: GEOMETRY and GEOD_GEOMETRY. The GEOMETRY column has data in the 2872 SRID, while the GEOD_GEOMETRY column has data in longitude and latitude. As this is a geodetic system, the tolerance for such systems is required to be in meters. So, a tolerance of 0.05 means a tolerance of 5cm. For geodetic data, it is recommended that the tolerance should not be less than 5cm for all of the topology and distance-based operations. Insert Into USER_SDO_GEOM_METDATA Values(‘BASE_ADDRESSES’, ‘GEOD_GEOM’, SDO_DIM_ARRAY((SDO_DIM_ELEMENT(‘Longitude’, -122.51436, -122.36638, .05), SDO_DIM_ELEMENT(‘Latitude’, 37.7081463, 37.8309382, .05)), 8307); As this is a geodetic system, the longitude range goes from -180 to 180 and the latitude range goes from -90 to 90. Even though it is normal practice to use these ranges for the metadata entry, many developers use the actual ranges spanned by the SDO_GEOMETRY object. Mapping tools and applications typically use this extent from the metadata to compute the initial extent of the data in each column of the spatial data. Any application looking for all the spatial columns in the database should select the data from the ALL_SDO_GEOM_METADATA view. This will return one row for each column of spatial data in the database that is visible to the current user. OGC-defined metadata views Open Geospatial Consortium (OGC) defines a different set of standardized metadata views. OGC standard metadata can be defined using a new set of tables or views in Oracle Spatial. For a simple solution for the OGC metadata schema, we will show a view-based implementation using the Oracle Spatial metadata table. All Oracle supplied packages, functions, and types of Oracle Spatial are in the MDSYS schema. It is generally not recommended to create any user objects under this schema as it might cause problems during database upgrades. Oracle also supplies another predefined schema called MDDATA that can be used for Oracle Spatial-related user objects that are general purpose in nature. We use this MDDATA schema to create the OGC metadata views. This user comes locked and it is recommended that you do not unlock this user. But, it does require a few privileges to make the following code work, so grant those privileges as required. Connect to the database as a user with SYSDBA privileges and execute all the following steps as the MDDATA user by changing the current schema to MDDATA. We need to grant an explicit Select privilege on SDO_GEOM_METADATA_TABLE to MDDATA. Alter session set current_schema=MDDATA; GRANT Select on MDSYS.SDO_GEOM_METADATA_TABLE to MDDATA; The OGC standard requires the geometry type as part of the metadata view. But, this is not part of the MDSYS owned metadata view and has to be computed based on the geometry table information stored in the MDSYS table. So, first define a function that can compute the geometry type based on the rows in the spatial tables. Note that this function just looks at the first non-NULL geometry and returns the type of that geometry. Users can modify this to make it look at the whole table to decide on the geometry type, but it can be a very expensive operation. Create Or Replace Function MDDATA.GET_GEOMETRY_TYPE (tsname varchar2, tname varchar2, cname varchar2) Return Number IS gtype number; Begin Begin execute immediate ‘ Select a.’|| SYS.DBMS_ASSERT.ENQUOTE_NAME(cname, false)|| ‘.sdo_gtype From ‘|| SYS.DBMS_ASSERT.ENQUOTE_NAME(tsname, false)||’.’|| SYS.DBMS_ASSERT.ENQUOTE_NAME(tname, false)|| ‘ a Where a.’|| SYS.DBMS_ASSERT.ENQUOTE_NAME(cname, false)|| ‘ is not null and rownum < 2’ Into gtype; Return gtype MOD 100; EXCEPTION When OTHERS Then Return 4; End; End; Notice all the uses of the ENQUOTE_NAME function from the SYS.DBMS_ASSERT package. This is used to avoid any possible SQL injection issues typically associated with functions that create SQL statements using the user supplied SQL. As we are creating a general purpose function that can be invoked by any user directly or indirectly, it is a good idea to protect the function from any possible SQL injection. Next, we define an OGC metadata view to see all the rows from the MDSYS owned metadata table. Create Or Replace View MDDATA.OGC_GEOMETRY_COLUMNS As Select GM.SDO_OWNER As F_TABLE_SCHEMA, GM.SDO_TABLE_NAME As F_TABLE_NAME, GM.SDO_COLUMN_NAME As F_GEOMETRY_COLUMN, Get_Geometry_Type(GM.sdo_owner, GM.sdo_table_name, GM.sdo_column_name) As GEOMETRY_TYPE, (Select count(*) From Table(GM.SDO_DIMINFO) ) As COORD_DIMENSION, GM.SDO_SRID As SRID From MDSYS.SDO_GEOM_METADATA_TABLE GM; And finally, we define a user view that will show all the geometry columns that are visible to the current user. Create Or Replace View GEOMETRY_COLUMNS As Select b.F_TABLE_SCHEMA , b.F_TABLE_NAME , b.F_GEOMETRY_COLUMN, b.COORD_DIMENSION, b.SRID, b.GEOMETRY_TYPE From MDDATA.OGC_GEOMETRY_COLUMNS b, ALL_OBJECTS a Where b.F_TABLE_NAME = a.OBJECT_NAME And b.F_TABLE_SCHEMA = a.OWNER And a.OBJECT_TYPE in (‘TABLE’, ‘SYNONYM’, ‘VIEW’); Grant Select On MDDATA.GEOMETRY_COLUMNS to public; Create PUBLIC SYNONYM GEOMETRY_COLUMNS FOR MDDATA.GEOMETRY_COLUMNS; Tolerance in Oracle Spatial Tolerance is used in Oracle Spatial to associate a level of precision with the data and to check the validity of geometries among other things. Tolerance should be derived based on the resolution and accuracy of the data. If the devices or methods used to collect the spatial data are correct up to a five-meter resolution, the tolerance for that layer should be set to 5 meters. The actual tolerance value, inserted into the metadata view depends on the real-world tolerance value and the unit of measurement used in the coordinate system is used for the column of spatial data. For example, let the tolerance for the spatial data be 5 centimeters and the unit of measurement of the coordinate system used for the spatial column is feet. Then, the five-centimeter value should first be converted to feet (1 centimeter is 0.032 feet)—this comes out to be 0.164 feet. So, you use a value of 0.164 for tolerance in the metadata. In practice, Oracle Spatial uses the following rules based on tolerance to determine if the geometry is valid or not. These are in addition to other topological consistency rules (as described by the OGC Simple Feature Specification) used to check the validity of geometries: If the distance between two consecutive vertices in the geometry is less than the tolerance value, the geometry is invalid. This rule applies to line-string and polygon type geometries. If the distance between a vertex and the nearest edge to that vertex in a polygon is less than the tolerance value, the geometry is invalid. This rule only applies to the polygon type geometries. Managing homogeneous and heterogeneous data If a spatial column in a table contains data of one single geometry type (for example, polygon or line-string, but not both), we can say that spatial data in that column is homogeneous. In other situations, a column may contain data from one or more geometry types (heterogeneous representation). For example, the spatial description of a rare and endangered flora object may normally be a single plant (a plant via a point), but in other situations, it may be an area (a patch via a polygon). Consider the CITY_FURNITURE table that is used for storing city assets like benches, trashcans, and streetlights. The geometry for the benches is represented using line-strings, while streetlights and trashcans are represented with points. It is perfectly correct, semantically, to store different types of observations within a single geometry column. However, while some mapping software systems can cope with multiple geometry types per SDO_GEOMETRY column, others, such as some traditional GIS packages, require homogeneity. We will describe how to achieve this next, when a column in the table has heterogeneous data. We define three views on top of the CITY_FURNITURE table corresponding to each of the types of data stored in the table. This table has three classes of objects: benches, trashcans, and streetlights. After the views are defined, we also need to create metadata entries for these views in USER_SDO_GEOM_METADATA so that any GIS tool can discover these views as if they are tables. Create the database views corresponding to each of the three types of data stored in the CITY_FURNITURE table. -- DDL for View CITY_FURN_BENCHES Create Or Replace FORCE VIEW CITY_FURN_BENCHES (FID, FEATTYPE, GEOM) As Select FID, FEATTYPE, GEOM From CITY_FURNITURE Where FEATTYPE=’BENCH’; -- DDL for View CITY_FURN_LIGHTS Create Or Replace FORCE VIEW CITY_FURN_LIGHTS (FID, FEATTYPE, GEOM) As Select FID, FEATTYPE, GEOM From CITY_FURNITURE Where FEATTYPE=’LIGHT’; -- DDL for View CITY_FURN_TRASHCANS Create Or Replace FORCE VIEW CITY_FURN_TRASHCANS (FID, FEATTYPE, GEOM) As Select FID, FEATTYPE, GEOM From CITY_FURNITURE Where FEATTYPE=’TRASHCAN’; The preceding examples show how to use other relational attributes to create the required views. Another way to do this is to constrain based on the SDO_GTYPE attribute of the SDO_GEOMETRY column. The following example shows how to do this for one of the preceding views, as the rest can be done with similar SQL: -- DDL for View CITY__FURN_BENCHES Create Or Replace FORCE VIEW CITY_FURN_BENCHES (FID, FEATTYPE, GEOM) AS Select FID, FEATTYPE, GEOM From CITY_FURNITURE A Where A.GEOM.SDO_GTYPE = 2002; Now create the metadata for each of these views so that any GIS can access this as if it is stored in a separate table. Note that these additional metadata entries are not required for the correct usage of Oracle Spatial. They are created only to facilitate the GIS tools that don’t support heterogeneous data in spatial columns. Insert Into USER_SDO_GEOM_METADATA Values ( ‘CITY_FURN_BENCHES’, ‘GEOM’, SDO_DIM_ARRAY(SDO_DIM_ELEMENT(‘X’, 5900000, 6100000, .05), SDO_DIM_ELEMENT(‘Y’, 2000000, 2200000, .05)), 2872); Insert Into USER_SDO_GEOM_METADATA Values( ‘CITY_FURN_LIGHTS’, ‘GEOM’, SDO_DIM_ARRAY(SDO_DIM_ELEMENT(‘X’, 5900000, 6100000, .05), SDO_DIM_ELEMENT(‘Y’, 2000000, 2200000, .05)), 2872); Insert Into USER_SDO_GEOM_METADATA Values( ‘CITY_FURN_TRASHCANS’, ‘GEOM’, SDO_DIM_ARRAY(SDO_DIM_ELEMENT(‘X’, 5900000, 6100000, .05), SDO_DIM_ELEMENT(‘Y’, 2000000, 2200000, .05)), 2872); How metadata is used Applications typically look at the ALL_SDO_GEOM_METADATA view to see the spatial tables available in the database for a given user. If you select the data from this view, now you will see 11 rows returned: 8 rows corresponding to tables and 3 rows corresponding to the views defined in the CITY_FURNITURE table. From an application point of view, it does not make any difference whether this data is stored in a view or a table. It will all look the same to the application. Sometimes it is useful to constrain the type of spatial data stored in the table to be homogeneous. For example, the ROAD_CLINES table should contain only linear geometries, as the roads are usually geometries of line type. This can be done by constraints that can be imposed by the spatial index defined in the ROAD_CLINES table. While creating the spatial index, provide the LAYER_GTYPE keyword and specify the type of data that will be stored in this table. -- DDL for Index ROAD_CLINES_SIDX Create Index ROAD_CLINES_SIDX ON ROAD_CLINES (GEOM) INDEXTYPE IS MDSYS.SPATIAL_INDEX PARAMETERS (‘LAYER_GTYPE=LINE’); Now, if you try to insert a row with a geometry that has a different SDO_GTYPE attribute than 2002, it will raise an error. Insert Into ROAD_CLINES Values ( 198999, 402, 0, 0, 2300, 2498, 190201, 23564000, 23555000, 94107, 10, ‘Y’, ‘DPW’, ‘Potrero Hill’, ‘190201’, ‘03RD ST’, ‘3’, ‘3RD’, SDO_GEOMETRY(2001, 2872, NULL, SDO_ELEM_INFO_ARRAY(1, 1, 1), SDO_ORDINATE_ARRAY( 6015763.86, 2104882.29))); * ERROR at line 1: ORA-29875: failed in the execution of the ODCIINDEXInsert routine ORA-13375: the layer is of type [2002] while geometry inserted has type [2001] ORA-06512: at “MDSYS.SDO_INDEX_METHOD_10I”, line 720 ORA-06512: at “MDSYS.SDO_INDEX_METHOD_10I”, line 225 The error message clearly indicates that the row that is currently being inserted has geometry with the wrong SDO_GTYPE attribute. This is the easiest way to strictly enforce the GTYPE constraints on the spatial data. However, this has the problem of rejecting the whole row when the geometry type does not match the LAYER_GTYPE keyword. And it is also not easy to log these cases as the error is thrown and the database moves on to process the next Insert statement. In some cases, the user might still want to insert the row into the table, but record cthe fact that there is invalid data in the row. Users can then come back and look at all the invalid entries and fix the issues. We will describe a few methods to do this logging and error processing later in this article. Using database check constraints Specifying the layer_gtype keyword is not the only way to constrain the type in a spatial layer. One can also use table level constraints to achieve the same result. With constraints, users get the additional benefit of specifying more complex constraints, such as allowing only points and lines in a layer. However, if only a single geometry type constraint is required, it is better to implement that constraint using the LAYER_GTYPE method as this is more efficient than the check constraint. These constraints can also be enforced with database triggers, and these trigger-based constraints are discussed in a later section. Alter Table CITY_FURNITURE ADD Constraint city_furniture_gtype_ck CHECK ( geom.sdo_gtype in (2002, 2001) ); Insert Into CITY_FURNITURE Values (432432, 'BENCH', SDO_GEOMETRY(2003,2872,NULL, SDO_ELEM_INFO_ARRAY(1, 1003, 3), SDO_ORDINATE_ARRAY(6010548.53, 2091896.34, 6010550.45,2091890.11))); This will fail with the following error: ERROR at line 1: ORA-02290: check Constraint (BOOK.CITY_FURNITURE_GTYPE_CK) violated Similarly, this approach can be useful for an OBJTYPE column check in the CITY_FURNITURE table. Alter Table CITY_FURNITURE ADD Constraint city_furniture_type_ck CHECK ( feattype in (‘BENCH’,’LIGHT’,’TRASHCAN’) ); Insert Into CITY_FURNITURE Values (432432, ‘LIGHTS’, SDO_GEOMETRY(2001,2872, SDO_POINT_TYPE(6010548.53, 2091896.34, NULL), NULL, NULL)); ERROR at line 1: ORA-02290: check Constraint (BOOK.CITY_FURNITURE_TYPE_CK) violated Now these two constraints are checking two independent columns, but what we really need is a more complex check to ensure each value of OBJTYPE has the corresponding SDO_GEOMETRY with the right type. That is, we want to make sure that TRASHCAN and LIGHT types have a point geometry and BENCH has a line geometry. Alter Table CITY_FURNITURE Drop Constraint city_furniture_gtype_ck; Alter Table CITY_FURNITURE Drop Constraint city_furniture_type_ck; Alter Table CITY_FURNITURE ADD Constraint city_furniture_objtype_geom_ck CHECK ( ( (“FEATTYPE”=’TRASHCAN’ Or “FEATTYPE”=’LIGHT’) AND “GEOM”.”SDO_GTYPE”=2001 ) Or (“FEATTYPE”=’BENCH’ AND “GEOM”.”SDO_GTYPE”=2002) /* Else Invalid combination */ ) ; Insert Into CITY_FURNITURE Values (432432, ‘BENCH’, SDO_GEOMETRY(2001,2872, SDO_POINT_TYPE(6010548.53, 2091896.34, NULL), NULL, NULL)); ERROR at line 1: ORA-02290: check Constraint (BOOK.CITY_FURNITURE_TYPE_CK) violated Multiple representations for the same objects In some situations, it is beneficial to have multiple representations for the same geometric feature. For example, an address usually has a point representation for its location. If a footprint of a building is associated with the address, then that footprint will be represented as a polygon. In some cases, a building might have many different point locations associated with it. One point may allow a GIS application to draw an icon for the building depending on its function (for example, a fire station). Another point may allow the building to be labeled with its street address, and finally another one may show an alternate location that is used for main delivery or emergency services entry at the back of the building. Similarly, a land parcel table can have an interior point of the parcel represented as point geometry in addition to the polygon representation. For a visualization application, it is sometimes useful to represent the land parcel as a point. When a map is displayed at a smaller scale (city level), the map will be cluttered if each land parcel is displayed as a polygon. In such cases, if land parcels are displayed as points with a suitable icon, the map will be less cluttered. When the map is displayed at a larger scale (street level), the same land parcel can be displayed as a polygon. Oracle Spatial allows such multiple representations by allowing multiple SDO_GEOMETRY columns in the same table. We first start with the BUILDING_FOOTPRINTS table and alter it to add an additional SDO_GEOMETRY column to allow the street address to be represented at the center of the building via a point feature. We can use a spatial function that can compute a point inside a polygon automatically to populate this column. Summary A spatial database application should be designed just like any other database application. It should follow the same standard data model practices like any other database application. In this article, we introduced a data model that can be used in a city-wide spatial data management system. This data model is used throughout the article to illustrate different aspects of spatial database application development. We discussed the use of triggers and queues to manage spatial data in the database. We also showed how to design database level constraints for spatial data management. We introduced the concepts of database triggers and queues and showed how they can be used for spatial data management in the database. Resources for Article: Further resources on this subject: Remote Job Agent in Oracle 11g Database with Oracle Scheduler [Article] Introduction to Oracle Service Bus & Oracle Service Registry [Article] Configuration, Release and Change Management with Oracle [Article]

(For more resources related to this topic, see here.) Sprites Let's briefly discuss sprites. In gaming, sprites are usually used for animation sequences; a sprite is a single image in which individual frames of a character animation are stored. We are going use sprites in our animations. If you already have knowledge of graphics design, it's good for you because it is an edge for you to define how you want your game to look like and how you want to define animation sequences in sprites. You can try out tools such as Sprite Maker for making your own sprites with ease; you can get a copy of Sprite Maker at http://www.spriteland.com/sprites/sprite-maker.zip. The following is an sample animation sprite by Marc Russell, which is available for free at http://opengameart.org/content/gfxlib-fuzed you can find other open source sprites at http://opengameart.org/content/platformersidescroller-tiles: The preceding sprite will play the animation of the character moving to the right. The character sequence is well organized using an invisible grid, as shown in the following screenshot: The grid is 32 x 32; the size of our grid is very important in setting up the quads for our game. A quad in LÖVE is a specific part of an image. Because our sprite is a single image file, quads will be used to specify each of the sequences we want to draw per unit time and will be the largest part part of our animation algorithm. Animation The animation algorithm will simply play the sprite like a tape of film; we'll be using a basic technique here as LÖVE doesn't have an official module for that. Some members of the LÖVE forum have come up with different libraries to ease the way we play animations. First of all let us load our file: function love.load() sprite = love.graphics.newImage "sprite.png" end Then we create quads for each part of the sprite by using love.graphics. newQuad(x, y, width, height, sw, sh), where x is the top-left position of the quad along the x axis, y is the top-left position of the quad along the y axis, width is the width of the quad, height is the height of the quad, sw is the sprite's width, and sh is the sprite's height: love.graphics.newQuad(0, 0, 32, 32, 256, 32) --- first quad love.graphics.newQuad(32, 0, 32, 32, 256, 32) --- second quad love.graphics.newQuad(64, 0, 32, 32, 256, 32) --- Third quad love.graphics.newQuad(96, 0, 32, 32, 256, 32) --- Fourth quad love.graphics.newQuad(128, 0, 32, 32, 256, 32) --- Fifth quad love.graphics.newQuad(160, 0, 32, 32, 256, 32) --- Sixth quad love.graphics.newQuad(192, 0, 32, 32, 256, 32) --- Seventh quad love.graphics.newQuad(224, 0, 32, 32, 256, 32) --- Eighth quad The preceding code can be rewritten in a more concise loop as shown in the following code snippet: for i=1,8 do love.graphics.newQuad((i-1)*32, 0, 32, 32, 256, 32) end As advised by LÖVE, we shouldn't state our quads in the draw() or update() functions, because it will cause the quad data to be repeatedly loaded into memory with every frame, which is a bad practice. So what we'll do is pretty simple; we'll load our quad parameters in a table, while love.graphics.newQuad will be referenced locally outside the functions. So the new code will look like the following for the animation in the right direction: local Quad = love.graphics.newQuad function love.load() sprite = love.graphics.newImage "sprite.png" quads = {} quads['right'] ={} quads['left'] = {} for j=1,8 do quads['right'][j] = Quad((j-1)*32, 0, 32, 32, 256, 32); quads['left'][j] = Quad((j-1)*32, 0, 32, 32, 256, 32); -- for the character to face the opposite direction, the quad need to be flipped by using the Quad:flip(x, y) method, where x and why are Boolean. quads.left[j]:flip(true, false) --flip horizontally x = true, y = false end end Now that our animation table is set, it is important that we set a Boolean value for the state of our character. At the start of the game our character is idle, so we set idle to true. Also, there are a number of quads the algorithm should read in order to play our animation. In our case, we have eight quads, so we need a maximum of eight iterations, as shown in the following code snippet: local Quad = love.graphics.newQuad function love.load() character= {} character.player = love.graphics.newImage("sprite.png") character.x = 50 character.y = 50 direction = "right" iteration = 1 max = 8 idle = true timer = 0.1 quads = {} quads['right'] ={} quads['left'] = {} for j=1,8 do quads['right'][j] = Quad((j-1)*32, 0, 32, 32, 256, 32); quads['left'][j] = Quad((j-1)*32, 0, 32, 32, 256, 32); -- for the character to face the opposite direction, the quad need to be flipped by using the Quad:flip(x, y) method, where x and why are Boolean. quads.left[j]:flip(true, false) --flip horizontally x = true, y = false end end Now let us update our motion; if a certain key is pressed, the animation should play; if the key is released, the animation should stop. Also, if the key is pressed, the character should change position. We'll be using the love.keypressed callback function here, as shown in the following code snippet: function love.update(dt) if idle == false then timer = timer + dt if timer > 0.2 then timer = 0.1 -- The animation will play as the iteration increases, so we just write iteration = iteration + 1, also we'll stop reset our iteration at the maximum of 8 with a timer update to keep the animation smooth. iteration = iteration + 1 if love.keyboard.isDown('right') then sprite.x = sprite.x + 5 end if love.keyboard.isDown('left') then sprite.x = sprite.x - 5 end if iteration > max then iteration = 1 end end end end function love.keypressed(key) if quads[key] then direction = key idle = false end end function love.keyreleased(key) if quads[key] and direction == key then idle = true iteration = 1 direction = "right" end end Finally, we can draw our character on the screen. Here we'll be using love.graphics.drawq(image, quad, x, y), where image is the image data, quad will load our quads table, x is the position in x axis and y is the position in the y axis: function love.draw() love.graphics.drawq(sprite.player, quads[direction][iteration], sprite.x, sprite.y) end So let's package our game and run it to see the magic in action by pressing the left or right navigation key: Summary That is all for this article. We have learned how to draw 2D objects on the screen and move the objects in four directions. We have delved into the usage of sprites for animations and how to play these animations with code. Resources for Article: Further resources on this subject: Panda3D Game Development: Scene Effects and Shaders [Article] Microsoft XNA 4.0 Game Development: Receiving Player Input [Article] Introduction to Game Development Using Unity 3D [Article]

(For more resources related to this topic, see here.) Using Capybara outside of Cucumber Capybara is by no means coupled to Cucumber, and can be used in any setting you wish, within Test::Unit, RSpec, or just from vanilla Ruby code. In fact, if you are using Cucumber but want to abstract some logic out of your step definitions and into Page Objects, then you will still need to consider how to use Capybara outside of Cucumber's world (https://github.com/cucumber/cucumber/wiki/A-Whole-New-World). Including the modules The first option you have for using Capybara outside of Cucumber is to include the DSL modules into your own modules or classes: require 'capybara/dsl' require 'rspec/expectations' Capybara.default_driver = :selenium module MyModule include Capybara::DSL include RSpec::Matchers def play_song visit 'http://localhost/html/html5.html' click_on 'Play' find('#log').should have_text 'playing' end end class Runner include MyModule def run play_song end end Runner.new.run It is important to require the Capybara DSL file, as this contains all the Capybara methods that need to be "mixed in". In this example, we have our own Ruby module and crucially within this, the relevant Capybara module Capybara::DSL is included. In addition, the RSpec::Matchers module has also been included, which allows us to utilize standard RSpec Matchers (obviously you do not have to use RSpec; you could choose a different way to assert behavior). If you follow this pattern in your own code, you can now mix in any of the standard Capybara methods into your own module or class methods. It is worth remembering that if you include modules in a base class, then all subclasses will inherit the ability to use those module methods. This would come in handy, for example, if you were using a Page Object pattern, where the only class that would have to include the DSL module would be the base page. Using the session directly The other option for mixing Capybara into your code is to use the session directly, that is to say, you instantiate a new instance of the session object and then call the DSL methods on it. The following example implements the same test as before, but this time by using a session instance and raising a simple exception if the expected content is not found: require 'capybara' session = Capybara::Session.new :selenium session.visit('http://localhost/html/html5.html') session.click_on 'Play' raise 'song not playing' unless session.find('#log') == 'playing' If you are using an object-oriented model for building your tests, you will need to pass the session instance around or find an appropriate strategy to deal with this, as you do not have the benefit of the modules mixing in the DSL methods globally. Capybara and popular test frameworks Capybara provides out of the box integration with a number of popular test frameworks; this is not a subject we will cover in depth, simply because they are covered very well in the Capybara README, which can be found at https://github.com/jniklas/capybara. Cucumber Throughout this article, examples have been set in the context of Cucumber, so you should be happy with how to implement Capybara's API in step definitions and do some simple setup in the env.rb file, such as setting the default driver. There are a couple of additional pieces of functionality that Capybara adds when using in conjunction with Cucumber, which are worth examining. The first is that Capybara hooks into Cucumber's Before do block as follows: Before do Capybara.reset_sessions! Capybara.use_default_driver end Apart from setting the default driver, this crucially makes a call to reset_sessions!, and this in turn will invoke some code in the underlying driver. In the case of Selenium, this deletes all cookies to ensure that you start each scenario with no pollution from the previous one. For Rack::Test, this will destroy the browser instance, so a new one gets created each time; for any other drivers, you will need to check the implementation of the reset! method to see what they do. Finally, Capybara also hooks into any Cucumber scenarios you have tagged with @javascript and automatically switches you to the driver you have set up to handle JavaScript. For example, you may use Rack::Test as your default driver, but have set the following in your env.rb file: Capybara.javascript_dirver = :selenium In this case, any scenarios tagged with @javascript will result in Capybara starting a browser using Selenium as the driver. RSpec Outside of Cucumber, RSpec is one of the most popular Ruby test frameworks, lending itself well to unit, integration, and acceptance tests, and used inside and outside of Rails. Capybara adds the following features to RSpec: Lets you mix in the DSL to your specs by adding require 'capybara/rspec' into your spec_helper.rb file Lets you use :js => true to invoke the JavaScript driver Adds a DSL for writing descriptive "feature style" acceptance tests using RSpec For more details on these features, check out the README, which is available at https://github.com/jniklas/capybara. Test::Unit If you are using Ruby's basic unit test library outside of Rails, then using Capybara simply means mixing in the DSL module via include Capybara::DSL, as you saw earlier. As noted in the README, it makes a lot of sense to reset the browser session in your teardown method, for example: def teardown Capybara.reset_sessions! Capybara.use_default_driver end If you are using Rails, then there will be other considerations, such as turning off transactional fixtures, as these will not work with Selenium; again, the Capybara README details this behavior fully. MiniTest::Spec MiniTest is a new unit test framework introduced in Ruby 1.9, which also has the ability to support BDD style tests. Capybara does not have built-in support for MiniTest, because MiniTest does not use RSpec but rather uses its own matchers. There is another gem named capybara_minitest_spec (https://github.com/ordinaryzelig/capybara_minitest_spec), which adds support for these matchers to Capybara. Advanced interactions and accessing the driver directly Although we have covered a great deal of Capybara's API, there are still a few interactions that we have not addressed, for example, hovering over an element or dragging elements around. Capybara does provide support for a lot of these more advanced interactions; for example, a recent addition (Capybara 2.1) to methods you can call on an element is hover: find('#box1').hover In Selenium, this results in a call to the mouse.move_to method and works for both elements using CSS's hover property or JavaScript's mouseenter / mouseleave methods. Other drivers may implement this differently, and obviously in some it may not be supported at all, either where JavaScript support is non-existent (Rack::Test) or rudimentary (Celerity). You can also emulate drag-and-drop using the following line of code: find('#mydiv').drag_to '#droplocation' Again, driver support is likely to be patchy, but of course this will work fine in Selenium WebDriver. Despite all the bells and whistles offered by Capybara, there may still be occasions where you need to access the API that exists in the underlying driver, but that has not been mapped in Capybara. In this instance, you have two options: Call Capybara's native method on any Caybara::Element, and then call the driver method Use page.driver.browser.manage to call the driver methods that are not called on elements Using the native method If you wish to call a method in the underlying driver, and that method is one that is called on a retrieved DOM element, then you can use the native method. A good example is retrieving a computed CSS style value. Elements in the DOM can obtain CSS properties in a couple of ways; firstly, there is the inline style: <p style="font-weight:bold;">Bold Paragraph Text</p> This information could be easily retrieved by accessing the style attribute via the methods. The other way in which an element obtains CSS properties is via style elements or stylesheets that are referenced via link tags in the page. When the browser loads the stylesheets and applies styles to the specified DOM elements, these are known as computed styles. Capybara has no direct API for retrieving the computed style of an element, which was most likely a deliberate design decision as only a few drivers would ever support this. However, Selenium WebDriver does have this capability, and it is possible that you would want to access this information. Consider the following code, where we apply a CSS hover property to a div element, so that when the user hovers over the element, it changes color. <html> <head> <title>Hover Examples</title> <style> .box { height: 200px; width: 200px; margin: 10px; background-color: blue; } .box:hover { background-color: green; } </style> </head> <body> <div id="main"> <div id="box1" class="box"> </div> </div> </body> </html> The Cucumber step definitions that follow use Capybara and Selenium WebDriver to assert that the color has changed: When(/^I hover over an element whose color changes on hover using CSS$/) do visit 'http://capybara.local/html/chapter5/hover.html' find('#box1').hover end Then(/^I see the color change$/) do find('#box1').native.style('background-color').should == 'rgba(0, 128, 0, 1)' end Here, the style method within Selenium WebDriver is accessed via Capybara's native method, and the value can then be validated. Accessing driver methods using browser.manage The other use case is when we wish to access functionality provided in the underlying driver that is neither mapped by Capybara nor related to a specific element on the page. A good example of this is accessing cookie information. There has been a long running debate on the Capybara forums and GitHub issue tracker, about whether Capybara should expose an API for cookie getters and setters, but the overriding feeling has always been that this should not be exposed (arguably, you should not set cookies in your tests, as this is changing the state of the application as a user never would). Nevertheless, it is something you may well need to do, and given that Selenium WebDriver and a number of other drivers support this functionality, you will need to access the driver methods directly. Consider this page that sets a cookie using JavaScript: <html> <head> <title>Cookie Examples</title> <script> document.cookie = 'mycookie=foobar'; </script> </head> <body></body> </html> The following steps simply visits the page and then outputs to the console all the cookies that are available on the current page: When(/^I visit a page that sets a Cookie$/) do visit 'http://localhost/html/cookie.html' end Then(/^I can access the cookie using Selenium$/) do puts page.driver.browser.manage.all_cookies end We can access any method in the underlying driver by using page.driver.browser.manage, and then the method we wish to call; in this instance, we call Selenium WebDriver's all_cookies method, and the output will be as follows: [{:name=>"mycookie", :value=>"foobar", :path=>"/html/chapter5", :domain=>"localhost", :expires=>nil, :secure=>false}] Selenium WebDriver exposes a full API for accessing and setting cookies. The documentation can be found at http://rubydoc.info/gems/selenium-webdriver/0.0.28/Selenium/WebDriver/Driver. Advanced driver configuration So far, we have only set the default driver or the JavaScript driver using a symbol: Capybara.default_driver = :selenium It is quite likely that you will need to fine-tune the configuration of your driver or register multiple configurations, which you can select from at run time. An example of this might be that you are running tests from your office, and the corporate network sits behind an HTTP Proxy (the bane of a tester's life). If you are using Selenium WebDriver with Firefox, you could register a custom driver configuration in Capybara as follows: Capybara.register_driver :selenium_proxy do |app| profile = Selenium::WebDriver::Firefox::Profile.new profile["network.proxy.type"] = 1 profile["network.proxy.no_proxies_on"] = "capybara.local" profile["network.proxy.http"] = "cache-mycompany.com" profile["network.proxy.ssl"] = 'securecache-mycompany.com' profile["network.proxy.http_port"] = 9999 profile["network.proxy.ssl_port"] = 9999 profile.native_events = true Capybara::Selenium::Driver.new(app, :browser => :firefox, :profile => profile) end Capybara.default_driver = :selenium_proxy This configuration uses the Selenium WebDriver API to construct a custom Firefox profile, set the proxy details programmatically, register the driver with the name :selenium-proxy, and then make it the default driver. Obviously the possibilities here are endless, and in browsers such as Firefox and Chrome, the amount of options you can customize runs into hundreds, so knowing how to set these is important. For example, you could use this technique to create profiles with JavaScript disabled or Cookies disabled to ensure your application behaves correctly in these cases. The driver ecosystem Capybara bundles two drivers, which as you know are Rack::Test and Selenium WebDriver. However, Capybara is architected in such a way to make it easy for developers to implement other drivers, and indeed, there is a healthy ecosystem of pluggable drivers, which offer interesting alternatives to the two built-in options. Capybara-WebKit Pretty much every developer I know who is passionate about test automation is desperate for one thing—a headless browser with great JavaScript support. A headless browser is one that runs without a UI. Headless browsers typically run tests faster than opening a real browser, and make it easy to run in Continuous Integration (CI) environments, where often, a windowing environment (either MS Windows or X11 on Linux) may not be available. You can refer to the following links to find out more about Capybara-WebKit: https://github.com/thoughtbot/capybara-webkit http://qt.digia.com/ Capybara-WebKit is a driver that wraps QtWebKit and is maintained by the guys at Thoughtbot. The Qt project provides a cross-platform framework for building native GUI applications, and as part of this, it provides a WebKit browser implementation that lends itself to headless implementations. There is a slight catch with this particular implantation of QtWebKit; you will need to install the Qt system libraries separately, and there is still a reliance on X11 being present on Linux distributions despite the browser being headless. Poltergeist Poltergeist is another driver, which, in the background, will use QtWebKit. The difference here is that it wraps PhantomJS, which brings a couple of potential advantages over Capybara-WebKit. You can refer to the following links to find out more about Poltergeist: https://github.com/jonleighton/poltergeist http://phantomjs.org/ You will still need to install PhantomJS as a system dependency, but the PhantomJS project has tackled some of the issues around making QtWebKit purely headless, so you will not need X11, and you will not need to install the entire Qt framework, because PhantomJS bundles this for you. Capybara-Mechanize Mechanize is a headless browser implemented purely in Ruby, and has long been a stalwart for the Ruby community; it uses Nokogiri at its core to provide DOM access, and then builds browser capabilities around this. You can refer to the following links to find out more about Capybara-Mechanize: https://github.com/jeroenvandijk/capybara-mechanize https://github.com/sparklemotion/mechanize Capybara-Mechanize is a driver that allows you to use Mechanize to run your tests. It is a very powerful option, but there are some important aspects to consider, such as: No JavaScript support: Mechanize does not contain a JavaScript engine; therefore, none of the JavaScript on any of your pages will be run No rendering engine: Mechanize does not contain a rendering engine; therefore, no computed styles will be evaluated Fast: Because it's not attempting to evaluate JavaScript and do graphical rendering, it will be super fast The benefits of using Mechanize may not be obvious at first sight, and a lot will depend on how your site is implemented. For sites whose functionality is wholly dependent on JavaScript, this option is clearly not feasible; however, if your site follows the principles of "progressive enhancement", where JavaScript is simply used to enrich the user's experience, Mechanize is a great option. You can implement all the core functional tests using Mechanize, which will be ensure they run with minimal latency and will be far less fragile than using Selenium, and then just implement a sprinkling of Selenium or WebKit tests to test the JavaScript dependent features. Capybara-Celerity The final driver that is worth considering is Capybara-Celerity, which wraps the Ruby library Celerity. This is especially worth a look if you are running your test code on JRuby (Ruby running on the JVM) as Celerity itself wraps the Java library HtmlUnit. You can refer to the following links to find out more about Capybara-Celerity: https://github.com/sobrinho/capybara-celerity https://github.com/jarib/celerity http://htmlunit.sourceforge.net/ Going back a few years, HtmlUnit would have been a serious consideration for anybody wanting a headless browser, indeed it was the default headless browser for the Selenium WebDriver project. HtmlUnit is a pure Java implementation of a browser and uses Rhino (https://developer.mozilla.org/en/docs/Rhino) to run JavaScript. Unfortunately, you may find that the JavaScript support still fails when attempting to evaluate heavy-weight JS libraries, as the project is not actively maintained, and keeping pace with the demands of modern browsers is unrealistic for a small project. If you are looking for a headless alternative to Mechanize, Celerity is worth a look, but don't rely on it for JavaScript support. Summary This article has taken you from confidently implementing Capybara tests against your application to being a Capybara ninja. By understanding how to use Capybara outside the comfort zone of Cucumber, you can now use it in almost any setting and even write your own custom framework. This is important even if you use Cucumber, because as your Cucumber tests grow, you will probably want to implement some Page Objects and mix Capybara::DSL into these. Another important aspect of writing effective tests is being able to configure the underlying driver and access it directly when required; there is nothing magical about this, and it means you can harness the full power of your chosen driver. Finally, we introduced some alternative drivers that should whet your appetite and prove why Capybara is such a powerful framework. You write your tests once and then run them in multitude of compatible drivers. Win! Resources for Article: Further resources on this subject: First Steps with Selenium RC [Article] Behavior-driven Development with Selenium WebDriver [Article] Setting up environment for Cucumber BDD Rails [Article]

(For more resources related to this topic, see here.) We have learned how to create snippets, how to work with forms, and Ajax, how to test your code, and how to create a REST API, and all this is awesome! However, we did not learn how to persist data to make it durable. In this article, we will see how to use Mapper, an object-relational mapping ( ORM ) system for relational databases, included with Lift. The idea is to create a map of database tables into a well-organized structure of objects; for example, if you have a table that holds users, you will have a class that represents the user table. Let's say that a user can have several phone numbers. Probably, you will have a table containing a column for the phone numbers and a column containing the ID of the user owning that particular phone number. This means that you will have a class to represent the table that holds phone numbers, and the user class will have an attribute to hold the list of its phone numbers; this is known as a one-to-many relationship . Mapper is a system that helps us build such mappings by providing useful features and abstractions that make working with databases a simple task. For example, Mapper provides several types of fields such as MappedString, MappedInt, and MappedDate which we can use to map the attributes of the class versus the columns in the table being mapped. It also provides useful methods such as findAll that is used to get a list of records or save, to persist the data. There is a lot more that Mapper can do, and we'll see what this is through the course of this article. Configuring a connection to database The first thing we need to learn while working with databases is how to connect the application that we will build with the database itself. In this recipe, we will show you how to configure Lift to connect with the database of our choice. For this recipe, we will use PostgreSQL; however, other databases can also be used: Getting ready Start a new blank project. Edit the build.sbt file to add the lift-mapper and PostgreSQL driver dependencies: "net.liftweb" %% "lift-mapper" % liftVersion % "compile", "org.postgresql" % "postgresql" % "9.2-1003-jdbc4" % "compile" Create a new database. Create a new user. How to do it... Now carry out the following steps to configure a connection with the database: Add the following lines into the default.props file: db.driver=org.postgresql.Driver db.url=jdbc:postgresql:liftbook db.user=<place here the user you've created> db.password=<place here the user password> Add the following import statement in the Boot.scala file: import net.liftweb.mapper._ Create a new method named configureDB() in the Boot.scala file with the following code: def configureDB() { for { driver <- Props.get("db.driver") url <- Props.get("db.url") } yield { val standardVendor = new StandardDBVendor(driver, url, Props.get("db.user"), Props.get("db.password")) LiftRules.unloadHooks.append(standardVendor.closeAllConnections_! _) DB.defineConnectionManager(DefaultConnectionIdentifier, standardVendor) } } Then, invoke configureDB() from inside the boot method. How it works... Lift offers the net.liftweb.mapper.StandardDBVendor class, which we can use to create connections to the database easily. This class takes four arguments: driver, URL, user, and password. These are described as follows: driver : The driver argument is the JDBC driver that we will use, that is, org.postgresql.Driver URL : The URL argument is the JDBC URL that the driver will use, that is, jdbc:postgresql:liftbook user and password : The user and password arguments are the values you set when you created the user in the database. After creating the default vendor, we need to bind it to a Java Naming and Directory Interface (JNDI) name that will be used by Lift to manage the connection with the database. To create this bind, we invoked the defineConnectionManager method from the DB object. This method adds the connection identifier and the database vendor into a HashMap, using the connection identifier as the key and the database vendor as the value. The DefaultConnectionIdentifier object provides a default JNDI name that we can use without having to worry about creating our own connection identifier. You can create your own connection identifier if you want. You just need to create an object that extends ConnectionIdentifier with a method called jndiName that should return a string. Finally, we told Lift to close all the connections while shutting down the application by appending a function to the unloadHooks variable. We did this to avoid locking connections while shutting the application down. There's more... It is possible to configure Lift to use a JNDI datasource instead of using the JDBC driver directly. In this way, we can allow the container to create a pool of connections and then tell Lift to use this pool. To use a JNDI datasource, we will need to perform the following steps: Create a file called jetty-env.xml in the WEB-INF folder under src/main/webapp/ with the following content: <!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure.dtd"> <Configure class="org.eclipse.jetty.webapp.WebAppContext"> <New id="dsliftbook" class="org.eclipse.jetty.plus.jndi.Resource"> <Arg>jdbc/dsliftbook</Arg> <Arg> <New class="org.postgresql.ds.PGSimpleDataSource"> <Set name="User">place here the user you've created</Set> <Set name="Password">place here the user password</Set> <Set name="DatabaseName">liftbook</Set> <Set name="ServerName">localhost</Set> <Set name="PortNumber">5432</Set> </New> </Arg> </New> </Configure> Add the following line into the build .sbt file: env in Compile := Some(file("./src/main/webapp/WEB-INF/jetty-env.xml") asFile) Remove all the jetty dependencies and add the following: "org.eclipse.jetty" % "jetty-webapp" % "8.0.4.v20111024" % "container", "org.eclipse.jetty" % "jetty-plus" % "8.0.4.v20111024" % "container", Add the following code into the web.xml file. <resource-ref> <res-ref-name>jdbc/dsliftbook</res-ref-name> <res-type>javax.sql.DataSource</res-type> <res-auth>Container</res-auth> </resource-ref> Remove the configureDB method. Replace the invocation of configureDB method with the following line of code: DefaultConnectionIdentifier.jndiName = "jdbc/dsliftbook" The creation of the file jetty-envy.xml and the change we made in the web.xml file were to create the datasource in jetty and make it available to Lift. Since the connections will be managed by jetty now, we don't need to append hooks so that Lift can close the connections or any other configuration when shutting down the application. All we need to do is tell Lift to get the connections from the JNDI datasource that we have configured into the jetty. We do this by setting the jndiName variable of the default connection identifier, DefaultConnectionIdentifier as follows: DefaultConnectionIdentifier.jndiName = "jdbc/dsliftbook" The change we've made to the build.sbt file was to make the jetty-env.xml file available to the embedded jetty. So, we can use it when we get the application started by using the container:start command. See also... You can learn more about how to configure a JNDI datasource in Jetty at the following address: http://wiki.eclipse.org/Jetty/Howto/Configure_JNDI_Datasource Mapping a table to a Scala class Now that we know how to connect Lift applications to the database, the next step is to learn how to create mappings between a database table and a Scala object using Mapper. Getting ready We will re-use the project we created in the previous recipe since it already has the connection configured. How to do it... Carry out the following steps to map a table into a Scala object using Mapper: Create a new file named Contact.scala inside the model package under src/main/scala/code/ with the following code: package code.model import net.liftweb.mapper.{MappedString, LongKeyedMetaMapper, LongKeyedMapper, IdPK} class Contact extends LongKeyedMapper[Contact] with IdPK { def getSingleton = Contact object name extends MappedString(this, 100) } object Contact extends Contact with LongKeyedMetaMapper[Contact] { override def dbTableName = "contacts" } Add the following import statement in the Boot.scala file: import code.model.Contact Add the following code into the boot method of the Boot class: Schemifier.schemify( true, Schemifier.infoF _, Contact ) Create a new snippet named Contacts with the following code: package code.snippet import code.model.Contact import scala.xml.Text import net.liftweb.util.BindHelpers._ class Contacts { def prepareContacts_!() { Contact.findAll().map(_.delete_!) val contactsNames = "John" :: "Joe" :: "Lisa" :: Nil contactsNames.foreach(Contact.create.name(_).save()) } def list = { prepareContacts_!() "li *" #> Contact.findAll().map { c => { c.name.get } } } } Edit the index.html file by replacing the content of the div element with main as the value of id using the following code : <div data-list="Contacts.list"> <ul> <li></li> </ul> </div> Start the application. Access your local host (http://localhost:8080), and you will see a page with three names, as shown in the following screenshot: How it works... To work with Mapper, we use a class and an object. The first is the class that is a representation of the table; in this class we define the attributes the object will have based on the columns of the table we are mapping. The second one is the class' companion object where we define some metadata and helper methods. The Contact class extends the LongKeyedMapper[Contact] trait and mixes the trait IdPK. This means that we are defining a class that has an attribute called id (the primary key), and this attribute has the type Long. We are also saying that the type of this Mapper is Contact. To define the attributes that our class will have, we need to create objects that extend "something". This "something" is the type of the column. So, when we say an object name extends MappedString(this, 100), we are telling Lift that our Contact class will have an attribute called name, which will be a string that can be 100 characters long. After defining the basics, we need to tell our Mapper where it can get the metadata about the database table. This is done by defining the getSingleton method. The Contact object is the object that Mapper uses to get the database table metadata. By default, Lift will use the name of this object as the table name. Since we don't want our table to be called contact but contacts, we've overridden the method dbTableName. What we have done here is created an object called Contact, which is a representation of a table in the database called contacts that has two columns: id and name. Here, id is the primary key and of the type Long, and name is of the type String, which will be mapped to a varchar datatype. This is all we need to map a database table to a Scala class, and now that we've got the mapping done, we can use it. To demonstrate how to use the mapping, we have created the snippet Contacts. This snippet has two methods. The list method does two things; it first invokes the prepareContacts_!() method, and then invokes the findAll method from the Contact companion object. The prepareContacts_!() method also does two things: it first deletes all the contacts from the database and then creates three contacts: John, Joe, and Lisa. To delete all the contacts from the database, it first fetches all of them using the findAll method, which executes a select * from contacts query and returns a list of Contact objects, one for each existing row in the table. Then, it iterates over the collection using the foreach method and for each contact, it invokes the delete_! method which as you can imagine will execute a delete from contacts where id = contactId query is valid. After deleting all the contacts from the database table, it iterates the contactsNames list, and for each element it invokes the create method of the Contact companion object, which then creates an instance of the Contact class. Once we have the instance of the Contact class, we can set the value of the name attribute by passing the value instance.name(value). You can chain commands while working with Mapper objects because they return themselves. For example, let's say our Contact class has firstName and lastName as attributes. Then, we could do something like this to create and save a new contact: Contact.create.firstName("John").lastName("Doe").save() Finally, we invoke the save() method of the instance, which will make Lift execute an insert query, thus saving the data into the database by creating a new record. Getting back to the list method, we fetch all the contacts again by invoking the findAll method, and then create a li tag for each contact we have fetched from the database. The content of the li tags is the contact name, which we get by calling the get method of the attribute we want the value of. So, when you say contact.name.get, you are telling Lift that you want to get the value of the name attribute from the contact object, which is an instance of the Contact class. There's more... Lift comes with a variety of built-in field types that we can use; MappedString is just one of them. The others include, MappedInt, MappedLong, and MappedBoolean. All these fields come with some built-in features such as the toForm method, which returns the HTML needed to generate a form field, and the validate method that validates the value of the field. By default, Lift will use the name of the object as the name of the table's column; for example, if you define your object as name—as we did—Lift will assume that the column name is name. Lift comes with a built-in list of database-reserved words such as limit, order, user, and so on. If the attribute you are mapping is a database-reserved word, Lift will append _c at the end of the column's name when using Schemifier. For example, if you create an attribute called user, Lift will create a database column called user_c. You can change this behavior by overriding the dbColumnName method, as shown in the following code: object name extends MappedString(this, 100) { override def dbColumnName = "some_new_name" } In this case, we are telling Lift that the name of the column is some_new_name. We have seen that you can fetch data from the database using the findAll method. However, this method will fetch every single row from the database. To avoid this, you can filter the result using the By object; for example, let's say you want to get only the contacts with the name Joe. To accomplish this, you would add a By object as the parameter of the findAll method as follows: Contact.findAll(By(Contact.name, "Joe")). There are also other filters such as ByList and NotBy. And if for some reason the features offered by Mapper to build select queries aren't enough, you can use methods such as findAllByPreparedStatement and findAllByInsecureSQL where you can use raw SQL to build the queries. The last thing left to talk about here is how this example would work if we didn't create any table in the database. Well, I hope you remember that we added the following lines of code to the Boot.scala file: Schemifier.schemify( true, Schemifier.infoF _, Contact ) As it turns out, the Schemifier object is a helper object that assures that the database has the correct schema based on a list of MetaMappers. This means that for each MetaMapper we pass to the Schemifier object, the object will compare the MetaMapper with the database schema and act accordingly so that both the MetaMapper and the database schema match. So, in our example, Schemifier created the table for us. If you change MetaMapper by adding attributes, Schemifier will create the proper columns in the table. See also... To learn more about query parameters, you can find more at: https://www.assembla.com/spaces/liftweb/wiki/Mapper#querying_the_database Creating one-to-many relationships In the previous recipe, we learned how to map a Scala class to a database table using Mapper. However, we have mapped a simple class with only one attribute, and of course, we will not face this while dealing with real-world applications. We will probably need to work with more complex data such as the one having one-to-many or many-to-many relationships. An example of this kind of relationship would be an application for a store where you'll have customers and orders, and we need to associate each customer with the orders he or she placed. This means that one customer can have many orders. In this recipe, we will learn how to create such relationships using Mapper. Getting ready We will modify the code from the last section by adding a one-to-many relationship into the Contact class. You can use the same project from before or duplicate it and create a new project. How to do it... Carry out the following steps: Create a new class named Phone into the model package under src/main/scala/code/ using the following code: package code.model import net.liftweb.mapper._ class Phone extends LongKeyedMapper[Phone] with IdPK { def getSingleton = Phone object number extends MappedString(this, 20) object contact extends MappedLongForeignKey(this, Contact) } object Phone extends Phone with LongKeyedMetaMapper[Phone] { override def dbTableName = "phones" } Change the Contact class declaration from: class Contact extends LongKeyedMapper[Contact] with IdPK To: class Contact extends LongKeyedMapper[Contact] with IdPK with OneToMany[Long, Contact] Add the attribute that will hold the phone numbers to the Contact class as follows: object phones extends MappedOneToMany(Phone, Phone.contact, OrderBy(Phone.id, Ascending)) with Owned[Phone] with Cascade[Phone] Insert the new class into the Schemifier list of parameters. It should look like the following code: Schemifier.schemify( true, Schemifier.infoF _, Contact, Phone ) In the Contacts snippet, replace the import statement from the following code: import code.model.Contact To: import code.model._ Modify the Contacts.prepareContacts_! method to associate a phone number to each contact. Your method should be similar to the one in the following lines of code: def prepareContacts_!() { Contact.findAll().map(_.delete_!) val contactsNames = "John" :: "Joe" :: "Lisa" :: Nil val phones = "5555-5555" :: "5555-4444" :: "5555-3333" :: "5555-2222" :: "5555-1111" :: Nil contactsNames.map(name => { val contact = Contact.create.name(name) val phone = Phone.create.number(phones((new Random()).nextInt(5))).saveMe() contact.phones.append(phone) contact.save() }) } Replace the list method's code with the following code: def list = { prepareContacts_!() ".contact *" #> Contact.findAll().map { contact => { ".name *" #> contact.name.get & ".phone *" #> contact.phones.map(_.number.get) } } } In the index.html file, replace the ul tag with the one in the following code snippet: <ul> <li class="contact"> <span class="name"></span> <ul> <li class="phone"></li> </ul> </li> </ul> Start the application. Access http://localhost:8080. Now you will see a web page with three names and their phone numbers, as shown in the following screenshot: How it works... In order to create a one-to-many relationship, we have mapped a second table called phone in the same way we've created the Contact class. The difference here is that we have added a new attribute called contact. This attribute extends MappedLongForeignKe y, which is the class that we need to use to tell Lift that this attribute is a foreign key from another table. In this case, we are telling Lift that contact is a foreign key from the contacts table in the phones table. The first parameter is the owner, which is the class that owns the foreign key, and the second parameter is MetaMapper, which maps the parent table. After mapping the phones table and telling Lift that it has a foreign key, we need to tell Lift what the "one" side of the one-to-many relationship will be. To do this, we need to mix the OneToMany trait in the Contact class. This trait will add features to manage the one-to-many relationship in the Contact class. Then, we need to add one attribute to hold the collection of children records; in other words, we need to add an attribute to hold the contact's phone numbers. Note that the phones attribute extends MappedOneToMany, and that the first two parameters of the MappedOneToMany constructor are Phone and Phone.contact. This means that we are telling Lift that this attribute will hold records from the phones table and that it should use the contact attribute from the Phone MetaMapper to do the join. The last parameter, which is optional, is the OrderBy object. We've added it to tell Lift that it should order, in an ascending order, the list of phone numbers by their id. We also added a few more traits into the phones attribute to show some nice features. One of the traits is Owned; this trait tells Lift to delete all orphan fields before saving the record. This means that if, for some reason, there are any records in the child table that has no owner in the parent table, Lift will delete them when you invoke the save method, helping you keep your databases consistent and clean. Another trait we've added is the Cascade trait. As its name implies, it tells Lift to delete all the child records while deleting the parent record; for example, if you invoke contact.delete_! for a given contact instance, Lift will delete all phone records associated with this contact instance. As you can see, Lift offers an easy and handy way to deal with one-to-many relationships. See also... You can read more about one-to-many relationships at the following URL: https://www.assembla.com/wiki/show/liftweb/Mapper#onetomany

(For more resources related to this topic, see here.) Dependencies can be injected in a consumer class using different patterns and injecting them into a constructor is just one of them. While there are some patterns that can be followed for injecting dependencies, there are also some patterns that are recommended to be avoided, as they usually lead to undesirable results. In this article, we will examine only those patterns and antipatterns that are somehow relevant to Ninject features. Constructor Injection Constructor Injection is the most common and recommended pattern for injecting dependencies in a class. Generally this pattern should always be used as the primary injection pattern unless we have to use other ones. In this pattern, a list of all class dependencies should be introduced in the constructor. The question is what if the class has more than one constructor. Although Ninject's strategy for selecting constructor is customizable, its default behavior is selecting the constructor with more parameters, provided all of them are resolvable by Ninject. So, although in the following code the second constructor introduces more parameters, Ninject will select the first one if it cannot resolve IService2 and it will even use the default constructor if IService1 is not registered either. But if both dependencies are registered and resolvable, Ninject will select the second constructor because it has more parameters: public class Consumer { private readonly IService1 dependency1; private readonly IService2 dependency2; public Consumer(IService1 dependency1) { this.dependency1 = dependency1; } public Consumer(IService1 dependency1, IService2 dependency2) { this.dependency1 = dependency1; this.dependency2 = dependency2; } } If the preceding class had another constructor with two resolvable parameters, Ninject would throw an ActivationException exception notifying that several constructors had the same priority. There are two approaches to override this default behavior and explicitly select a constructor. The first approach is to indicate the desired constructor in a binding as follows: Bind<Consumer>().ToConstructor(arg => new Consumer(arg.Inject<IService1>())); In the preceding example, we explicitly selected the first constructor. Using the Inject<T> method that the arg argument provides, we requested Ninject to resolve IService1 in order to be injected into the specified constructor. The second method is to indicate the desired constructor using the [Inject] attribute: [Inject] public Consumer(IService1 dependency1) { this.dependency1 = dependency1; } In the preceding example, we applied the Ninject's [Inject] attribute on the first constructor to explicitly specify that we need to initialize the class by injecting dependencies into this constructor; even though the second constructor has more parameters and the default strategy of Ninject would be to select the second one. Note that applying this attribute on more than one constructor will result in the ActivationException. Ninject is highly customizable and it is even possible to substitute the default [Inject] attribute with another one, so that we don't need to add reference to the Ninject library from our consumer classes just because of an attribute: kernel.Settings.Set("InjectAttribute",typeof(MyAttribute)); Initializer methods and properties Apart from constructor injection, Ninject supports the injection of dependencies using initializer methods and property setters. We can specify as many methods and properties as required using the [Inject] attribute to inject dependencies. Although the dependencies will be injected to them as soon as the class is constructed, it is not possible to predict in which order they will receive their dependencies. The following code shows how to specify a property for injection: [Inject]public IService Service{ get { return dependency; } set { dependency = value; }} Here is an example of injecting dependencies using an injector method: [Inject]public void Setup(IService dependency){ this.dependency = dependency;} Note that only public members and constructors will be injected and even the internals will be ignored unless Ninject is configured to inject nonpublic members. In Constructor Injection, the constructor is a single point where we can consume all of the dependencies as soon as the class is activated. But when we use initializer methods the dependencies will be injected via multiple points in an unpredictable order, so we cannot decide in which method all of the dependencies will be ready to consume. In order to solve this problem, Ninject offers the IInitializable interface. This interface has an Initialize method which will be called once all of the dependencies have been injected: public class Consumer:IInitializable{ private IService1 dependency1; private IService2 dependency2; [Inject] public IService Service1 { get { return dependency1; } set { dependency1 = value; } } [Inject] public IService Service2 { get { return dependency2; } set { dependency2 = value; } } public void Initialize() { // Consume all dependencies here }} Although Ninject supports injection using properties and methods, Constructor Injection should be the superior approach. First of all, Constructor Injection makes the class more reusable, because a list of all class dependencies are visible, while in the initializer property or method the user of the class should investigate all of the class members or go through the class documentations (if any), to discover its dependencies. Initialization of the class is easier while using Constructor Injection because all the dependencies get injected at the same time and we can easily consume them at the same place where the constructor initializes the class. As we have seen in the preceding examples the only case where the backing fields could be readonly was in the Constructor Injection scenario. As the readonly fields are initializable only in the constructor, we need to make them writable to be able to use initializer methods and properties. This can lead to potential mutation of backing fields. Service Locator Service Locator is a design pattern introduced by Martin Fowler regarding which there have been some controversies. Although it can be useful in particular circumstances, it is generally considered as an antipattern and preferably should be avoided. Ninject can easily be misused as a Service Locator if we are not familiar to this pattern. The following example demonstrates misusing the Ninject kernel as a Service Locator rather than a DI container: public class Consumer{ public void Consume() { var kernel = new StandardKernel(); var depenency1 = kernel.Get<IService1>(); var depenency2 = kernel.Get<IService2>(); ... }} There are two significant downsides with the preceding code. The first one is that although we are using a DI container, we are not at all implementing DI. The class is tied to the Ninject kernel while it is not really a dependency of this class. This class and all of its prospective consumers will always have to drag their unnecessary dependency on the kernel object and Ninject library. On the other hand, the real dependencies of class (IService1 and IService2) are invisible from the consumers, and this reduces its reusability. Even if we change the design of this class to the following one, the problems still exist: public class Consumer{ private readonly IKernel kernel; public Consumer(IKernel kernel) { this.kernel = kernel; } public void Consume() { var depenency1 = kernel.Get<IService1>(); var depenency2 = kernel.Get<IService2>(); ... }} The preceding class still depends on the Ninject library while it doesn't have to and its actual dependencies are still invisible to its consumers. It can easily be refactored using the Constructor Injection pattern: public Consumer(IService1 dependency1, IService2 dependency2){ this.dependency1 = dependency1; this.dependency2 = dependency2;} Summary In this article we studied the most common DI patterns and anti-patterns related to Ninject. Resources for Article: Further resources on this subject: Introduction to JBoss Clustering [Article] Configuring Clusters in GlassFish [Article] Designing Secure Java EE Applications in GlassFish [Article]

Representational State Transfer Representational State Transfer (REST) is a style of information application architecture that aligns the distributed applications to the HTTP request and response protocols, in particular matching Hypermedia to the HTTP request methods and Uniform Resource Identifiers (URI). Hypermedia is the term that describes the ability of a system to deliver self-referential content, where related contextual links point to downloadable or stream-able digital media, such as photographs, movies, documents, and other data. Modern systems, especially web applications, demonstrate through display of text that a certain fragment of text is a link to the media. Hypermedia is the logical extension of the term hypertext, which is a text that contains embedded references to other text. These embedded references are called links, and they immediately transfer the user to the other text when they are invoked. Hypermedia is a property of media, including hypertext, to immediately link other media and text. In HTML, the anchor tag <a> accepts a href attribute, the so-called hyperlink parameter. The World Wide Web is built on the HTTP standards, Versions 1.0 and 1.1, which define specific enumerations to retrieve data from a web resource. These operations, sometimes called Web Methods, are GET, POST, PUT, and DELETE. Representational State Transfer also reuses these operations to form semantic interface to a URI. Representational State Transfer, then, is both a style and architecture for building network enabled distributed applications. It is governed by the following constraints: Client/Server: A REST application encourages the architectural robust principle of separation of concerns by dividing the solution into clients and servers. A standalone, therefore, cannot be a RESTful application. This constraint ensures the distributed nature of RESTful applications over the network. Stateless: A REST application exhibits stateless communication. Clients cannot and should not take advantage of any stored context information in the server and therefore the full data of each request must be sent to the server for processing. Cache: A REST application is able to declare which data is cacheable or not cacheable. This constraint allows the architect to set the performance level for the solution, in other words, a trade-off. Caching data to the web resource, allows the business to achieve a sense of latency, scalability, and availability. The counter point to improved performance through caching data is the issue of expiration of the cache at the correct time and sequence, when do we delete stale data? The cache constraint also permits successful implementation providers to develop optimal frameworks and servers. Uniform Interface: A REST application emphasizes and maintains a unique identifier for each component and there are a set of protocols for accessing data. The constraint allows general interaction to any REST component and therefore anyone or anything can manipulate a REST component to access data. The drawback is the Uniform Interface may be suboptimal in ease-of-use and cognitive load to directly provide a data structure and remote procedure function call. Layered Style: A REST application can be composed of functional processing layers in order to simplify complex flows of data between clients and servers. Layered style constraint permits modularization of function with the data and in itself is another sufficient example of separation of concerns. The layered style is an approach that benefits load-balancing servers, caching content, and scalability. Code-on-Demand: A REST application can optimally supply downloadable code on demand for the client to execute. The code could be the byte-codes from the JVM, such as a Java Applet, or JavaFX WebStart application, or it could be a JavaScript code with say JSON data. Downloadable code is definitely a clear security risk that means that the solution architect must assume responsibility of sandboxing Java classes, profiling data, and applying certificate signing in all instances. Therefore, code-on-demand, is a disadvantage in a public domain service, and this constraint in REST application is only seen inside the firewalls of corporations. In terms of the Java platform, the Java EE standard covers REST applications through the specification JAX-RS and this article covers Version 2.0. JAX-RS 2.0 features For Java EE 7, the JAX-RS 2.0 specification has the following new features: Client-side API for invoking RESTful server-side remote endpoint Support for Hypermedia linkage Tighter integration with the Bean Validation framework Asynchronous API for both server and client-side invocations Container filters on the server side for processing incoming requests and outbound responses Client filter on the client side for processing outgoing request and incoming responses Reader and writer interceptors to handle specific content types Architectural style The REST style is simply a Uniform Resource Identifier and the application of the HTTP request methods, which invokes resources that generate a HTTP response. Although Fielding, himself, says that REST does not necessarily require the HTTP communication as a networker layer, and the style of architecture can be built on any other network protocol. Let's look at those methods again with a fictional URL (http://fizzbuzz.com/) Method Description POST A REST style application creates or inserts an entity with the supplied data. The client can assume new data has been inserted into the underlying backend database and the server returns a new URI to reference the data. PUT A REST style application replaces the entity into the database with the supplied data. GET A REST style application retrieves the entity associated with the URI, and it can be a collection of URI representing entities or it can be the actual properties of the entity DELETE A REST style application deletes the entity associated with the URI from the backend database. The user should note that PUT and DELETE are idempotent operations, meaning they can be repeated endlessly and the result is the same in steady state conditions. The GET operation is a safe operation; it has no side effects to the server-side data. REST style for collections of entities Let's take a real example with the URL http://fizzbuzz.com/resources/, which represents the URI of a collection of resources. Resources could be anything, such as books, products, or cast iron widgets. Method Description GET Retrieves the collection entities by URI under the link http://fizzbuzz.com/resources and they may include other more data. POST Creates a new entity in the collection under the URI http://fizzbuzz.com/resources. The URI is automatically assigned and returned by this service call, which could be something like http://fizzbuzz.com/resources/WKT54321. PUT Replaces the entire collection of entities under the URI http://fizzbuzz.com/resources. DELETE Deletes the entire collection of entities under the URI http://fizzbuzz.com/resources. As a reminder, a URI is a series of characters that identifies a particular resource on the World Wide Web. A URI, then, allows different clients to uniquely identify a resource on the web, or a representation. A URI is combination of a Uniform Resource Name (URN) and a Uniform Resource Locator (URL). You can think of a URN like a person's name, as way of naming an individual and a URL is similar to a person's home address, which is the way to go and visit them sometime. In the modern world, non-technical people are accustomed to desktop web browsing as URL. However, the web URL is a special case of a generalized URI. A diagram that illustrates HTML5 RESTful communication between a JAX RS 2.0 client and server, is as follows: REST style for single entities Assuming we have a URI reference to a single entity like http://fizzbuzz.com/resources/WKT54321. Method Description GET Retrieves the entity with reference to URI under the link http://fizzbuzz.com/resources/WKT54321. POST Creates a new sub entity under the URI http://fizzbuzz.com/resources/WKT54321. There is a subtle difference here, as this call does something else. It is not often used, except to create Master-Detail records. The URI of the subentity is automatically assigned and returned by this service call, which could be something like http://fizzbuzz.com/resources/WKT54321/D1023 PUT Replaces the referenced entity's entire collection with the URI http://fizzbuzz.com/resources/WKT54321 . If the entity does not exist then the service creates it. DELETE Deletes the entity under the URI references http://fizzbuzz.com/resources/WKT54321 Now that we understand the REST style we can move on to the JAX-RS API properly. Consider carefully your REST hierarchy of resources The key to build a REST application is to target the users of the application instead of blindly converting the business domain into an exposed middleware. Does the user need the whole detail of every object and responsibility in the application? On the other hand is the design not spreading enough information for the intended audience to do their work? Servlet mapping In order to enable JAX-RS in a Java EE application the developer must set up the configuration in the web deployment descriptor file. JAX-RS requires a specific servlet mapping to be enabled, which triggers the provider to search for annotated classes. The standard API for JAX-RS lies in the Java package javax.ws.rs and in its subpackages. Interestingly, the Java API for REST style interfaces sits underneath the Java Web Services package javax.ws. For your web applications, you must configure a Java servlet with just fully qualified name of the class javax.ws.rs.core.Application. The Servlet must be mapped to a root URL pattern in order to intercept REST style requests. The following is an example code of such a configuration: <?xml version="1.0" encoding="UTF-8"?><web-app xsi:schemaLocation="http://metadata-complete="false"><display-name>JavaEE Handbook 7 JAX RS Basic</display-name><servlet><servlet-name>javax.ws.rs.core.Application</servlet-name><load-on-startup>1</load-on-startup></servlet><servlet-mapping><servlet-name>javax.ws.rs.core.Application</servlet-name><url-pattern>/rest/*</url-pattern></servlet-mapping></web-app> In the web deployment descriptor we just saw, only the servlet name is defined, javax.ws.rs.core.Application. Do not define the servlet class. The second step maps the URL pattern to the REST endpoint path. In this example, any URL that matches the simplified Glob pattern /rest/* is mapped. This is known as the application path. A JAX-RS application is normally packaged up as a WAR file and deployed to a Java EE application server or a servlet container that conforms to the Web Profile. The application classes are stored in the /WEB-INF/classes and required libraries are found under /WEB-INF/lib folder. Therefore, JAX-RS applications share the consistency of configurations as servlet applications. The JAX-RS specification does recommend, but does not enforce, that conformant providers follow the general principles of Servlet 3.0 plug-ability and discoverability of REST style endpoints. Discoverability is achieved in the recommendation through class scanning of the WAR file, and it is up to the provider to make this feature available. An application can create a subclass of the javax.ws.rs.core.Application class. The Application class is a concrete class and looks like the following code: public class Application {public Application() { /* ... */ }public java.util.Set<Class<?>> getClasses() {/* ... */ }public java.util.Set<Object> getSingletons() {/* ... */ }public java.util.Map<String,Object> getProperties() {/* ... */ }} Implementing a custom Application subclass is a special case of providing maximum control of RESTful services for your business application. The developer must provide a set collection of classes that represent JAX-RS end points. The engineer must supply a list of Singleton object, if any, and do something useful with the properties. The default implementation of javax.ws.rs.core.Application and the methods getClasses() and getSingletons() return empty sets. The getProperties() method returns an empty map collection. By returning empty sets, the provider assumes that all relevant JAX-RS resource and provider classes that are scanned and found will be added to the JAX-RS application. Majority of the time, I suspect, you, the developer will want to rely on annotations to specify the REST end points and there are Servlet Context listeners and Servlet Filters to configure application wide behavior, for example the startup sequence of a web application. So how can we register our own custom Application subclass? The answer is just subclass the core class with your own class. The following code explains what we just read: package com.fizbuzz.services;@javax.ws.rs.ApplicationPath("rest")public class GreatApp extends javax.ws.rs.core.Application {// Do your custom thing here} In the custom GreatApp class, you can now configure logic for initialization. Note the use of @ApplicationPath annotation to configure the REST style URL. You still have to associate your custom Application subclass into the web deployment descriptor with a XML Servlet name definition. Remember the Application Configuration A very common error for first time JAX-RS developers is to forget that the web deployment descriptor really requires a servlet mapping to a javax.ws.js.core.Application type. Now that we know how to initialize a JAX-RS application, let us dig deeper and look at the defining REST style endpoints. Mapping JAX-RS resources JAX-RS resources are configured through the resource path, which suffixes the application path. Here is the constitution of the URL. http://<hostname>:<port>/<web_context>/<application_path>/<resource_path> The <hostname> is the host name of the server. The <port> refers to the port number, which is optional and the default port is 80. The <web_context> is the Servlet context, which is the context for the deployed web application. The <application_path> is the configuration URI pattern as specified in the web deployment descriptor @ApplicationPath or the Servlet configuration of Application type. The <resource_path> is the resource path to REST style resource. The final fragment <resource_path> defines the URL pattern that maps a REST style resource. The resource path is configured by annotation javax.ws.rs.Path. Test-Driven Development with JAX-RS Let us write a unit test to verify the simplest JAX-RS service. It follows a REST style resource around a list of books. There are only four books in this endpoint and the only thing the user/client can do at the start is to access the list of books by author and title. The client invokes the REST style endpoint, otherwise known as a resource with an HTTP GET request. The following is the code for the class RestfulBookService: package je7hb.jaxrs.basic;import javax.annotation.*;import javax.ws.rs.*;import java.util.*;@Path("/books")public class RestfulBookService {private List<Book> products = Arrays.asList(new Book("Sir Arthur Dolan Coyle","Sherlock Holmes and the Hounds of the Baskervilles"),new Book("Dan Brown", "Da Vinci Code"),new Book("Charles Dickens", "Great Expectations"),new Book("Robert Louis Stevenson", "Treasure Island"));@GET@Produces("text/plain")public String getList() {StringBuffer buf = new StringBuffer();for (Book b: products) {buf.append(b.title); buf.append('n'); }return buf.toString();}@PostConstructpublic void acquireResource() { /* ... */ }@PreDestroypublic void releaseResource() { /* ... */ }static class Book {public final String author;public final String title;Book(String author, String title) {this.author = author;this.title = title;}}} The annotation @javax.ws.rs.Path declares the class as a REST style end-point for a resource. The @Path annotation is assigned to the class itself. The path argument defines the relative URL pattern for this resource, namely/books. The method getList() is the interesting one. It is annotated with both @javax.ws.rs.GET and @javax.ws.rs.Produces. The @GET is one of the six annotations that conform to the HTTP web request methods. It indicates that the method is associated with HTTP GET protocol request. The @Produces annotation indicates the MIME content that this resource will generate. In this example, the MIME content is text/plain. The other methods on the resource bring CDI into the picture. In the example, we are injecting post construction and pre destruction methods into the bean. This is the only class that we require for a simple REST style application from the server side. With an invoking of the web resource with a HTTP GET Request like http://localhost:8080/mywebapp/rest/books we should get a plain text output with list of titles like the following: Sherlock Holmes and the Hounds of the BaskervillesDa Vinci CodeGreat ExpectationsTreasure Island So do we test this REST style interface then? We could use Arquillian Framework directly, but this means our tests have to be built specifically in a project and it complicates the build process. Arquillian uses another open source project in the JBoss repository called ShrinkWrap. The framework allows the construction of various types of virtual Java archives in a programmatic fashion. Let's look at the unit test class RestfulBookServiceTest in the following code: package je7hb.jaxrs.basic;// import omittedpublic class RestfulBookServiceTest {@Testpublic void shouldAssembleAndRetrieveBookList()throws Exception {WebArchive webArchive =ShrinkWrap.create(WebArchive.class, "test.war").addClasses(RestfulBookService.class).setWebXML(new File("src/main/webapp/WEB-INF/web.xml")).addAsWebInfResource(EmptyAsset.INSTANCE, "beans.xml");File warFile = new File(webArchive.getName());new ZipExporterImpl(webArchive).exportTo(warFile, true);SimpleEmbeddedRunner runner =SimpleEmbeddedRunner.launchDeployWarFile(warFile, "mywebapp", 8080);try {URL url = new URL("http://localhost:8080/mywebapp/rest/books");InputStream inputStream = url.openStream();BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));List<String> lines = new ArrayList<>();String text = null;int count=0;while ( ( text = reader.readLine()) != null ) {lines.add(text);++count;System.out.printf("**** OUTPUT ****text[%d] = %sn", count, text );}assertFalse( lines.isEmpty() );assertEquals("Sherlock Holmes and the Houndsof the Baskervilles", lines.get(0));assertEquals("Da Vinci Code", lines.get(1));assertEquals("Great Expectations", lines.get(2));assertEquals( "Treasure Island", lines.get(3) );}finally {runner.stop();}}} In the unit test method, shouldAssembleAndRetrieveBookList(), we first assemble a virtual web archive with an explicit name test.war. The WAR file contains the RestfulBookService services, the Web deployment descriptor file web.xml and an empty beans.xml file, which if you remember is only there to trigger the CDI container into life for this web application. With the virtual web archive, we export the WAR as a physical file with the utility ZipExporterImpl class from ShinkWrap library, which creates the file test.war in the project root folder. Next, we fire up the SimpleEmbeddedRunner utility. It deploys the web archive to an embedded GlassFish container. Essentially, this is the boilerplate to get to deliver a test result. We, then, get to the heart of the test itself; we construct a URL to invoke the REST style endpoint, which is http://localhost:8080/mywebapp/rest/books. We read the output from the service endpoint with Java standard I/O line by line into a list collection of Strings. Once we have a list of collections, then we assert each line against the expected output from the rest style service. Because we acquired an expensive resource, like an embedded Glassfish container, we are careful to release it, which is the reason, why we surround critical code with a try-finally block statement. When the execution comes to end of test method, we ensure the embedded GlassFish container is shut down.

(For more resources related to this topic, see here.) Getting ready This recipe is going to log a large JSON message to the console, so large that it will actually overflow STS's default console buffer. To ensure that we don't miss anything, we're going to change STS's configuration to have an unlimited console buffer. Open the Preferences dialog by selecting Window | Preferences from the menu if you are running Windows, or by pressing ?+, if you are using a Mac. Open the Run/Debug item and select the Console item. Uncheck the Limit console output checkbox, and then click on the OK button. How to do it... Let's start by creating our project and updating its dependencies. Launch the New Spring Project wizard by navigating to File | New | Spring Project in the menu. In the dialog box, enter the project name WebServiceProject, expand the Integration folder, and select Spring Integration Project (Standalone) – Simple. Click on the Next button. Allow STS to download the project template if you are prompted to do so. Enter the groupId as com.example, artifactId as web-service-project, and top-level package as com.example.webserviceproject. Click on the Finish button to create the project. Open the Maven pom.xml file. Click on the Dependencies tab, and add the following dependencies: org.springframework.integration : spring-integrationhttp : ${spring.integration.version} org.springframework : spring-test : 3.1.3.RELEASE (Spring Integration 2.2.4.RELEASE depends on this version of Spring, and it's a good idea to be consistent) org.codehaus.jackson : jackson-core-asl : 1.9.12 org.codehaus.jackson : jackson-mapper-asl : 1.9.12 Save the file to update the project's dependencies. Now let's create an integration test. Expand the src/test/java source folder, right-click on the top-level package under it, and select New | JUnit Test Case from the pop-up menu. Name the test WebServiceTest, and click on the Finish button. We're creating a Spring context test, so add the following annotations to the top of the class: @ContextConfiguration(locations = "classpath:META-INF/spring/integration/spring-integration-context.xml")@RunWith(SpringJUnit4ClassRunner.class) Our test is going to send a Spring Integration message down one channel and then wait for a response on another channel. We need to inject the following two channels into our test: @Autowired private MessageChannel weatherRequest;@Autowired private SubscribableChannel weatherResponse; Create the following test case: @Testpublic void shouldRetrieveWeatherDataFromWebService() {weatherResponse.subscribe(new MessageHandler() {@Overridepublic void handleMessage(Message<?> message)throws MessagingException {Map payload = (Map) message.getPayload();Map observations = (Map) payload.get("observations");List data = (List) observations.get("data");Map latestData = (Map) data.get(0);assertNotNull(latestData);assertEquals("Sydney - Observatory Hill",latestData.get("name"));assertNotNull(latestData.get("air_temp"));assertNotNull(latestData.get("local_date_time_full"));}});weatherRequest.send(MessageBuilder.withPayload(new WeatherRequest("N", "94768")).build());} Add all required imports using Quick Fixes. The instantiation of the WeatherRequest object will cause a compilation error because the class does not yet exist. Use a Quick Fix to create the class, taking care to change its source folder to WebServiceProject/src/main/java. Create the class as follows: package com.example.webserviceproject;public class WeatherRequest {private final String stateCode;private final String locationId;public WeatherRequest(String stateCode, String locationId) {this.stateCode = stateCode;this.locationId = locationId;}public String getStateCode() {return stateCode;}public String getLocationId() {return locationId;}} Run the test. It will fail, complaining that no Spring beans could be injected into our test. We have our failing test. Now it's time to create our integration pipeline. Open spring-integration-context.xml (found in the src/main/ resources/META-INF/spring/integration folder). Click on the Namespaces tab. You will find that the int namespace is already checked. Check the int-http namespace as well. Click on the integration-graph tab and maximize the editor to give yourself more space. This template project has already created an integration pipeline for us, but we want to create our own, so select all of the components in the workspace area by dragging a box around them, and then press the Delete key to remove them. Expand the Channels category and drag a channel component into the workspace. Double-click on it to bring up its properties in the Properties view. Change its id to weatherRequest. Remember to close the Properties view after editing the properties of each integration component. Drag another channel component into the workspace, and use the Properties view to set its id to weatherServiceResponse. Drag a publish-subscribe-channel component into the workspace. Give it the id weatherResponse. Drag a logging-channel-adapter component into the workspace. Give it the id logger, set its level to INFO, and its logger-name to com.example. webserviceproject.webserviceresponse. Now expand the http category and drag an outbound-gateway component into the workspace. Set its id property to weatherService, its url property to http://www.bom.gov.au/fwo/ID{stateCode}60901/ ID{stateCode}60901.{locationId}.json, its http-method to GET, and expected-response-type to java.lang.String. Expand the Transformation category and drag a json-to-object-transformer component into the workspace. Set its type property to java.util.HashMap. Now we need to connect everything together. Select the connection tool from the palette and make the following connections: weatherRequest channel to weatherService gateway weatherService gateway to weatherServiceResponse channel weatherServiceResponse channel to json-to-object-transformer json-to-object-transformer to weatherResponse channel Select the mapping/recipient/wire-tap tool from the palette and create a connection from the weatherServiceResponse channel to the logging-channel-adapter channel. Our pipeline should now look like the following figure: Select the integration tab. In the Integration Overview panel, expand all of the beans and then right-click on the int-http:outbound-gateway bean. Select Insert <int-http:uri-variable> element from the pop-up menu. This will create a new sub element and show its properties on the right side of editor. Set its name property to stateCode and its expression to payload.stateCode. Create another uri-variable property, setting its name to locationId, and its expression to payload.locationId. And we're done! Save the file, then re-run the WebServiceTest. If all was configured correctly, the test will take a few seconds to run (it has to contact the web service), and then you should be rewarded with a green bar. Maximize the Console view, and you will find the JSON string returned from the web service. How it works... In this recipe, we used a few different Spring Integration components to make a request to a web service and to transform its response to a Map class so that our test could pick out pieces of data. We also used a wire tap component to log messages to the console. Channels We have seen a couple of different channel implementations in action in this recipe. The weatherRequest and weatherService channels are examples of the default Spring Integration channel, DirectChannel , which is a simple point-to-point channel (that is, there is one sender on one side of the channel and one receiver on the other). By contrast, the weatherRequest channel is a PublishSubscribeChannel , which allows for multiple receivers to register for message events (in our test, we only had the one).Spring Integration provides a number of other channel types designed for different requirements. See the Spring Integration Reference Manual ( http://static.springsource.org/spring-integration/reference/htmlsingle ) for further information. URI variables You would have noticed when setting the URL property of the HTTP outbound gateway that the value included two placeholders, namely {stateCode} and {locationId}. We configured a URL variable for each of these, giving them the values payload.stateCode and payload.locationId respectively. These are simply JavaBean property references. In our case, the payload was a WeatherRequest instance, and we had defined both the stateCode and locationId JavaBean properties on that class. The values of those properties are substituted into the placeholders in the configured URL string to determine the actual URL requested. Wire taps and logging When we connected the weatherServiceResponse channel with logging-channel-adapter, under the hood, STS configured a special type of channel interceptor called a wire tap. A wire tap takes messages passing through a channel, and sends a copy to another channel. This makes them excellent for logging and auditing purposes. When we configured our logging-channel-adapter , we told it to log at the INFO level using the logger name com.example.webserviceproject.webserviceresponse . We could have used any name, of course, but the log4j configuration provided in the project template (the log4j.xml file under src/main/resources ) already had an INFO level logging threshold configured for com.example.webserviceproject loggers, so it was convenient for us to choose the logger name that we did. Summary In this article, we retrieved weather data from the Australian Bureau of Meteorology and also learned about how STS and Spring Integration allow us to interact with web services with very little effort. Resources for Article : Further resources on this subject: Vaadin project with Spring and Handling login with Spring [Article] So, what is Spring for Android? [Article] Migration to Spring Security 3 [Article]

(For more resources related to this topic, see here.) An introduction to image filtering Morphological operations and edge detection are actually types of image filtering, even though we used them in a black box sense, without really looking under the hood. Hopefully, this approach will get you accustomed to the details of image filtering a little faster. First of all, let's give a general definition of image filtering; it can be explained as the process of modifying the values of the pixels using a function that is typically applied on a local neighborhood of the image. In many situations, applying the function on a neighborhood involves a special operation, called convolution, with an operand called kernel. In this sense, you have already applied such a process in the case of erosion or dilation and even in the case of edge detection. The former processes used the strel function to create a kernel, while the latter used a kernel based on your choice of the edge detection method. But let's not get ahead of ourselves. We will try to take things one step at a time, starting by explaining neighborhood processing. Processing neighborhoods of pixels In the previous paragraph, we mentioned that the filtering process typically takes place on a specific neighborhood of pixels. When this neighborhood process is applied for all pixels, it is called sliding neighborhood operation. In it, we slide a rectangular neighborhood window through all possible positions of the image and modify its central pixel using a function of the pixels in the neighborhood. Let's see how this is done, using a numeric example. We'll start with something simple, like a linear filtering process, that is, averaging. Let's suppose that we have a small image, sized 8x8 pixels and we want to modify its pixel values, so that they get assigned with the rounded average of the pixels' values in their 3x3 neighborhoods. This will be easier to explain by using a real numeric example. Let's explain what happens in the step shown in the following image, in which the central pixel of the highlighted 3x3 neighborhood (in the fourth row and sixth column) will be replaced by the average value of all the pixels in the neighborhood (rounded to the nearest integer): Let the image be called I, the result in pixel I(4,6) will be: Substituting the values of the pixels, we can calculate the average value: Hence, the value of the central pixel of the neighborhood will become 121 (the closest integer to 120.89). By repeating the process described previously for all the pixels of the image, we get a result commonly known as mean filtering or average filtering. The final result of the entire process is shown in the following figure: You may be wondering now; the choice of neighborhood, for the example, was very convenient, but what happens when we want to change the value of a pixel on the borders of the image such as let's say pixel I(1,4)? Why was it set to 77 as shown in the image? This is indeed a valid and natural question, and you are very intuitive if you already thought about it. The answer is that the way to tackle this problem when you want your resulting image to be the same size as your original image is to involve only the neighboring pixels that exist in your calculations. However, since in our example, the calculation that has to be performed is averaging the neighborhood pixels, the denominator will still be 9, hence, it will be like we pad the rest of the neighborhood with zeros. Let's demonstrate this example as well: As shown in the previous image, the central pixel value gets evaluated as follows: Of course, since there is no 0th line, the first three operands of the addition are non-existent, hence set to zero: Therefore, the result of the averaging process for the aforementioned neighborhood will be equal to 77 (as shown in the image). This approach is not the only one we have for the image borders. We could assign the maximum possible value (255 for our example) to the non-existent pixels, or assign them the mean value of the rest of the neighborhood, and so on. The choice we make affects the quality of the borders of the image, as we will see in real pictures later on. The basics of convolution The process described previously was performed in overlapping neighborhoods of the image, but no use of a kernel was mentioned. So, what is this all about? And how does the convolution fit in this framework? Well, the truth is that the process described previously is actually describing the essence of convolution, which is passing a kernel over all possible equally sized neighborhoods of the image and using it to modify the value of the central pixel. The only problem in our case is that we did not use a specific kernel in the process described. Or did we? Let's try to find out using MATLAB code to perform two-dimensional convolution. The 3x3 neighborhood we used for the described process can be replaced by a 3x3 kernel, as long as the final result remains the same. The kernel that accomplishes this effect is a 3x3 matrix with all pixels set to 1/9. Convolving this kernel with the original image produces the same result as the aforementioned example. To demonstrate the process, we can use the two-dimensional convolution MATLAB function conv2 as follows, to get the result: >> original = [132 101 101 107 115 121 110 92 120 124 122 120 129 123 121 129 134 146 144 134 134 132 134 138 143 147 136 121 121 115 107 107 145 147 138 129 119 113 113 122 162 155 152 149 142 129 118 122 127 122 115 113 117 102 95 94 67 74 78 80 89 89 107 109]; % Create original image >> kernel = ones(3,3)*(1/9); % Create kernel >> conv_result = conv2(original, kernel,'same'); % Perform convolution >> final_result = round(conv_result) % Rounding of result The final result obtained is as follows: final_result = 53 78 75 77 79 80 77 50 84 125 122 123 124 124 122 80 90 135 133 129 125 124 123 82 96 142 138 131 124 121 120 80 100 147 142 134 126 120 116 77 95 140 136 130 124 116 112 74 79 117 115 115 112 110 107 72 43 65 65 66 66 67 66 45 As expected, the result is the same as the one calculated using the analytical process described before. The convolution kernel has done its job. In our process, we used a 8x8 original image and a 3x3 kernel with the values of all pixels as 1/9 (this is what happens when you get a 3x3 matrix with all instances of 1 and multiply it by 1/9, as we did) and finally ordered the conv2 function to produce the result using the padding process described earlier for the borders, hence calculating a result with the same dimensions as the original. But how did it do it? What exactly is convolution? Now it is time to fully understand convolution. But first, you must get acquainted with its mathematical equations. Since learning math is not the purpose of this book, we will try to give you just the basics, so that you get an idea of what this operation is all about, as it is invaluable for image filtering. The ugly mathematical truth Let's start with the mathematical definition of convolution for discrete functions (since in digital image processing all functions are discrete). To form our problem in a signal processing sense, we can define it as passing an input image I, through a Linear Space Invariant (LSI) system, performing convolution with a kernel h (also called a filter), to produce an output image, g. Hence, we get the following block diagram: This process is described mathematically by the following equation: where * is the symbol for convolution and the large Σ denotes a sum. The reason we have two sums is because our process is two-dimensional. Without going into too much detail, we can summarize the process described previously using the following steps, which are also followed in the implementation of conv2: Rotate the convolution kernel by 180 degrees to abide by the process in the double sum of the equation. Determine the central pixel of the neighborhood. This is straightforward when the neighborhood has an odd number of rows and columns, but must be based on some rule if either of the dimensions is even. Apply the rotated kernel to each pixel of the input image. This is a multiplication of each pixel in the rotated kernel by the corresponding pixel on the image neighborhood processed. It can be thought of as the weighted sum of the neighborhood pixels. The result of conv2 can be either of the following choices: full: Larger than the original image, taking into account all the pixels that can be computed using the convolution kernel, even if their center falls out of the image. This is the default choice for the function. same: Same size as the original image, using zeros to calculate border pixel values. valid: Smaller than the original image, so that it uses only pixels that have full valid neighbors in the computations. This means that when you want to produce a convolution result with the same size as the original image, you will have to use same as an input, as we did in our previous example. By now, those of you that are not very much into math may be tempted to stop reading. So, let's stop the mathematical jargon and dive into the practical examples. We know what a convolution does and we have seen an example on the pixels of a very small image, using an averaging convolution kernel. So, what does this process really do to an image? Time for action – applying averaging filters in images We will start off with an easy-to-follow example, so that all the theory described previously is demonstrated. In this example, we will also introduce some new MATLAB functions, to facilitate your understanding. Let's start: First, we load our image, which is holiday_image2.bmp: >> img = imread('holiday_image2.bmp'); Then, we generate our convolution kernel, using function fspecial and then rotate it 180 degrees: >> kernel = fspecial('average',3); >> kernel = rot90(kernel,2) The output of the code will be as follows: kernel = 0.1111 0.1111 0.1111 0.1111 0.1111 0.1111 0.1111 0.1111 0.1111 Now, it is time to use the three different ways of convolving our image: >> con1 = conv2(img,kernel); % Default usage ('full') >> con2 = conv2(img,kernel,'same'); % convolution using 'same' >> con3 = conv2(img,kernel,'valid'); % convolution using 'valid' In the previous step, you probably got a warning saying: Warning: CONV2 on values of class UINT8 is obsolete. Use CONV2(DOUBLE(A),DOUBLE(B)) or CONV2(SINGLE(A),SINGLE(B)) instead. This actually means that UNIT8 type will not be supported by conv2 in the future. To be on the safe side, you might want to use the suggestion by MATLAB and convert your image to single prior to convolving it: >> img = single(img); >> kernel = fspecial('average',3); % Create 3x3 averaging kernel >> con1 = conv2(img,kernel); % Default usage ('full') >> con2 = conv2(img,kernel,'same'); % convolution using 'same' >> con3 = conv2(img,kernel,'valid'); % convolution using 'valid' Now, we can show our results in one figure, along with the original image. This time, we are going to use an empty matrix as the second argument in imshow, to avoid having to convert our results to UNIT8: >> figure;subplot(2,2,1),imshow(img,[]),title('Original') >> subplot(2,2,2),imshow(con1,[]),title('full') >> subplot(2,2,3),imshow(con2,[]),title('same') >> subplot(2,2,4),imshow(con3,[]),title('valid') It is obvious that the three results are identical, but there is a small detail. Their size is not. So let's see if we got what we expected. In the Workspace window, you can see the difference in sizes: Let's now discuss the physical, qualitative meaning of averaging an image. What does it exactly do? The answer is; it performs blurring of the image. To examine this effect, we can crop the tower from our original and averaged image and display the result. The tower can be cropped using the following coordinates: >> tower_original = img(51:210,321:440); >> tower_blurred = con2(51:210,321:440); figure >> subplot(1,2,1),imshow(tower_original),title('Original tower') >> subplot(1,2,2),imshow(tower_blurred),title('Blurred tower') The original image and the blurred image are as follows: What just happened? The process described in the previous example demonstrated the usage of convolution in its various implementations, using the averaging kernel produced using fspecial. This function is designed to generate kernels for popular filtering tasks, as we will further analyze in the following sections. In our case, we created a 3x3 kernel of values equal to 1/9 (which is almost equal to 0.1111, hence the result in step 2). Then, the three different choices of convolution were applied and the results were displayed along with the original image. Of course, a detail such as the size of the borders cannot be easily observed in full scale, so we observed the difference in the sizes of the results. Finally, we displayed a part of the original image next to the same part of the same convolution result, to prove that the result of the averaging process is a blurring of the image. Alternatives to convolution Convolution is not the only way to perform image filtering. There is also correlation, which gives us the same result. Filtering an image using correlation can be accomplished by using the MATLAB function called filter2, which performs, as its name implies, a two-dimensional filtering of two images. The first input in this case is a kernel (filter) and the second input is an image (or in a more general case a two-dimensional matrix). We will not go into detail here, just point out that one main difference between the two methods is that correlation does not need the kernel to be rotated. The border issue remains, having the same three approaches as in the case of convolution using conv2. A demonstration on the equivalence of the two functions is given if we type in the following commands: >> img = imread('holiday_image2.bmp'); >> img = img(51:210,321:440); >> kernel = fspecial('average',3); >> kernel180 = rot90(kernel,3); >> conv_result = conv2(img,kernel180,'same'); >> corr_result = filter2(kernel,img,'same'); >> subplot(1,3,1),imshow(img),title('Original') >> subplot(1,3,2),imshow(uint8(conv_result)),title('Blurred - conv2') >> subplot(1,3,3),imshow(uint8(corr_result)),title('Blurred - filter2') The result of the preceding code is displayed as follows: In our example, the two kernels used for conv2 and filter2 are identical, since the averaging filter used is square (3x3) and all its elements are equal. The generalized process shown will be useful when we have a more complex kernel. Using imfilter The two alternative solutions for performing image filtering presented so far have their origin in general two-dimensional signal processing theory. This means that they should be expanded for three-dimensional signals when we have to deal with colored image filtering. The process is pretty straightforward and involves repeating the process for all three separate colored channels. But why do that, when we have a function that takes care of checking the image before applying the filter and then selecting the correct method? This specialized function is called imfilter and it is designed for handling images, regardless if they are grayscale or color. This function can implement both filtering methods described in previous paragraphs and it can also define the result to be same or full. Its extra functionality comes in the selection of the way it handles boundary values, and the automatic processing of color images. Furthermore, this function performs the needed conversions, in case the image input is integer-valued. Combined with the fspecial function, this will probably be your most valuable tool in MATLAB when it comes to image filtering.

(For more resources related to this topic, see here.) The NetBeans IDE also supports C/C++, PHP, HTML5, CSS3, JavaScript, Groovy, and Ruby. Nowadays nearly all the companies are using NetBeans as the basic tool for the development. And it is very popular in all industrial segments. NetBeans IDE 6.9 introduces the JavaFX Composer, support for JavaFX SDK 1.3, OSGi interoperability, support for the PHP Zend framework and Ruby on Rails 3.0, and more. “NetBeans” basically stands for “Network JavaBeans”. The journey of NetBeans started in 1996. NetBeans started as a final year student project and group of seven students started coding in C++ later they switched on to Java 1.0.2 towards framing NetBeans. Initially it was named as “Xelfi” (meaning Delphi for UNIX). After completing their studies, they started selling it on the internet for approximately $20 USD as the Shareware and got successful. In 1997, Roman Stanek who was a Czech entrepreneur founded NetBeans and started selling Xelfi with the name of NetBeans IDE now. Later on during Java One in 1998, NetBeans was presented in front of Java Community and Sun Microsystems acquired NetBeans. Jaroslav Tulach (alias Yarda Tulach) is one of the founders of NetBeans is now working with Oracle to improve NetBeans IDE and to make it more and more powerful but performance driven IDE for specially for different kind of Java application development, it is his and his teams efforts that NetBeans is the most powerful IDE for Java application development and people are able to create, run, deploy Java applications so easily. I don’t understand that how people say that Java has so many restrictions, it is not easy to develop Java applications, don’t have drag-drop features for GUI based Java applications. People who still says such things, I personally feel that they are still leaving in stone-age. NetBeans gives you everything you require to design, develop, run and deploy any Java application. One can develop any Java application as we have already mentioned above and it gives you visual designer for Java Desktop applications, Web applications, Mobile applications, etc. Layout of NetBeans IDE : Launch the NetBeans IDE and create a DemoProject i.e. simple java application to understand detailed layout of the IDE. Here is the snapshot of NetBeans IDE 7.2. Use steps here to get following output : Launch NetBeans IDE Create a simple java project as we have studied before named “DemoProject” with a Main.java which will help you to understand the layout in detail. It will open IDE with different child windows as shown in below figure. Layout of NetBeans As shown in figure above, Layout of NetBeans is divided into different small windows as follows: Project Window: Open projects and source code will be listed here in different tab for each source file. Editor Window: In Editor Window, Source tab allows you to write, generate code and business logics as per the suggestions provided by NetBeans IDE. Whereas Design tab allows you to design windows, forms, panels, dialogs, etc. with the visual designer feature provided by Design tab in NetBeans. Design tab also allows you to customize the menus and components added to the Editor area and Design tab has a great feature of drag and drop. Properties Window: Allows you to display and customize the properties of components selected in Design tab. Palette Window: List and display the windows, dialogs, menu bar, tool bar, components, etc. available for respective technology and to drag and drop. Navigator Window: Displays the hierarchical structure for all the components like class, functions or methods, objects, variables, text field, label, etc. i.e. visual and non-visual components. Output Window: Displays the output and other messages from the NetBeans, also displays logs, etc. Every child window in NetBeans IDE is designed with purpose which helps developers to reduce the amount of time required for development, their efforts, chances of errors, bugs, etc.The Properties Window, Palette Window and Output Window are hidden by default and are not visible by default. To open Properties Window : Go to Window | Properties or use shortcut Ctrl + Shift + 7 To open Palette Window : Go to Window | Palette or use shortcut Ctrl + Shift + 8 To open Output Window : Go to Window | Output | Output or use shortcut Ctrl + 4 UI Builder in NetBeans IDE The UI Builder is further divided into three main tabs: Design Tab : The Design tab provides Editor area where you get visual view of the components where you can add components directly from palette; place, align, resize components. Source Tab :The Source tab allows you to see the source code you have written and generated after the use of Design tab. History Tab :The History tab provides the details about the changes made in the file. UI Builder Here we have created a blank java application, created package and then we have added a JFrame form which provides Editor Area as shown in fig. above. UI Builder in NetBeans IDE allows you to drag and drop components you need from categories specified in Palette to the Canvas and design the UI as per requirements. Before we start designing projects, we will have a look at some design related concepts and some tips which may help you in designing great applications using NetBeans IDE. Free Form Design : With the Editor area in design tab, you can go on adding components freely and place the components here and there wherever you feel. Placement of Components : Since the Editor area allows you to add components the place you need, it also provides hints, using these hints you can automatically place components. And the code will be generated automatically. Visual Help : Design tab visually helps you a lot by providing hints, anchors, insets, offsets, fills, etc. with which you can design professional UIs very easily. Shortcuts to NetBeans : Even if you are new to NetBeans, you can design and develop applications like a professional or an expert by using the various shortcuts available in NetBeans IDE. For shortcuts, Go to Help | Keyboard Shortcuts Card. It will open PDF file containing shortcuts applicable to respective version of the NetBeans IDE. The file will contain the various shortcuts related to coding, debugging, compiling, testing, running, navigating through the source code, etc. Summary Helps you understand the layout of NetBeans IDE and how easily we can design projects in NetBeans IDE. Resources for Article: Further resources on this subject: Getting Started with NetBeans [Article] NetBeans IDE 7: Building an EJB Application [Article] iReport in NetBeans [Article]