Programming | 1081 articles | Tech News, Tutorials & Expert Insights

article-image-setting-software-infrastructure-cloud

23 Sep 2014

42 min read

Setting up of Software Infrastructure on the Cloud

23 Sep 2014

0
0
1996

Packt

19 Sep 2014

5 min read

Event-driven BPEL Process

Packt

19 Sep 2014

5 min read

0
0
1301

Packt

18 Sep 2014

8 min read

Redis in Autosuggest

Packt

18 Sep 2014

8 min read

0
0
3301

Packt

18 Sep 2014

6 min read

Waiting for AJAX, as always…

Packt

18 Sep 2014

6 min read

In this article, by Dima Kovalenko, author of the book, Selenium Design Patterns and Best Practices, we will learn how test automation have progressed over the period of time. Test automation was simpler in the good old days, before asynchronous page loading became mainstream. Previously the test would click on a button causing the whole page to reload; after the new page load we could check if any errors were displayed. The act of waiting for the page to load guaranteed that all of the items on the page are already there, and if expected element was missing our test could fail with confidence. Now, an element might be missing for several seconds, and magically show up after an unspecified delay. The only thing for a test to do is become smarter! (For more resources related to this topic, see here.) Filling out credit card information is a common test for any online store. Let’s take a look at a typical credit card form: Our form has some default values for user to fill out, and a quick JavaScript check that the required information was entered into the field, by adding Done next to a filled out input field, like this: Once all of the fields have been filled out and seem correct, JavaScript makes the Purchase button clickable. Clicking on the button will trigger an AJAX request for the purchase, followed by successful purchase message, like this: Very simple and straight forward, anyone who has made an online purchase has seen some variation of this form. Writing a quick test to fill out the form and make sure the purchase is complete should be a breeze! Testing AJAX with sleep method Let’s take a look at a simple test, written to test this form. Our tests are written in Ruby for this demonstration for easy of readability. However, this technique will work in Java or any other programming language you may choose to use. To follow along with this article, please make sure you have Ruby and selenium-webdriver gem installed. Installers for both can be found here https://www.ruby-lang.org/en/installation/ and http://rubygems.org/gems/selenium-webdriver. Our test file starts like this: If this code looks like a foreign language to you, don’t worry we will walk through it until it all makes sense. First three lines of the test file specify all of the dependencies such as selenium-webdriver gem. On line five, we declare our test class as TestAjax which inherits its behavior from the Test::Unit framework we required on line two. The setup and teardown methods will take care of the Selenium instance for us. In the setup we create a new instance of Firefox browser and navigate to a page, which contains the mentioned form; the teardown method closes the browser after the test is complete. Now let’s look at the test itself: Lines 17 to 21 fill out the purchase form with some test data, followed by an assertion that Purchase complete! text appears in the DIV with ID of success. Let’s run this test to see if it passes. The following is the output result of our test run; as you can see it’s a failure: Our test fails because it was expecting to see Purchase complete! right here: But no text was found, because the AJAX request took a much longer time than expected. The AJAX request in progress indicator is seen here: Since this AJAX request can take anywhere from 15 to 30 seconds to complete, the most logical next step is to add a pause in between the click on the Purchase button and the test assertion; shown as follows: However, this obvious solution is really bad for two reasons: If majority of AJAX requests take 15 seconds to run, than our test is wasting another 15 seconds waiting for things instead continuing. If our test environment is under heavy load, the AJAX request can take as long as 45 seconds to complete, so our test will fail. The better choice is to make our tests smart enough to wait for AJAX request to complete, instead of using a sleep method. using smart AJAX waits To solve the shortcomings of the sleep methods we will create a new method called wait_for_ajax, seen here: In this method, we use the Wait class built into the WebDriver. The until method in the Wait class allows us to pause the test execution for an arbitrary reason. In this case to sleep for 1 second, on line 29, and to execute a JavaScript command in the browser with the help of the execute_script method. This method allows us to run a JavaScript snippet in the current browser window on the current page, which gives us access to all of the variables and methods that JavaScript has. The snippet of JavaScript that we are sending to the browser is a query against jQuery framework. The active method in jQuery returns an integer of currently active AJAX requests. Zero means that the page is fully loaded, and there are no background HTTP requests happening. On line 30, we ask the execute_script to return the current active count of AJAX requests happening on the page, and if the returned value equals 0 we break out of the Wait loop. Once the loop is broken, our tests can continue on their way. Note that the upper limit of the wait_for_ajax method is set to 60 seconds on line 28. This value can be increased or decreased, depending on how slow the test environment is. Let’s replace the sleep method call with our newly created method, shown here: And run our tests one more time, to see this passing result: Now that we stabilized our test against slow and unpredictable AJAX requests, we need to add a method that will wait for JavaScript animations to finish. These animations can break our tests just as much as the AJAX requests. Also, are tests are incredibly vulnerable due third party slowness; such as when the Facebook Like button takes long time to load. Summary This article introduced you to using a simple method that intelligently waits for all of the AJAX requests to complete, we have increased the overall stability of our test and test suite. Furthermore, we have removed a wasteful delay, which adds unnecessary delay in our test execution. In conclusion, we have improved the test stability while at the same time making our test run faster! Resources for Article: Further resources on this subject: Quick Start into Selenium Tests [article] Behavior-driven Development with Selenium WebDriver [article] Exploring Advanced Interactions of WebDriver [article]

0
0
825

article-image-index-item-sharding-and-projection-dynamodb

Packt

17 Sep 2014

13 min read

Index, Item Sharding, and Projection in DynamoDB

Packt

17 Sep 2014

13 min read

Understanding the secondary index and projections should go hand in hand because of the fact that a secondary index cannot be used efficiently without specifying projection. In this article by Uchit Vyas and Prabhakaran Kuppusamy, authors of DynamoDB Applied Design Patterns, we will take a look at local and global secondary indexes, and projection and its usage with indexes. (For more resources related to this topic, see here.) The use of projection in DynamoDB is pretty much similar to that of traditional databases. However, here are a few things to watch out for: Whenever a DynamoDB table is created, it is mandatory to create a primary key, which can be of a simple type (hash type), or it can be of a complex type (hash and range key). For the specified primary key, an index will be created (we call this index the primary index). Along with this primary key index, the user is allowed to create up to five secondary indexes per table. There are two kinds of secondary index. The first is a local secondary index (in which the hash key of the index must be the same as that of the table) and the second is the global secondary index (in which the hash key can be any field). In both of these secondary index types, the range key can be a field that the user needs to create an index for. Secondary indexes A quick question: while writing a query in any database, keeping the primary key field as part of the query (especially in the where condition) will return results much faster compared to the other way. Why? This is because of the fact that an index will be created automatically in most of the databases for the primary key field. This the case with DynamoDB also. This index is called the primary index of the table. There is no customization possible using the primary index, so the primary index is seldom discussed. In order to make retrieval faster, the frequently-retrieved attributes need to be made as part of the index. However, a DynamoDB table can have only one primary index and the index can have a maximum of two attributes (hash and range key). So for faster retrieval, the user should be given privileges to create user-defined indexes. This index, which is created by the user, is called the secondary index. Similar to the table key schema, the secondary index also has a key schema. Based on the key schema attributes, the secondary index can be either a local or global secondary index. Whenever a secondary index is created, during every item insertion, the items in the index will be rearranged. This rearrangement will happen for each item insertion into the table, provided the item contains both the index's hash and range key attribute. Projection Once we have an understanding of the secondary index, we are all set to learn about projection. While creating the secondary index, it is mandatory to specify the hash and range attributes based on which the index is created. Apart from these two attributes, if the query wants one or more attribute (assuming that none of these attributes are projected into the index), then DynamoDB will scan the entire table. This will consume a lot of throughput capacity and will have comparatively higher latency. The following is the table (with some data) that is used to store book information: Here are few more details about the table: The BookTitle attribute is the hash key of the table and local secondary index The Edition attribute is the range key of the table The PubDate attribute is the range key of the index (let's call this index IDX_PubDate) Local secondary index While creating the secondary index, the hash and range key of the table and index will be inserted into the index; optionally, the user can specify what other attributes need to be added. There are three kinds of projection possible in DynamoDB: KEYS_ONLY: Using this, the index consists of the hash and range key values of the table and index INCLUDE: Using this, the index consists of attributes in KEYS_ONLY plus other non-key attributes that we specify ALL: Using this, the index consists of all of the attributes from the source table The following code shows the creation of a local secondary index named Idx_PubDate with BookTitle as the hash key (which is a must in the case of a local secondary index), PubDate as the range key, and using the KEYS_ONLY projection: private static LocalSecondaryIndex getLocalSecondaryIndex() { ArrayList<KeySchemaElement> indexKeySchema = newArrayList<KeySchemaElement>(); indexKeySchema.add(new KeySchemaElement() .withAttributeName("BookTitle") .withKeyType(KeyType.HASH)); indexKeySchema.add(new KeySchemaElement() .withAttributeName("PubDate") .withKeyType(KeyType.RANGE)); LocalSecondaryIndex lsi = new LocalSecondaryIndex() .withIndexName("Idx_PubDate") .withKeySchema(indexKeySchema) .withProjection(new Projection() .withProjectionType("KEYS_ONLY")); return lsi; } The usage of the KEYS_ONLY index type will create the smallest possible index and the usage of ALL will create the biggest possible index. We will discuss the trade-offs between these index types a little later. Going back to our example, let us assume that we are using the KEYS_ONLY index type, so none of the attributes (other than the previous three key attributes) are projected into the index. So the index will look as follows: You may notice that the row order of the index is almost the same as that of the table order (except the second and third rows). Here, you can observe one point: the table records will be grouped primarily based on the hash key, and then the records that have the same hash key will be ordered based on the range key of the index. In the case of the index, even though the table's range key is part of the index attribute, it will not play any role in the ordering (only the index's hash and range keys will take part in the ordering). There is a negative in this approach. If the user is writing a query using this index to fetch BookTitle and Publisher with PubDate as 28-Dec-2008, then what happens? Will DynamoDB complain that the Publisher attribute is not projected into the index? The answer is no. The reason is that even though Publisher is not projected into the index, we can still retrieve it using the secondary index. However, retrieving a nonprojected attribute will scan the entire table. So if we are sure that certain attributes need to be fetched frequently, then we must project it into the index; otherwise, it will consume a large number of capacity units and retrieval will be much slower as well. One more question: if the user is writing a query using the local secondary index to fetch BookTitle and Publisher with PubDate as 28-Dec-2008, then what happens? Will DynamoDB complain that the PubDate attribute is not part of the primary key and hence queries are not allowed on nonprimary key attributes? The answer is no. It is a rule of thumb that we can write queries on the secondary index attributes. It is possible to include nonprimary key attributes as part of the query, but these attributes must at least be key attributes of the index. The following code shows how to add non-key attributes to the secondary index's projection: private static Projection getProjectionWithNonKeyAttr() { Projection projection = new Projection() .withProjectionType(ProjectionType.INCLUDE); ArrayList<String> nonKeyAttributes = new ArrayList<String>(); nonKeyAttributes.add("Language"); nonKeyAttributes.add("Author2"); projection.setNonKeyAttributes(nonKeyAttributes); return projection; } There is a slight limitation with the local secondary index. If we write a query on a non-key (both table and index) attribute, then internally DynamoDB might need to scan the entire table; this is inefficient. For example, consider a situation in which we need to retrieve the number of editions of the books in each and every language. Since both of the attributes are non-key, even if we create a local secondary index with either of the attributes (Edition and Language), the query will still result in a scan operation on the entire table. Global secondary index A problem arises here: is there any way in which we can create a secondary index using both the index keys that are different from the table's primary keys? The answer is the global secondary index. The following code shows how to create the global secondary index for this scenario: private static GlobalSecondaryIndex getGlobalSecondaryIndex() { GlobalSecondaryIndex gsi = new GlobalSecondaryIndex() .withIndexName("Idx_Pub_Edtn") .withProvisionedThroughput(new ProvisionedThroughput() .withReadCapacityUnits((long) 1) .withWriteCapacityUnits((long) 1)) .withProjection(newProjection().withProjectionType ("KEYS_ONLY")); ArrayList<KeySchemaElement> indexKeySchema1 = newArrayList<KeySchemaElement>(); indexKeySchema1.add(new KeySchemaElement() .withAttributeName("Language") .withKeyType(KeyType.HASH)); indexKeySchema1.add(new KeySchemaElement() .withAttributeName("Edition") .withKeyType(KeyType.RANGE)); gsi.setKeySchema(indexKeySchema1); return gsi; } While deciding the attributes to be projected into a global secondary index, there are trade-offs we must consider between provisioned throughput and storage costs. A few of these are listed as follows: If our application doesn't need to query a table so often and it performs frequent writes or updates against the data in the table, then we must consider projecting the KEYS_ONLY attributes. The global secondary index will be minimum size, but it will still be available when required for the query activity. The smaller the index, the cheaper the cost to store it and our write costs will be cheaper too. If we need to access only those few attributes that have the lowest possible latency, then we must project only those (lesser) attributes into a global secondary index. If we need to access almost all of the non-key attributes of the DynamoDB table on a frequent basis, we can project these attributes (even the entire table) into the global secondary index. This will give us maximum flexibility with the trade-off that our storage cost would increase, or even double if we project the entire table's attributes into the index. The additional storage costs to store the global secondary index might equalize the cost of performing frequent table scans. If our application will frequently retrieve some non-key attributes, we must consider projecting these non-key attributes into the global secondary index. Item sharding Sharding, also called horizontal partitioning, is a technique in which rows are distributed among the database servers to perform queries faster. In the case of sharding, a hash operation will be performed on the table rows (mostly on one of the columns) and, based on the hash operation output, the rows will be grouped and sent to the proper database server. Take a look at the following diagram: As shown in the previous diagram, if all the table data (only four rows and one column are shown for illustration purpose) is stored in a single database server, the read and write operations will become slower and the server that has the frequently accessed table data will work more compared to the server storing the table data that is not accessed frequently. The following diagram shows the advantage of sharding over a multitable, multiserver database environment: In the previous diagram, two tables (Tbl_Places and Tbl_Sports) are shown on the left-hand side with four sample rows of data (Austria.. means only the first column of the first item is illustrated and all other fields are represented by ..).We are going to perform a hash operation on the first column only. In DynamoDB, this hashing will be performed automatically. Once the hashing is done, similar hash rows will be saved automatically in different servers (if necessary) to satisfy the specified provisioned throughput capacity. Have you ever wondered about the importance of the hash type key while creating a table (which is mandatory)? Of course we all know the importance of the range key and what it does. It simply sorts items based on the range key value. So far, we might have been thinking that the range key is more important than the hash key. If you think that way, then you may be correct, provided we neither need our table to be provisioned faster nor do we need to create any partitions for our table. As long as the table data is smaller, the importance of the hash key will be realized only while writing a query operation. However, once the table grows, in order to satisfy the same provision throughput, DynamoDB needs to partition our table data based on this hash key (as shown in the previous diagram). This partitioning of table items based on the hash key attribute is called sharding. It means the partitions are created by splitting items and not attributes. This is the reason why a query that has the hash key (of table and index) retrieves items much faster. Since the number of partitions is managed automatically by DynamoDB, we cannot just hope for things to work fine. We also need to keep certain things in mind, for example, the hash key attribute should have more distinct values. To simplify, it is not advisable to put binary values (such as Yes or No, Present or Past or Future, and so on) into the hash key attributes, thereby restricting the number of partitions. If our hash key attribute has either Yes or No values in all the items, then DynamoDB can create only a maximum of two partitions; therefore, the specified provisioned throughput cannot be achieved. Just consider that we have created a table called Tbl_Sports with a provisioned throughput capacity of 10, and then we put 10 items into the table. Assuming that only a single partition is created, we are able to retrieve 10 items per second. After a point of time, we put 10 more items into the table. DynamoDB will create another partition (by hashing over the hash key), thereby satisfying the provisioned throughput capacity. There is a formula taken from the AWS site: Total provisioned throughput/partitions = throughput per partition OR No. of partitions = Total provisioned throughput/throughput per partition In order to satisfy throughput capacity, the other parameters will be automatically managed by DynamoDB. Summary In this article, we saw what the local and global secondary indexes are. We walked through projection and its usage with indexes. Resources for Article: Further resources on this subject: Comparative Study of NoSQL Products [Article] Ruby with MongoDB for Web Development [Article] Amazon DynamoDB - Modelling relationships, Error handling [Article]

0
0
5324

article-image-security-settings-salesforce

Packt

11 Sep 2014

10 min read

Security Settings in Salesforce

Packt

11 Sep 2014

10 min read

0
0
1047

article-image-introducing-llvm-intermediate-representation

Packt

26 Aug 2014

18 min read

Introducing LLVM Intermediate Representation

Packt

26 Aug 2014

18 min read

In this article by Bruno Cardoso Lopez and Rafael Auler, the authors of Getting Started with LLVM Core Libraries, we will look into some basic concepts of the LLVM intermediate representation (IR). (For more resources related to this topic, see here.) LLVM IR is the backbone that connects frontends and backends, allowing LLVM to parse multiple source languages and generate code to multiple targets. Frontends produce the IR, while backends consume it. The IR is also the point where the majority of LLVM target-independent optimizations takes place. Overview The choice of the compiler IR is a very important decision. It determines how much information the optimizations will have to make the code run faster. On one hand, a very high-level IR allows optimizers to extract the original source code intent with ease. On the other hand, a low-level IR allows the compiler to generate code tuned for a particular hardware more easily. The more information you have about the target machine, the more opportunities you have to explore machine idiosyncrasies. Moreover, the task at lower levels must be done with care. As the compiler translates the program to a representation that is closer to machine instructions, it becomes increasingly difficult to map program fragments to the original source code. Furthermore, if the compiler design is exaggerated using a representation that represents a specific target machine very closely, it becomes awkward to generate code for other machines that have different constructs. This design trade-off has led to different choices among compilers. Some compilers, for instance, do not support code generation for multiple targets and focus on only one machine architecture. This enables them to use specialized IRs throughout their entire pipeline that make the compiler efficient with respect to a single architecture, which is the case of the Intel C++ Compiler (icc). However, writing compilers that generate code for a single architecture is an expensive solution if you aim to support multiple targets. In these cases, it is unfeasible to write a different compiler for each architecture, and it is best to design a single compiler that performs well on a variety of targets, which is the goal of compilers such as GCC and LLVM. For these projects, called retargetable compilers, there are substantially more challenges to coordinate the code generation for multiple targets. The key to minimizing the effort to build a retargetable compiler lies in using a common IR, the point where different backends share the same understanding about the source program to translate it to a divergent set of machines. Using a common IR, it is possible to share a set of target-independent optimizations among multiple backends, but this puts pressure on the designer to raise the level of the common IR to not overrepresent a single machine. Since working at higher levels precludes the compiler from exploring target-specific trickery, a good retargetable compiler also employs other IRs to perform optimizations at different, lower levels. The LLVM project started with an IR that operated at a lower level than the Java bytecode, thus, the initial acronym was Low Level Virtual Machine. The idea was to explore low-level optimization opportunities and employ link-time optimizations. The link-time optimizations were made possible by writing the IR to disk, as in a bytecode. The bytecode allows the user to amalgamate multiple modules in the same file and then apply interprocedural optimizations. In this way, the optimizations will act on multiple compilation units as if they were in the same module. LLVM, nowadays, is neither a Java competitor nor a virtual machine, and it has other intermediate representations to achieve efficiency. For example, besides the LLVM IR, which is the common IR where target-independent optimizations work, each backend may apply target-dependent optimizations when the program is represented with the MachineFunction and MachineInstr classes. These classes represent the program using target-machine instructions. On the other hand, the Function and Instruction classes are, by far, the most important ones because they represent the common IR that is shared across multiple targets. This intermediate representation is mostly target-independent (but not entirely) and the official LLVM intermediate representation. To avoid confusion, while LLVM has other levels to represent a program, which technically makes them IRs as well, we do not refer to them as LLVM IRs; however, we reserve this name for the official, common intermediate representation by the Instruction class, among others. This terminology is also adopted by the LLVM documentation. The LLVM project started as a set of tools that orbit around the LLVM IR, which justifies the maturity of the optimizers and the number of optimizers that act at this level. This IR has three equivalent forms: An in-memory representation (the Instruction class, among others) An on-disk representation that is encoded in a space-efficient form (the bitcode files) An on-disk representation in a human-readable text form (the LLVM assembly files) LLVM provides tools and libraries that allow you to manipulate and handle the IR in all forms. Hence, these tools can transform the IR back and forth, from memory to disk as well as apply optimizations, as illustrated in the following diagram: Understanding the LLVM IR target dependency The LLVM IR is designed to be as target-independent as possible, but it still conveys some target-specific aspects. Most people blame the C/C++ language for its inherent, target-dependent nature. To understand this, consider that when you use standard C headers in a Linux system, for instance, your program implicitly imports some header files from the bits Linux headers folder. This folder contains target-dependent header files, including macro definitions that constrain some entities to have a particular type that matches what the syscalls of this kernel-machine expect. Afterwards, when the frontend parses your source code, it needs to also use different sizes for int, for example, depending on the intended target machine where this code will run. Therefore, both library headers and C types are already target-dependent, which makes it challenging to generate an IR that can later be translated to a different target. If you consider only the target-dependent, C standard library headers, the parsed AST for a given compilation unit is already target-dependent, even before the translation to the LLVM IR. Furthermore, the frontend generates IR code using type sizes, calling conventions, and special library calls that match the ones defined by each target ABI. Still, the LLVM IR is quite versatile and is able to cope with distinct targets in an abstract way. Exercising basic tools to manipulate the IR formats We mention that the LLVM IR can be stored on disk in two formats: bitcode and assembly text. We will now learn how to use them. Consider the sum.c source code: int sum(int a, int b) { return a+b; } To make Clang generate the bitcode, you can use the following command: $ clang sum.c -emit-llvm -c -o sum.bc To generate the assembly representation, you can use the following command: $ clang sum.c -emit-llvm -S -c -o sum.ll You can also assemble the LLVM IR assembly text, which will create a bitcode: $ llvm-as sum.ll -o sum.bc To convert from bitcode to IR assembly, which is the opposite, you can use the disassembler: $ llvm-dis sum.bc -o sum.ll The llvm-extract tool allows the extraction of IR functions, globals, and also the deletion of globals from the IR module. For instance, extract the sum function from sum.bc with the following command: $ llvm-extract -func=sum sum.bc -o sum-fn.bc Nothing changes between sum.bc and sum-fn.bc in this particular example since sum is already the sole function in this module. Introducing the LLVM IR language syntax Observe the LLVM IR assembly file, sum.ll: target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128" target triple = "x86_64-apple-macosx10.7.0" define i32 @sum(i32 %a, i32 %b) #0 { entry: %a.addr = alloca i32, align 4 %b.addr = alloca i32, align 4 store i32 %a, i32* %a.addr, align 4 store i32 %b, i32* %b.addr, align 4 %0 = load i32* %a.addr, align 4 %1 = load i32* %b.addr, align 4 %add = add nsw i32 %0, %1 ret i32 %add } attributes #0 = { nounwind ssp uwtable ... } The contents of an entire LLVM file, either assembly or bitcode, are said to define an LLVM module. The module is the LLVM IR top-level data structure. Each module contains a sequence of functions, which contains a sequence of basic blocks that contain a sequence of instructions. The module also contains peripheral entities to support this model, such as global variables, the target data layout, and external function prototypes as well as data structure declarations. LLVM local values are the analogs of the registers in the assembly language and can have any name that starts with the % symbol. Thus, %add = add nsw i32 %0, %1 will add the local value %0 to %1 and put the result in the new local value, %add. You are free to give any name to the values, but if you are short on creativity, you can just use numbers. In this short example, we can already see how LLVM expresses its fundamental properties: It uses the Static Single Assignment (SSA) form. Note that there is no value that is reassigned; each value has only a single assignment that defines it. Each use of a value can immediately be traced back to the sole instruction responsible for its definition. This has an immense value to simplify optimizations, owing to the trivial use-def chains that the SSA form creates, that is, the list of definitions that reaches a user. If LLVM had not used the SSA form, we would need to run a separate data flow analysis to compute the use-def chains, which are mandatory for classical optimizations such as constant propagation and common subexpression elimination. Code is organized as three-address instructions. Data processing instructions have two source operands and place the result in a distinct destination operand. It has an infinite number of registers. Note how LLVM local values can be any name that starts with the % symbol, including numbers that start at zero, such as %0, %1, and so on, that have no restriction on the maximum number of distinct values. The target datalayout construct contains information about endianness and type sizes for target triple that is described in target host. Some optimizations depend on knowing the specific data layout of the target to transform the code correctly. Observe how the layout declaration is done: target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128" target triple = "x86_64-apple-macosx10.7.0" We can extract the following facts from this string: The target is an x86_64 processor with macOSX 10.7.0. It is a little-endian target, which is denoted by the first letter in the layout (a lowercase e). Big-endian targets need to use an uppercase E. The information provided about types is in the format type:<size>:<abi>:<preferred>. In the preceding example, p:64:64:64 represents a pointer that is 64 bits wide in size, with the abi and preferred alignments set to the 64-bit boundary. The ABI alignment specifies the minimum required alignment for a type, while the preferred alignment specifies a potentially larger value, if this will be beneficial. The 32-bit integer types i32:32:32 are 32 bits wide in size, 32-bit abi and preferred alignment, and so on. The function declaration closely follows the C syntax: define i32 @sum(i32 %a, i32 %b) #0 { This function returns a value of the type i32 and has two i32 arguments, %a and %b. Local identifiers always need the % prefix, whereas global identifiers use @. LLVM supports a wide range of types, but the most important ones are the following: Arbitrary-sized integers in the iN form; common examples are i32, i64, and i128. Floating-point types, such as the 32-bit single precision float and 64-bit double precision double. Vectors types in the format <<# elements> x <elementtype>>. A vector with four i32 elements is written as <4 x i32>. The #0 tag in the function declaration maps to a set of function attributes, also very similar to the ones used in C/C++ functions and methods. The set of attributes is defined at the end of the file: attributes #0 = { nounwind ssp uwtable "less-precise-fpmad"="false""no-frame-pointer-elim"="true" "no-frame-pointer-elim-non-leaf"="true""no-infs-fp-math"="false" "no-nans-fp-math"="false" "unsafe-fp-math"="false""use-soft-float"="false" } For instance, nounwind marks a function or method as not throwing exceptions, and ssp tells the code generator to use a stack smash protector in an attempt to increase the security of this code against attacks. The function body is explicitly divided into basic blocks (BBs), and a label is used to start a new BB. A label relates to a basic block in the same way that a value identifier relates to an instruction. If a label declaration is omitted, the LLVM assembler automatically generates one using its own naming scheme. A basic block is a sequence of instructions with a single entry point at its first instruction, and a single exit point at its last instruction. In this way, when the code jumps to the label that corresponds to a basic block, we know that it will execute all of the instructions in this basic block until the last instruction, which will change the control flow by jumping to another basic block. Basic blocks and their associated labels need to adhere to the following conditions: Each BB needs to end with a terminator instruction, one that jumps to other BBs or returns from the function The first BB, called the entry BB, is special in an LLVM function and must not be the target of any branch instructions Our LLVM file, sum.ll, has only one BB because it has no jumps, loops, or calls. The function start is marked with the entry label, and it ends with the return instruction, ret: entry: %a.addr = alloca i32, align 4 %b.addr = alloca i32, align 4 store i32 %a, i32* %a.addr, align 4 store i32 %b, i32* %b.addr, align 4 %0 = load i32* %a.addr, align 4 %1 = load i32* %b.addr, align 4 %add = add nsw i32 %0, %1 ret i32 %add The alloca instruction reserves space on the stack frame of the current function. The amount of space is determined by element type size, and it respects a specified alignment. The first instruction, %a.addr = alloca i32, align 4, allocates a 4-byte stack element, which respects a 4-byte alignment. A pointer to the stack element is stored in the local identifier, %a.addr. The alloca instruction is commonly used to represent local (automatic) variables. The %a and %b arguments are stored in the stack locations %a.addr and %b.addr by means of store instructions. The values are loaded back from the same memory locations by load instructions, and they are used in the addition, %add = add nsw i32 %0, %1. Finally, the addition result, %add, is returned by the function. The nsw flag specifies that this add operation has "no signed wrap", which indicates instructions that are known to have no overflow, allowing for some optimizations. If you are interested in the history behind the nsw flag, a worthwhile read is the LLVMdev post at http://lists.cs.uiuc.edu/pipermail/llvmdev/2011-November/045730.html by Dan Gohman. In fact, the load and store instructions are redundant, and the function arguments can be used directly in the add instruction. Clang uses -O0 (no optimizations) by default, and the unnecessary loads and stores are not removed. If we compile with -O1 instead, the outcome is a much simpler code, which is reproduced here: define i32 @sum(i32 %a, i32 %b) ... { entry: %add = add nsw i32 %b, %a ret i32 %add } ... Using the LLVM assembly directly is very handy when writing small examples to test target backends and as a means to learn basic LLVM concepts. However, a library is the recommended interface for frontend writers to build the LLVM IR, which is the subject of our next section. You can find a complete reference to the LLVM IR assembly syntax at http://llvm.org/docs/LangRef.html. Introducing the LLVM IR in-memory model The in-memory representation closely models the LLVM language syntax that we just presented. The header files for the C++ classes that represent the IR are located at include/llvm/IR. The following is a list of the most important classes: The Module class aggregates all of the data used in the entire translation unit, which is a synonym for "module" in LLVM terminology. It declares the Module::iterator typedef as an easy way to iterate across the functions inside this module. You can obtain these iterators via the begin() and end() methods. View its full interface at http://llvm.org/docs/doxygen/html/classllvm_1_1Module.html. The Function class contains all objects related to a function definition or declaration. In the case of a declaration (use the isDeclaration() method to check whether it is a declaration), it contains only the function prototype. In both cases, it contains a list of the function parameters accessible via the getArgumentList() method or the pair of arg_begin() and arg_end(). You can iterate through them using the Function::arg_iterator typedef. If your Function object represents a function definition, and you iterate through its contents via the for (Function::iterator i = function.begin(), e = function.end(); i != e; ++i) idiom, you will iterate across its basic blocks. View its full interface at http://llvm.org/docs/doxygen/html/classllvm_1_1Function.html. The BasicBlock class encapsulates a sequence of LLVM instructions, accessible via the begin()/end() idiom. You can directly access its last instruction using the getTerminator() method, and you also have a few helper methods to navigate the CFG, such as accessing predecessor basic blocks via getSinglePredecessor(), when the basic block has a single predecessor. However, if it does not have a single predecessor, you need to work out the list of predecessors yourself, which is also not difficult if you iterate through basic blocks and check the target of their terminator instructions. View its full interface at http://llvm.org/docs/doxygen/html/classllvm_1_1BasicBlock.html. The Instruction class represents an atom of computation in the LLVM IR, a single instruction. It has some methods to access high-level predicates, such as isAssociative(), isCommutative(), isIdempotent(), or isTerminator(), but its exact functionality can be retrieved with getOpcode(), which returns a member of the llvm::Instruction enumeration, which represents the LLVM IR opcodes. You can access its operands via the op_begin() and op_end() pair of methods, which are inherited from the User superclass that we will present shortly. View its full interface at http://llvm.org/docs/doxygen/html/classllvm_1_1Instruction.html. We have still not presented the most powerful aspect of the LLVM IR (enabled by the SSA form): the Value and User interfaces; these allow you to easily navigate the use-def and def-use chains. In the LLVM in-memory IR, a class that inherits from Value means that it defines a result that can be used by others, whereas a subclass of User means that this entity uses one or more Value interfaces. Function and Instruction are subclasses of both Value and User, while BasicBlock is a subclass of just Value. To understand this, let's analyze these two classes in depth: The Value class defines the use_begin() and use_end() methods to allow you to iterate through Users, offering an easy way to access its def-use chain. For every Value class, you can also access its name through the getName() method. This models the fact that any LLVM value can have a distinct identifier associated with it. For example, %add1 can identify the result of an add instruction, BB1 can identify a basic block, and myfunc can identify a function. Value also has a powerful method called replaceAllUsesWith(Value *), which navigates through all of the users of this value and replaces it with some other value. This is a good example of how the SSA form allows you to easily substitute instructions and write fast optimizations. You can view the full interface at http://llvm.org/docs/doxygen/html/classllvm_1_1Value.html. The User class has the op_begin() and op_end() methods that allows you to quickly access all of the Value interfaces that it uses. Note that this represents the use-def chain. You can also use a helper method called replaceUsesOfWith(Value *From, Value *To) to replace any of its used values. You can view the full interface at http://llvm.org/docs/doxygen/html/classllvm_1_1User.html. Summary In this article, we acquainted ourselves with the concepts and components related to the LLVM intermediate representation. Resources for Article: Further resources on this subject: Creating and Utilizing Custom Entities [Article] Getting Started with Code::Blocks [Article] Program structure, execution flow, and runtime objects [Article]

0
0
18274

Packt

26 Aug 2014

14 min read

Using OSGi Services

Packt

26 Aug 2014

14 min read

0
0
1019

Packt

25 Aug 2014

11 min read

The NServiceBus Architecture

Packt

25 Aug 2014

11 min read

In this article by Rich Helton, the author of Mastering NServiceBus and Persistence, we will focus on the NServiceBus architecture. We will discuss the different message and storage types supported in NSB. This discussion will include an introduction to some of the tools and advantages of using NSB. We will conceptually look at how some of the pieces fit together while backing up the discussions with code examples. (For more resources related to this topic, see here.) NSB is the cornerstone of automation. As an Enterprise Service Bus (ESB), NSB is the most popular C# ESB solution. NSB is a framework that is used to provide many of the benefits of implementing a service-oriented architecture (SOA). It uses an IBus and its ESB bus to handle messages between NSB services, without having to create custom interaction. This type of messaging between endpoints creates the bus. The services, which are autonomous Windows processes, use both Windows and NSB hosting services. NSB-hosting services provide extra functionalities, such as creating endpoints; setting up Microsoft Queuing (MSMQ), DTC for transactions across queues, subscription storage for publish/subscribe message information, NSB sagas; and much more. Deploying these pieces for messaging manually can lead to errors and a lot of work is involved to get it correct. NSB takes care of provisioning its needed pieces. NSB is not a frontend framework, such as Microsoft's Model-View-Controller (MVC). It is not used as an Object-to-Relationship Mapper (ORM), such as Microsoft's Entity Frameworks, to map objects to SQL Server tables. It is also not a web service framework, such as Microsoft's Windows Communication Foundation (WCF). NSB is a framework to provide the communication and support for services to communicate with each other and provide an end-to-end workflow to process all of these pieces. Benefits of NSB NSB provides many components needed for automation that are only found in ESBs. ESBs provide the following: Separation of duties: From the frontend to the backend by allowing the frontend to fire a message to a service and continue with its processing not worrying about the results until it needs an update. Also, you can separate workflow responsibilities by separating NSB services. One service could be used to send payments to a bank, and another service can be used to provide feedback of the current status of the payment to the MVC-EF database so that a user may see the status of their payment. Message durability: Messages are saved in queues between services so that if the services are stopped, they can start from the messages saved in the queues when they are restarted. This is done so that the messages will persist, until told otherwise. Workflow retries: Messages, or endpoints, can be told to retry a number of times until they completely fail and send an error. The error is automated to return to an error queue. For instance, a web service message can be sent to a bank, and it can be set to retry the web service every 5 minutes for 20 minutes before giving up completely. This is useful while fixing any network or server issues. Monitoring: NSB's ServicePulse can keep a check on the heartbeat of its services. Other monitoring checks can be easily performed on NSB queues to report the number of messages. Encryption: Messages between services and endpoints can be easily encrypted. High availability: Multiple services, or subscribers, could be processing the same or similar messages from various services that live on different servers. When one server, or a service, goes down, others could be made available to take over that are already running. More on endpoints While working with a service-to-service interaction, messages are transmitted in the form of XML through queues that are normally part of Microsoft Server such as MSMQ, SQL Server such as SQL queuing, or even part of Microsoft Azure queues for cloud computing. There are other endpoints that services use to process resources that are not part of service-to-service communications. These endpoints are used to process commands and messages as well, for instance, sending a file to non-NSB-hosted services, sending SFTP files to non-NSB-hosted services, or sending web services, such as payments, to non-NSB services. While at the other end of these communications are non-NSB-hosted services, NSB offers a lot of integrity by checking how these endpoints were processed. NSB provides information on whether a web service was processed or not, with or without errors, and provides feedback and monitoring, and maintains the records through queues. It also provides saga patterns to provide feedback to the originating NSB services of the outcome while storing messages from a particular NSB service to the NSB service of everything that has happened. In many NSB services, an audit queue is used to keep a backup of each message that occurred successfully, and the error queue is used to keep track of any message that was not processed successfully. The application security perspective From the application security perspective, OWASP's top ten list of concerns, available at https://www.owasp.org/index.php/Top_10_2013-Top_10, seems to always surround injection, such as SQL injection, broken authentication, and cross-site scripting (XSS). Once an organization puts a product in production, they usually have policies in place for the company's security personnel to scan the product at will. Not all organizations have these policies in place, but once an organization attaches their product to the Internet, there are armies of hackers that may try various methods to attack the site, depending on whether there is money to be gained or not. Money comes in a new economy these days in the form of using a site as a proxy to stage other attacks, or to grab usernames and passwords that a user may have for a different system in order to acquire a user's identity or financial information. Many companies have suffered bankruptcy over the last decades thinking that they were secure. NSB offers processing pieces to the backend that would normally be behind a firewall to provide some protection. Firewalls provide some protection as well as Intrusion Detection Systems (IDSes), but there is so much white noise for viruses and scans that many real hack attacks may go unnoticed, except by very skilled antihackers. NSB offers additional layers of security by using queuing and messaging. The messages can be encrypted, and the queues may be set for limited authorization from production administrators. NSB hosting versus self-hosting NServiceBus.Host is an executable that will deploy the NSB service. When the NSB service is compiled, it turns into a Windows DLL that may contain all the configuration settings for the IBus. If there are additional settings needed for the endpoint's configuration that are not coded in the IBus's configuration, then it can be resolved by setting these configurations in the Host command. However, NServiceBus.Host need not be used to create the program that is used in NServiceBus. As a developer, you can create a console program that is run by a Window's task scheduler, or even create your own services that run the NSB IBus code as an endpoint. Not using the NSB-hosting engine is normally referred to as self-hosting. The NServiceBus host streamlines service development and deployment, allows you to change technologies without code, and is administrator friendly when setting permissions and accounts. It will deploy your application as an NSB-hosted solution. It can also add configurations to your program at the NServiceBus.Host.exe command line. If you develop a program with the NServiceBus.Host reference, you can use EndpoinConfig.cs to define your IBus configuration in this code, or add it as part of the command line instead of creating your own Program.cs that will do a lot of the same work with more code. When debugging with the NServiceBus.Host reference, the Visual Studio project is creating a windows DLL program that is run by the NserviceBus.Host.exe command. Here's an example form of the properties of a Visual Studio project: The NServiceBus.Host.exe command line has support for deploying Window's services as NSB-hosted services: These configurations are typically referred to as the profile for which the service will be running. Here are some of the common profiles: MultiSite: This turns on the gateway. Master: This makes the endpoint a "master node endpoint". This means that it runs the gateway for multisite interaction, the timeout manager, and the distributor. It also starts a worker that is enlisted with the distributor. It cannot be combined with the worker or distributor profiles. Worker: This makes the current endpoint enlist as a worker with its distributor running on the master node. It cannot be combined with the master or distributor profiles. Distributor: This starts the endpoint only as a distributor. This means that the endpoint does no actual work and only distributes the load among its enlisted workers. It cannot be combined with the Master and Worker profiles. Performance counters: This turns on the NServiceBus-specific performance counters. Performance counters are installed by default when you run a Production profile. Lite: This keeps everything in memory with the most detailed logging. Integration: This uses technologies closer to production but without a scale-out option and less logging. It is used in testing. Production: This uses scale-out-friendly technologies and minimal file-based logging. It is used in production. Using Powershell commands Many items can be managed in the Package Manager console program of Visual Studio 2012. Just as we add commands to the NServiceBus.Host.exe file to extend profiles and configurations, we may also use VS2012 Package Manager to extend some of the functionalities while debugging and testing. We will use the ScaleOut solution discussed later just to double check that the performance counters are installed correctly. We need to make sure that the PowerShell commandlets are installed correctly first. We do this by using Package Manager: Install the package, NServiceBus.PowerShell Import the module, .packagesNServiceBus.PowerShell.4.3.0libnet40NServiceBus.PowerShell.dll Test NServiceBusPerformanceCountersInstallation The "Import module" step is dependent on where NService.PowerShell.dll was installed during the "Install package" process. The "Install-package" command will add the DLL into a package directory related to the solution. We can find out more on PowerShell commandlets at http://docs.particular.net/nservicebus/managing-nservicebus-using-powershell and even by reviewing the help section of Package Manager. Here, we see that we can insert configurations into App.config when we look at the help section, PM> get-help about_NServiceBus. Message exchange patterns Let's discuss the various exchange patterns now. The publish/subscribe pattern One of the biggest benefits of using the ESB technology is the benefits of the publish/subscribe message pattern; refer to http://en.wikipedia.org/wiki/Publish-subscribe_pattern. The publish/subscribe pattern has a publisher that sends messages to a queue, say a MSMQ MyPublisher queue. Subscribers, say Subscriber1 and Subscriber2, will listen for messages on the queue that the subscribers are defined to take from the queue. If MyPublisher cannot process the messages, it will return them to the queue or to an error queue, based on the reasons why it could not process the message. The queue that the subscribers are looking for on the queue are called endpoint mappings. The publisher endpoint mapping is usually based on the default of the project's name. This concept is the cornerstone to understand NSB and ESBs. No messages will be removed, unless they are explicitly told to be removed by a service. Therefore, no messages will be lost, and all are accounted for from the services. The configuration data is saved to the database. Also, the subscribers can respond back to MyPublisher with messages indicating that everything was alright or not using the queue. So why is this important? It's because all the messages can then be accounted for, and feedback can be provided to all the services. A service is a Windows service that is created and hosted by the NSB host program. It could also be a Windows command console program or even an MVC program, but the service program is always up and running on the server, continuously checking queues and messages that are sent to it from other endpoints. These messages could be commands, such as instructions to go and look at the remote server to see whether it is still running, or data messages such as sending a particular payment to the bank through a web service. For NSB, we formalize that events are used in publish/subscribe, and commands are used in a request-response message exchange pattern. Windows Server could have too many services, so some of these services could just be standing by, waiting to take over if one service is not responding or processing messages simultaneously. This provides a very high availability.

0
0
3977

article-image-new-functionality-opencv-30

Packt

25 Aug 2014

5 min read

New functionality in OpenCV 3.0

Packt

25 Aug 2014

5 min read

In this article by Oscar Deniz Suarez, coauthor of the book OpenCV Essentials, we will cover the forthcoming Version 3.0, which represents a major evolution of the OpenCV library for Computer Vision. Currently, OpenCV already includes several new techniques that are not available in the latest official release (2.4.9). The new functionality can be already used by downloading and compiling the latest development version from the official repository. This article provides an overview of some of the new techniques implemented. Other numerous lower-level changes in the forthcoming Version 3.0 (updated module structure, C++ API changes, transparent API for GPU acceleration, and so on) are not discussed. (For more resources related to this topic, see here.) Line Segment Detector OpenCV users have had the Hough transform-based straight line detector available in the previous versions. An improved method called Line Segment Detector (LSD) is now available. LSD is based on the algorithm described at http://dx.doi.org/10.5201/ipol.2012.gjmr-lsd. This method has been shown to be more robust and faster than the best previous Hough-based detector (the Progressive Probabilistic Hough Transform). The detector is now part of the imgproc module. OpenCV provides a short sample code ([opencv_source_code]/samples/cpp/lsd_lines.cpp), which shows how to use the LineSegmentDetector class. The following table shows the main components of the class: Method Function <constructor> The constructor allows to enter parameters of the algorithm; particularly; the level of refinements we want in the result detect This method detects line segments in the image drawSegments This method draws the segments in a given image compareSegments This method draws two sets of segments in a given image. The two sets are drawn with blue and red color lines Connected components The previous versions of OpenCV have included functions for working with image contours. Contours are the external limits of connected components (that is, regions of connected pixels in a binary image). The new functions, connectedComponents and connectedComponentsWithStats retrieve connected components as such. The connected components are retrieved as a labeled image with the same dimensions as the input image. This allows drawing the components on the original image easily. The connectedComponentsWithStats function retrieves useful statistics about each component shown in the following table: CC_STAT_LEFT The leftmost (x) coordinate, which is the inclusive start of the bounding box in the horizontal direction CC_STAT_TOP The topmost (y) coordinate, which is the inclusive start of the bounding box in the vertical direction CC_STAT_WIDTH The horizontal size of the bounding box CC_STAT_HEIGHT The vertical size of the bounding box CC_STAT_AREA The total area (in pixels) of the connected component Scene text detection Text recognition is a classic problem in Computer Vision. Thus, Optical Character Recognition (OCR) is now routinely used in our society. In OCR, the input image is expected to depict typewriter black text over white background. In the last years, researchers aim at the more challenging problem of recognizing text "in the wild" on street signs, indoor signs, with diverse backgrounds and fonts, colors, and so on. The following figure shows and example of the difference between the two scenarios. In this scenario, OCR cannot be applied to the input images. Consequently, text recognition is actually accomplished in two steps. The text is first localized in the image and then character or word recognition is performed on the cropped region. OpenCV now provides a scene text detector based on the algorithm described in Neumann L., Matas J.: Real-Time Scene Text Localization and Recognition, CVPR 2012 (Providence, Rhode Island, USA). The implementation of OpenCV makes use of additional improvements found at http://158.109.8.37/files/GoK2013.pdf. OpenCV includes an example ([opencv_source_code]/samples/cpp/textdetection.cpp) that detects and draws text regions in an input image. The KAZE and AKAZE features Several 2D features have been proposed in the computer vision literature. Generally, the two most important aspects in feature extraction algorithms are computational efficiency and robustness. One of the latest contenders is the KAZE (Japanese word meaning "Wind") and Accelerated-KAZE (AKAZE) detector. There are reports that show that KAZE features are both robust and efficient, as compared with other widely-known features (BRISK, FREAK, and so on). The underlying algorithm is described in KAZE Features, Pablo F. Alcantarilla, Adrien Bartoli, and Andrew J. Davison, in European Conference on Computer Vision (ECCV), Florence, Italy, October 2012. As with other keypoint detectors in OpenCV, the KAZE implementation allows retrieving both keypoints and descriptors (that is, a feature vector computed around the keypoint neighborhood). The detector follows the same framework used in OpenCV for other detectors, so drawing methods are also available. Computational photography One of the modules with most improvements in the forthcoming Version 3.0 is the computational photography module (photo). The new techniques include the functionalities mentioned in the following table: Functionality Description HDR imaging Functions for handling High-Dynamic Range images (tonemapping, exposure alignment, camera calibration with multiple exposures, and exposure fusion) Seamless cloning Functions for realistically inserting one image into other image with an arbitrary-shape region of interest. Non-photorealistic rendering This technique includes non-photorealistic filters (such as pencil-like drawing effect) and edge-preserving smoothing filters (those are similar to the bilateral filter). New modules Finally, we provide a list with the new modules in development for version 3.0: Module name Description videostab Global motion estimation, Fast Marching method softcascade Implements a stageless variant of the cascade detector, which is considered more accurate shape Shape matching and retrieval. Shape context descriptor and matching algorithm, Hausdorff distance and Thin-Plate Splines cuda<X> Several modules with CUDA-accelerated implementations of other functions in the library Summary In this article, we learned about the different functionalities in OpenCV 3.0 and its different components. Resources for Article: Further resources on this subject: Wrapping OpenCV [article] A quick start – OpenCV fundamentals [article] Linking OpenCV to an iOS project [article]

0
0
6409

Packt

19 Aug 2014

8 min read

BPMS Components

Packt

19 Aug 2014

8 min read

In this article by Mariano Nicolas De Maio, the author of jBPM6 Developer Guide, we will look into the various components of a Business Process Management (BPM) system. (For more resources related to this topic, see here.) BPM systems are pieces of software created with the sole purpose of guiding your processes through the BPM cycle. They were originally monolithic systems in charge of every aspect of a process, where they had to be heavily migrated from visual representations to executable definitions. They've come a long way from there, but we usually relate them to the same old picture in our heads when a system that runs all your business processes is mentioned. Nowadays, nothing is further from the truth. Modern BPM Systems are not monolithic environments; they're coordination agents. If a task is finished, they will know what to do next. If a decision needs to be made regarding the next step, they manage it. If a group of tasks can be concurrent, they turn them into parallel tasks. If a process's execution is efficient, they will perform the processing 0.1 percent of the time in the process engine and 99.9 percent of the time on tasks in external systems. This is because they will have no heavy executions within, only derivations to other systems. Also, they will be able to do this from nothing but a specific diagram for each process and specific connectors to external components. In order to empower us to do so, they need to provide us with a structure and a set of tools that we'll start defining to understand how BPM systems' internal mechanisms work, and specifically, how jBPM6 implements these tools. Components of a BPMS All big systems become manageable when we divide their complexities into smaller pieces, which makes them easier to understand and implement. BPM systems apply this by dividing each function in a different module and interconnecting them within a special structure that (in the case of jBPM6) looks something like the following figure: BPMS' internal structure Each component in the preceding figure resolves one particular function inside the BPMS architecture, and we'll see a detailed explanation on each one of them. The execution node The execution node, as seen from a black box perspective, is the component that receives the process definitions (a description of each step that must be followed; from here on, we'll just refer to them as processes). Then, it executes all the necessary steps in the established way, keeping track of each step, variable, and decision that has to be taken in each process's execution (we'll start calling these process instances). The execution node along with its modules are shown in the following figure: The execution node is composed of a set of low-level modules: the semantic module and the process engine. The semantic module The semantic module is in charge of defining each of the specific language semantics, that is, what each word means and how it will be translated to the internal structures that the process engine can execute. It consists of a series of parsers to understand different languages. It is flexible enough to allow you to extend and support multiple languages; it also allows the user to change the way already defined languages are to be interpreted for special use cases. It is a common component of most of the BPMSes out there, and in jBPM6, it allows you to add the extensions of the process interpretations to the module. This is so that you can add your own language parsers, and define your very own text-based process definition language or extend existing ones. The process engine The process engine is the module that is in charge of the actual execution of our business processes. It creates new process instances and keeps track of their state and their internal steps. Its job is to expose methods to inject process definitions and to create, start, and continue our process instances. Understanding how the process engine works internally is a very important task for the people involved in BPM's stage 4, that is, runtime. This is where different configurations can be used to improve performance, integrate with other systems, provide fault tolerance, clustering, and many other functionalities. Process Engine structure In the case of jBPM6, process definitions and process instances have similar structures but completely different objectives. Process definitions only show the steps it should follow and the internal structures of the process, keeping track of all the parameters it should have. Process instances, on the other hand, should carry all of the information of each process's execution, and have a strategy for handling each step of the process and keep track of all its actual internal values. Process definition structures These structures are static representations of our business processes. However, from the process engine's internal perspective, these representations are far from the actual process structure that the engine is prepared to handle. In order for the engine to get those structures generated, it requires the previously described semantic module to transform those representations into the required object structure. The following figure shows how this parsing process happens as well as the resultant structure: Using a process modeler, business analysts can draw business processes by dragging-and-dropping different activities from the modeler palette. For jBPM6, there is a web-based modeler designed to draw Scalable Vector Graphics (SVG) files; this is a type of image file that has the particularity of storing the image information using XML text, which is later transformed into valid BPMN2 files. Note that both BPMN2 and jBPM6 are not tied up together. On one hand, the BPMN2 standard can be used by other process engine provides such as Activiti or Oracle BPM Suite. Also, because of the semantic module, jBPM6 could easily work with other parsers to virtually translate any form of textual representation of a process to its internal structures. In the internal structures, we have a root component (called Process in our case, which is finally implemented in a class called RuleFlowProcess) that will contain all the steps that are represented inside the process definition. From the jBPM6 perspective, you can manually create these structures using nothing but the objects provided by the engine. Inside the jBPM6-Quickstart project, you will find a code snippet doing exactly this in the createProcessDefinition() method of the ProgrammedProcessExecutionTest class: //Process Definition RuleFlowProcess process = new RuleFlowProcess(); process.setId("myProgramaticProcess"); //Start Task StartNode startTask = new StartNode(); startTask.setId(1); //Script Task ActionNode scriptTask = new ActionNode(); scriptTask.setId(2); DroolsAction action = new DroolsAction(); action.setMetaData("Action", new Action() { @Override public void execute(ProcessContext context) throws Exception { System.out.println("Executing the Action!!"); } }); scriptTask.setAction(action); //End Task EndNode endTask = new EndNode(); endTask.setId(3); //Adding the connections to the nodes and the nodes to the processes new ConnectionImpl(startTask, "DROOLS_DEFAULT", scriptTask, "DROOLS_DEFAULT"); new ConnectionImpl(scriptTask, "DROOLS_DEFAULT", endTask, "DROOLS_DEFAULT"); process.addNode(startTask); process.addNode(scriptTask); process.addNode(endTask); Using this code, we can manually create the object structures to represent the process shown in the following figure: This process contains three components: a start node, a script node, and an end node. In this case, this simple process is in charge of executing a simple action. The start and end tasks simply specify a sequence. Even if this is a correct way to create a process definition, it is not the recommended one (unless you're making a low-level functionality test). Real-world, complex processes are better off being designed in a process modeler, with visual tools, and exported to standard representations such as BPMN 2.0. The output of both the cases is the same; a process object that will be understandable by the jBPM6 runtime. While we analyze how the process instance structures are created and how they are executed, this will do. Process instance structures Process instances represent the running processes and all the information being handled by them. Every time you want to start a process execution, the engine will create a process instance. Each particular instance will keep track of all the activities that are being created by its execution. In jBPM6, the structure is very similar to that of the process definitions, with one root structure (the ProcessInstance object) in charge of keeping all the information and NodeInstance objects to keep track of live nodes. The following code shows a simplification of the methods of the ProcessInstance implementation: public class RuleFlowProcessInstance implements ProcessInstance { public RuleFlowProcess getRuleFlowProcess() { ... } public long getId() { ... } public void start() { ... } public int getState() { ... } public void setVariable(String name, Object value) { ... } public Collection<NodeInstance> getNodeInstances() { ... } public Object getVariable(String name) { ... } } After its creation, the engine calls the start() method of ProcessInstance. This method seeks StartNode of the process and triggers it. Depending on the execution of the path and how different nodes connect between each other, other nodes will get triggered until they reach a safe state where the execution of the process is completed or awaiting external data. You can access the internal parameters that the process instance has through the getVariable and setVariable methods. They provide local information from the particular process instance scope. Summary In this article, we saw what are the basic components required to set up a BPM system. With these components in place, we are ready to explore, in more detail, the structure and working of a BPM system. Resources for Article: Further resources on this subject: jBPM for Developers: Part 1 [Article] Configuring JBoss Application Server 5 [Article] Boss jBPM Concepts and jBPM Process Definition Language (jPDL) [Article]

0
0
2858

article-image-program-structure-execution-flow-and-runtime-objects

Packt

30 Jul 2014

8 min read

Program structure, execution flow, and runtime objects

Packt

30 Jul 2014

8 min read

(For more resources related to this topic, see here.) Unfortunately, C++ isn't an easy language at all. Sometimes, you can think that it is a limitless language that cannot be learned and understood entirely, but you don't need to worry about that. It is not important to know everything; it is important to use the parts that are required in specific situations correctly. Practice is the best teacher, so it's better to understand how to use as many of the features as needed. In examples to come, we will use author Charles Simonyi's Hungarian notation. In his PhD in 1977, he used Meta-Programming – A software production method to make a standard for notation in programming, which says that the first letter of type or variable should represent the data type. For the example class that we want to call, a Test data type should be CTest, where the first letter says that Test is a class. This is good practice because a programmer who is not familiar with the Test data type will immediately know that Test is a class. The same standard stands for primitive types, such as int or double. For example, iCount stands for an integer variable Count, while dValue stands for a double variable Value. Using the given prefixes, it is easy to read code even if you are not so familiar with it. Getting ready Make sure Visual Studio is up and running. How to do it... Now, let's create our first program by performing the following steps and explaining its structure: Create a new C++ console application named TestDemo. Open TestDemo.cpp. Add the following code: #include "stdafx.h" #include <iostream> using namespace std; int _tmain(int argc, _TCHAR* argv[]) { cout << "Hello world" << endl; return 0; } How it works... The structure of the C++ program varies due to different programming techniques. What most programs must have is the #include or preprocessor directives. The #include <iostream> header tells the compiler to include the iostream.h header file where the available function prototypes reside. It also means that libraries with the functions' implementations should be built into executables. So, if we want to use some API or function, we need to include an appropriate header file, and maybe we will have to add an additional input library that contains the function/API implementation. One more important difference when including files is <header> versus "header". The first (<>) targets solution-configured project paths, while the other ("") targets folders relative to the C++ project. The using command instructs the compiler to use the std namespace. Namespaces are packages with object declarations and function implementations. Namespaces have a very important usage. They are used to minimize ambiguity while including third-party libraries when the same function name is used in two different packages. We need to implement a program entry point—the main function. As we said before, we can use main for an ANSI signature, wmain for a Unicode signature, or _tmain where the compiler will resolve its signature depending on the preprocessor definitions in the project property pages. For a console application, the main function can have the following four different prototypes: int _tmain(int argc, TCHAR* argv[]) void _tmain(int argc, TCHAR* argv[]) int _tmain(void) void _tmain(void) The first prototype has two arguments, argc and argv. The first argument, argc, or the argument count, says how many arguments are present in the second argument, argv, or the argument values. The argv parameter is an array of strings, where each string represents one command-line argument. The first string in argv is always the current program name. The second prototype is the same as the first one, except the return type. This means that the main function may or may not return a value. This value is returned to the OS. The third prototype has no arguments and returns an integer value, while the fourth prototype neither has arguments nor returns a value. It is good practice to use the first format. The next command uses the cout object. The cout object is the name of the standard output stream in C++, and the meaning of the entire statement is to insert a sequence of characters (in this case, the Hello world sequence of characters) into the standard output stream (usually corresponds to the screen). The cout object is declared in the iostream standard file within the std namespace. This is why we need to include the specific file and declare that we will use this specific namespace earlier in our code. In our usual selection of the (int _tmain(int,_TCHAR**)) prototype, _tmain returns an integer. We must specify some int value after the return command, in this case 0. When returning a value to the operating system, 0 usually means success, but this is operating system dependent. This simple program is very easy to create. We use this simple example to demonstrate the basic program structure and usage of the main routine as the entry point for every C++ program. Programs with one thread are executed sequentially line-by-line. This is why our program is not user friendly if we place all code into one thread. After the user gives the input, control is returned to the application. Only now can the application continue the execution. In order to overcome such an issue, we can create concurrent threads that will handle the user input. In this way, the application does not stall and is responsive all the time. After a thread handles its task, it can signal that a user has performed the requested operation to the application. There's more... Every time we have an operation that needs to be executed separately from the main execution flow, we have to think about a separate thread. The simplest example is when we have some calculations, and we want to have a progress bar where the calculation progress will be shown. If the same thread was responsible for calculation as well as to update the progress bar, probably it wouldn't work. This occurs because, if both work and a UI update are performed from a single thread, such threads can't interact with OS painting adequately; so almost always, the UI thread is separate from the working threads. Let's review the following example. Assume that we have created a function where we will calculate something, for example, sines or cosines of some angle, and we want to display progress in every step of the calculation: void CalculateSomething(int iCount) { int iCounter = 0; while (iCounter++ < iCount) { //make calculation //update progress bar } } As commands are executed one after another inside each iteration of the while loop, the operating system doesn't have the required time to properly update the user interface (in this case, the progress bar), so we would see an empty progress bar. After the function returns, a fully filled progress bar appears. The solution for this is to create a progress bar on the main thread. A separate thread should execute the CalculateSomething function; in each step of iteration, it should somehow signal the main thread to update the progress bar step by step. As we said before, threads are switched extremely fast on the CPU, so we can get the impression that the progress bar is updated at the same time at the calculation is performed. To conclude, each time we have to make a parallel task, wait for some kind of user input, or wait for an external dependency such as some response from a remote server, we will create a separate thread so that our program won't hang and be unresponsive. In our future examples, we will discuss static and dynamic libraries, so let's say a few words about both. A static library (*.lib) is usually some code placed in separate files and already compiled for future use. We will add it to the project when we want to use some of its features. When we wrote #include <iostream> earlier, we instructed the compiler to include a static library where implementations of input-output stream functions reside. A static library is built into the executable in compile time before we actually run the program. A dynamic library (*.dll) is similar to a static library, but the difference is that it is not resolved during compilation; it is linked later when we start the program, in another words—at runtime. Dynamic libraries are very useful when we have functions that a lot of programs will use. So, we don't need to include these functions into every program; we will simply link every program at runtime with one dynamic library. A good example is User32.dll, where the Windows OS placed a majority of GUI functions. So, if we create two programs where both have a window (GUI form), we do not need to include CreateWindow in both programs. We will simply link User32.dll at runtime, and the CreateWindow API will be available. Summary Thus the article covers the four main paradigms: imperative, declarative, functional (or structural), and object-oriented. Resources for Article: Further resources on this subject: OpenGL 4.0: Building a C++ Shader Program Class [Article] Application Development in Visual C++ - The Tetris Application [Article] Building UI with XAML for Windows 8 Using C [Article]

0
0
2698

Packt

22 Jul 2014

17 min read

The .NET Framework Primer

Packt

22 Jul 2014

17 min read

0
0
1434

Packt

21 Jul 2014

12 min read

Configuration

Packt

21 Jul 2014

12 min read

0
0
1468

Packt

21 Jul 2014

5 min read

Design patterns

Packt

21 Jul 2014

5 min read

(For more resources related to this topic, see here.) Design patterns are ways to solve a problem and the way to get your intended result in the best possible manner. So, design patterns are not only ways to create a large and robust system, but they also provide great architectures in a friendly manner. In software engineering, a design pattern is a general repeatable and optimized solution to a commonly occurring problem within a given context in software design. It is a description or template for how to solve a problem, and the solution can be used in different instances. The following are some of the benefits of using design patterns: Maintenance Documentation Readability Ease in finding appropriate objects Ease in determining object granularity Ease in specifying object interfaces Ease in implementing even for large software projects Implements the code reusability concept If you are not familiar with design patterns, the best way to begin understanding is observing the solutions we use for commonly occurring, everyday life problems. Let's take a look at the following image: Many different types of power plugs exist in the world. So, we need a solution that is reusable, optimized, and cheaper than buying a new device for different power plug types. In simple words, we need an adapter. Have a look at the following image of an adapter: In this case, an adapter is the best solution that's reusable, optimized, and cheap. But an adapter does not provide us with a solution when our car's wheel blows out. In object-oriented languages, we the programmers use the objects to do whatever we want to have the outcome we desire. Hence, we have many types of objects, situations, and problems. That means we need more than just one approach to solving different kinds of problems. Elements of design patterns The following are the elements of design patterns: Name: This is a handle we can use to describe the problem Problem: This describes when to apply the pattern Solution: This describes the elements, relationships, responsibilities, and collaborations, in a way that we follow to solve a problem Consequences: This details the results and trade-offs of applying the pattern Classification of design patterns Design patterns are generally divided into three fundamental groups: Creational patterns Structural patterns Behavioral patterns Let's examine these in the following subsections. Creational patterns Creational patterns are a subset of design patterns in the field of software development; they serve to create objects. They decouple the design of an object from its representation. Object creation is encapsulated and outsourced (for example, in a factory) to keep the context of object creation independent from concrete implementation. This is in accordance with the rule: "Program on the interface, not the implementation." Some of the features of creational patterns are as follows: Generic instantiation: This allows objects to be created in a system without having to identify a specific class type in code (Abstract Factory and Factory pattern) Simplicity: Some of the patterns make object creation easier, so callers will not have to write large, complex code to instantiate an object (Builder (Manager) and Prototype pattern) Creation constraints: Creational patterns can put bounds on who can create objects, how they are created, and when they are created The following patterns are called creational patterns: The Abstract Factory pattern The Factory pattern The Builder (Manager) pattern The Prototype pattern The Singleton pattern Structural patterns In software engineering, design patterns structure patterns facilitate easy ways for communications between various entities. Some of the examples of structures of the samples are as follows: Composition: This composes objects into a tree structure (whole hierarchies). Composition allows customers to be uniformly treated as individual objects according to their composition. Decorator: This dynamically adds options to an object. A Decorator is a flexible alternative embodiment to extend functionality. Flies: This is a share of small objects (objects without conditions) that prevent overproduction. Adapter: This converts the interface of a class into another interface that the clients expect. Adapter lets those classes work together that would normally not be able to because of the different interfaces. Facade: This provides a unified interface meeting the various interfaces of a subsystem. Facade defines a higher-level interface to the subsystem, which is easier to use. Proxy: This implements the replacement (surrogate) of another object that controls access to the original object. Bridge: This separates an abstraction from its implementation, which can then be independently altered. Behavioral patterns Behavioral patterns are all about a class' objects' communication. Behavioral patterns are those patterns that are most specifically concerned with communication between objects. The following is a list of the behavioral patterns: Chain of Responsibility pattern Command pattern Interpreter pattern Iterator pattern Mediator pattern Memento pattern Observer pattern State pattern Strategy pattern Template pattern Visitor pattern If you want to check out the usage of some patterns in the Laravel core, have a look at the following list: The Builder (Manager) pattern: IlluminateAuthAuthManager and IlluminateSessionSessionManager The Factory pattern: IlluminateDatabaseDatabaseManager and IlluminateValidationFactory The Repository pattern: IlluminateConfigRepository and IlluminateCacheRepository The Strategy pattern: IIlluminateCacheStoreInterface and IlluminateConfigLoaderInterface The Provider pattern: IIlluminateAuthAuthServiceProvider and IlluminateHashHashServiceProvider Summary In this article, we have explained the fundamentals of design patterns. We've also introduced some design patterns that are used in the Laravel Framework. Resources for Article: Further resources on this subject: Laravel 4 - Creating a Simple CRUD Application in Hours [article] Your First Application [article] Creating and Using Composer Packages [article]

0
0
1378

How-To Tutorials - Programming

Setting up of Software Infrastructure on the Cloud

Event-driven BPEL Process

Redis in Autosuggest

Waiting for AJAX, as always…

Index, Item Sharding, and Projection in DynamoDB

Security Settings in Salesforce

Introducing LLVM Intermediate Representation

Using OSGi Services

The NServiceBus Architecture

New functionality in OpenCV 3.0

Trending Topics

BPMS Components

Program structure, execution flow, and runtime objects

The .NET Framework Primer

Configuration

Design patterns