How-To Tutorials

If you want to learn how to build artificial intelligence systems, the first step is simple: forget all about artificial intelligence. Instead focus your attention on machine learning. That way, you can be sure you’re in the domain of the practical rather than the domain of hype. Okay, this position might sound a little too dramatic. But there are a number of jokes doing the rounds on Twitter along these lines. Mat Velloso, an adviser to Satya Nadella at Microsoft, wrote late last year that “if it’s written in Python, it’s machine learning. If it’s written in PowerPoint, it’s probably AI.” https://twitter.com/matvelloso/status/1065778379612282885 There are similar jokes that focus on the use of the different words depending on whether you’re talking to investors or colleagues - either way, it’s clear that if you’re starting to explore artificial intelligence and machine learning, understanding what’s important and what you can ignore will help you to get a better view on where you need to go as your learning journey unfolds. So, once you understand that artificial intelligence is merely the word describing the end goal we’re trying to achieve, and machine learning is a means of achieving that goal, you can begin to start trying to develop intelligent systems yourself. Clearly, a question will keep cropping up: where next? Well, this post should go some way to helping you. Do you want to learn artificial intelligence? Read Packt's extensive Learning Path Python: Beginner's Guide to Artificial Intelligence. For a more advanced guide, check out Python: Advanced Guide to Artificial Intelligence. The basics of machine learning If you want to build artificial intelligence, you need to start by learning the basics of machine learning. Follow these steps: Get to grips with the basics of Python and core programming principles - if you’re reading this, you probably know enough to begin, but if you don’t there are plenty of resources to get you started. (We suggest you start with Learning Python) Make sure you understand basic statistical principles - machine learning is really just statistics, automated by code. Venturing further into machine learning and artificial intelligence The next step builds on those foundations. This is where you begin thinking about the sorts of problems you want to solve and the types of questions you want to ask. This is actually a creative step where you set the focus for your project - whatever kind of pattern or relationship you want to understand, this is where you can do just that. One of the difficulties, however, is making sure you have access to the data you need to actually do what you want. Sometimes, you might need to do some serious web scraping or data mining to get hold of the data you want - that’s beyond the scope of this piece, but there are plenty of resources out there to help you do just that. But there are also plenty of ready made data sets available for you to use in your machine learning project in whichever way you wish. You can find 50 data sets for machine learning here, all for a range of different uses. (If you’re trying machine learning for the first time, we’d suggest using one of these data sets and playing around to save you collecting data). Getting to grips with data modelling Although machine learning modelling is the next step in the learning journey, arguably it should happen at the same time as you’re thinking about both the questions you’re asking and the different data sources you might require. This is because the model - or models - you decide to employ in your project will follow directly from the problems you’re trying to tackle and, indeed, the nature and size of the data sets you eventually use. It’s important to note that no model is perfect. There’s a rule in the data science and machine learning world called the ‘no free lunch’ rule - basically, there’s no model that offers a short cut. There will always be trade offs between different algorithms in how they perform in various factors. To manage this issue you need to understand what’s important to you - maybe you’re not worried about speed, for example? Or perhaps accuracy isn’t crucial, you just want to build something that runs quickly. Broadly, the models you use will fall into these categories: supervised or unsupervised. Supervised machine learning algorithms Supervised learning is where you have an input and an output and you use an algorithm to better understand the relationship between the two. Ultimately, you want to get to a point when your machine learning system understands the relationship in such a way that you could predict an output. Supervised learning can also be broken down into regression or classification. Regression is where the output is a number or value, while classification is a specific category, or descriptor. Some algorithms can be used for both regression and classification problems, such as random forest, while others can be used for one or the other. For example, support vector machines can be used for classification problems, while linear regression algorithms can, as the name indicates, be used for regression problems. Unsupervised machine learning algorithms Unsupervised machine learning contrasts from supervised machine learning in that there are no outputs on which the algorithm works. If supervised learning 'tells' the algorithm the answers from which it then needs to understand how those answers were generated, unsupervised learning aims to understand the underlying structure within a given set of data. There aren’t any answers to guide the machine learning algorithm. As above, there are a couple of different approaches to unsupervised machine learning: clustering and association. Clustering helps you understand different groups within a set of data, while association is simply a way of understanding relationship or rules: if this happens, then this will happen too. Okay, so what about artificial intelligence? By now you will have a solid foundation of knowledge in machine learning. However, this is only the tip of the iceberg - machine learning at its most basic provides a very limited form of artificial intelligence. Advances in artificial intelligence are possible through ever more powerful algorithms - artificial or deep neural networks - that have additional layers of complexity (quite literally additional neurons). These are the algorithms that are used to power sophisticated applications and tools. From image recognition to image identification, through to speech to text and machine translation, the applications of these algorithms are radically transforming our relationship with technology. But you probably already knew that. The important question is how you actually go about doing it. Well, luckily in many ways, if you know the core components of machine learning, more advanced elements of deep learning and artificial neural networks shouldn’t actually be as complex as you might at first think. There are, however, a couple of considerations that become more important as you move deeper into deep learning. Hardware considerations for deep learning One of the most important considerations for any deep learning projects you want to try is the hardware you’re using. For a basic machine learning problem, this shouldn’t be an issue. However, but as the computations on which your deep learning system is working become more extensive, the hardware you use to run will become a challenge you need to resolve. This is too big an issue to explore here, but you can look in detail at our comparison of different processors here. Getting started with deep learning frameworks One of the reasons the whole world is talking about artificial intelligence is because it’s easier to do. And this is thanks, in part, to the growth of new deep learning frameworks that make it relatively straightforward to build complex deep learning models. The likes of TensorFlow, Keras, and PyTorch are all helping engineers and data scientists build deep learning models of considerable sophistication. Although they each have their own advantages, and it’s well worth spending some time comparing them, there’s certainly a lot to be said for simply getting started with them yourself. What about cloud's impact on machine learning and artificial intelligence? An interesting development in the machine learning space is the impact of cloud based solutions. The likes of Azure, AWS and Google Cloud Platform are all offering a number of different services and tools from within their overarching cloud products that make performing machine and deep learning tasks much easier. While this is undoubtedly going to be an important development, and, indeed, one you may have encountered already, there is no substitute for simply getting your hands dirty with the data and seeing how the core principles behind machine learning and artificial intelligence actually work. Conclusion: Don’t be scared, take machine learning and artificial intelligence one step at a time Clearly, with so much hype around artificial intelligence its easy to get stuck before you begin. However, by focusing on the core principles and practical application of machine learning you will be well on your way to helping drive the future of artificial intelligence. Learn artificial intelligence from scratch with Python: Beginner's Guide to Artificial Intelligence. Dive deeper into deep learning and artificial intelligence with Python: Advanced Guide to Artificial Intelligence.  

To implement a use-case, we need to use a well-known Spring module, Spring Web and Spring Web MVC. Our application will not use the new features of Spring 5, so it will run similarly on Spring Framework 4.x. This article is an excerpt taken from the book Hands-On Reactive Programming in Spring 5 by Oleh Dokuka and Igor Lozynskyi. This book covers the difference between a reactive system and reactive programming, the basics of reactive programming in Spring 5 and much more. In this article, you will learn how to bootstrap a Spring application, implement business logic, and much more. To bootstrap our application, we may configure and download a Gradle project from the Spring Initializer website at start.spring.io. For now, we need to select the preferred Spring Boot version and dependency for the web (the actual dependency identifier in Gradle config will be org.springframework.boot:spring-boot-starter-web), as shown in the following screenshot: Diagram 2.4 Web-based Spring Initializer simplifies the bootstrapping of a new Spring Boot application Alternatively, we may generate a new Spring Boot project using cURL and the HTTP API of the Spring Boot Initializer site. The following command will effectively create and download the same empty project with all the desired dependencies: curl https://start.spring.io/starter.zip \ -d dependencies=web,actuator \ -d type=gradle-project \ -d bootVersion=2.0.2.RELEASE \ -d groupId=com.example.rpws.chapters \ -d artifactId=SpringBootAwesome \ -o SpringBootAwesome.zip Implementing business logic We may now outline the design of our system in the following diagram: Diagram 2.5 Events flow from a temperature sensor to a user In this use case, the domain model will consist only of the Temperature class with the only double value inside. For simplicity purposes, it is also used as an event object, as shown in the following code: final class Temperature { private final double value; // constructor & getter... } To simulate the sensor, let's implement the TemperatureSensor class and decorate it with a @Component annotation to register the Spring bean, as follows: @Component public class TemperatureSensor { private final ApplicationEventPublisher publisher; // (1) private final Random rnd = new Random(); // (2) private final ScheduledExecutorService executor = // (3) Executors.newSingleThreadScheduledExecutor(); public TemperatureSensor(ApplicationEventPublisher publisher) { this.publisher = publisher; } @PostConstruct public void startProcessing() { // (4) this.executor.schedule(this::probe, 1, SECONDS); } private void probe() { // (5) double temperature = 16 + rnd.nextGaussian() * 10; publisher.publishEvent(new Temperature(temperature)); // schedule the next read after some random delay (0-5 seconds) executor .schedule(this::probe, rnd.nextInt(5000), MILLISECONDS); // (5.1) } } So, our simulated temperature sensor only depends on the ApplicationEventPublisher class (1), provided by Spring Framework. This class makes it possible to publish events to the system. It is a requirement to have a random generator (2) to contrive temperatures with some random intervals. An event generation process happens in a separate ScheduledExecutorService (3), where each event's generation schedules the next round of an event's generation with a random delay (5.1). All that logic is defined in the probe() method (5).  In turn, the mentioned class has the startProcessing() method annotated with @PostConstruct (4), which is called by Spring Framework when the bean is ready and triggers the whole sequence of random temperature values. Asynchronous HTTP with Spring Web MVC The introduced in Servlet 3.0 asynchronous support expands the ability to process an HTTP request in non-container threads. Such a feature is pretty useful for long-running tasks. With those changes, in Spring Web MVC we can return not only a value of type T in @Controller but also a Callable<T> or a DeferredResult<T>. The Callable<T> may be run inside a non-container thread, but still, it would be a blocking call. In contrast, DeferredResult<T> allows an asynchronous response generation on a non-container thread by calling the setResult(T result) method so it could be used within the event-loop. Starting from version 4.2, Spring Web MVC makes it possible to return ResponseBodyEmitter, which behaves similarly to DeferredResult, but can be used to send multiple objects, where each object is written separately with an instance of a message converter (defined by the HttpMessageConverter interface). The SseEmitter extends ResponseBodyEmitter and makes it possible to send many outgoing messages for one incoming request in accordance with SSE's protocol requirements. Alongside ResponseBodyEmitter and SseEmitter, Spring Web MVC also respects the StreamingResponseBody interface. When returned from @Controller, it allows us to send raw data (payload bytes) asynchronously. StreamingResponseBody may be very handy for streaming large files without blocking Servlet threads. Exposing the SSE (Server Sent Events) endpoint The next step requires adding the TemperatureController class with the @RestController annotation, which means that the component is used for HTTP communication, as shown in the following code: @RestController public class TemperatureController { private final Set<SseEmitter> clients = // (1) new CopyOnWriteArraySet<>(); @RequestMapping( value = "/temperature-stream", // (2) method = RequestMethod.GET) public SseEmitter events(HttpServletRequest request) { // (3) SseEmitter emitter = new SseEmitter(); // (4) clients.add(emitter); // (5) // Remove emitter from clients on error or disconnect emitter.onTimeout(() -> clients.remove(emitter)); // (6) emitter.onCompletion(() -> clients.remove(emitter)); // (7) return emitter; // (8) } @Async // (9) @EventListener // (10) public void handleMessage(Temperature temperature) { // (11) List<SseEmitter> deadEmitters = new ArrayList<>(); // (12) clients.forEach(emitter -> { try { emitter.send(temperature, MediaType.APPLICATION_JSON); // (13) } catch (Exception ignore) { deadEmitters.add(emitter); // (14) } }); clients.removeAll(deadEmitters); // (15) } } Now, to understand the logic of the TemperatureController class, we need to describe the SseEmitter. Spring Web MVC provides that class with the sole purpose of sending SSE events. When a request-handling method returns the SseEmitter instance, the actual request processing continues until SseEnitter.complete(), an error, or a timeout occurs. The TemperatureController provides one request handler (3) for the URI /temperature-stream (2) and returns the SseEmitter (8). In the case when a client requests that URI, we create and return the new SseEmitter instance (4) with its previous registration in the list of the active clients (5). Furthermore, the SseEmitter constructor may consume the timeout parameter. For the clients' collection, we may use the CopyOnWriteArraySet class from the java.util.concurrent package (1). Such an implementation allows us to modify the list and iterate over it at the same time. When a web client opens a new SSE session, we add a new emitter to the clients' collection. The SseEmitter removes itself from the clients' list when it has finished processing or has reached timeout (6) (7). Now, having a communication channel with clients means that we need to be able to receive events about temperature changes. For that purpose, our class has a handleMessage() method (11). It is decorated with the @EventListener annotation (10) in order to receive events from Spring. This framework will invoke the handleMessage() method only when receiving Temperature events, as this type of method's argument is known as temperature. The @Async annotation (9) marks a method as a candidate for the asynchronous execution, so it is invoked in the manually configured thread pool. The handleMessage() method receives a new temperature event and asynchronously sends it to all clients in JSON format in parallel for each event (13). Also, when sending to individual emitters, we track all failing ones (14) and remove them from the list of the active clients (15). Such an approach makes it possible to spot clients that are not operational anymore. Unfortunately, SseEmitter does not provide any callback for handling errors, and can be done by handling errors thrown by the send() method only. Configuring asynchronous support To run everything, we need an entry point for our application with the following customized methods: @EnableAsync // (1) @SpringBootApplication // (2) public class Application implements AsyncConfigurer { public static void main(String[] args) { SpringApplication.run(Application.class, args); } @Override public Executor getAsyncExecutor() { // (3) ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();// (4) executor.setCorePoolSize(2); executor.setMaxPoolSize(100); executor.setQueueCapacity(5); // (5) executor.initialize(); return executor; } @Override public AsyncUncaughtExceptionHandler getAsyncUncaughtExceptionHandler(){ return new SimpleAsyncUncaughtExceptionHandler(); // (6) } } As we can see, the example is a Spring Boot application (2), with an asynchronous execution enabled by the @EnableAsync annotation (1). Here, we may configure an exception handler for exceptions thrown from the asynchronous execution (6). That is also where we prepare Executor for asynchronous processing. In our case, we use ThreadPoolTaskExecutor with two core threads that may be increased to up to one hundred threads. It is important to note that without a properly configured queue capacity (5), the thread pool is not able to grow. That is because the SynchronousQueue would be used instead, limiting concurrency. Building a UI with SSE support The last thing that we need in order to complete our use case is an HTML page with some JavaScript code to communicate with the server. For the sake of conciseness, we will strip all HTML tags and leave only the minimum that is required to achieve a result, as follows: <body> <ul id="events"></ul> <script type="application/javascript"> function add(message) { const el = document.createElement("li"); el.innerHTML = message; document.getElementById("events").appendChild(el); } var eventSource = new EventSource("/temperature-stream"); // (1) eventSource.onmessage = e => { // (2) const t = JSON.parse(e.data); const fixed = Number(t.value).toFixed(2); add('Temperature: ' + fixed + ' C'); } eventSource.onopen = e => add('Connection opened'); // (3) eventSource.onerror = e => add('Connection closed'); // </script> </body> Here, we are using the EventSource object pointed at /temperature-stream (1). This handles incoming messages by invoking the onmessage() function (2), error handling, and reaction to the stream opening, which are done in the same fashion (3). We should save this page as index.html and put it in the src/main/resources/static/ folder of our project. By default, Spring Web MVC serves the content of the folder through HTTP. Such behavior could be changed by providing a configuration that extends the WebMvcConfigurerAdapter class. Verifying application functionality After rebuilding and completing our application's startup, we should be able to access the mentioned web page in a browser at the following address: http://localhost:8080 (Spring Web MVC uses port 8080 for the web server as the default one. However, this can be changed in the application.properties file using the configuration line server.port=9090). After a few seconds, we may see the following output: Connection opened Temperature: 14.71 C Temperature: 9.67 C Temperature: 19.02 C Connection closed Connection opened Temperature: 18.01 C Temperature: 16.17 C As we can see, our web page reactively receives events, preserving both client and server resources. It also supports auto-reconnect in the case of network issues or timeouts. As the current solution is not exclusive to JavaScript, we may connect with other clients for example, curl. By running the next command in a terminal, we receive the following stream of raw, but not formatted, events: > curl http://localhost:8080/temperature-stream data:{"value":22.33210856124129} data:{"value":13.83133638119636} In this article, we learned how to bootstrap a Spring application, implement business logic, and much more. To know more about the difference between a reactive system and reactive programming, check out the book Hands-On Reactive Programming in Spring 5 by Oleh Dokuka and Igor Lozynskyi. Netflix adopts Spring Boot as its core Java framework Implementing Dependency Injection in Spring [Tutorial] How to recover deleted data from an Android device [Tutorial]

Last Saturday, Google presented a paper in the Munich Security Conference titled How Google Fights Disinformation. In the paper, they explain what steps they’re taking against disinformation and detail their strategy for their platforms Google Search, News, YouTube, and Google Ads. We take a look at the key strategies that Google is taking against disinformation. Disinformation has become widespread in recent years. It directly affects Google’s mission of organizing the world’s information and making it accessible. Disinformation, misinformation, or fake new are deliberate attempts by acting parties to mislead people in believing things that aren’t true by spreading such content over the internet. Disinformation is deliberate attempts to mislead people where the creator knows that the information is false, misinformation is where the creator has their facts wrong and spreads wrong information unintentionally. The motivations behind it can be financial, political, or just for entertainment (trolls). Motivations can overlap with the content produced, moreover, the disinformation could also be for a good cause, making the fight against fake news very complex. A common solution for all platforms is not possible as different platforms pose different challenges. Making standards that exercise deep deliberation for individual cases is also not practical. There are three main principles that Google is outlining to combat disinformation, shown as follows. #1 Make quality content count Google products sort through a lot of information to display the most useful content first. They want to deliver quality content and legitimate commercial messages are prone to rumors. While the content is different on different Google platforms, the principles are similar: Organizing information by ranking algorithms. The algorithms are aimed to ensure that the information benefits users and is measured by user testing #2 Counter malicious actors Algorithms cannot determine if a piece of content is true or false based on current events. Neither can it determine the true intents of the content creator. For this, Google products have policies that prohibit certain behaviors like misinterpreting ownership of content. Certain users try to get a better ranking by practicing spam, such behavior is also shown by people who engage in spreading disinformation. Google has algorithms in place that can reduce such content and it’ll also be supported by human reviews for further filtering. #3 Giving users more choices Giving users different perspectives is important before they choose a link and proceed reading content or viewing a video. Hence, Google provides multiple links for a topic searched. Google search and other products now have additional UI elements to segregate information into different sections for an organized view of content. They also have a feedback button on their services via which users can submit their thoughts. Partnership with external experts Google cannot do this alone, hence they have partnered with supporting new organizations to create quality content that can uproot disinformation. They mention in the paper: “In March 2018, we launched the Google News Initiative (GNI) 3 to help journalism thrive in the digital age. With a $300 million commitment over 3 years, the initiative aims to elevate and strengthen quality journalism.” Preparing for the future People who create fake news will always try new methods to propagate it. Google is investing in research and development against it, now especially before the elections. They intend to stay ahead of the malicious actors who may use new technologies or tactics which can include deepfakes. They want to protect so that polling booths etc are easily available, guard against phishing, mitigate DDoS attacks on political websites. YouTube and conspiracy theories Recently, there have been a lot of conspiracy theories floating around on YouTube. In the paper, they say that: “YouTube has been developing products that directly address a core vulnerability involving the spread of disinformation in the immediate aftermath of a breaking news event.” Making a legitimate video with correct facts takes time, while disinformation can be created quickly for spreading panic/negativity etc,. In conclusion they, note that “fighting disinformation is not a straightforward endeavor. Disinformation and misinformation can take many shapes, manifest differently in different products, and raise significant challenges when it comes to balancing risks of harm to good faith, free expression, with the imperative to serve users with information they can trust.” Public reactions People think that only the platforms themselves can take actions against disinformation propaganda. https://twitter.com/halhod/status/1097640819102691328 Users question Google’s efforts in cases where the legitimate website is shown after the one with disinformation with an example of Bitcoin. https://twitter.com/PilotDaveCrypto/status/1097395466734653440 Some speculate that corporate companies should address their own bias of ranking pages first: https://twitter.com/PaulJayzilla/status/1097822412815646721 https://twitter.com/Darin_T80/status/1097203275483426816 To read the complete research paper with Google product-specific details on fighting disinformation, you can head on to the Google Blog. Fake news is a danger to democracy. These researchers are using deep learning to model fake news to understand its impact on elections. Defending Democracy Program: How Microsoft is taking steps to curb increasing cybersecurity threats to democracy Is Anti-trust regulation coming to Facebook following fake news inquiry made by a global panel in the House of Commons, UK?

When dealing with security, it makes sense to keep those levels in mind in order to approach security-related issues in an organized way. Bind addresses: listen_addresses in the postgresql.conf file Host-based access control: The pg_hba.conf file Instance-level permissions: Users, roles, database creation, login, and replication Database-level permissions: Connecting, creating schemas, and so on Schema-level permissions: Using schema and creating objects inside a schema Table-level permissions: Selecting, inserting, updating, and so on Column-level permissions: Allowing or restricting access to columns Row-level security: Restricting access to rows In order to read a value, PostgreSQL has to ensure that we have sufficient permissions on every level. The entire chain of permissions has to be correct. This article is an excerpt taken from the book Mastering PostgreSQL 11 - Second Edition by Hans-Jürgen Schönig. In this book, you will learn the approach to get to grips with advanced PostgreSQL 11 features and SQL functions, master replication and failover techniques, configure database security and more. In this article, you will learn the process of handling SSL, column-level security and configuring default privileges and much more. Understanding bind addresses and connections When configuring a PostgreSQL server, one of the first things that needs to be done is define remote access. By default, PostgreSQL does not accept remote connections. The important thing here is that PostgreSQL does not even reject the connection because it simply does not listen on the port. If we try to connect, the error message will actually come from the operating system because PostgreSQL does not care at all. Assuming that there is a database server using the default configuration on 192.168.0.123, the following will happen: iMac:~ hs$ telnet 192.168.0.123 5432 Trying 192.168.0.123... telnet: connect to address 192.168.0.123: Connection refused telnet: Unable to connect to remote host Telnet tries to create a connection on port 5432 and is instantly rejected by the remote box. From the outside, it looks as if PostgreSQL is not running at all. The key to success can be found in the postgresql.conf file: # - Connection Settings - # listen_addresses = 'localhost' # what IP address(es) to listen on; # comma-separated list of addresses; # defaults to 'localhost'; use '*' for all # (change requires restart) The listen_addresses setting will tell PostgreSQL which addresses to listen on. Technically speaking, those addresses are bind addresses. What does that actually mean? Suppose we have four network cards in our machine. We can listen on, say, three of those IP addresses. PostgreSQL takes requests to those three cards into account and does not listen on the fourth one. The port is simply closed. If we put an * in, PostgreSQL will listen to every IP assigned to your machine. However, there are more settings related to connection management that are highly important to understand. They are as follows: #port = 5432 # (change requires restart) max_connections = 100 # (change requires restart) # Note: Increasing max_connections costs ~400 bytes of # shared memory per # connection slot, plus lock space # (see max_locks_per_transaction). #superuser_reserved_connections = 3 # (change requires restart) #unix_socket_directories = '/tmp' # comma-separated list of directories # (change requires restart) #unix_socket_group = '' # (change requires restart) #unix_socket_permissions = 0777 # begin with 0 to use octal notation # (change requires restart) First of all, PostgreSQL listens to a single TCP port, the default value of which is is 5432. Keep in mind that PostgreSQL will listen on a single port only. Whenever a request comes in, the postmaster will fork and create a new process to handle the connection. By default, up to 100 normal connections are allowed. On top of that, three additional connections are reserved for superusers. This means that we can either have 97 connections plus three superusers or 100 superuser connections. Handling SSL PostgreSQL allows us to encrypt the transfer between the server and the client. Encryption is highly beneficial, especially if we are communicating over long distances. SSL offers a simple and secure way to ensure that nobody is able to listen to your communication. In this section, we will learn how to set up SSL. The first thing to do is to set the ssl parameter to on in the postgresql.conf file when the server starts. In the next step, we can put SSL certificates into the $PGDATA directory. If we don't want the certificates to be in some other directory, change the following parameters: #ssl_cert_file = 'server.crt' # (change requires restart) #ssl_key_file = 'server.key' # (change requires restart) #ssl_ca_file = '' # (change requires restart) #ssl_crl_file = '' # (change requires restart) If we want to use self-signed certificates, perform the following step: openssl req -new -text -out server.req Answer the questions asked by OpenSSL. Make sure that we enter the local hostname as common name. We can leave the password empty. This call will generate a key that is passphrase protected; it will not accept a passphrase that is less than four characters long. To remove the passphrase (as you must if you want automatic startup of the server), run the following commands: openssl rsa -in privkey.pem -out server.key rm privkey.pem Enter the old passphrase to unlock the existing key. Now, do this to turn the certificate into a self-signed certificate and to copy the key and certificate to where the server will look for them: openssl req -x509 -in server.req -text -key server.key -out server.crt After doing this, make sure that the files have the right set of permissions: chmod og-rwx server.key Once the proper rules have been put into the pg_hba.conf file, we can use SSL to connect to your server. To verify that we are indeed using SSL, consider checking out the pg_stat_ssl function. It will tell us every connection and whether it uses SSL or not, and it will provide some important information about encryption: test=# \d pg_stat_sslView "pg_catalog.pg_stat_ssl" Column | Type | Modifiers -------------+----------+----------- pid | integer | ssl | boolean | version | text | cipher | text | bits | integer | compression | boolean | clientdn | text | If the ssl field for a process contains true; PostgreSQL does what we would expect it to do: postgres=# select * from pg_stat_ssl; -[ RECORD 1 ] ---------------------------- pid | 20075 ssl | t version | TLSv1.2 cipher | ECDHE-RSA-AES256-GCM-SHA384 bits | 256 compression | f clientdn | Handling instance-level security So far, we have configured bind addresses and we have told PostgreSQL which means of authentication to use for which IP ranges. Up to now, the configuration was purely network-related. In the next step, we can shift our attention to permissions at the instance level. The most important thing to know is that users in PostgreSQL exist at the instance level. If we create a user, it is not just visible inside one database; it can be seen by all the databases. A user might have permissions to access just a single database, but basically users are created at the instance level. To those of you who are new to PostgreSQL, there is one more thing you should keep in mind: users and roles are the same thing. CREATE ROLE and CREATE USER clauses have different default values (literally, the only difference is that roles do not get the LOGIN attribute by default), but at the end of the day, users and roles are the same. Therefore, CREATE ROLE and CREATE USER clauses support the very same syntax: test=# \h CREATE USER Command: CREATE USER Description: define a new database role Syntax: CREATE USER name [ [ WITH ] option [ ... ] ] where option can be: SUPERUSER | NOSUPERUSER | CREATEDB | NOCREATEDB | CREATEROLE | NOCREATEROLE | INHERIT | NOINHERIT | LOGIN | NOLOGIN | REPLICATION | NOREPLICATION | BYPASSRLS | NOBYPASSRLS | CONNECTION LIMIT connlimit | [ ENCRYPTED ] PASSWORD 'password' | VALID UNTIL 'timestamp' | IN ROLE role_name [, ...] | IN GROUP role_name [, ...] | ROLE role_name [, ...] | ADMIN role_name [, ...] | USER role_name [, ...] | SYSID uid Let's discuss those syntax elements one by one. The first thing we see is that a user can be a superuser or a normal user. If somebody is marked as a SUPERUSER , there are no longer any restrictions that a normal user has to face. A SUPERUSER can drop objects (databases and so on) as they wish. The next important thing is that it takes permissions on the instance level to create a new database. The rule is this: the creator is always automatically the owner of an object (unless specified otherwise, as can be done with the CREATE DATABASE clause). The beauty is that object owners can also drop an object again. The next important thing is the INHERIT/NOINHERIT clause. If the INHERIT clause is set (which is the default value), a user can inherit permissions from some other user. Using inherited permissions allows us to use roles, which is as a good way to abstract permissions. For example, we can create the role of bookkeeper and make many other roles inherit from bookkeeper. The idea is that we only have to tell PostgreSQL once what a bookkeeper is allowed to do, even if we have many people working in accounting. The LOGIN/NOLOGIN clause defines whether a role is allowed to log in to the instance. After this theoretical introduction, it is time to actually create users and see how things can be used in a practical example: test=# CREATE ROLE bookkeeper NOLOGIN; CREATE ROLE test=# CREATE ROLE joe LOGIN; CREATE ROLE test=# GRANT bookkeeper TO joe; GRANT ROLE The first thing done here is that a role called bookkeeper is created. Note that we don't want people to log in as bookkeeper, so the role is marked as NOLOGIN. Also note that NOLOGIN is the default value if you use the CREATE ROLE clause. If you prefer the CREATE USER clause, the default setting is LOGIN. Then, the joe role is created and marked as LOGIN. Finally, the bookkeeper role is assigned to the joe role so that he can do everything a bookkeeper is actually allowed to do. Once the users are in place, we can test what we have so far: [hs@zenbook ~]$ psql test -U bookkeeper psql: FATAL: role "bookkeeper" is not permitted to log in As expected, the bookkeeper role is not allowed to log in to the system. What happens if the joe role tries to log in? [hs@zenbook ~]$ psql test -U joe ... test=> This will actually work as expected. However, note that Command Prompt has changed. This is just a way for PostgreSQL to show you that you are not logged in as a superuser. Once a user has been created, it might be necessary to modify it. One thing we might want to change is the password. In PostgreSQL, users are allowed to change their own passwords. Here is how it works: test=> ALTER ROLE joe PASSWORD 'abc'; ALTER ROLE test=> SELECT current_user; current_user -------------- joe (1 row) The ALTER ROLE clause (or ALTER USER) will allow us to change most settings which can be set during user creation. However, there is even more to managing users. In many cases, we want to assign special parameters to a user. The ALTER USER clause gives us the means to do that: ALTER ROLE { role_specification | ALL } [ IN DATABASE database_name ] SET configuration_parameter { TO | = } { value | DEFAULT } ALTER ROLE { role_specification | ALL } [ IN DATABASE database_name ] SET configuration_parameter FROM CURRENT ALTER ROLE { role_specification | ALL } [ IN DATABASE database_name ] RESET configuration_parameter ALTER ROLE { role_specification | ALL } [ IN DATABASE database_name ] RESET ALL The syntax is fairly simple and pretty straightforward. To depict why this is really useful, I have added a real-world example. Let's suppose that Joe happens to live on the island of Mauritius. When he logs in, he wants to be in his own time zone, even if his database server is located in Europe: test=> ALTER ROLE joe SET TimeZone = 'UTC-4'; ALTER ROLE test=> SELECT now(); now ------------------------------- 2017-01-09 20:36:48.571584+01 (1 row) test=> q [hs@zenbook ~]$ psql test -U joe ... test=> SELECT now(); now ------------------------------- 2017-01-09 23:36:53.357845+04 (1 row) The ALTER ROLE clause will modify the user. As soon as joe reconnects, the time zone will already be set for him. The time zone is not changed immediately. You should either reconnect or use a SET ... TO DEFAULT clause. The important thing here is that this is also possible for some memory parameters, such as work_mem and so on. Security at the database level After configuring users at the instance level, it is possible to dig deeper and see what can be done at the database level. The first major question that arises is: we explicitly allowed Joe to log in to the database instance, but who or what allowed Joe to actually connect to one of the databases? Maybe we don't want Joe to access all the databases in your system. Restricting access to certain databases is exactly what we can achieve on this level. For databases, the following permissions can be set using a GRANT clause: GRANT { { CREATE | CONNECT | TEMPORARY | TEMP } [, ...] | ALL [ PRIVILEGES ] } ON DATABASE database_name [, ...] TO role_specification [, ...] [ WITH GRANT OPTION ] There are two major permissions on the database level that deserve close attention: CREATE: This allows somebody to create a schema inside the database. Note that a CREATE clause does not allow for the creation of tables; it is about schemas. In PostgreSQL, a table resides inside a schema, so you have to get to the schema level first to be able to create a table. CONNECT: This allows somebody to connect to a database. The question now is: nobody has explicitly assigned CONNECT permissions to the joe role, so where do those permissions actually come from? The answer is this: there is a thing called public, which is similar to the Unix world. If the world is allowed to do something, so is joe, who is part of the general public. The main thing is that public is not a role in the sense that it can be dropped and renamed. We can simply see it as the equivalent for everybody on the system. So, to ensure that not everybody can connect to any database at any time, CONNECT may have to be revoked from the general public. To do so, we can connect as superuser and fix the problem: [hs@zenbook ~]$ psql test -U postgres ... test=# REVOKE ALL ON DATABASE test FROM public; REVOKE test=# \q [hs@zenbook ~]$ psql test -U joe psql: FATAL: permission denied for database "test" DETAIL: User does not have CONNECT privilege. As we can see, the joe role is not allowed to connect anymore. At this point, only superusers have access to test. In general, it is a good idea to revoke permissions from the postgres database even before other databases are created. The idea behind this concept is that those permissions won't be in all those newly created databases anymore. If somebody needs access to a certain database, rights have to be explicitly granted. Rights are not automatically there anymore. If we want to allow the joe role to connect to the test database, try the following line as superuser: [hs@zenbook ~]$ psql test -U postgres ... test=# GRANT CONNECT ON DATABASE test TO bookkeeper; GRANT test=# \q [hs@zenbook ~]$ psql test -U joe ... test=> Basically, there are two choices here: We can allow the joe role directly so that only the joe role will be able to connect. Alternatively, we can grant permissions to the bookkeeper role. Remember, the joe role will inherit all the permissions from the bookkeeper role, so if we want all accountants to be able to connect to the database, assigning permissions to the bookkeeper role seems such as an attractive idea. If we grant permissions to the bookkeeper role, it is not risky because the role is not allowed to log in to the instance in the first place, so it purely serves as a source of permissions. Handling column-level security In some cases, not everybody is allowed to see all the data. Just imagine a bank. Some people might see the entire information about a bank account, while others might be limited to only a subset of the data. In a real-world situation, somebody might not be allowed to read the balance column or somebody might not see the interest rates of people's loans. Another example would be that people are allowed to see people's profiles but not their pictures or some other private information. The question now is: how can column-level security be used? To demonstrate that, we will add a column to the existing table belonging to the joe role: test=> ALTER TABLE t_useful ADD COLUMN name text; ALTER TABLE The table now consists of two columns. The goal of the example is to ensure that a user can see only one of those columns: test=> \d t_useful Table "public.t_useful" Column | Type | Modifiers --------+---------+----------- id | integer | name | text | As a superuser, let's create a user and give it access to the schema containing our table: test=# CREATE ROLE paul LOGIN; CREATE ROLE test=# GRANT CONNECT ON DATABASE test TO paul; GRANT test=# GRANT USAGE ON SCHEMA public TO paul; GRANT CONNECT was revoked from public. Explicit granting is therefore absolutely necessary to ensure that we can even get to the table. The SELECT permissions can be given to the paul role: test=# GRANT SELECT (id) ON t_useful TO paul; GRANT Basically, this is already enough. It is already possible to connect to the database as user paul and read the column: [hs@zenbook ~]$ psql test -U paul ... test=> SELECT id FROM t_useful; id ---- (0 rows) If we are using column-level permissions, there is an important thing to keep in mind; we should stop using SELECT *, as it does not work anymore: test=> SELECT * FROM t_useful; ERROR: permission denied for relation t_useful * still means all columns, but as there is no way to access all columns, things will error out instantly. Configuring default privileges So far, a lot of stuff has already been configured. What happens if new tables are added to the system? It can be quite painful and risky to process these tables one by one and to set proper permissions. Wouldn't it be nice if those things would just happen automatically? This is exactly what the ALTER DEFAULT PRIVILEGES clause does. The idea is to give users an option to make PostgreSQL automatically set the desired permissions as soon as an object comes into existence. It cannot happen anymore that somebody simply forgets to set those rights. The following listing shows the first part of the syntax specification: postgres=# \h ALTER DEFAULT PRIVILEGES Command: ALTER DEFAULT PRIVILEGES Description: define default access privileges Syntax: ALTER DEFAULT PRIVILEGES [ FOR { ROLE | USER } target_role [, ...] ] [ IN SCHEMA schema_name [, ...] ] abbreviated_grant_or_revoke where abbreviated_grant_or_revoke is one of: GRANT { { SELECT | INSERT | UPDATE | DELETE | TRUNCATE | REFERENCES | TRIGGER } [, ...] | ALL [ PRIVILEGES ] } ON TABLES TO { [ GROUP ] role_name | PUBLIC } [, ...] [ WITH GRANT OPTION ] ... Basically, the syntax works similar to the GRANT clause and is therefore easy and intuitive to use. To show us how it works, I compiled a simple example. The idea is that if the joe role creates a table, the paul role will automatically be able to use it: test=# ALTER DEFAULT PRIVILEGES FOR ROLE joe IN SCHEMA public GRANT ALL ON TABLES TO paul; ALTER DEFAULT PRIVILEGES Let's connect as the joe role now and create a table: [hs@zenbook ~]$ psql test -U joe ... test=> CREATE TABLE t_user (id serial, name text, passwd text); CREATE TABLE Connecting as the paul role will prove that the table has been assigned to the proper set of permissions: [hs@zenbook ~]$ psql test -U paul ... test=> SELECT * FROM t_user; id | name | passwd ----+------+-------- (0 row In the article, we covered the process of handling SSL, column-level security and configuring default privileges and much more. To know about row-level security and get detailed information about advanced PostgreSQL 11 features, check out the book Mastering PostgreSQL 11 - Second Edition. PostgreSQL wins ‘DBMS of the year’ 2018 beating MongoDB and Redis in DB-Engines Ranking Devart releases standard edition of dbForge Studio for PostgreSQL PipelineDB 1.0.0, the high performance time-series aggregation for PostgreSQL, released!

A siamese network is a special type of neural network and it is one of the simplest and most popularly used one-shot learning algorithms. One-shot learning is a technique where we learn from only one training example per class. So, a siamese network is predominantly used in applications where we don't have many data points in each class. For instance, let's say we want to build a face recognition model for our organization and about 500 people are working in our organization. If we want to build our face recognition model using a Convolutional Neural Network (CNN) from scratch, then we need many images of all of these 500 people for training the network and attaining good accuracy. But apparently, we will not have many images for all of these 500 people and so it is not feasible to build a model using a CNN or any deep learning algorithm unless we have sufficient data points. So, in these kinds of scenarios, we can resort to a sophisticated one-shot learning algorithm such as a siamese network, which can learn from fewer data points. Siamese networks basically consist of two symmetrical neural networks both sharing the same weights and architecture and both joined together at the end using some energy function, E. The objective of our siamese network is to learn whether two input values are similar or dissimilar. We will understand the siamese network by building a face recognition model. The objective of our network is to understand whether two faces are similar or dissimilar. We use the AT&T Database of Faces, which can be downloaded from the Cambridge University Computer Laboratory website. This article is an excerpt from a book written by Sudharsan Ravichandiran titled Hands-On Meta-Learning with Python. In this book, you will learn how to build relation networks and matching networks from scratch. Once you have downloaded and extracted the archive, you can see the folders s1, s2, up to s40, as shown here:   Each of these folders has 10 different images of a single person taken from various angles. For instance, let's open folder s1. As you can see, there are 10 different images of a single person:   We open and check folder s13: Siamese networks require input values as a pair along with the label, so we have to create our data in such a way. So, we will take two images randomly from the same folder and mark them as a genuine pair and we will take single images from two different folders and mark them as an imposite pair. A sample is shown in the following screenshot; as you can see, a genuine pair has images of the same person and the imposite pair has images of different people: Once we have our data as pairs along with their labels, we train our siamese network. From the image pair, we feed one image to network A and another image to network B. The role of these two networks is only to extract the feature vectors. So, we use two convolution layers with rectified linear unit (ReLU) activations for extracting the features. Once we have learned the features, we feed the resultant feature vector from both of the networks to the energy function, which measures the similarity; we use Euclidean distance as our energy function. So, we train our network by feeding the image pair to learn the semantic similarity between them. Now, we will see this step by step. For better understanding, you can check the complete code, which is available as a Jupyter Notebook with an explanation from GitHub. First, we will import the required libraries: import re import numpy as np from PIL import Image from sklearn.model_selection import train_test_split from keras import backend as K from keras.layers import Activation from keras.layers import Input, Lambda, Dense, Dropout, Convolution2D, MaxPooling2D, Flatten from keras.models import Sequential, Model from keras.optimizers import RMSprop Now, we define a function for reading our input image. The read_image function takes as input an image and returns a NumPy array: def read_image(filename, byteorder='>'): #first we read the image, as a raw file to the buffer with open(filename, 'rb') as f: buffer = f.read() #using regex, we extract the header, width, height and maxval of the image header, width, height, maxval = re.search( b"(^P5\s(?:\s*#.*[\r\n])*" b"(\d+)\s(?:\s*#.*[\r\n])*" b"(\d+)\s(?:\s*#.*[\r\n])*" b"(\d+)\s(?:\s*#.*[\r\n]\s)*)", buffer).groups() #then we convert the image to numpy array using np.frombuffer which interprets buffer as one dimensional array return np.frombuffer(buffer, dtype='u1' if int(maxval) < 256 else byteorder+'u2', count=int(width)*int(height), offset=len(header) ).reshape((int(height), int(width))) For an example, let's open one image: Image.open("data/orl_faces/s1/1.pgm") When we feed this image to our read_image function, it will return as a NumPy array: img = read_image('data/orl_faces/s1/1.pgm') img.shape (112, 92) Now, we define another function, get_data, for generating our data. As we know, for the siamese network, data should be in the form of pairs (genuine and imposite) with a binary label. First, we read the (img1, img2) images from the same directory and store them in the x_genuine_pair array and assign y_genuine to 1. Next, we read the (img1, img2) images from the different directory and store them in the x_imposite pair and assign y_imposite to 0. Finally, we concatenate both x_genuine_pair and x_imposite to X and y_genuine and y_imposite to Y: size = 2 total_sample_size = 10000 def get_data(size, total_sample_size): #read the image image = read_image('data/orl_faces/s' + str(1) + '/' + str(1) + '.pgm', 'rw+') #reduce the size image = image[::size, ::size] #get the new size dim1 = image.shape[0] dim2 = image.shape[1] count = 0 #initialize the numpy array with the shape of [total_sample, no_of_pairs, dim1, dim2] x_geuine_pair = np.zeros([total_sample_size, 2, 1, dim1, dim2]) # 2 is for pairs y_genuine = np.zeros([total_sample_size, 1]) for i in range(40): for j in range(int(total_sample_size/40)): ind1 = 0 ind2 = 0 #read images from same directory (genuine pair) while ind1 == ind2: ind1 = np.random.randint(10) ind2 = np.random.randint(10) # read the two images img1 = read_image('data/orl_faces/s' + str(i+1) + '/' + str(ind1 + 1) + '.pgm', 'rw+') img2 = read_image('data/orl_faces/s' + str(i+1) + '/' + str(ind2 + 1) + '.pgm', 'rw+') #reduce the size img1 = img1[::size, ::size] img2 = img2[::size, ::size] #store the images to the initialized numpy array x_geuine_pair[count, 0, 0, :, :] = img1 x_geuine_pair[count, 1, 0, :, :] = img2 #as we are drawing images from the same directory we assign label as 1. (genuine pair) y_genuine[count] = 1 count += 1 count = 0 x_imposite_pair = np.zeros([total_sample_size, 2, 1, dim1, dim2]) y_imposite = np.zeros([total_sample_size, 1]) for i in range(int(total_sample_size/10)): for j in range(10): #read images from different directory (imposite pair) while True: ind1 = np.random.randint(40) ind2 = np.random.randint(40) if ind1 != ind2: break img1 = read_image('data/orl_faces/s' + str(ind1+1) + '/' + str(j + 1) + '.pgm', 'rw+') img2 = read_image('data/orl_faces/s' + str(ind2+1) + '/' + str(j + 1) + '.pgm', 'rw+') img1 = img1[::size, ::size] img2 = img2[::size, ::size] x_imposite_pair[count, 0, 0, :, :] = img1 x_imposite_pair[count, 1, 0, :, :] = img2 #as we are drawing images from the different directory we assign label as 0. (imposite pair) y_imposite[count] = 0 count += 1 #now, concatenate, genuine pairs and imposite pair to get the whole data X = np.concatenate([x_geuine_pair, x_imposite_pair], axis=0)/255 Y = np.concatenate([y_genuine, y_imposite], axis=0) return X, Y Now, we generate our data and check our data size. As you can see, we have 20,000 data points and, out of these, 10,000 are genuine pairs and 10,000 are imposite pairs: X, Y = get_data(size, total_sample_size) X.shape (20000, 2, 1, 56, 46) Y.shape (20000, 1) Next, we split our data for training and testing with 75% training and 25% testing proportions: x_train, x_test, y_train, y_test = train_test_split(X, Y, test_size=.25) Now that we have successfully generated our data, we build our siamese network. First, we define the base network, which is basically a convolutional network used for feature extraction. We build two convolutional layers with ReLU activations and max pooling followed by a flat layer: def build_base_network(input_shape): seq = Sequential() nb_filter = [6, 12] kernel_size = 3 #convolutional layer 1 seq.add(Convolution2D(nb_filter[0], kernel_size, kernel_size, input_shape=input_shape, border_mode='valid', dim_ordering='th')) seq.add(Activation('relu')) seq.add(MaxPooling2D(pool_size=(2, 2))) seq.add(Dropout(.25)) #convolutional layer 2 seq.add(Convolution2D(nb_filter[1], kernel_size, kernel_size, border_mode='valid', dim_ordering='th')) seq.add(Activation('relu')) seq.add(MaxPooling2D(pool_size=(2, 2), dim_ordering='th')) seq.add(Dropout(.25)) #flatten seq.add(Flatten()) seq.add(Dense(128, activation='relu')) seq.add(Dropout(0.1)) seq.add(Dense(50, activation='relu')) return seq Next, we feed the image pair to the base network, which will return the embeddings, that is, feature vectors: input_dim = x_train.shape[2:] img_a = Input(shape=input_dim) img_b = Input(shape=input_dim) base_network = build_base_network(input_dim) feat_vecs_a = base_network(img_a) feat_vecs_b = base_network(img_b) feat_vecs_a and feat_vecs_b are the feature vectors of our image pair. Next, we feed these feature vectors to the energy function to compute the distance between them, and we use Euclidean distance as our energy function: def euclidean_distance(vects): x, y = vects return K.sqrt(K.sum(K.square(x - y), axis=1, keepdims=True)) def eucl_dist_output_shape(shapes): shape1, shape2 = shapes return (shape1[0], 1) distance = Lambda(euclidean_distance, output_shape=eucl_dist_output_shape)([feat_vecs_a, feat_vecs_b]) Now, we set the epoch length to 13, and we use the RMS prop for optimization and define our model: epochs = 13 rms = RMSprop() model = Model(input=[input_a, input_b], output=distance) Next, we define our loss function as the contrastive_loss function and compile the model: def contrastive_loss(y_true, y_pred): margin = 1 return K.mean(y_true * K.square(y_pred) + (1 - y_true) * K.square(K.maximum(margin - y_pred, 0))) model.compile(loss=contrastive_loss, optimizer=rms) Now, we train our model: img_1 = x_train[:, 0] img_2 = x_train[:, 1] model.fit([img_1, img_2], y_train, validation_split=.25, batch_size=128, verbose=2, nb_epoch=epochs) You can see how the loss decreases over epochs: Train on 11250 samples, validate on 3750 samples Epoch 1/13 - 60s - loss: 0.2179 - val_loss: 0.2156 Epoch 2/13 - 53s - loss: 0.1520 - val_loss: 0.2102 Epoch 3/13 - 53s - loss: 0.1190 - val_loss: 0.1545 Epoch 4/13 - 55s - loss: 0.0959 - val_loss: 0.1705 Epoch 5/13 - 52s - loss: 0.0801 - val_loss: 0.1181 Epoch 6/13 - 52s - loss: 0.0684 - val_loss: 0.0821 Epoch 7/13 - 52s - loss: 0.0591 - val_loss: 0.0762 Epoch 8/13 - 52s - loss: 0.0526 - val_loss: 0.0655 Epoch 9/13 - 52s - loss: 0.0475 - val_loss: 0.0662 Epoch 10/13 - 52s - loss: 0.0444 - val_loss: 0.0469 Epoch 11/13 - 52s - loss: 0.0408 - val_loss: 0.0478 Epoch 12/13 - 52s - loss: 0.0381 - val_loss: 0.0498 Epoch 13/13 - 54s - loss: 0.0356 - val_loss: 0.0363 Now, we make predictions with test data: pred = model.predict([x_test[:, 0], x_test[:, 1]]) Next, we define a function for computing accuracy: def compute_accuracy(predictions, labels): return labels[predictions.ravel() < 0.5].mean() Now, we compute the accuracy of model: compute_accuracy(pred, y_test) 0.9779092702169625 In this tutorial, we have learned to build face recognition models using siamese networks. The architecture of siamese networks, basically consists of two identical neural networks both having the same weights and architecture and the output of these networks is plugged into some energy function to understand the similarity. To learn more about meta-learning with Python, check out the book Hands-On Meta-Learning with Python. What is Meta-Learning? Introducing Open AI’s Reptile: The latest scalable meta-learning Algorithm on the block “Deep meta reinforcement learning will be the future of AI where we will be so close to achieving artificial general intelligence (AGI)”, Sudharsan Ravichandiran

REST is an architectural style for implementing communication between the application client and server over HTTP. RESTful APIs use HTTP verbs (POST, GET, PUT, DELETE, and so on) to dictate the operation to be performed (Create, Read, Update, Delete) by the server on the domain entity. The REST style has become the de facto standard for creating services in modern application development. This makes it easy to use and consume services in any technology and on any platform, such as web frontends, desktop applications, or other web services. This article is an excerpt from a book written by Tamir Dresher, Amir Zuker, and Shay Friedman titled Hands-On Full-Stack Web Development with ASP.NET Core. In this book, you will learn how to build RESTful APIs in C# with ASP.NET Core, web APIs, and Entity Framework. Overview — REST APIs with ASP.NET Core API A basic ASP.NET Core MVC application can be broken down into three layers: models, controllers, and views. RESTful APIs in ASP.NET Core work very similarly; the only difference is that, instead of returning responses as visual views, the API response is a payload of data (usually in JSON format). The data returned from the API is later consumed by clients, such as Angular-based applications that can render the data as views, or by headless clients that have no UI and simply process data. For example, consider a background process that periodically sends notifications to a user about their account status: Before ASP.NET Core, Microsoft created an explicit distinction between ASP.NET MVC and the ASP.NET Web API. The former was used to create web applications with views that are generated by the server, while the former was used to create services that contain only logic and can be consumed by any client. Over time, the distinction between the two frameworks caused duplication of code and added a burden on the developers who needed to learn and master two technologies. ASP.NET Core unified the two frameworks into the ASP.NET Core MVC suite, and made it simpler to create web applications, with or without visual responses. Let's start with a simple API that will be the foundation for our application called, say, GiveNTake application. Creating a simple API The GiveNTake application allows the user to see a catalog of available products. For this to be possible, our server needs to include a specific API method that the client application can call and get back the collection of available products. We will treat the product as a simple string in the format of [Product ID] - [Product Name]: Open the any basic project you may have created, and add a new controller class to the Controllers folder.  Right-click on the newly created Controllers folder and choose Add | Controller. In the list of available templates, choose API Controller - Empty and click Add: Set the name to ProductsController and click Add. Add the following method to the generated controller: public string[] GetProducts() { return new[] { "1 - Microwave", "2 - Washing Machine", "3 - Mirror" }; } Your controller should look like this: [Route("api/Products")] [ApiController] public class ProductsController : Controller { public string[] GetProducts() { return new[] { "1 - Microwave", "2 - Washing Machine", "3 - Mirror" }; } } Congratulations! You have completed coding your first API method. The GetProducts method returns the collection of the available products. To see it in action, build and run your project. This will open a browser with the base address of your ASP.NET application. Add the  /api/Products string to the base address in the browser's address bar and execute it. You should see the collection of strings appear on the screen like so: Inspecting your APIs using Fiddler and Postman Using your browser to execute your APIs is nice enough for simple APIs that retrieve data, but as you go along and extend your APIs, you'll soon find that you need other powerful tools to test and debug what you develop. There are many tools that let you inspect and debug your APIs, but I have chosen to teach you about Fiddler and Postman because they are both simple and powerful. Fiddler Fiddler is a free web debugging tool that works as a proxy, logging all HTTP(S) traffic that is executed by processes in your computer. Fiddler allows you to inspect the traffic to see that exact HTTP request that was sent and the exact HTTP response that was returned. You can also use other advanced features, such as setting breakpoints and overriding the data that is sent or received. To install Fiddler, navigate to https://www.telerik.com/fiddler and click the Free download button. Save and run the installer: The Fiddler main screen is built from these main parts: Sessions list: Shows the HTTP(S) requests that were sent from processes in your machine Fiddler tabs: Contains different tools for inspecting and controlling sessions Request inspector: When the Inspectors tab and inner Raw tab are selected, this section shows the request as it was sent over-the-wire Response inspector: When the Inspectors tab and inner Raw tab are selected, this section shows the response as it was sent over-the-wire Immediately after you run Fiddler, it starts collecting the HTTP sessions that are performed in your machine. If you refresh the browser that you used to navigate to the /api/Products API  you created, you should see this session in Fiddler's Sessions List, as shown in the preceding screenshot. If you run a .NET application that sends HTTP requests to an address in your localhost, you won't see the session appear in Fiddler. Changing the address to localhost.fiddler will force the request to be captured by Fiddler. Fiddler is a great tool for debugging the requests and responses that are made in your application, but it means that you need to have a client that sends those requests. Many times when debugging and experimenting with APIs, you want to create HTTP requests manually and inspect them. You can accomplish this task with Fiddler's Composer tab, but I want to teach you about another tool that is much more suitable for these scenarios—Postman. Postman Postman is an HTTP client that simplifies the testing of web services and RESTful APIs. Postman allows you to easily construct HTTP requests, send them, and inspect them. Download Postman from https://www.getpostman.com/, and then install and run it: On the introduction screen, click on the Request option: Enter GetProducts in the Request name field, and then type GiveNTake into the collection section and create a new collection. Press Save to create the new request: Enter the full URL of the GetProducts API (for example, http://localhost:5267/api/products) in the URL field: Make sure that the HTTP Verb is set to GET and click on the Send button. After a few moments, you should see the response that was received from your service, and you can now inspect the status code, response body, and headers: You will find Postman to be an indispensable development tool while you develop your APIs, and we will use it many times as we go deeper into ASP.NET Core in this book. At this point, you might be wondering, how come the GetProducts method was invoked when we navigated to /api/Products? To answer this question, we need to talk about how ASP.NET Core routes requests into controllers and actions. ASP.NET Core provides the necessary infrastructure you need to create powerful RESTful APIs. In this article, you learned how to create controllers and actions that respond to HTTP requests, and return HTTP responses that you control. We've introduced two popular tools: Fiddler and Postman, and you'll find them very useful when you create and debug your API applications. To know more about APIs in ASP.NET Core, check out the book Hands-On Full-Stack Web Development with ASP.NET Core. Google announces the general availability of a new API for Google Docs What to expect in ASP.NET Core 3.0 How to call an Azure function from an ASP.NET Core MVC application

JavaScript, being a flexible scripting language along with its dynamic type system, can become harder to maintain, the more a project scales up and as the team's staff changes. There are many tools and languages that can assist with this situation, one of which is TypeScript. This article is an excerpt from a book written by Tamir Dresher, Amir Zuker, and Shay Friedman titled Hands-On Full-Stack Web Development with ASP.NET Core. In this book, you will learn how to build web applications using Angular, React, and Vue. Back in the day when JavaScript ES5 was the active version, writing JavaScript code that adhered to encapsulation and modularity was not a trivial task. Things such as prototypes, namespaces, and self-executing functions could certainly improve your code; unfortunately, many JavaScript developers did not invest the necessary effort to improve the manageability of their code using those constructs. Consequently, new technologies surfaced to help with this matter; a popular example relevant at that time was CoffeeScript. Then, JavaScript ES6 was released. ES6, with its great added features, such as modules, classes, and more specific variable scoping, basically overturned CoffeeScript and similar alternatives at that time. Evidently, JavaScript is still missing a key feature, and that is type information. JavaScript's dynamic type system is a strong feature at times; unfortunately, the manageability and reusability of existing code suffers due to this fact. For that key purpose, every large-scale project should examine the use of tools that compensate the situation; those leading the field now are TypeScript and Flow. TypeScript, an open source scripting language, is developed and maintained by Microsoft. Its killer feature is bringing static typing to JavaScript-based systems and allowing you to annotate the code with type information. Additionally, it supports augmenting existing JavaScript code with external type information, similar to the header files pattern, as it's commonly known in C++. TypeScript is a superset of JavaScript, that meaning it has seamless integration with JavaScript, and makes JavaScript developers feel right at home. Being a JavaScript developer makes you a TypeScript developer as well since JavaScript code is actually valid TypeScript. Importantly, this makes the gradual upgrade of existing JavaScript code fairly easy. Another example of how great TypeScript is is the fact that Angular, a framework built by Google, adopts it extensively, and is actually implemented in TypeScript internally. The TypeScript compiler TypeScript is a superset of JavaScript, yet browsers and other platforms don't recognize it, nor are they able to execute it. For that purpose, TypeScript is processed by the TypeScript compiler, which transpiles your TypeScript code to JavaScript. Then, you can run the transpiled code basically everywhere JavaScript is supported. The tsc is available as an npm package that you can install globally. Open your favorite terminal and install it using the following command: npm install -g typescript Afterward, you can use the command tsc to execute the tsc: TypeScript files have a .ts file extension; this is where you write your TypeScript code. When needed, you use the TypeScript compiler to transpile your TypeScript code, which creates the JavaScript implementation of the code; you do that by executing the following command: tsc <typescript_filename> Types in Typescript One of the key features of TypeScript is bringing static typing to JavaScript. In TypeScript, you annotate your code with type information; every declaration can be defined with its associated typing. In the declaration, you do that by adding a colon following the type. For example, the following statement defines a variable of type number: const productsCount: number; Annotating your code with type information enables validation. The TypeScript compiler should throw errors if you try to do anything that is not supported by the specified type, thus productivity and discoverability benefit substantially through type safety. Additionally, with proper tooling support, you get IntelliSense support and automatic code completion. TypeScript supports a handful of types to reflect most constructs in JavaScript. We will cover these next, starting with basic types. Basic types TypeScript supports a handful of built-in types; the following are examples of the most common ones: const fullName: string = "John Doe"; const age: number = 6; const isDone: boolean = false; const d: Date = new Date(); const canBeAnything: any = 6; As you can see, TypeScript supports basic primitive types such as string, number, boolean, and date. Another thing to notice is the type any. The any type is a valid TypeScript type that indicates that the declaration can be of any given type and all operations on it should be allowed and considered safe. Arrays TypeScript arrays can be written in one of two ways. You can specify the item type followed by square brackets or you can use the generic Array type as follows: const list: number[] = [1, 2, 3]; const list: Array<number> = [1, 2, 3]; Enums Enums allow you to specify named values that correspond to numeric or string values. If you don't manually set the associated corresponding value, the named values in the enum are numbered, starting at 0 by default. Enums are extremely useful, and are commonly used to represent a set of predefined options or lookup values. In TypeScript, you define enums by using the enum keyword as follows: enum Color {Red, Green, Blue} const c: Color = Color.Red; Objects In JavaScript, objects contain key/value pairs that form the shape of the object. TypeScript supports object types as well, very similar to how you write plain JavaScript objects. Consider the following plain JavaScript object: const obj = { x: 5, y: 6 } In the preceding code, obj is a plain object defined with two keys, x and y, both with number values. To define an object type in TypeScript, you use a similar format — curly braces to represent an object, inside the keys, and their type information: { x:number, y:number } Together, this is how you associate a type to an object: const obj: { x:number, y:number } = { x: 5, y: 6 } In the preceding code, the obj object is initialized the same as before, only now along with its type information, which is marked in bold. Functions Functions have type information as well. Functions are comprised of parameters and a return value: function buildName(firstName: string, lastName: string): string { return firstName + " " + lastName; } Each parameter is annotated with its type, and then the return value's type is added at the end of the function signature. Unlike JavaScript, TypeScript enforces calls to functions to adhere to the defined signature. For example, if you try to call the function with anything other than two string parameters, the TypeScript compiler will not allow it. You can define more flexible function signatures if you like. TypeScript supports defining optional parameters by using the question mark symbol, ?: function buildName(firstName: string, lastName: string, title?: string): string { return title + " " + firstName + " " + lastName; } const name = buildName('John', 'Doe'); // valid call The preceding function can now be called with either 2 or 3 string parameters. Everything in JavaScript is still supported, of course. You can still use ES6 default parameter values and rest parameters as well. Type inference Until now, the illustrated code included the type information in the same statement where the actual value assignment took place. TypeScript supports type inference, meaning that it tries to resolve the relevant type information if you don't specify any. Let's look at some examples: const count = 2; In the preceding simple example, TypeScript is smart enough to realize that the type of count is a number without you having to explicitly specify that: function sum(x1: number, x2: number) { return x1 + x2; } const mySum = sum(1, 2); The preceding example demonstrates the power of type inference in TypeScript. If you pay close attention, the code doesn't specify the return value type of the sum function. TypeScript evaluates the code and determines that the return type is number in this case. As a result, the type of the variable mySum is also a number. Type inference is an extremely useful feature since it saves us the incredible time that would be otherwise spent in adding endless type information. Type casting Having the code statically typed leads to type safety. TypeScript does not let you perform actions that are not considered safe and supported. In the following example, the loadProducts function is defined with a return value of type any, thus the type of products is inferred as any: function loadProducts(): any { return [{name: 'Book1'}, {name: 'Book2'}]; } const products = loadProducts(); // products is of type 'any' In some cases, you may actually know the expected type. You can instruct TypeScript of the type by using a cast. Typecasting in TypeScript is supported by using the as an operator or the use of angle brackets, as follows: const products = loadProducts() as {name: string}[]; const products = <{name: string}[]>loadProducts(); Using the preceding cast, you have full support and recognition by TypeScript that products is now an array of objects with a name key. Type aliasing TypeScript supports defining new types via type aliases. You can use type aliases to reuse type information in multiple declarations, for example: type person = {name: string}; let employee: person; let contact: person; Given the preceding example, the code defines a type alias named person as an object with a name key of type string. Afterwards, the rest of the code can reference this type where needed. Type aliases are somewhat similar to interfaces (covered next), but can name primitives, unions, and tuples. Unlike interfaces, type aliases cannot be extended or implemented from, and so interfaces are generally preferred. Unions and intersections allow you to construct a type from multiple existing types, while tuples express arrays with a fixed number of known elements. You can read more about these at https://www.typescriptlang.org/docs/handbook/basic-types.html and https://www.typescriptlang.org/docs/handbook/advanced-types.html. TypeScript is one of the most popular scripting languages in respect to JavaScript-based code bases. At its very core, it brings static typing and assists tremendously with maintainability and productivity. In this article, we covered TypeScript types. TypeScript advances at a steady pace and shows great promise for using it in large projects. It has more features that weren't addressed in this tutorial, such as generics, mapped types, abstract classes, project references, and much more that will help you for ASP.NET core development. To know more about interfaces and compilers of TypeScript, check out the book Hands-On Full-Stack Web Development with ASP.NET Core. Typescript 3.3 is finally released! Future of ESLint support in TypeScript Introducing ReX.js v1.0.0 a companion library for RegEx written in TypeScript

In a paper published on February 4, Google engineers drafted out plans to forward federated learning at a scale. It showcases the high-level plans, challenges, solutions, and applications. Federated learning was first introduced in 2017 by Google. The idea is to use data from a number of computing devices like smartphones instead of a centralized data source. Federated learning can help with privacy Federated learning can be beneficial as it addresses the privacy concern. Android phones are used for the system where the data is only used but never uploaded to any server. A deep neural network is trained by using TensorFlow on the data stored in the Android phone. The Federated averaging algorithm by Brendan McMahan uses a similar approach as synchronous training. The weights of the neural network are combined in the cloud using Federated Averaging. This creates a global model which is then pushed back to the phones as results/desirable actions. To enhance privacy approaches like differential privacy and Secure aggregation are taken. The paper addresses challenges like time zone differences, connectivity issues, interrupted execution etc,. Their work is mature enough to deploy the system in production for tens of millions of devices. They are working towards supporting billions of devices now. The training protocol The system involves devices and the Federated Learning server communicating availability and the server selecting devices to run a task. A subset of the available devices are selected for a task. The Federated Learning server instructs the devices what computing task to run with a plan. A plan would consist a TensorFlow graph and instructions to execute it. There are three phases for the training to take place: Selection of the devices that meet eligibility criteria Configuring the server with simple or Secure Aggregation Reporting from the devices where reaching a certain number would get the training round started Source: Towards Federated Learning at Scale: System Design The devices are supposed to maintain a repository of the collected data and the applications are responsible to provide data to the Federated Learning runtime as an example store. The Federated Learning server is designed to operate on orders of many magnitudes. Each round can mean updates from devices in the range of KBs to tens of MBs coming going the server. Data collection To avoid harming the phone’s battery life and performance, various analytics are collected in the cloud. The logs don’t contain any personally identifiable information. Secure aggregation Secure aggregation uses encryption to make individual device updates uninspectable. They plant to use it for protection against threats in data centers. Secure aggregation would ensure data encryption even when it is in-memory. Challenges of federated learning Compared to a centralized dataset, federated learning poses a number of challenges. The training data is not inspectable, tooling is required to work with proxy data. Models cannot be run interactively and must be compiled to be deployed in the Federated Learning server. Model resource consumption and runtime compatibility also come into the picture when working with many devices in real-time. Applications of Federated Learning It is best for cases where the data on devices is more relevant than data on servers. Ranking items for better navigation, suggestions for on-device keyboard, and next word prediction. This has already been implemented on Google pixel and Gboard. Future work is to eliminate bias caused be restrictions in device selection, algorithms to support better parallelism (more devices in one round), avoiding retraining already trained tasks on devices, and compression to save bandwidth. Federated computation, not federated learning The authors do no mention machine learning explicitly anywhere in the paper. They believe that the applications of such a model are not limited to machine learning. Federated Computation is the term they want to use for this concept. Federated computation and edge computing Federated learning and edge computing are very similar, there are but subtle differences in the purpose of these two. Federated learning is used to solve problems with specific tasks assigned to endpoint smartphones. Edge computing is for predefined tasks to be processed at end nodes, for example, IoT cameras. Federated learning decentralizes the data used while edge computing decentralizes the task computation to various devices. For more details on the architecture and its working, you can check out the research paper. Technical and hidden debts in machine learning – Google engineers’ give their perspective Researchers introduce a machine learning model where the learning cannot be proved What if AIs could collaborate using human-like values? DeepMind researchers propose a Hanabi platform.

Open source software in general, and JavaScript in particular, can seem like a place where boom and bust is the rule of law: rapid growth before everyone moves on to the next big thing. But Node.js is different. Although it certainly couldn’t be described as new, and it's growth hasn't been dramatic by any measure, over the last few years it has managed to push itself forward as one of the most widely used JavaScript tools on the planet. Do you want to learn Node.js? Popularity, however can only tell you so much. The key question, if you’re reading this, is whether you should learn Node.js. So, to help you decide if it’s time to learn the JavaScript library, here’s a list of the biggest reasons why you should start learning Node.js... Learn everything you need to know about Node.js with Packt's Node.js Complete Reference Guide Book Learning Path. Node.js lets you write JavaScript on both client and server Okay, let’s get the obvious one out of the way first: Node.js is worth learning because it allows you to write JavaScript on the server. This has arguably transformed the way we think about JavaScript. Whereas in the past it was a language specifically written on the client, backed by the likes of PHP and Java, it’s now a language that you can use across your application. Read next: The top 5 reasons Node.js could topple Java This is important because it means teams can work much more efficiently together. Using different languages for backend and frontend is typically a major source of friction. Unless you have very good polyglot developers, a team is restricted to their core skills, while tooling is also more inflexible. If you’re using JavaScript across the stack, it’s easier to use a consistent toolchain. From a personal perspective, learning Node.js is a great starting point for full stack development. In essence, it's like an add-on that immediately expands what you can do with JavaScript. In terms of your career, then, it could well make you an invaluable asset to a development team. Read next: How is Node.js changing web development? Node.js allows you to build complex and powerful applications without writing complex code Another strong argument for Node.js is that it is built for performance. This is because of 2 important things - Node.js' asynchronous-driven architecture, and the fact that it uses the V8 JavaScript engine. The significance of this is that V8 is one of the fastest implementations of JavaScript, used to power many of Google’s immensely popular in-browser products (like Gmail). Node.js is powerful because it employs an asynchronous paradigm for handling data between client and server. To clarify what this means, it’s worth comparing to the typical application server model that uses blocking I/O - in this instance, the application has to handle each request sequentially, suspending threads until they can be processed. This can add complexity to an application and, of course, slows an application down. In contrast, Node.js allows you to use non-blocking I/O in which threads (in this case sequential, not concurrent), which can manage multiple requests. If one can’t be processed, it’s effectively ‘withheld’ as a promise, which means it can be executed later without holding up other threads. This means Node.js can help you build applications of considerable complexity without adding to the complexity of your code. Node.js is well suited to building microservices Microservices have become a rapidly growing architectural style that offer increased agility and flexibility over the traditional monolith. The advantages of microservices are well documented, and whether or not they’re right for you now, it’s likely that they’re going to dominate the software landscape as the world moves away from monolithic architecture. This fact only serves to strengthen the argument that you should learn Node.js because the library is so well suited to developing in this manner. This is because it encourages you to develop in a modular and focused manner, quite literally using specific modules to develop an application. This is distinct and almost at odds with the monolithic approach to software architecture. At this point, it’s probably worth highlighting that it’s incredibly easy to package and publish the modules you build thanks to npm (node package manager). So, even if you haven’t yet worked with microservices, learning Node.js is a good way to prepare yourself for a future where they are going to become even more prevalent. Node.js can be used for more than just web development We know by now that Node.js is flexible. But it’s important to recognise that its flexibility means it can be used for a wide range of different purposes. Yes, the library's community are predominantly building applications for the web, but it’s also a useful tool for those working in ops or infrastructure. This is because Node.js is a great tool for developing other development tools. If you’re someone working to support a team of developers, or, indeed, to help manage an entire distributed software infrastructure, it could be vital in empowering you to get creative and build your own support tools. Even more surprisingly, Node.js can be used in some IoT projects. As this post from 2016 suggests, the two things might not be quite such strange bedfellows. Node.js is a robust project that won't be going anywhere As I’ve already said, in the JavaScript world frameworks and tools can appear and disappear quickly. That means deciding what to learn, and, indeed, what to integrate into your stack, can feel like a bit of a gamble. However, you can be sure that Node.js is here to stay. There are a number of reasons for this. For starters, there’s no other tool that brings JavaScript to the server. But more than that, with Google betting heavily on V8 - which is, as we’ve seen, such an important part of the project - you can be sure it’s only going to go from strength to strength. It’s also worth pointing out that Node.js went through a small crisis when io.js broke away from the main Node.js project. This feud was as much personal as it was technical, but with the rift healed, and the Node.js Foundation now managing the whole project, helping to ensure that the software is continually evolving with other relevant technological changes and that the needs of the developers who use it continue to be met. Conclusion: spend some time exploring Node.js before you begin using it at work That’s just 5 reasons why you should learn Node.js. You could find more, but broadly speaking these all underline its importance in today’s development world. If you’re still not convinced, there’s a caveat. If Node.js isn’t yet right for you, don’t assume that it’s going to fix any technological or cultural issues that have been causing you headaches. It probably won’t. In fact, you should probably tackle those challenges before deciding to use it. But that all being said, even if you don’t think it’s the right time to use Node.js professionally, that doesn’t mean it isn’t worth learning. As you can see, it’s well worth your time. Who knows where it might take you? Ready to begin learning? Purchase Node.js Complete Reference Guide or read it for free with a subscription free trial.

In this article, you will learn how to use lambda expressions in Java 11. This article is an excerpt from a book written by Nick Samoylov and Mohamed Sanaulla titled Java 11 Cookbook - Second Edition. In this book, you will learn how to build graphical user interfaces using JavaFX. Getting ready Creating and using lambda expressions is actually much simpler than writing a method. One just needs to list the input parameters, if any, and the code that does what has to be done. Let's see an implementation of standard functional interfaces rewritten using lambda expressions. Here's how we have implemented the four main functional interfaces using anonymous classes: Function<Integer, Double> ourFunc = new Function<Integer, Double>(){ public Double apply(Integer i){ return i * 10.0; } }; System.out.println(ourFunc.apply(1)); //prints: 10.0 Consumer<String> consumer = new Consumer<String>() { public void accept(String s) { System.out.println("The " + s + " is consumed."); } }; consumer.accept("Hello!"); //prints: The Hello! is consumed. Supplier<String> supplier = new Supplier<String>() { public String get() { String res = "Success"; //Do something and return result—Success or Error. return res; } }; System.out.println(supplier.get()); //prints: Success Predicate<Double> pred = new Predicate<Double>() { public boolean test(Double num) { System.out.println("Test if " + num + " is smaller than 20"); return num < 20; } }; System.out.println(pred.test(10.0)? "10 is smaller":"10 is bigger"); //prints: Test if 10.0 is smaller than 20 // 10 is smaller And here's how they look with lambda expressions: Function<Integer, Double> ourFunc = i -> i * 10.0; System.out.println(ourFunc.apply(1)); //prints: 10.0 Consumer<String> consumer = s -> System.out.println("The " + s + " is consumed."); consumer.accept("Hello!"); //prints: The Hello! is consumed. Supplier<String> supplier = () - > { String res = "Success"; //Do something and return result—Success or Error. return res; }; System.out.println(supplier.get()); //prints: Success Predicate<Double> pred = num -> { System.out.println("Test if " + num + " is smaller than 20"); return num < 20; }; System.out.println(pred.test(10.0)? "10 is smaller":"10 is bigger"); //prints: Test if 10.0 is smaller than 20 // 10 is smaller The examples of specialized functional interfaces we have presented are as follows: IntFunction<String> ifunc = new IntFunction<String>() { public String apply(int i) { return String.valueOf(i * 10); } }; System.out.println(ifunc.apply(1)); //prints: 10 BiFunction<String, Integer, Double> bifunc = new BiFunction<String, Integer, Double >() { public Double apply(String s, Integer i) { return (s.length() * 10d) / i; } }; System.out.println(bifunc.apply("abc",2)); //prints: 15.0 BinaryOperator<Integer> binfunc = new BinaryOperator<Integer>(){ public Integer apply(Integer i, Integer j) { return i >= j ? i : j; } }; System.out.println(binfunc.apply(1,2)); //prints: 2 IntBinaryOperator intBiFunc = new IntBinaryOperator(){ public int applyAsInt(int i, int j) { return i >= j ? i : j; } }; System.out.println(intBiFunc.applyAsInt(1,2)); //prints: 2 And here's how they look with lambda expressions: IntFunction<String> ifunc = i -> String.valueOf(i * 10); System.out.println(ifunc.apply(1)); //prints: 10 BiFunction<String, Integer, Double> bifunc = (s,i) -> (s.length() * 10d) / i; System.out.println(bifunc.apply("abc",2)); //prints: 15.0 BinaryOperator<Integer> binfunc = (i,j) -> i >= j ? i : j; System.out.println(binfunc.apply(1,2)); //prints: 2 IntBinaryOperator intBiFunc = (i,j) -> i >= j ? i : j; System.out.println(intBiFunc.applyAsInt(1,2)); //prints: 2 As you can see, the code is less cluttered and more readable. How to form lambda expressions Those who have some traditional code-writing experience, when starting functional programming, equate functions with methods.  They try to create functions first because that was how we all used to write traditional code—by creating methods. Yet, functions are just smaller pieces of functionality that modify some aspects of the behavior of the methods or provide the business logic for the otherwise non-business-specific code. In functional programming, as in traditional programming, methods continue to provide the code structure, while functions are the nice and helpful additions to it. So, in functional programming, creating a method comes first, before the functions are defined. Let's demonstrate this. The following are the basic steps of code writing. First, we identify the well-focused block of code that can be implemented as a method. Then, after we know what the new method is going to do, we can convert some pieces of its functionality into functions: Create the calculate() method: void calculate(){ int i = 42; //get a number from some source double res = 42.0; //process the above number if(res < 42){ //check the result using some criteria //do something } else { //do something else } } The preceding pseudocode outlines the idea of the calculate() method's functionality. It can be implemented in a traditional style—by using methods, as follows: int getInput(){ int result; //getting value for result variable here return result; } double process(int i){ double result; //process input i and assign value to result variable } boolean checkResult(double res){ boolean result = false; //use some criteria to validate res value //and assign value to result return result; } void processSuccess(double res){ //do something with res value } void processFailure(double res){ //do something else with res value } void calculate(){ int i = getInput(); double res = process(i); if(checkResult(res)){ processSuccess(res); } else { processFailure(res); } } But some of these methods may be very small, so the code becomes fragmented and less readable with so many additional indirections. This disadvantage becomes especially glaring in the case when the methods come from outside the class where the calculate() method is implemented: void calculate(){ SomeClass1 sc1 = new SomeClass1(); int i = sc1.getInput(); SomeClass2 sc2 = new SomeClass2(); double res = sc2.process(i); SomeClass3 sc3 = new SomeClass3(); SomeClass4 sc4 = new SomeClass4(); if(sc3.checkResult(res)){ sc4.processSuccess(res); } else { sc4.processFailure(res); } } As you can see, in the case where each of the external methods is small, the amount of plumbing code may substantially exceed the payload it supports. Besides, the preceding implementation creates many tight dependencies between classes. Let's look at how we can implement the same functionality using functions. The advantage is that the functions can be as small as they need to be, but the plumbing code will never exceed the payload because there is no plumbing code. Another reason to use functions is when we need the flexibility to change sections of the functionality on the fly, for the algorithm's research purpose. And if these pieces of functionality have to come from outside the class, we do not need to build other classes just for the sake of passing a method into calculate(). We can pass them as functions: void calculate(Supplier<Integer> souc e, Function<Integer, Double> process, Predicate<Double> condition, Consumer<Double> success, Consumer<Double> failure){ int i = source.get(); double res = process.apply(i); if(condition.test(res)){ success.accept(res); } else { failure.accept(res); } } Here's how the functions may look: Supplier<Integer> source = () -> 4; Function<Integer, Double> before = i -> i * 10.0; Function<Double, Double> after = d -> d + 10.0; Function<Integer, Double> process = before.andThen(after); Predicate<Double> condition = num -> num < 100; Consumer<Double> success = d -> System.out.println("Success: "+ d); Consumer<Double> failure = d -> System.out.println("Failure: "+ d); calculate(source, process, condition, success, failure); The result of the preceding code is going to be as follows: Success: 50.0 How lambda expressions work The lambda expression acts as a regular method, except when you think about testing each function separately. How to do it? There are two ways to address this issue. First, since the functions are typically small, there is often no need to test them separately, and they are tested indirectly when the code that uses them is tested. Second, if you still think the function has to be tested, it is always possible to wrap it in the method that returns the function, so you can test that method like any other method. Here is an example of how it can be done: public class Demo { Supplier<Integer> source(){ return () -> 4;} Function<Double, Double> after(){ return d -> d + 10.0; } Function<Integer, Double> before(){return i -> i * 10.0; } Function<Integer, Double> process(){return before().andThen(after());} Predicate<Double> condition(){ return num -> num < 100.; } Consumer<Double> success(){ return d -> System.out.println("Failure: " + d); } Consumer<Double> failure(){ return d-> System.out.println("Failure: " + d); } void calculate(Supplier<Integer> souce, Function<Integer, Double> process, Predicate<Double> condition, Consumer<Double> success, Consumer<Double> failure){ int i = source.get(); double res = process.apply(i); if(condition.test(res)){ success.accept(res); } else { failure.accept(res); } } void someOtherMethod() { calculate(source(), process(), condition(), success(), failure()); } Now we can write the function unit tests as follows: public class DemoTest { @Test public void source() { int i = new Demo().source().get(); assertEquals(4, i); } @Test public void after() { double d = new Demo().after().apply(1.); assertEquals(11., d, 0.01); } @Test public void before() { double d = new Demo().before().apply(10); assertEquals(100., d, 0.01); } @Test public void process() { double d = new Demo().process().apply(1); assertEquals(20., d, 0.01); } @Test public void condition() { boolean b = new Demo().condition().test(10.); assertTrue(b); } } Typically, lambda expressions (and functions in general) are used for specializing otherwise generic functionalities—by adding business logic to a method. A good example is stream operations. The library authors have created them to be able to work in parallel, which required a lot of expertise. And now the library users can specialize the operations by passing into them the lambda expressions (functions) that provide the application's business logic. Function inlining Since, as we have mentioned already, functions are often simple one-liners, they are often inlined when passed in as parameters, for example: Consumer<Double> success = d -> System.out.println("Success: " + d); Consumer<Double> failure = d-> System.out.println("Failure: " + d); calculate(() -> 4, i -> i * 10.0 + 10, n -> n < 100, success, failure); But one should not push it too far, as such inlining may decrease code readability. In this tutorial, we learned how to use lambda expressions in Java 11. To learn more Java 11 recipes, check out the book Java 11 Cookbook - Second Edition. Brian Goetz on Java futures at FOSDEM 2019 7 things Java programmers need to watch for in 2019 Clojure 1.10 released with Prepl, improved error reporting and Java compatibility

In this tutorial, we will learn new ways to interact with network devices using Python. We will understand how to configure network devices using configuration templates, and also write a modular code to ensure high reusability of the code to perform repetitive tasks. We will also see the benefits of parallel processing of tasks and the efficiency that can be gained through multithreading. This Python tutorial has been taken from the second edition of Practical Network Automation. Read it here. Interacting with network devices Python is widely used to perform network automation. With its wide set of libraries (such as Netmiko and Paramiko), there are endless possibilities for network device interactions for different vendors. Let us understand one of the most widely used libraries for network interactions. We will be using Netmiko to perform our network interactions. Python provides a well-documented reference for each of the modules, and, for our module, the documentation can be found at pypi.org.  For installation, all we have to do is go into the folder from the command line where python.exe is installed or is present. There is a subfolder in that location called scripts. Inside the folder, we have two options that can be used for installing the easy_install.exe or pip.exe modules. Installing the library for Python can be done in two ways: The syntax of easy_install is as follows: easy_install <name of module> For example, to install Netmiko, the following command is run: easy_install netmiko The syntax of pip install is as follows: pip install <name of module> For example: pip install netmiko Here's an example of a simple script to log in to the router (an example IP is 192.168.255.249 with a username and password of cisco) and show the version: from netmiko import ConnectHandler device = ConnectHandler(device_type='cisco_ios', ip='192.168.255.249', username='cisco', password='cisco') output = device.send_command("show version") print (output) device.disconnect() The output of the execution of code against a router is as follows: As we can see in the sample code, we call the ConnectHandler function from the Netmiko library, which takes four inputs (platform type, IP address of device, username, and password): Netmiko works with a variety of vendors. Some of the supported platform types and their abbreviations to be called in Netmiko are as follows: a10: A10SSH, accedian: AccedianSSH, alcatel_aos: AlcatelAosSSH, alcatel_sros: AlcatelSrosSSH, arista_eos: AristaSSH, aruba_os: ArubaSSH, avaya_ers: AvayaErsSSH, avaya_vsp: AvayaVspSSH, brocade_fastiron: BrocadeFastironSSH, brocade_netiron: BrocadeNetironSSH, brocade_nos: BrocadeNosSSH, brocade_vdx: BrocadeNosSSH, brocade_vyos: VyOSSSH, checkpoint_gaia: CheckPointGaiaSSH, ciena_saos: CienaSaosSSH, cisco_asa: CiscoAsaSSH, cisco_ios: CiscoIosBase, cisco_nxos: CiscoNxosSSH, cisco_s300: CiscoS300SSH, cisco_tp: CiscoTpTcCeSSH, cisco_wlc: CiscoWlcSSH, cisco_xe: CiscoIosBase, cisco_xr: CiscoXrSSH, dell_force10: DellForce10SSH, dell_powerconnect: DellPowerConnectSSH, eltex: EltexSSH, enterasys: EnterasysSSH, extreme: ExtremeSSH, extreme_wing: ExtremeWingSSH, f5_ltm: F5LtmSSH, fortinet: FortinetSSH, generic_termserver: TerminalServerSSH, hp_comware: HPComwareSSH, hp_procurve: HPProcurveSSH, huawei: HuaweiSSH, juniper: JuniperSSH, juniper_junos: JuniperSSH, linux: LinuxSSH, mellanox_ssh: MellanoxSSH, mrv_optiswitch: MrvOptiswitchSSH, ovs_linux: OvsLinuxSSH, paloalto_panos: PaloAltoPanosSSH, pluribus: PluribusSSH, quanta_mesh: QuantaMeshSSH, ubiquiti_edge: UbiquitiEdgeSSH, vyatta_vyos: VyOSSSH, vyos: VyOSSSH Depending upon the selection of the platform type, Netmiko can understand the returned prompt and the correct way to SSH into the specific device. Once the connection is made, we can send commands to the device using the send_command method. Once we get the return value, the value stored in the output variable is displayed, which is the string output of the command that we sent to the device. The last line, which uses the disconnect function, ensures that the connection is terminated cleanly once we are done with our task. For configuration (for example, we need to provide a description to the FastEthernet 0/0 router interface), we use Netmiko, as shown in the following example: from netmiko import ConnectHandler print ("Before config push") device = ConnectHandler(device_type='cisco_ios', ip='192.168.255.249', username='cisco', password='cisco') output = device.send_command("show running-config interface fastEthernet 0/0") print (output) configcmds=["interface fastEthernet 0/0", "description my test"] device.send_config_set(configcmds) print ("After config push") output = device.send_command("show running-config interface fastEthernet 0/0") print (output) device.disconnect() The output of the execution of the preceding code is as follows: As we can see, for config push, we do not have to perform any additional configurations but just specify the commands in the same order as we send them manually to the router in a list, and pass that list as an argument to the send_config_set function. The output in Before config push is a simple output of the FastEthernet0/0 interface, but the output under After config push now has the description that we configured using the list of commands. In a similar way, we can pass multiple commands to the router, and Netmiko will go into configuration mode, write those commands to the router, and exit config mode. If we want to save the configuration, we use the following command after the send_config_set command: device.send_command("write memory") This ensures that the router writes the newly pushed configuration in memory. Additionally, for reference purposes across the book, we will be referring to the following GNS3 simulated network: In this topology, we have connected four routers with an Ethernet switch. The switch is connected to the local loopback interface of the computer, which provides the SSH connectivity to all the routers. We can simulate any type of network device and create topology based upon our specific requirements in GNS3 for testing and simulation. This also helps in creating complex simulations of any network for testing, troubleshooting, and configuration validations. The IP address schema used is the following: rtr1: 192.168.20.1 rtr2: 192.168.20.2 rtr3: 192.168.20.3 rtr4: 192.168.20.4 Loopback IP of computer: 192.168.20.5 The credentials used for accessing these devices are the following: Username: test Password: test Let us start from the first step by pinging all the routers to confirm their reachability from the computer. The code is as follows: import socket import os s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) for n in range(1, 5): server_ip="192.168.20.{0}".format(n) rep = os.system('ping ' + server_ip) if rep == 0: print ("server is up" ,server_ip) else: print ("server is down" ,server_ip) The output of running the preceding code is as follows: As we can see in the preceding code, we use the range command to iterate over the IPs 192.168.20.1-192.168.20.4. The server_ip variable in the loop is provided as an input to the ping command, which is executed for the response. The response stored in the rep variable is validated with a value of 0 stating that the router can be reached, and a value of 1 means the router is not reachable. As a next step, to validate whether the routers can successfully respond to SSH, let us fetch the value of uptime from the show version command: show version | in uptime The code is as follows: from netmiko import ConnectHandler username = 'test' password="test" for n in range(1, 5): ip="192.168.20.{0}".format(n) device = ConnectHandler(device_type='cisco_ios', ip=ip, username='test', password='test') output = device.send_command("show version | in uptime") print (output) device.disconnect() The output of running the preceding command is as follows: Using Netmiko, we fetched the output of the command from each of the routers and printed a return value. A return value for all the devices confirms SSH attainability, whereas failure would have returned an exception, causing the code to abruptly end for that particular router. If we want to save the configuration, we use the following command after the send_config_set command: device.send_command("write memory") This ensures that the router writes the newly pushed configuration in memory. Network device configuration using template With all the routers reachable and accessible through SSH, let us configure a base template that sends the Syslog to a Syslog server and additionally ensures that only information logs are sent to the Syslog server. Also, after configuration, a validation needs to be performed to ensure that logs are being sent to the Syslog server. The logging server info is as follows: Logging server IP: 192.168.20.5  Logging port: 514 Logging protocol: TCP Additionally, a loopback interface (loopback 30) needs to be configured with the {rtr} loopback interface description. The code lines for the template are as follows: logging host 192.168.20.5 transport tcp port 514 logging trap 6 interface loopback 30 description "{rtr} loopback interface" To validate that the Syslog server is reachable and that the logs sent are informational, use the show logging command. In the event that  the output of the command contains the text: Trap logging: level informational: This confirms that the logs are sent as informational Encryption disabled, link up: This confirms that the Syslog server is reachable The code to create the configuration, push it on to the router and perform the validation, is as follows: from netmiko import ConnectHandler template="""logging host 192.168.20.5 transport tcp port 514 logging trap 6 interface loopback 30 description "{rtr} loopback interface\"""" username = 'test' password="test" #step 1 #fetch the hostname of the router for the template for n in range(1, 5): ip="192.168.20.{0}".format(n) device = ConnectHandler(device_type='cisco_ios', ip=ip, username='test', password='test') output = device.send_command("show run | in hostname") output=output.split(" ") hostname=output[1] generatedconfig=template.replace("{rtr}",hostname) #step 2 #push the generated config on router #create a list for generateconfig generatedconfig=generatedconfig.split("\n") device.send_config_set(generatedconfig) #step 3: #perform validations print ("********") print ("Performing validation for :",hostname+"\n") output=device.send_command("show logging") if ("encryption disabled, link up"): print ("Syslog is configured and reachable") else: print ("Syslog is NOT configured and NOT reachable") if ("Trap logging: level informational" in output): print ("Logging set for informational logs") else: print ("Logging not set for informational logs") print ("\nLoopback interface status:") output=device.send_command("show interfaces description | in loopback interface") print (output) print ("************\n") The output of running the preceding command is as follows: Another key aspect to creating network templates is understanding the type of infrastructure device for which the template needs to be applied. As we generate the configuration form templates, there are times when we want to save the generated configurations to file, instead of directly pushing on devices. This is needed when we want to validate the configurations or even keep a historic repository for the configurations that are to be applied on the router. Let us look at the same example, only this time, the configuration will be saved in files instead of writing back directly to routers. The code to generate the configuration and save it as a file is as follows: from netmiko import ConnectHandler import os template="""logging host 192.168.20.5 transport tcp port 514 logging trap 6 interface loopback 30 description "{rtr} loopback interface\"""" username = 'test' password="test" #step 1 #fetch the hostname of the router for the template for n in range(1, 5): ip="192.168.20.{0}".format(n) device = ConnectHandler(device_type='cisco_ios', ip=ip, username='test', password='test') output = device.send_command("show run | in hostname") output=output.split(" ") hostname=output[1] generatedconfig=template.replace("{rtr}",hostname) #step 2 #create different config files for each router ready to be pushed on routers. configfile=open(hostname+"_syslog_config.txt","w") configfile.write(generatedconfig) configfile.close() #step3 (Validation) #read files for each of the router (created as routername_syslog_config.txt) print ("Showing contents for generated config files....") for file in os.listdir('./'): if file.endswith(".txt"): if ("syslog_config" in file): hostname=file.split("_")[0] fileconfig=open(file) print ("\nShowing contents of "+hostname) print (fileconfig.read()) fileconfig.close() The output of running the preceding command is as follows: In a similar fashion to the previous example, the configuration is now generated. However, this time, instead of being pushed directly on routers, it is stored in different files with filenames based upon router names for all the routers that were provided in input. In each case, a .txt file is created (here is a sample filename that will be generated during execution of the script: rtr1_syslog_config.txt for the rtr1 router). As a final validation step, we read all the .txt files and print the generated configuration for each of the text files that has the naming convention containing syslog_config in the filename. There are times when we have a multi-vendor environment, and to manually create a customized configuration is a difficult task. Let us see an example in which we leverage a library (PySNMP) to fetch details regarding the given devices in the infrastructure using Simple Network Management Protocol (SNMP). For our test, we are using the SNMP community key mytest on the routers to fetch their model/version. The code to get the version and model of router, is as follows: #snmp_python.py from pysnmp.hlapi import * for n in range(1, 3): server_ip="192.168.20.{0}".format(n) errorIndication, errorStatus, errorIndex, varBinds = next( getCmd(SnmpEngine(), CommunityData('mytest', mpModel=0), UdpTransportTarget((server_ip, 161)), ContextData(), ObjectType(ObjectIdentity('SNMPv2-MIB', 'sysDescr', 0))) ) print ("\nFetching stats for...", server_ip) for varBind in varBinds: print (varBind[1]) The output of running the preceding command is as follows: As we see in this, the SNMP query was performed on a couple of routers (192.168.20.1 and 192.168.20.2). The SNMP query was performed using the standard Management Information Base (MIB), sysDescr. The return value of the routers against this MIB request is the make and model of the router and the current OS version it is running on. Using SNMP, we can fetch many vital statistics of the infrastructure and can generate configurations based upon the return values. This ensures that we have standard configurations even with a multi-vendor environment. As a sample, let us use the SNMP approach to determine the number of interfaces that a particular router has and, based upon the return values, we can dynamically generate a configuration irrespective of any number of interfaces available on the device. The code to fetch the available interfaces in a router is as follows: #snmp_python_interfacestats.py from pysnmp.entity.rfc3413.oneliner import cmdgen cmdGen = cmdgen.CommandGenerator() for n in range(1, 3): server_ip="192.168.20.{0}".format(n) print ("\nFetching stats for...", server_ip) errorIndication, errorStatus, errorIndex, varBindTable = cmdGen.bulkCmd( cmdgen.CommunityData('mytest'), cmdgen.UdpTransportTarget((server_ip, 161)), 0,25, '1.3.6.1.2.1.2.2.1.2' ) for varBindTableRow in varBindTable: for name, val in varBindTableRow: print('%s = Interface Name: %s' % (name.prettyPrint(), val.prettyPrint())) The output of running the preceding command is as follows: Using the snmpbulkwalk, we query for the interfaces on the router. The result from the query is a list that is parsed to fetch the SNMP MIB ID for the interfaces, along with the description of the interface. Multithreading A key focus area while performing operations on multiple devices is how quickly we can perform the actions. To put this into perspective, if each router takes around 10 seconds to log in, gather the output, and log out, and we have around 30 routers that we need to get this information from, we would need 10*30 = 300 seconds for the program to complete the execution. If we are looking for more advanced or complex calculations on each output, which might take up to a minute, then it will take 30 minutes for just 30 routers. This starts becoming very inefficient when our complexity and scalability grows. To help with this, we need to add parallelism to our programs.  Let us log in to each of the routers and fetch the show version using a parallel calling (or multithreading): #parallel_query.py from netmiko import ConnectHandler from datetime import datetime from threading import Thread startTime = datetime.now() threads = [] def checkparallel(ip): device = ConnectHandler(device_type='cisco_ios', ip=ip, username='test', password='test') output = device.send_command("show run | in hostname") output=output.split(" ") hostname=output[1] print ("\nHostname for IP %s is %s" % (ip,hostname)) for n in range(1, 5): ip="192.168.20.{0}".format(n) t = Thread(target=checkparallel, args= (ip,)) t.start() threads.append(t) #wait for all threads to completed for t in threads: t.join() print ("\nTotal execution time:") print(datetime.now() - startTime) The output of running the preceding command is as follows: The calling to the same set of routers being done in parallel takes approximately 8 seconds to fetch the results. Summary In this tutorial, we learned how to interact with Network devices through Python and got familiar with an extensively used library of Python (Netmiko) for network interactions. You also learned how to interact with multiple network devices using a simulated lab in GNS3 and got to know the device interaction through SNMP. Additionally, we also touched base on multithreading, which is a key component in scalability through various examples. To learn how to make your network robust by leveraging the power of Python, Ansible and other network automation tools, check out our book Practical Network Automation - Second Edition. AWS announces more flexibility its Certification Exams, drops its exam prerequisites Top 10 IT certifications for cloud and networking professionals in 2018 What matters on an engineering resume? Hacker Rank report says skills, not certifications

Modular programming enables one to organize code into independent, cohesive modules, which can be combined to achieve the desired functionality. This article is an excerpt from a book written by Nick Samoylov and Mohamed Sanaulla titled Java 11 Cookbook - Second Edition. In this book, you will learn how to implement object-oriented designs using classes and interfaces in Java 11. The complete code for the examples shown in this tutorial can be found on GitHub. You should be wondering what this modularity is all about, and how to create a modular application in Java. In this article, we will try to clear up the confusion around creating modular applications in Java by walking you through a simple example. Our goal is to show you how to create a modular application; hence, we picked a simple example so as to focus on our goal. Our example is a simple advanced calculator, which checks whether a number is prime, calculates the sum of prime numbers, checks whether a number is even, and calculates the sum of even and odd numbers. Getting ready We will divide our application into two modules: The math.util module, which contains the APIs for performing the mathematical calculations The calculator module, which launches an advanced calculator How to do it Let's implement the APIs in the com.packt.math.MathUtil class, starting with the isPrime(Integer number) API: public static Boolean isPrime(Integer number){ if ( number == 1 ) { return false; } return IntStream.range(2,num).noneMatch(i -> num % i == 0 ); } Implement the sumOfFirstNPrimes(Integer count) API: public static Integer sumOfFirstNPrimes(Integer count){ return IntStream.iterate(1,i -> i+1) .filter(j -> isPrime(j)) .limit(count).sum(); } Let's write a function to check whether the number is even: public static Boolean isEven(Integer number){ return number % 2 == 0; } The negation of isEven tells us whether the number is odd. We can have functions to find the sum of the first N even numbers and the first N odd numbers, as shown here: public static Integer sumOfFirstNEvens(Integer count){ return IntStream.iterate(1,i -> i+1) .filter(j -> isEven(j)) .limit(count).sum(); } public static Integer sumOfFirstNOdds(Integer count){ return IntStream.iterate(1,i -> i+1) .filter(j -> !isEven(j)) .limit(count).sum(); } We can see in the preceding APIs that the following operations are repeated: An infinite sequence of numbers starting from 1 Filtering the numbers based on some condition Limiting the stream of numbers to a given count Finding the sum of numbers thus obtained Based on our observation, we can refactor the preceding APIs and extract these operations into a method, as follows: Integer computeFirstNSum(Integer count, IntPredicate filter){ return IntStream.iterate(1,i -> i+1) .filter(filter) .limit(count).sum(); } Here, count is the limit of numbers we need to find the sum of, and filter is the condition for picking the numbers for summing. Let's rewrite the APIs based on the refactoring we just did: public static Integer sumOfFirstNPrimes(Integer count){ return computeFirstNSum(count, (i -> isPrime(i))); } public static Integer sumOfFirstNEvens(Integer count){ return computeFirstNSum(count, (i -> isEven(i))); } public static Integer sumOfFirstNOdds(Integer count){ return computeFirstNSum(count, (i -> !isEven(i))); So far, we have seen a few APIs around mathematical computations. These APIs are part of our com.packt.math.MathUtil class. The complete code for this class can be found at Chapter03/2_simple-modular-math-util/math.util/com/packt/math, in the codebase downloaded for this book. Let's make this small utility class part of a module named math.util. The following are some conventions we use to create a module: Place all the code related to the module under a directory named math.util and treat this as our module root directory. In the root folder, insert a file named module-info.java. Place the packages and the code files under the root directory. What does module-info.java contain? The following: The name of the module The packages it exports, that is, the one it makes available for other modules to use The modules it depends on The services it uses The service for which it provides implementation Our math.util module doesn't depend on any other module (except, of course, the java.base module). However, it makes its API available for other modules (if not, then this module's existence is questionable). Let's go ahead and put this statement into code: module math.util{ exports com.packt.math; } We are telling the Java compiler and runtime that our math.util module is exporting the code in the com.packt.math package to any module that depends on math.util. The code for this module can be found at Chapter03/2_simple-modular-math-util/math.util. Now, let's create another module calculator that uses the math.util module. This module has a Calculator class whose work is to accept the user's choice for which mathematical operation to execute and then the input required to execute the operation. The user can choose from five available mathematical operations: Prime number check Even number check Sum of N primes Sum of N evens Sum of N odds Let's see this in code: private static Integer acceptChoice(Scanner reader){ System.out.println("************Advanced Calculator************"); System.out.println("1. Prime Number check"); System.out.println("2. Even Number check"); System.out.println("3. Sum of N Primes"); System.out.println("4. Sum of N Evens"); System.out.println("5. Sum of N Odds"); System.out.println("6. Exit"); System.out.println("Enter the number to choose operation"); return reader.nextInt(); } Then, for each of the choices, we accept the required input and invoke the corresponding MathUtil API, as follows: switch(choice){ case 1: System.out.println("Enter the number"); Integer number = reader.nextInt(); if (MathUtil.isPrime(number)){ System.out.println("The number " + number +" is prime"); }else{ System.out.println("The number " + number +" is not prime"); } break; case 2: System.out.println("Enter the number"); Integer number = reader.nextInt(); if (MathUtil.isEven(number)){ System.out.println("The number " + number +" is even"); } break; case 3: System.out.println("How many primes?"); Integer count = reader.nextInt(); System.out.println(String.format("Sum of %d primes is %d", count, MathUtil.sumOfFirstNPrimes(count))); break; case 4: System.out.println("How many evens?"); Integer count = reader.nextInt(); System.out.println(String.format("Sum of %d evens is %d", count, MathUtil.sumOfFirstNEvens(count))); break; case 5: System.out.println("How many odds?"); Integer count = reader.nextInt(); System.out.println(String.format("Sum of %d odds is %d", count, MathUtil.sumOfFirstNOdds(count))); break; } The complete code for the Calculator class can be found at Chapter03/2_simple-modular-math-util/calculator/com/packt/calculator/Calculator.java. Let's create the module definition for our calculator module in the same way we created it for the math.util module: module calculator{ requires math.util; } In the preceding module definition, we mentioned that the calculator module depends on the math.util module by using the required keyword. The code for this module can be found at Chapter03/2_simple-modular-math-util/calculator. Let's compile the code: javac -d mods --module-source-path . $(find . -name "*.java") The preceding command has to be executed from Chapter03/2_simple-modular-math-util. Also, you should have the compiled code from across both the modules, math.util and calculator, in the mods directory. Just a single command and everything including the dependency between the modules is taken care of by the compiler. We didn't require build tools such as ant to manage the compilation of modules. The --module-source-path command is the new command-line option for javac, specifying the location of our module source code. Let's execute the preceding code: java --module-path mods -m calculator/com.packt.calculator.Calculator The --module-path command, similar to --classpath, is the new command-line option  java, specifying the location of the compiled modules. After running the preceding command, you will see the calculator in action: Congratulations! With this, we have a simple modular application up and running. We have provided scripts to test out the code on both Windows and Linux platforms. Please use run.bat for Windows and run.sh for Linux. How it works Now that you have been through the example, we will look at how to generalize it so that we can apply the same pattern in all our modules. We followed a particular convention to create the modules: |application_root_directory |--module1_root |----module-info.java |----com |------packt |--------sample |----------MyClass.java |--module2_root |----module-info.java |----com |------packt |--------test |----------MyAnotherClass.java We place the module-specific code within its folders with a corresponding module-info.java file at the root of the folder. This way, the code is organized well. Let's look into what module-info.java can contain. From the Java language specification (http://cr.openjdk.java.net/~mr/jigsaw/spec/lang-vm.html), a module declaration is of the following form: {Annotation} [open] module ModuleName { {ModuleStatement} } Here's the syntax, explained: {Annotation}: This is any annotation of the form @Annotation(2). open: This keyword is optional. An open module makes all its components accessible at runtime via reflection. However, at compile-time and runtime, only those components that are explicitly exported are accessible. module: This is the keyword used to declare a module. ModuleName: This is the name of the module that is a valid Java identifier with a permissible dot (.) between the identifier names—similar to math.util. {ModuleStatement}: This is a collection of the permissible statements within a module definition. Let's expand this next. A module statement is of the following form: ModuleStatement: requires {RequiresModifier} ModuleName ; exports PackageName [to ModuleName {, ModuleName}] ; opens PackageName [to ModuleName {, ModuleName}] ; uses TypeName ; provides TypeName with TypeName {, TypeName} ; The module statement is decoded here: requires: This is used to declare a dependency on a module. {RequiresModifier} can be transitive, static, or both. Transitive means that any module that depends on the given module also implicitly depends on the module that is required by the given module transitively. Static means that the module dependence is mandatory at compile time, but optional at runtime. Some examples are requires math.util, requires transitive math.util, and requires static math.util. exports: This is used to make the given packages accessible to the dependent modules. Optionally, we can force the package's accessibility to specific modules by specifying the module name, such as exports com.package.math to claculator. opens: This is used to open a specific package. We saw earlier that we can open a module by specifying the open keyword with the module declaration. But this can be less restrictive. So, to make it more restrictive, we can open a specific package for reflective access at runtime by using the opens keyword—opens com.packt.math. uses: This is used to declare a dependency on a service interface that is accessible via java.util.ServiceLoader. The service interface can be in the current module or in any module that the current module depends on. provides: This is used to declare a service interface and provide it with at least one implementation. The service interface can be declared in the current module or in any other dependent module. However, the service implementation must be provided in the same module; otherwise, a compile-time error will occur. We will look at the uses and provides clauses in more detail in the Using services to create loose coupling between the consumer and provider modules recipe. The module source of all modules can be compiled at once using the --module-source-path command-line option. This way, all the modules will be compiled and placed in their corresponding directories under the directory provided by the -d option. For example, javac -d mods --module-source-path . $(find . -name "*.java") compiles the code in the current directory into a mods directory. Running the code is equally simple. We specify the path where all our modules are compiled into using the command-line option --module-path. Then, we mention the module name along with the fully qualified main class name using the command-line option -m, for example, java --module-path mods -m calculator/com.packt.calculator.Calculator. In this tutorial, we learned to create a simple modular Java application. To learn more Java 11 recipes, check out the book Java 11 Cookbook - Second Edition. Brian Goetz on Java futures at FOSDEM 2019 7 things Java programmers need to watch for in 2019 Clojure 1.10 released with Prepl, improved error reporting and Java compatibility

A Gaussian prototypical network is a variant of a prototypical network. A prototypical network learns the embeddings of the data points and how it builds the class prototype by taking the mean embeddings of each class and use the class prototype for performing classification. This article is an excerpt from a book written by Sudharsan Ravichandiran titled Hands On Meta-Learning with Python. In this book, you will learn the prototypical network along with its variants. In a Gaussian prototypical network, along with generating embeddings for the data points, we add a confidence region around them, characterized by a Gaussian covariance matrix. Having a confidence region helps in characterizing the quality of individual data points and would be useful in the case of noisy and less homogeneous data. So, in Gaussian prototypical networks, the output of the encoder will be embeddings, as well as the covariance matrix. Instead of using the full covariance matrix, we either include a radius or diagonal component from the covariance matrix along with the embeddings: Radius component: If we use the radius component of the covariance matrix, then the dimension of our covariance matrix would be 1, as the radius is just a single number. Diagonal component: If we use the diagonal component of the covariance matrix, then the dimension of our covariance matrix would be the same as the embedding matrix dimension. Also, instead of using the covariance matrix directly, we use the inverse of a covariance matrix. We can convert the raw covariance matrix into the inverse covariance matrix using any of the following methods. Let Sraw be the covariance matrix and S be the inverse covariance matrix: S = 1 + Softplus(Sraw) S = 1 + sigmoid(Sraw) S = 1+ 4 * sigmoid(Sraw) S = offset + scale * softplus(Sraw/div), where offset and scale are trainable parameters So, the encoder, along with generating embedding for the input, also returns the covariance matrix. We use either the diagonal or radius components of the covariance matrix. Also, instead of using a covariance matrix directly, we use the inverse covariance matrix. But what is the use of having the covariance matrix along with the embeddings? As said earlier, it adds the confidence region around the data points and is very useful in the case of noisy data. Look at the following diagram. Let's say we have two classes, A and B. The dark dots represent the embeddings of the data point, and the circles around the dark dots indicate the covariance matrices. A big dotted circle represents the overall covariance matrix for a class. A star in the middle indicates the class prototype. As you can see, having this covariance matrix around the embeddings gives us a confidence region around the data point and for class prototypes: Let's better understand this by looking at the code. Let's say we have an image, X, and we want to generate embeddings for the image. Let's represent the covariance matrix by sigma. First, we select what component of the covariance matrix we want to use—that is, whether we want to use the diagonal or radius component. If we use the radius component, then our covariance matrix dimension would be just one. If we opt for the diagonal component, then the size of the covariance matrix would be same as the embedding dimension: if component =='radius': covariance_matrix_dim = 1 else: covariance_matrix_dim = embedding_dim Now, we define our encoder. Since our input is an image, we use a convolutional block as our encoder. So, we define the size of filters, a number of filters, and the pooling layer size: filters = [3,3,3,3] num_filters = [64,64,64,embedding_dim +covariance_matrix_dim] pools = [2,2,2,2] We initialize embeddings as our image, X: previous_channels = 1 embeddings = X weight = [] bias = [] conv_relu = [] conv = [] conv_pooled = [] Then, we perform the convolutional operation and get the embeddings: for i in range(len(filters)): filter_size = filters[i] num_filter = num_filters[i] pool = pools[i] weight.append(tf.get_variable("weights_"+str(i), shape=[filter_size, filter_size, previous_channels, num_filter]) bias.append(tf.get_variable("bias_"+str(i), shape=[num_filter])) conv.append(tf.nn.conv2d(embeddings, weight[i], strides=[1,1,1,1], padding='SAME') + bias[i]) conv_relu.append(tf.nn.relu(conv[i])) conv_pooled.append(tf.nn.max_pool(conv_relu[i], ksize = [1,pool,pool,1], strides=[1,pool,pool,1], padding = "VALID")) previous_channels = num_filter embeddings = conv_pooled [i] We take the output of the last convolutional layer as our embeddings and reshape the result to have embeddings, as well as the covariance matrix: X_encoded = tf.reshape(embeddings,[-1,embedding_dim + covariance_matrix_dim ]) Now, we split the embeddings and raw covariance matrix, as we need to convert the raw covariance matrix into the inverse covariance matrix: embeddings, raw_covariance_matrix = tf.split(X_encoded, [embedding_dim, covariance_matrix_dim], 1) Next, we calculate the inverse of a covariance matrix using any of the discussed methods: if inverse_transform_type == "softplus": offset = 1.0 scale = 1.0 inv_covariance_matrix = offset + scale * tf.nn.softplus(raw_covariance_matrix) elif inverse_transform_type == "sigmoid": offset = 1.0 scale = 1.0 inv_covariance_matrix = offset + scale * tf.sigmoid(raw_covariance_matrix) elif inverse_transform_type == "sigmoid_2": offset = 1.0 scale = 4.0 inv_covariance_matrix = offset + scale * tf.sigmoid(raw_covariance_matrix) elif inverse_transform_type == "other": init = tf.constant(1.0) scale = tf.get_variable("scale", initializer=init) div = tf.get_variable("div", initializer=init) offset = tf.get_variable("offset", initializer=init) inv_covariance_matrix = offset + scale * tf.nn.softplus(raw_covariance_matrix/div) So far, we have seen that we calculate the covariance matrix along with embeddings of an input. What's next? How can we compute the class prototype? The class prototype, , can be computed as follows: In this equation, is the diagonal of the inverse covariance matrix, denotes the embeddings and superscript c denotes the class. After computing the prototype for each of the classes, we learn the embedding of the query point. Let  be the embedding of a query point. Then, we compute the distance between the query point embedding and class prototype as follows: Finally, we predict the class of a query set ( ), which has the minimum distance with the class prototype: The algorithm for Gaussian prototypical networks Now, we will better understand the Gaussian prototypical network by going through it step by step: Let's say we have a dataset, D = {(x1, y1,), (x2, y2), ... (xi, yi)}, where x is the feature and y is the label. Let's say we have a binary label, which means we have only two classes, 0 and 1. We will sample data points at random without replacement from each of the classes from our dataset, D, and create our support set, S. Similarly, we sample data points at random per class and create the query set, Q. We will pass the support set to our embedding function, f(). The embedding function will generate the embeddings for our support set, along with the covariance matrix. We calculate the inverse of the covariance matrix. We compute the prototype of each class in the support set as follows: In this equation, is the diagonal of the inverse covariance matrix, denotes the embeddings of the support set and superscript c denotes the class. After computing the prototype of each class in the support set, we learn the embeddings for the query set, Q. Let's say x' is the embedding of the query point. We calculate the distance of the query point embeddings to the class prototypes as follows: After calculating the distance between the class prototype and query set embeddings, we predict the class of the query set as a class that has a minimum distance, as follows: In this tutorial, we learned about the Gaussian prototypical network, which, uses embeddings, and the covariance matrix to compute the class prototype. To learn more about meta-learning in Python, check out the book Hands-On Meta-Learning with Python. What is Meta-Learning? Introducing Open AI’s Reptile: The latest scalable meta-learning Algorithm on the block “Deep meta reinforcement learning will be the future of AI where we will be so close to achieving artificial general intelligence (AGI)”, Sudharsan Ravichandiran

React Native was developed by Facebook, along with the lines of the React framework. Instead of rendering components to a browser's DOM, React Native (RN) invokes native APIs to create internal components that are handled through your JS code. There are some differences between the usual HTML elements and RN's components, but they are not too hard to overcome. With this tool, you are actually building a native app that looks and behaves exactly like any other native application, except that you use a single language, JS, for both Android and iOS development. This article is taken from the book  Modern JavaScript Web Development Cookbook by Federico Kereki.  This book is a perfect blend of solutions for traditional JavaScript development and modern areas that developers have recently been exploring with JavaScript. This problem-solving guide teaches you popular problems solving techniques for JavaScript on servers, browsers, mobile phones, and desktops. To follow along with the examples implemented in this article, you can download the code from the book's GitHub repository. In this article, we'll see how to install and use React Native to build a mobile application. We will also see how to add development tools like ESLint, Flow, and Prettier. Setting up a RN application There are three ways to set up a RN application: manually, which you won't want to do; secondly, with packages, using the react-native-cli command-line interface; or lastly, by using a package very similar to create-react-native-app (or CRAN). We start by getting a command-line utility, which will include plenty of other packages: npm install create-react-native-app -g Afterward, we can create and run a simple project with just three commands: create-react-native-app yourprojectname cd yourprojectname npm start How it works... When you run your app, it starts a server at your machine, at port 19000 or 19001, to which you will connect using the Expo application. You can download Expo from its official website, which is available for both Android or iOS. Install it by following the instructions onscreen: When you open the Expo app for the first time, it will look like the following screenshot: Note that both the phone and your machine must be in the same local network, and your machine must also allow connections to ports 19000 and 19001; you may have to modify your firewall for this to work. After you use the Scan QR Code option, there will be some synchronization, and soon you'll get to see your basic code running with no problems: Furthermore, if you modify the App.js source code, the changes will be immediately reflected in your device, which means all is well! To make sure this happens, shake the phone to enable the debugging menu, and make sure that Live Reload and Hot Reloading are enabled. You'll also require Remote JS Debugging for later. Your phone should look as follows: Adding development tools Next, we need to add all the development tools required. We want to have ESLint for code checking, Prettier for formatting, and Flow for data types. CRAN takes care of including Babel and Jest, so we won't have to do anything for those two. How to do it... As opposed React, where we need to add a special rewiring package in order to work with specific configurations, in RN, we can just add some packages and configuration files, and we'll be ready to go. Adding ESLint For ESLint, we'll have quite a list of packages we want: npm install --save-dev \ eslint eslint-config-recommended eslint-plugin-babel \ eslint-plugin-flowtype eslint-plugin-react eslint-plugin-react-native We'll require a separate .eslintrc file, as in the case with React. The appropriate contents include the following: { "parser": "babel-eslint", "parserOptions": { "ecmaVersion": 2017, "sourceType": "module", "ecmaFeatures": { "jsx": true } }, "env": { "node": true, "browser": true, "es6": true, "jest": true, "react-native/react-native": true }, "extends": [ "eslint:recommended", "plugin:flowtype/recommended", "plugin:react/recommended", "plugin:react-native/all" ], "plugins": ["babel", "flowtype", "react", "react-native"], "rules": { "no-console": "off", "no-var": "error", "prefer-const": "error", "flowtype/no-types-missing-file-annotation": 0 } } Adding Flow Having completed that, ESLint is set to recognize our code, but we have to configure Flow as well: npm install --save-dev flow flow-bin flow-coverage-report flow-typed We'll have to add a couple of lines to the scripts section of package.json: "scripts": { "start": "react-native-scripts start", . . . "flow": "flow", "addTypes": "flow-typed install" }, Then, we have to initialize the working directories of Flow: npm run flow init The contents of the .flowconfig file look like this: [ignore] .*/node_modules/.* [include] [libs] [lints] all=warn untyped-type-import=off unsafe-getters-setters=off [options] include_warnings=true [strict] Adding Prettier There's not much to installing Prettier, all we need is an npm command, plus the .prettierrc file. For the former, just use the following command: npm install --save-dev prettier For configuration, we can use the contents of this .prettierrc file: { "tabWidth": 4, "printWidth": 75 } How it works... Let's check that everything is OK. We'll start by looking at the App.js file that was created by CRAN, and we can immediately verify that the tools work—because a problem is detected! Have a look at the following screenshot: The rule that fails is a new one, from eslint-plugin-react-native: no-color-literals, because we are using constants in styling, which could prove to be a maintenance headache in the future. We can solve that by adding a variable, and we'll use a type declaration to make sure Flow is also running. The new code should be as follows: // Source file: App.original.fixed.js /* @flow */ import React from "react"; import { StyleSheet, Text, View } from "react-native"; export default class App extends React.Component<> { render() { return ( <View style={styles.container}> <Text>Open up App.js to start working on your app!</Text> <Text>Changes you make will automatically reload.</Text> <Text>Shake your phone to open the developer menu.</Text> </View> ); } } const white: string = "#fff"; const styles = StyleSheet.create({ container: { flex: 1, backgroundColor: white, alignItems: "center", justifyContent: "center" } }); Using native components Working with RN is very much like working with React—there are components, state, props, life cycle events, and so on—but there is a key difference: your own components won't be based on HTML but on specific RN ones. For instance, you won't be using <div> elements, but rather <View> ones, which will be then mapped by RN to a UIView for iOS, or to an Android.View for Android. Getting ready We will start with an example of countries and regions page, which you can find in the book's GitHub repository.  Since we are using PropTypes, we'll need that package. Install it with the following command: npm install prop-types --save Then, we'll have to install some packages, starting with Redux and relatives. Actually, CRAN already includes redux and react-redux, so we don't need those, but redux-thunk isn't included.  We can install it using the following command: npm install react react-redux redux-thunk --save We'll also be using axios for async calls: npm install axios --save Our final step will be to run the server code (you can find it in the GitHub repo) so that our app will be able to do async calls. After downloading the server code from the GitHub repo, go to the directory, and just enter the following command: node out/restful_server.js. Let's now see how we can modify our code to make it appropriate for RN. How to do it... Since RN uses its own components, your HTML experience will be of little use. Here, we'll see some changes, but in order to derive the full benefits of all of RN's possibilities, you'll have to study its components on your own. Let's start with the <RegionsTable> component, which is rather simple: // Source file: src/regionsApp/regionsTable.component.js . . . render() { if (this.props.list.length === 0) { return ( <View> <Text>No regions.</Text> </View> ); } else { const ordered = [...this.props.list].sort( (a, b) => (a.regionName < b.regionName ? -1 : 1) ); return ( <View> {ordered.map(x => ( <View key={x.countryCode + "-" + x.regionCode}> <Text>{x.regionName}</Text> </View> ))} </View> ); } } Notice that there are no changes in the rest of the component, and all your React knowledge is still valid; you just have to adjust the output of your rendering method. Next, we'll change the <CountrySelect> component to use <Picker>, which is sort of similar, but we'll require some extra modifications. Let's take a look at our component, highlighting the parts where changes are needed: // Source file: src/regionsApp/countrySelect.component.js /* @flow */ import React from "react"; import PropTypes from "prop-types"; import { View, Text, Picker } from "react-native"; export class CountrySelect extends React.PureComponent<{ dispatch: ({}) => any }> { static propTypes = { loading: PropTypes.bool.isRequired, currentCountry: PropTypes.string.isRequired, list: PropTypes.arrayOf(PropTypes.object).isRequired, onSelect: PropTypes.func.isRequired, getCountries: PropTypes.func.isRequired }; componentDidMount() { if (this.props.list.length === 0) { this.props.getCountries(); } } onSelect = value => this.props.onSelect(value); render() { if (this.props.loading) { return ( <View> <Text>Loading countries...</Text> </View> ); } else { const sortedCountries = [...this.props.list].sort( (a, b) => (a.countryName < b.countryName ? -1 : 1) ); return ( <View> <Text>Country:</Text> <Picker onValueChange={this.onSelect} prompt="Country" selectedValue={this.props.currentCountry} > <Picker.Item key={"00"} label={"Select a country:"} value={""} /> {sortedCountries.map(x => ( <Picker.Item key={x.countryCode} label={x.countryName} value={x.countryCode} /> ))} </Picker> </View> ); } } } Lots of changes! Let's go through them in the order they occur: An unexpected change: if you want a <Picker> component to display its current value, you must set its selectedValue property; otherwise, even if the user selects a country, the change won't be seen onscreen. We'll have to provide an extra prop, currentCountry, which we'll get from the store, so we can use it as the selectedValue for our list. The fired event when the user selects a value is also different; the event handler will be called directly with the chosen value, instead of with an event from which to work with event.target.value. We have to replace the <select> element with <Picker>, and provide a prompt text prop that will be used when the expanded list is shown onscreen. We have to use <Item> elements for the individual options, noting that the label to be displayed is now a prop. Let's not forget the change when connecting the list of countries to the store; we'll only have to add an extra property to the getProps() function: // Source file: src/regionsApp/countrySelect.connected.js const getProps = state => ({ list: state.countries, currentCountry: state.currentCountry, loading: state.loadingCountries }); Now, all we need to do is see how the main app is set up. Our App.js code will be quite simple: // Source file: App.js /* @flow */ import React from "react"; import { Provider } from "react-redux"; import { store } from "./src/regionsApp/store"; import { Main } from "./src/regionsApp/main"; export default class App extends React.PureComponent<> { render() { return ( <Provider store={store}> <Main /> </Provider> ); } } This is pretty straightforward. The rest of the setup will be in the main.js file, which has some interesting details: // Source file: src/regionsApp/main.js /* @flow */ import React from "react"; import { View, StatusBar } from "react-native"; import { ConnectedCountrySelect, ConnectedRegionsTable } from "."; export class Main extends React.PureComponent<> { render() { return ( <View> <StatusBar hidden /> <ConnectedCountrySelect /> <ConnectedRegionsTable /> </View> ); } } Apart from the usage of <View> wherever we would previously have used <div> (a change to which you should already have gotten used to), there's an added detail: we don't want the status bar to show, so we use the <StatusBar> element, and make sure to hide it. How it works... Just for variety, instead of using my mobile phone, as I did earlier in this article, I decided to use an emulated device. After starting the application with npm start, I started my device, and soon got the following: If the user touches the <Picker> element, a popup will be displayed, listing the countries that were received from our Node server, as shown in the following screenshot: When the user actually taps on a country, the onValueChange event is fired, and after calling the server, the list of regions is displayed, as follows: Everything works, and is using native components; great! By the way, if you were not very sure about the selectedValue problem we described, just omit that prop, and when the user picks on a country, you'll get a bad result: This article walked you through the installation and set up the process of React Native and other development tools for developing the mobile version of a web app. If you found this post useful, do check out the book, Modern JavaScript Web Development Cookbook.  You will learn how to create native mobile applications for Android and iOS with React Native, build client-side web applications using React and Redux, and much more. React Native 0.59 RC0 is now out with React Hooks, and more The React Native team shares their open source roadmap, React Suite hits 3.4.0 How to create a desktop application with Electron [Tutorial]

According to the Tor Project website: “Tor is free software and an open network that helps you defend against traffic analysis, a form of network surveillance that threatens personal freedom and privacy, confidential business activities and relationships, and state security. The Tor network is a group of volunteer-operated servers that allows people to improve their privacy and security on the Internet. Tor's users employ this network by connecting through a series of virtual tunnels rather than making a direct connection, thus allowing both organizations and individuals to share information over public networks without compromising their privacy. Along the same line, Tor is an effective censorship circumvention tool, allowing its users to reach otherwise blocked destinations or content. Tor can also be used as a building block for software developers to create new communication tools with built-in privacy features.” This article is an excerpt taken from the book, Hands-On Dark Web Analysis written by Sion Retzkin. In this book, you will learn how to install operating systems and Tor Browser for privacy, security, and anonymity while accessing them. In this article, we will understand what Tor and the Tor browser is and how to install it in several ways. Tor (which is an acronym for The Onion Router, by the way) is a privacy-focused network that hides your traffic, by routing it through multiple random servers on the Tor network. So, instead of the packets that make up your communication with another party (person or organization), going from point A to B directly, using Tor, they will jump all over the place, between multiple servers, before reaching point B, hiding the trail. Additionally, the packets that make up the traffic (or communication) in the Tor network are wrapped in special layers, which only show the previous server or step that the packet came from, and the next step, hiding the entire route effectively. Tor Browser is a web browser, based on Firefox that was created for the purpose of accessing the Tor network, securely and privately. Even if you use Tor, this doesn't mean that you're secure. Why is that? Because Tor Browser has software vulnerabilities, the same as every other browser. It's also based on Firefox, so it inherits some of its vulnerabilities from there as well. You can minimize attack vectors by applying common security sense, and by employing various tools to try to limit or prevent malicious activity, related to infecting the Tor Browser or the host running it. Installing Tor on Linux Let's start with a classic installation, by accessing the Tor Project website, via a browser. The default browser that ships with Ubuntu is Firefox, which is what we'll use. Although you might think that this would be the best way to install Tor Browser, it's actually the least secure, since the Tor Project website is continuously targeted by hackers and might have any number of security or privacy issues on it. Instead of just downloading Tor Browser and immediately installing it (which is dangerous), you can either download the file and verify its hash (to verify that it is indeed the correct one), or you could install it through other methods, for example, via the Terminal, by using Linux commands, or from the Ubuntu Software Center. We'll start by going over the steps to download Tor Browser from the Tor Project website: After booting your Linux installation, open your browser Enter the following address and navigate to it: https://www.torproject.org/download/download-easy.html.en#linux. Notice that the URL takes you directly to the Linux download section of the Tor Project website. I usually prefer this direct method, rather than starting with Google (or any other search engine), searching for Tor, and then accessing the Tor Project website, since, as you may know, Google collects information about users accessing it, and the whole idea of this book is to maintain our privacy and security. Also, always verify that you're accessing the Tor Project website via HTTPS. Choose the correct architecture (32 or 64 bit), and click the Download link. You'll be able to choose what you want to do with the file—open it with Ubuntu's Archive Manager, or save it to a location on the disk: Downloading Tor Browser   Again, the quickest way to go would be to open the compressed file, but the more secure way would be to download the file and to verify its hash, before doing anything else. The Tor Project provides GNU Privacy Guard (GPG) signature files, with each version of Tor Browser. You will need to install GnuPG on your Linux OS, if it isn't there already, in order to be able to verify the hash of the browser package. To do so, just open the Terminal and type in the following: sudo apt install gnupg Enter your password when required, and the installation will commence. Most Linux installations already include gnupg, as can be seen in the following screenshot: Installing GnuPG    After installing GnuPG, you need to import the key that signed the package. According to the Tor Project website, the Tor Browser import key is 0x4e2C6e8793298290. The Tor Project updates and changes the keys from time to time, so you can always navigate to:  https://www.torproject.org/docs/verifying-signatures.html.en to find the current import key if the one in the book doesn't work. The command to import the key is as follows: gpg --keyserver pool.sks-keyservers.net --recv-keys 0x4e2C6e8793298290 This is followed by this: gpg --fingerprint 0x4e2C6e8793298290 This will tell you whether the key fingerprint is correct. You should see the following: Verify key fingerprint Now, you need to download the .asc file, which is found on the Tor Browser Downloads page, next to the relevant package of the browser (it appears as sig, short for signature): ASC file location   You can find the Tor Browser download page here: https://www.torproject.org/projects/torbrowser.html Now, you can verify the signature of the package, using the ASC file. To do so, enter the following command in the Terminal: gpg --verify tor-browser-linux64-7.5.6_en-US.tar.xz.asc tor-browser-linux64-7.5.6_en-US.tar.xz Note the 64 that I marked in bold. If your OS is 32-bit, change the number to 32. The result you should get is as follows: Verifying the signature   After verifying the hash (signature) of the Tor Browser package, you can install it. You can do so by either: Double-clicking the Tor Browser package file (which will open up the Archive Manager program), clicking Extract, and choosing the location of your choice. Right-clicking the file and choosing Extract here or Extract to and choosing a location. After extracting, perform the following steps: Navigate to the location you defined. Double-click on the Start-tor-browser.desktop file to launch Tor Browser. Press Trust and Launch in the window that appears: Launching Tor   Notice that the filename and icon changed to Tor Browser. Press Connect and you will be connected to the Tor network, and will be able to browse it, using Tor Browser: Connecting to Tor   Before we discuss using Tor Browser, let's talk about alternative ways to install it, for example, by using the Ubuntu Software application. Start by clicking on the Ubuntu Software icon: Ubuntu Software Search for Tor Browser, then click on the relevant result: Tor Browser in Ubuntu Software Then, click Install. After entering your password, the installation process will start. When it ends, click Launch to start Tor Browser. Installing Tor Browser via the Terminal, from the downloaded package Another way to install Tor is to use commands, via the Terminal. There are several ways to do so, as follows: First, download the required Tor Browser package from the website Verify the download, as we discussed before, and then keep the Terminal open Navigate to the location where you downloaded Tor, by entering the following command: cd path/Tor_Browser_Directory For example, note the following: cd /downloads/tor-browser_en_US Then, launch Tor Browser by running the following: ./start-tor-browser.desktop Never launch Tor as root (or with the sudo command). Installing the Tor Browser entirely via the Terminal Next, we'll discuss how to install Tor entirely via the Terminal: First, launch the Terminal, as before. Then, execute the following command: sudo apt install torbrowser-launcher This command will install the Tor Browser. We need root access to install an app, not to launch it. You can then run Tor by executing the following command: ./start-tor-browser.desktop Thus, in this post, we talked about Tor, Tor Browser, how to install it in several ways, and how to use it. If you've enjoyed this post and want to know more about the concept of the Deep Web and the Dark Web and their significance in the security sector, head over to the book  Hands-On Dark Web Analysis. Tor Project gets its first official mobile browser for Android, the privacy-friendly Tor Browser Tor Browser 8.0 powered by Firefox 60 ESR released How to create a desktop application with Electron [Tutorial]