Packt+ | Advance your knowledge in tech

0

All Products

Best Sellers

New Releases

Books

Videos

Audiobooks

Free Learning

Python High Performance, Second Edition

You're reading from Python High Performance, Second Edition Build high-performing, concurrent, and distributed applications

Product type Paperback

Published in May 2017

Publisher Packt

ISBN-13 9781787282896

Length 270 pages

Edition 2nd Edition

Languages

Python

Tools

Dask

Concepts

High Performance Programming

Author (1):

Dr. Gabriele Lanaro

View More author details

Table of Contents (9) Chapters

Benchmarking and Profiling FREE CHAPTER

Pure Python Optimizations

Fast Array Operations with NumPy and Pandas

C Performance with Cython

Exploring Compilers

Implementing Concurrency

Parallel Processing

Distributed Processing

Designing for High Performance

Using PySpark

Nowadays, Apache Spark is one of the most popular projects for distributed computing. Developed in Scala, Spark was released in 2014, and integrates with HDFS and provides several advantages and improvements over the Hadoop MapReduce framework.

Contrary to Hadoop MapReduce, Spark is designed to process data interactively and supports APIs for the Java, Scala, and Python programming languages. Given its different architecture, especially by the fact that Spark keep results in memory, Spark is generally much faster than Hadoop MapReduce.

Setting up Spark and PySpark

Setting up PySpark from scratch requires the installation of the Java and Scala runtimes, the compilation of the project from source, and the configuration of Python and Jupyter notebook so that they can be used alongside the Spark installation. An easier and less error-prone way to set up PySpark is to use an already configured Spark cluster made available through a Docker container.

Note

Docker can be downloaded at...

The rest of the chapter is locked

Register for a free Packt account to unlock a world of extra content!

A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.

Unlock this book and the full library FREE for 7 days

Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of

Start free trial

Renews at $15.99/month. Cancel anytime

Authors (1)

Dr. Gabriele Lanaro

Dr. Gabriele Lanaro

Dr. Gabriele Lanaro is passionate about good software and is the author of the chemlab and chemview open source packages. His interests span machine learning, numerical computing visualization, and web technologies. In 2013, he authored the first edition of the book High Performance Python Programming. He has been conducting research to study the formation and growth of crystals using medium and large-scale computer simulations. In 2017, he obtained his PhD in theoretical chemistry.

See other products by Dr. Gabriele Lanaro

Other recommended products

Related to this chapter

Deep Learning with Theano

Deep Learning with Theano

This book covers a complete overview of Deep Learning with Theano, a Python-based library that makes optimizing numerical expressions easy. Practical code examples address supervised, unsupervised, generative and reinforcement learning for image recognition, natural language processing, or game strategy, with best performing nets and principles.

Jul 2017 10h 0m

Python Parallel Programming Cookbook

Python Parallel Programming Cookbook

Python Parallel Programming Cookbook, Second Edition, covers recipes that will help you how to build multithreaded, multiprocess and asynchronous applications in Python. The book will help you build applications for the GPU using CUDA and PyOPENCL and implement effective debugging and testing techniques.

Sep 2019 12h 20m

IPython Interactive Computing and Visualization Cookbook

IPython Interactive Computing and Visualization Cookbook

IPython Interactive Computing and Visualization Cookbook, Second Edition shows you how to analyze and visualize data in the Jupyter Notebook. It will help you become an expert in high-performance computing and visualization for data analysis and scientific modeling.

Jan 2018 18h 16m

Learning Concurrency in Python

Learning Concurrency in Python

Python is a very high level, general purpose language that is utilized heavily in fields such as data science and research, as well as being one of the top choices for general purpose programming for programmers around the world. It features a wide number of powerful, high and low-level libraries and frameworks that complement its delightful syntax and enable Python programmers to create.

Aug 2017 12h 0m

Mastering Numerical Computing with NumPy

Mastering Numerical Computing with NumPy

Mastering Numerical Computing with Python guides you in performing complex computing with cutting-edge coverage on advanced concepts such as exploratory data analysis and clustering algorithms. You'll become an expert in addressing matrix calculations, and write efficient NumPy codes for implementing algorithms with real-world examples.

Jun 2018 8h 16m

Daniel Arbuckle's Mastering Python

Daniel Arbuckle's Mastering Python

Daniel Arbuckle's Mastering Python covers the basics of operating in a Python development environment, as well as advanced topics. Daniel presents you with real-world solutions to Python 3.6 and advanced-level concepts such as reactive programming and microservices, introducing ctypes and Cython tools.

Python Data Analysis

Python Data Analysis

This book will show data analysis tasks, ranging from data retrieval, cleaning, manipulation, visualization, and storage to complex analysis and modeling using a variety of modules such as NumPy, SciPy, matplotlib, pandas, scikit-learn, and NLTK. You will be able to analyze different kinds of data including numeric, text, time-series, graph, and social media.

Mar 2017 11h 0m

Expert Python Programming

Expert Python Programming

Expert Python Programming, Fourth Edition is a collection of actionable Python programming insights that will help you effectively solve challenging problems. This Python book provides you with a thorough understanding of the complete process of building and maintaining Python apps.

May 2021 21h 0m

Expert Python Programming

Expert Python Programming

Expert Python Programming, Third Edition is a collection of valuable tips that will help you build robust and scalable applications in Python. These will help you become confident at solving challenging programming problems effectively.

Apr 2019 21h 32m

Julia 1.0 High Performance

Julia 1.0 High Performance

Julia is a high-level, high-performance dynamic programming language for numerical computing. This book will help you understand the performance characteristics of your Julia programs and achieve near-C levels of performance in Julia.

Jun 2019 7h 16m

Secret Recipes of the Python Ninja

Secret Recipes of the Python Ninja

Do you feel you've mastered the Python language and you know everything it takes to write applications that are a class apart? Well, you're in for a surprise! This book covers the darkest secrets of Python, delving into its depths and uncovering things you never would have thought could be done.

May 2018 12h 40m

Personalised recommendations for you

Based on your interests and search pattern

LLVM Code Generation

LLVM Code Generation

LLVM is a key technology for building modern compilers. You'll learn how to use its infrastructure to create and extend backends (X86 and AArch64), and contribute to the LLVM open-source project, equipping you with key skills in compiler design.

May 2025 20h 40m

Domain-Driven Refactoring

Domain-Driven Refactoring

This book is your exhaustive guide to applying DDD practices for building robust applications. You'll learn key patterns, refine business logic, and solve real-world problems with DDD principles, refactoring, and event-driven architectures.

May 2025 10h 48m

The Power Platform Playbook for Digital Transformation

The Power Platform Playbook for Digital Transformation

Transform your business with Power Apps, Power Automate, Power BI, and Copilot Studio. This guide covers automation, AI insights, governance, and security, helping leaders, IT pros, and analysts build scalable, low-code enterprise solutions.

Apr 2025 11h 44m

Microservices with Go

Microservices with Go

The book covers Go microservice development, covering the basics for beginners and best practices for structuring Go apps. Novice as well seasoned developers will gain insights into managing and scaling microservices.

Jun 2025 14h 16m