In-Memory Analytics with Apache Arrow: Perform fast and efficient data analytics on both flat and hierarchical structured data

There aren't a lot of other books right now on Apache Arrow at the moment, and this book fills a nice void in cutting edge tech. Couple that with the fun prose that Matt uses, it was entertaining as well as met the needs for a data geek like me.The book focused on the Arrow libraries quite a bit and as a whole section on contributing to the open-source Arrow community. It wasn't my particular specialty but was able to understand and follow. The git repos helped as well if you wanted to reference examples out of book like I did.I found the chapter on Real World Applications most useful for those looking to modernize and where we could actually apply the concepts and for ideas on the book. This was the most helpful to me and I found myself referencing this section for some of my firm's future projects.Overall, learned many things and this book does a great job promoting to you on the benefits of Arrow and Flight in your data stack. As someone new to Apache Arrow, this book does a great job explaining the benefits over the "legacy" technologies. It has definitely sold me on the analytics modernization that's to come!

This book makes a very dense subject matter accessible. Topol's conversational tone and frequent quips made it a fun read.The book goes into depth on many Arrow libraries in multiple languages. It also discusses memory management in detail to explain the pros and cons of Arrow and other formats, such as Profobuf, JSON, CSV, and FlatBuffers. And it also provides examples of integrating with other common data platforms, such as Spark and ElasticSearch.While all of the book is insightful, I'd recommend picking and choosing the chapters that are most relevant for your use case. I've read through it all once, but I think I'll need to read it again to let some of the more advanced topics sink in.

The book is logically structured starting, as a course in Arrow, with basic concepts, data types, I/O, filesystems and interaction with other libraries, such as Pandas, leading on to more advanced features such as compute functions and Flight.The book is illustrated throughout with example code, in both C++ and Python, which can be downloaded from the publisher's Github site to try yourself (see screenshot).By following through the text, and examples, a user with an understanding of C++ and/or Python can quickly get up to speed with Apache Arrow and start writing efficient in-memory analytical code.Highly recommended for beginners and intermediate level Arrow programmers who want to develop their skills.

Remember those rare times when a friend or colleague explained complex concepts in terms so simple that your brain lit up like a Christmas tree from ‘Ah, ha’ moments?This book was one of those times for me. Apache Arrow is a technology you need to know. It’s a critical piece of the Modern Data Stack used in projects like pandas, Dremio, Spark, and Spice.ai. Matt is a master of coupling words with diagrams making seemingly complex concepts very simple to understand. I wish we had this book a year ago when we first created Spice.ai; an open-source runtime that helps developers build AI-driven applications, and Spice.xyz; data and AI infrastructure for web3. Both are built on a core of Apache Arrow.**About the Book Author**I first got to know Matt, a core contributor to the Go Apache Arrow libraries when we started building upon them. We reached out to Matt for help, advice, and to participate in the community, and he was always incredibly knowledgeable and helpful. He’s also given several presentations on building data systems at conferences like Dremio’s Subsurface. My respect for him, his expertise, and his contributions to the project and community have only grown with each interaction and release.**Book Review**To my surprise, Matt not only covered Arrow through the use of Go, but several other languages including Python, C++, and the underlying C libraries. This is one way the books shines because Arrow concepts are demonstrated in multiple ways, across multiple languages, which really help you get them. He also walks you through hardware concepts, including SIMD and branch prediction, to core Arrow concepts, and language implementations, and why they are important to the technology. This pulls back some of the black magic of using Arrow as an engineer and you realize just how well-designed it is for high-performance in-memory analytics.The book starts with the basics of getting started with Arrow and the specification. It then jumps into Arrow for Data Science using projects like Apache Spark and Jupyter as examples, and how it compares with older technologies like ODBC. Next up is core memory format and handling, including practical examples and interactive GitHub hosted code snippets, so you can understand how formats like JSON, CSV, Parquet, and Protocol Buffers fit together and reconcile with the core in-memory Arrow columnar format. This leads into a discussion of the C Data API, which makes Arrow truly language agnostic, and the compute and dataset APIs for high-performance, in-memory analytics.Working across a variety of runtimes and systems is a huge benefit of Arrow, so it’s important to efficiently move data between them. Arrow Flight maintains the same on-the-wire representation of the in-memory format, enabling use of data in runtimes without serialization overhead. The book has an entire chapter on Flight, before demonstrating several real-world use cases for Arrow for analytics, data science, and ML.Community is so important to projects like these that are core dependencies across so many other projects and products. I like how Matt included a section on how to get involved, contribute, and make Arrow better for everyone. The Go library for example, is rapidly expanding, and Matt details new improvements in v8 in the later half of the book.The book wraps up with future development plans for the project, including discussion of exciting Flight SQL which reconciles JDBC, ODBC, and Flight into one high-performance Arrow-native interface.**Areas of improvement**While many diagrams are color, in at least my e-book copy, all the code snippets are grayscale without syntax highlighting. Call me spoiled by modern code editors, so while Matt has done a great job of including a lot of code snippets and examples and at the same time, without color, they are harder to parse.The other thing I would have liked to see is more examples of idiomatic use of Arrow and best-practices to really understand the most effective way to leverage Arrow and Flight. Perhaps this might make enough content for a dedicated book.**Merits**The book really shines in three ways. First, it’s worth repeating, Matt makes deep, seemingly complex concepts, like SIMD, branch prediction, vectorization, and columnar formats really simple and easy to understand. This is helpful not just for Arrow, but any high-performance software engineering. By the end of the book, these concepts will just be basic to you.Second, the book is well-structured. Concepts build upon concepts, as do the chapters. This helps you incrementally render your understanding as you read the book. Many sections have call outs and important notes to provide additional clarity.Finally, it’s very inclusive of different ways of approaching Arrow, including as mentioned, examples across several programming languages, but also notes and tips across Windows, Linux, and macOS, different real-world use-cases and adjacent technologies like ODBC.**Conclusion**Application development has transitioned across paradigms from on-premises, to cloud, and to cloud-native. The next paradigm of application development is data-native where the best applications will be data and AI-driven. Apache Arrow is core to data-native applications; it’s a technology you need to know. Matt has written a book that provides you a deep understanding of Arrow and its applications, in the easiest way to understand. If you want an edge in data-native, this book is a must-read.

This book explains the C++ part much better than I could find in the Apache Arrow C++ docs.