The cure

However, if we are looking to implement a distributed computing solution using Go, then we're in luck.

Let's take a quick look at the features of a new Go package for performing distributed MapReduce for data processing at scale.

Gleam - distributed MapReduce for Golang

"First, generics are needed. Of course, we can use reflection. But it is noticeably slower, to the point that I do not want to show the performance numbers. Second, dynamic remote code execution is also needed if we want to dynamically adjust the execution plan. We could pre-build all the execution DAGs first and choose one of them during runtime. But it is very limiting. As everyone else here, I enjoyed the beauty of Go. How to make it work for big data?"

- Chris Lu

That's the right question.

Chris resolved the performance issues of reflection and the lack of Generics using a scripting language named LuaJIT. Rather than building the entire directed acyclic graph (DAG) and then choosing one branch during runtime, the scripting...