Ursa Labs March 2019 Report

ursa labs
work
Author

Wes McKinney

Published

April 4, 2019

The first quarter of 2019 has now wrapped up. In March we spent a good amount of time focused on getting the 0.13.0 Apache Arrow release out of the door. I will mention a few development highlights from the month and provide the full changelog of patches later in the post.

Development Highlights

We are continuing to set up our physical build and test cluster which we’ll use to run integration tests, GPU-enabled builds, benchmark comparisons, and other automated tests to help with Arrow development.

Some highlights from our work in the Apache Arrow codebase:

  • C++ CMake Revamp: we collaborated with Uwe Korn, Kouhei Sutou, and other parts of the Arrow community on a major revamp of the CMake build system for C++, with associated improvements and fixes to the downstream packages
  • C++ expression algebra: we have begun prototyping an expression algebra to use for query engine development in C++. This work is loosely modeled after our prior work in the Ibis project.
  • Arrow Flight: the C++ build dependencies (including gRPC) for the new Arrow Flight messaging and RPC framework are now available in conda-forge. We have also added a URI library for file paths to our build toolchain to help describe network locations and protocols for Flight. We have been working closely with Two Sigma and Dremio on this important effort.
  • C++ Query Engine Discussions: we wrote a 10-page discussion document for the design of an embeddable Arrow-native analytical query engine in C++
  • R packaging support: we are working to get the Arrow R package submitted to CRAN for R users
  • Gandiva packaging: Gandiva (LLVM expression compiler) is now shipped in the Arrow 0.13.0 Python wheels

We have much work ahead of us and look forward to seeing you on GitHub, JIRA, and the dev@arrow.apache.org developer mailing list.

Team Changelog

The team had 68 commits merged into Apache Arrow in March. You can click on the ASF JIRA links to learn more about the discussion on a particular issue or the commit hash to see each patch.