Blog
Nix in the Wild: Looker (GCP)
Flox Team | 18 Aug 2023
Nix in the Wild is a series where we dive into the stories of Nix users across the industry, covering everything from the dotfiles of crafty developers to the processes of engineering leaders in large organizations. Learn where Nix is used, how it came to be, and why it works the way it does.
In the world of software engineering, champions often emerge that help evolve the way we build and manage our projects. One such champion, Farid "FarNix" Zakaria, an engineering manager at Google, has been leading a remarkable transformation in Looker, a business intelligence platform.
Farid has helped the Looker team transition from a heterogeneous set of build targets to a harmonious ecosystem with consistent builds, all based on the power of Nix.
Before being acquired by Google, Looker was a somewhat typical tech startup. Farid describes their build systems as a "proliferation of scripts that mostly work”, similar to the ones you would find elsewhere. These scripts did, indeed, serve their needs. However, as the team began to introduce Google's software development standards, the urgency for consistent builds became more critical. They realized they needed a solution that would guarantee reliable and consistent builds from source, one they could implement in a non-invasive way.
About Nix at Looker
Google has a unique self-empowering model that enables engineers like Farid to bring about transformative changes. He and his team turned to Nix, an open-source package manager. However, they didn't want to force a "hardcore" Nix setup upon their developers. Instead, they adopted a methodical approach, slowly integrating Nix, tackling the biggest pain points, and embracing it at a steady cadence.
With Nix, a build process only has access to explicitly declared dependencies. That means they can be confident that they are aware of everything that a successful build contains. If a build succeeds, Nix ensures that the software bill of materials (SBOM) is accurate and comprehensive.
Because the ability to recreate any build from original sources is important to them, they do not rely on the Nix public binary cache; they build everything they ship. This allows them to be confident in their builds, helps them keep track of their software supply chain, and adds a layer of resilience against potential cache outages or dependency issues.
Addressing challenges
Farid admits that Nix comes with a learning curve, but he and the team saw it as a worthwhile investment. He believes that, with a bit of effort, it’s not as hard as it may seem to win people over.
He suggests taking a holistic approach to evangelizing Nix. Imagine trying to flatten a waterbed with a broomstick: pushing down in one area will just cause another to pop up. The same is true for the pain points of a new technology, explains Farid, which must be addressed by teaching big concepts instead of small workarounds. He believes that after considering the entire surface area of Nix and teaching its principals, he is able to help users quickly find comfort with them and see the potential across the entire software development lifecycle.
In fact, because Nix provides the ability to consistently restart from scratch, the engineers he worked with were able to explore and experiment with less risk than with their previous systems. The way that Nix works makes it easy to try new things and revert back to a known-good state. Once engineers are comfortable with that basic interaction, Farid finds that their learning accelerates quickly.
Sometimes users needed specific packages or versions that were not available in Nixpkgs. Farid’s team was ready to offer "white glove" service to engineers, sitting down one-on-one with them, showcasing new tools, and collaboratively reviewing their existing build scripts. Because of this dedication, he is called “FarNix” by colleagues who know him as the person who can help with their Nix questions.
Outcomes
Building everything from original sources is a practice with intrinsic operational benefits. From time to time, an original source will disappear - perhaps a hosting system has gone offline, a file is temporarily unavailable, or an upstream project no longer exists. When this happens, it’s important to resolve the issue so that services that are running in production can always be traced back to their source. Thanks to Nix, they are notified of this during the build process, giving them a head start towards resolution.
There have been clear efficiency benefits as well. Their build process has evolved from a giant README with tons of detail to a single command that addresses all of the necessary corner cases. This streamlined workflow has allowed them to reduce the onboarding time for new engineers significantly, even during a period of rapid expansion.
Conclusion
Farid is a strong champion for Nix, both inside and outside Google. He hosts many Nix meetups in his local community, and was a driving force behind Google’s sponsorship of NixCon and the NixOS organization.
For Google, the journey doesn't end here. Farid dreams of a day when Nix becomes an integral part of Google's entire pipeline. While they have achieved remarkable consistency within Looker, he envisions a future where more teams at Google are leveraging Nix.
Naturally, this vision is not limited to the build process. Farid also imagines a world where Nix is used to rethink basic primitives within the Linux landscape such as compilers, linkers, loaders and object file formats. He’s already published a paper on some of this research on the topic while he pursues a PhD at University of California Santa Cruz and often writes about his musings on his blog. You can also check out his talk from NixCon 2022.
But these are stories for future editions of Nix in the Wild. Stay tuned!