Blog
Nix in the Wild: PDT Partners
Flox Team | 27 June 2023
Nix in the Wild is a series where we dive into the stories of Nix users across the industry, covering everything from the dotfiles of crafty developers to the processes of engineering leaders in large organizations. Learn where Nix is used, how it came to be, and why it works the way it does.
In this post we had the pleasure to interview Graham Bennett, the head of Software Infrastructure at PDT Partners, a prominent quantitative investment manager, who shed light on how Nix has revolutionized their workflow. His team focuses on high-performance computing, real-time trading, and full-stack development. Controlling configurations and ensuring reproducibility are critical aspects of their work, making Nix an ideal fit for their needs.
Graham's journey with Nix began in 2015, but it wasn't until 2018 that they fully embraced it. “It looked like the correct way to do things from a theoretical perspective,” Graham mentioned. Initially, they felt they didn’t have the internal expertise to adopt Nix, and explored alternative packaging solutions. Hiring individuals experienced in Nix proved to be a turning point, allowing them to navigate and apply its principles to their use cases effectively.
The adoption of Nix at PDT followed a gradual process. They started by using Nix to provision development tools, enabling them to access the latest versions easily. They then moved on to their Python scientific computing environment, where they maintained and released software with fine-grained control over compilers and libraries. Replicating their existing system in Nix proved to be a game-changer, significantly reducing the time it took to produce builds and eliminating version conflicts.
As their confidence in Nix grew, PDT progressed to building and packaging their own software using Nix. Today, most of their production systems and containers are built using Nix, providing consistency and reliability across their entire stack. Additionally, Nix's seamless integration with their CI/CD pipeline streamlines the testing process and ensures a smooth development experience.
While Nix has proven to be a valuable asset for PDT, they did encounter some hurdles along the way. “The biggest hurdle was getting a large group of developers to work with Nix”. The team's initial focus on capabilities rather than user experience resulted in a steeper learning curve. However, with the advantage of hindsight, they now recognize the importance of emphasizing user experience from the start.
Despite these challenges, Graham enthusiastically encourages others to explore Nix. He acknowledges its initial complexity, but highlights the long-term benefits and the satisfaction that comes with doing things the right way. “It’s easy to look at Nix and dismiss it, to underestimate it, but then to burn so much time working around problems that Nix would solve. Doing things the right way is worth it in the end”. What once took months to achieve in terms of reproducibility and version control can now be accomplished in a matter of hours.
Looking towards the future, PDT hopes to extend Nix further across their development environment, production build and release processes. They aspire to create standardized and user-friendly workflows, leveraging Nix's capabilities to provide reproducible development environments across different projects. They hope that Nix will become the premier choice for the scientific computing and ML communities.
Graham’s insights and experiences at PDT are just one example of the success stories surrounding Nix. By raising awareness and celebrating these achievements, we can establish Nix as a clear choice for modern enterprises. We recognize it as a powerful tool that empowers teams to manage complex systems, achieve reproducibility, and elevate the user experience.
Nix has indeed transformed the way we approach software development, making it more efficient, reliable, and enjoyable. As more organizations and individuals embrace its capabilities, Nix is poised to become a cornerstone of the technology ecosystem, enabling groundbreaking achievements and propelling innovation forward.