Custom packages¶
Up until now we were only consuming packages in flox environments. But in your project you can also define new custom packages.
While defining custom packages can be a tedious task, it is an important step in the journey to be able to let others use your package in their environment.
If you haven't already, please install flox before continuing.
Advanced Nix knowledge required
At flox we are aware of the steep learning curve of Nix. It is part of our mission to to flatten this learning curve.
We don't want to discurage you from reading this tutorial, but we feel it is our duty to inform you that advanced Nix knowledge is required to write build recipes with Nix.
The good news is that we are aware of this and are already building a solution that will make writing build recipes as easy as writing a Dockerfile.
We welcome you to share your experiences and suggestions with us on our discourse.
1. Prepare example project¶
In this tutorial we package a simple hello-world
app written in python from the
ground up. The source to the example can be found here:
flox-examples/hello-python.
$ git clone https://github.com/flox-examples/hello-python.git --branch "pre-flox" # (1)!
$ cd hello-python
- The tutorial starts with the project in the tutorial/pre-flox branch state.
The repo is structured like this:
What does the code in this hello-python
look like?
If we examine the content we can see that this is a very simple hello-world application written in python.
2. flox init
- Create an initial project structure¶
We start by invoking flox init
to generate an initial build recipe for your
project. The process begins by prompting you to choose an appropriate template,
and in this case we will choose package-python
.
$ flox init
Found git repo: /home/USER/dev/hello-python
wrote: /home/USER/dev/hello-python/flake.nix
> Enter package name (hello-python) <enter>
? Select a template for flox init
package-bazel: Bazel package template
package-go: Go package tempalate
package-java: Java package template
package-js-yarn: Javascript(yarn) package template
package-mix: Mix package template
package-perl: Perl package template
> package-python: Python package template
package-ruby: Ruby package template
package-rust: Rust package template
package-simple: Simple package template
> Enter package name hello-python
> Select a template for flox init package-python: Python package template
HINT: avoid selecting a template next time with:
$ flox init --template package-python
wrote: /home/USER/dev/hello-python/pkgs/__PACKAGE_NAME__/default.nix
wrote: /home/USER/dev/hello-python/pkgs/__PACKAGE_NAME__/flox.nix
wrote: /home/USER/dev/hello-python/pkgs/__PACKAGE_NAME__
wrote: /home/USER/dev/hello-python/pkgs
[INFO] [flox_rust_sdk::models::project] moved: /home/USER/dev/hello-python/pkgs/__PACKAGE_NAME__ -> /home/USER/dev/hello-python/pkgs/hello-python
After flox initializes the package, we find that the repository has changed:
Let's inspect the content of the files just generated:
{
description = "A flox project"; # (1)!
inputs.flox-floxpkgs.url = "github:flox/floxpkgs"; # (2)!
outputs = args @ {flox-floxpkgs, ...}: flox-floxpkgs.project args (_: {}); # (3)!
}
-
A description of the project.
-
A definition of an external resource.
flox/floxpkgs
is a collection of utilities and packages that provide an easy way to start working with flox. -
Glue code that scans your project for package and environment configurations.
{
self,
python3Packages,
}: # (1)!
python3Packages.buildPythonPackage { # (2)!
pname = "hello-python"; # (3)!
version = "0.0.0"; # (4)!
src = self; # (5)!
propagatedBuildInputs = with python3Packages; [ # (6)!
requests
];
meta.description = "An example hello-python flox package."; # (7)!
}
-
Section of the file where we import things that we will later need in the build recipe.
In our case we import:
self
: An object that holds reference to our project's content.python3Packages
: A Nixpkgs collection of python 3 packages and utilities. -
A
buildPythonPackage
utility function from Nixpkgs which is used to describe a python package in Nix. -
A package name.
-
A package version.
-
Source code of your project.
-
Runtime dependencies.
propagatedBuildInputs
in Nix this concept refers loosely to runtime dependencies.buildPythonPackage
function does not know how to extract dependencies from setup.py. You need to provide them manually. -
Description of a package. It will appear in
flox search
results.
We will cover the purpose of the flox environment file (pkgs/hello-python/flox.nix
) in detail a
bit later in this tutorial when talking about developing a custom
package.
3. flox build
- Build the custom package¶
With both a build recipe (pkgs/hello-python/default.nix
) and the glue code
(flake.nix
) in place we can now build a package.
$ flox build # (1)!
warning: not writing modified lock file of flake 'git+file:///home/USER/dev/hello-python':
...
• Added input 'flox-floxpkgs/tracelinks/flox-floxpkgs':
follows 'flox-floxpkgs'
$ ./result/bin/hello # (2)!
Hello python
-
This command tells flox to build your project and then creates a symbolic link to the package at
./result
. -
Let's run just built
hello
binary. -
Running
hello
binary.
4. flox run
- Build and run the custom package¶
These 2 steps can be simplified into one command.
- Set the name of the main binary program in
pkgs/hello-python/default.nix
:
Index: pkgs/hello-python/default.nix
===================================================================
--- pkgs/hello-python/default.nix (revision 100644)
+++ pkgs/hello-python/default.nix (working copy)
@@ -12,4 +12,5 @@ python3Packages.buildPythonPackage {
requests
];
meta.description = "an example flox package";
+ meta.mainProgram = "hello";
}
Build and run:
$ flox run # (1)!
warning: not writing modified lock file of flake 'git+file:///home/USER/dev/hello-python':
...
• Added input 'flox-floxpkgs/tracelinks/flox-floxpkgs':
follows 'flox-floxpkgs'
Hello python
- In the background, this command runs
flox build
and `./result/bin/hello' for you.
4. flox develop
- Developing a package¶
With the build recipe (pkgs/hello-python/default.nix
) having all the required
information, we can provide a development environment for our package. Not only
that, we can extend the development environment with extra development
tools.
Adding extra development tools is done via the flox environment file
(pkgs/hello-python/default.nix
).
Lets add the black
package to our flox environment file
(pkgs/hello-python/flox.nix
) to be able to format our python code.
{
# flox environment configuration
#
# flox.nix options: https://flox.dev/docs/reference/flox-nix-config
# Getting started with flox: https://flox.dev/docs
# Get help: https://discourse.flox.dev
#
# Happy hacking!
packages.nixpkgs-flox.black = {}; # (1)!
environmentVariables.PIP_DISABLE_PIP_VERSION_CHECK = "1"; # (2)!
}
-
black
is a commonly used python formatter. -
The
PIP_DISABLE_PIP_VERSION_CHECK
environment variable makes sure that pip will not try to update itself.
Now lets enter the development environment for our hello-python
package.
$ flox develop
warning: creating lock file '/home/USER/dev/hello-python/flake.lock'
warning: not writing modified lock file of flake 'git+file:///home/USER/dev/hello-python':
• Updated input 'flox-floxpkgs/nixpkgs/nixpkgs':
follows 'flox-floxpkgs/nixpkgs/nixpkgs-stable'
→ 'github:flox/nixpkgs/1b1f50645af2a70dc93eae18bfd88d330bfbcf7f' (2023-01-23)
warning: not writing modified lock file of flake 'git+file:///home/USER/dev/hello-python':
• Updated input 'flox-floxpkgs/nixpkgs/nixpkgs':
follows 'flox-floxpkgs/nixpkgs/nixpkgs-stable'
→ 'github:flox/nixpkgs/1b1f50645af2a70dc93eae18bfd88d330bfbcf7f' (2023-01-23)
Executing setuptoolsShellHook
Obtaining file:///home/USER/dev/hello-python
Preparing metadata (setup.py) ... done
Installing collected packages: hello-python
Running setup.py develop for hello-python
Successfully installed hello-python-0.1
Finished executing setuptoolsShellHook
$ black hello # (1)!
reformatted hello/__main__.py
All done! ✨ 🍰 ✨
1 file reformatted, 1 file left unchanged.
$ python -c "import requests; print(requests.__name__)" # (2)!
requests
$ hello
Hello world!
$ sed -i -e 's/world/everyone/' hello/__init__.py
$ hello # (3)!
Hello everyone!
$ exit
-
Command
black
is now available in the environment. -
Python is configured with the required modules.
-
With Nix
$PATH
is configured to invokehello
directly from source code, which allows for rapid iterative developmentNote that changes to the source code are immediately reflected in subsequent invocations.
Where to next?¶
-
Publish custom packages for others to use.