Introduction

Welcome to Glistix! Glistix is a fork of the compiler for the Gleam language which adds a Nix backend, so that you can compile Gleam code into Nix and use it in your configurations!

This allows you to leverage Gleam's type-safety and simplicity to write reasonable and more correct code. You will also be able to use Gleam's tooling in your favor, such as unit tests and easy package management.

This book aims to explain many core concepts related to Glistix, possibly helping you use it more effectively for your projects.

Please note that this book is a work in progress! There's much more to come.

About Glistix

This chapter presents important general information regarding Glistix.

Check out the Getting Started chapter once you're ready to start using Glistix!

Goals & Roadmap

Glistix's Goals

Glistix has the following primary goals:

Provide and maintain a Nix compilation target for Gleam. That is, you should be able to write Gleam code and use it with Nix, aiming to improve the Nix development experience thanks to Gleam's compile-time guarantees, type-safety and great tooling, which should also improve the accessibility and lower the entry barrier of working with Nix (especially when it is needed to create complex and dynamic Nix configurations).
Integrate the Gleam ecosystem with the Nix ecosystem as much as possible. In particular, the existing packages for Gleam should be made usable within Nix where possible.

In addition, we have the following secondary goals:

Aid proper usage of the Gleam programming language among Nix users. The tools we are creating to use Glistix within Nix, such as builders for packages, should ideally be easily reusable for the upstream Gleam compiler as well, eventually including proper support for the Erlang and JavaScript targets, if possible.
Provide packages and bindings to aid in interacting with the Nix ecosystem from projects using Glistix. For example, we'd like to release bindings for the easy creation of NixOS modules, configuration of flakes etc. with pure Gleam, and maybe even integrate some of those into the compiler itself. Our first step so far has been the creation of the glistix_nix package.
Contribute back to the upstream Gleam compiler where possible. We have certain needs and challenges which are shared with Gleam users which use the Erlang and JavaScript targets. Therefore, where possible, we'd like to contribute our solutions to upstream and use them, in order to ensure we maintain some amount of uniformity with the rest of the Gleam ecosystem. This doesn't mean we can't add features exclusive to Glistix, but, if they aren't Nix-related, they should be kept to a minimum.

Finally, we want to clarify what we are NOT trying to achieve:

Glistix is not meant to replace or compete with Gleam. Rather, we are focused on extending and integrating Gleam to the Nix ecosystem. This includes the addition of the Nix target, but also the maintenance of tools to improve the Gleam on Nix experience as much as possible, as outlined above.
Glistix is not meant to replace all usage of the Nix language. We do want to provide a better experience when working with Nix, but it is absolutely not our goal (nor is it practical) to have Nix users switch to using exclusively Glistix for their configurations and other applications of Nix. This is also because Nix's (and nixpkgs) APIs are large and change relatively often, and thus it is not possible to always provide fully accurate type bindings for all aspects of the Nix ecosystem.

Roadmap

Here is a non-exhaustive list of tasks we want to eventually tackle within the Glistix project. This can change at any time.

Add more real-world examples and generally make the documentation more robust.
Add some form of tail-call optimization. This depends on changes to Nix itself to be fully practical, but there might be improvements we can make in the meantime.
Fully decide the semantics of Nix's lazy evaluation when using Glistix. In particular, we should take a final stance on how discarded expressions behave in this regard by default.
Improve the package management and patching story. We should cooperate with the upstream Gleam compiler to avoid future incompatibilities. Currently, our methods of overriding the Gleam standard library through Git submodules are quite manual and need replacement with proper Git dependencies together with requirement overriding.
Create a playground page in which you can compile Gleam to Nix online. Would be nice to be able to give Glistix a quick try in your web browser (maybe even in the documentation)!

Limitations

Here is a non-exhaustive list of relevant issues and warnings to consider when using Glistix.

Lack of requirement overriding and Git dependencies

The Gleam ecosystem, whose packages are mostly available through Hex, contains many useful packages which can also be used with Glistix. However, oftentimes you will find packages which do not work on Nix, because they rely on FFI with the usual Gleam targets (Erlang and JavaScript). This includes, most importantly, Gleam's standard library. The way to deal with this is to use forks of those packages patched for Nix support.

However, Gleam does not have a requirement overriding system yet (tracked at upstream issue #2899), so we depend on local and Git dependencies to those forks, as those kinds of dependencies have priority over transitive Hex dependencies (so we get some initial patching support that way). Additionally, however, Git dependencies aren't natively supported (tracked at upstream issue #1338), so we have to use local dependencies to Git submodules in order to use patches hosted in external Git repositories.

For more information, including steps to override a package, please check the page on "Overriding incompatible packages".

Missing tail-call optimization

Compared to Gleam's Erlang and JavaScript targets, Glistix's Nix target does not have tail-call optimization. This means that recursion continually grows the stack and, as such, it is possible to trigger stack overflows on Nix through deep enough recursion. As a consequence, at least for now, you should avoid relying on recursion to process very large data.

There are some ways to try to work around this limitation:

Gleam's built-in List type works as a linked list across all targets (including Nix). It is therefore ideal for recursion, but should be avoided when recursion needs to be avoided (e.g. because the list might have thousands of elements). In those cases, consider using Nix's arrays (called "lists" in Nix's official docs) instead. The glistix_nix package contains various helpful functions to aid you in using arrays. The array functions avoid recursion where possible (functions using recursion indicate they are doing so clearly in the docs - they are few).
It is possible to use Nix's builtins.genericClosure function to improve efficiency when traversing lots of data, as it allows creating arrays from existing (possibly recursive) structures, but is not recursive itself, thus allowing for some light "tail-call optimization" in some cases, particularly when you need to create arrays.
- For more information, check out this great NixOS Discourse thread by user sternenseemann: https://discourse.nixos.org/t/tail-call-optimization-in-nix-today/17763

Lazy evaluation

Nix code is evaluated lazily, meaning an expression isn't evaluated until requested. This has implications on side-effects, such as when using logging for debugging during development; more importantly, however, values which aren't evaluated lead to the creation of thunks in memory; Creating too many thunks can lead to slowdown and/or high RAM usage. Therefore, and especially when working with complex and/or heavy programs or code within Nix, try to reduce recursion and memory usage as much as possible to avoid surprises.

Discarded expressions

Due to the above, it is natural that expressions not bound to any variables would never be evaluated (in principle), due to laziness. To tackle that, Glistix ensures all discarded expressions are evaluated (at least shallowly). For example:

pub fn main() {
  // The panic below won't run due to laziness.
  // The variable is never used.
  let var = panic as "message"

  // The panic below, however, WILL run.
  // Glistix forces discarded expressions
  // to be evaluated before the function's
  // return value using Nix's `builtins.seq`
  // functionality.
  panic as "other message"

  // The panic below will also run.
  let _ = panic

  // However, the panic below won't run,
  // as the evaluation of discarded expressions
  // is shallow.
  // This might change in the future.
  #(panic as "inside tuple")

  // It is worth saying that any returned values
  // are evaluated, of course.
  Nil
}

To force deep evaluation of expressions, you can use the nix.deep_eval function from the glistix_nix package:

import glistix/nix

pub fn main() {
    nix.deep_eval(#(panic as "this will run"))

    Nil
}

Additionally, assertions are always evaluated:

pub fn main() {
    // This will cause a panic, even though
    // this expression doesn't bind any variables.
    let assert False = True

    Ok("finished")
}

Glistix does this as certain packages rely on this behavior, such as gleeunit (the default test runner), which generally encourages performing multiple assertions in one function, for example - however, said assertions are usually done through function calls not bound to any variables (or let assert expressions). In other words, if discarded expressions were ignored, the test suites of multiple packages would simply not do (almost) anything, as multiple assertions per test function are often used and their side effects (of panicking upon failure) relied upon.

Thanks

Thanks to Louis Pilfold and the rest of the Gleam team for their amazing work on the Gleam compiler and the Gleam ecosystem, which was fundamental for Glistix to come to exist.
Thanks to the Purenix project for being one of our main inspirations. It has a lot of goals in common with the Glistix project, and generally demonstrated that the initial idea which led to Glistix could actually have been something viable to do.
More generally, thanks to all contributors to Nix and the aforementioned projects as well.

Getting Started

The pages in this chapter will guide you with getting started with Glistix.

Installation

Glistix officially supports Linux, MacOS and Windows. (Note, however, that Nix doesn't support Windows yet, so you won't be able to test your projects, but glistix build should work at least.)

You can install Glistix in one of the following ways.

From GitHub Releases: Regardless of your OS or distribution, you can install Glistix by downloading the latest precompiled binary for your platform at https://github.com/glistix/glistix/releases.
With Nix flakes: Invoke the command below in the command line to download, compile and run a specific release of Glistix - here the latest at the time of writing (v0.4.0).
```
nix run 'github:Glistix/glistix/v0.4.0' -- --help
```
To install permanently, you can either add github:Glistix/glistix/v0.4.0 as an input to your system/Home Manager configuration, or use nix profile:
```
nix profile install 'github:Glistix/glistix/v0.4.0'
```
With Cargo: You can use Cargo to compile and install Glistix's latest release (v0.4.0 at the time of writing):
```
cargo install --git https://github.com/glistix/glistix --tag v0.4.0 --locked
```

Currently, Glistix cannot be installed through nixpkgs.

Basic Usage

After installing the Glistix compiler, here's how you can start working on a new Glistix project straight away:

Use the glistix new NAME command to create a new Glistix project.
- This command will set (almost) everything up for you, including initialize a Git repository, initialize the project structure (gleam.toml, src/, test/ etc.), prepare essential *.nix files, and even clone Glistix's standard library to external/stdlib as a Git submodule (this is a workaround which is currently needed while we don't have Git dependencies!).
- We also generate a default GitHub Actions CI workflow file which tries to build your project through flake.nix. You can add --skip-github to glistix new to opt out of the creation of this file (or just delete it).
You can edit src to customize the Gleam code, as well as edit gleam.toml to your liking.
- You can use glistix add name to add a dependency from Hex. For instance, you may want to use the glistix_nix package to easily access certain Nix built-in types from Gleam, which can be done with glistix add glistix_nix.
  - If your desired dependency doesn't support Nix, you will have to use a fork patched for Nix support. Check "Overriding incompatible packages" for more information.
- Note that Git dependencies are not yet supported, so you'll have to use git submodule add --name NAME URL external/NAME to clone each one, and then add it as a local dependency (package = { path = "./external/NAME" }). See Limitations for more information.
Run glistix build at least once, not only to make sure everything is working, but also to generate the manifest.toml (which should be checked into your repository).
Afterwards, to complete the Nix side of your setup, ensure you have Nix with flakes support available (the nix command), as well as run git add . so all relevant files are checked in, and then run the command below to generate your flake.lock.
```
nix flake update
```

Nice! Your project is now ready to be used by both Nix users (which will use your Gleam code compiled to Nix) and also other Glistix users.

To import a Gleam module in your project from within Nix, the default.nix and flake.nix files in your new project export a lib.loadGlistixPackage { module = "module/name"; } function, which, when used, will give you an attribute set with all names exported by that module, so you can use its record constructors, constants and functions from within Nix. See "Import a Gleam package in Nix" for more information.

Using the Compiler

This chapter describes some common ways to use the Glistix compiler.

Command-Line Interface

Here are some of the most common compiler commands.

glistix new: An essential command, creates a new Glistix project for you with batteries included, containing:
- Several basic directories and files of the project structure expected by the compiler (see "Project structure");
  - This includes an initial gleam.toml tuned for Glistix-specific defaults.
- An initial flake.nix (see "Import a Gleam package in Nix");
- An initial Git repository with a .gitignore file.
glistix build [--target nix]: Builds your project to build/dev/<target>, by default the target specified in your gleam.toml unless you specify --target <target>.
- Note that, if no target is specified in your gleam.toml, the compiler will default to the erlang target to be compatible with existing Gleam projects. As such, make sure to specify target = "nix" in your gleam.toml.
glistix run [--target nix]: Runs your project's main function in the target specified either by --target or by gleam.toml (or erlang by default, for the same reason as before).
- For the Nix target, this will call nix-instantiate to evaluate your packagename.gleam's main function.
glistix test [--target nix]: Similar to glistix run, but runs the main function compiled from test/packagename_test.gleam.
glistix format: Formats your Gleam code according to Gleam's standards.
glistix clean: Deletes your build/ directory. Useful to get rid of stale package versions.
glistix add name: Adds a Hex dependency to your project.
glistix docs build: Builds documentation for your package to build/docs.
glistix publish: Publishes your package to Hex.

Project Structure

When you run glistix new, the compiler will generate a project which should conform to the following structure (largely based on Gleam's structure, which by itself is based on Erlang's project structure):

gleam.toml contains all essential information regarding your project that the compiler should be aware of, including metadata (such as package name), your preferred target (defaults to nix, can also be javascript or erlang for compatibility with other Gleam projects), and also specifying your dependencies.
src/ contains the source code of your package. This can contain both .gleam files and also FFI files (.nix for the Nix target). A packagename.gleam file with a main public function with zero arguments is expected if your package is not a Gleam library (but rather made to be used within Nix). If present, you can check its output with gleam run.
test/ optionally contains a packagename_test.gleam file containing a single main function with zero arguments which is called when running gleam test. Use this with glistix_gleeunit or some other test runner.
priv/ is an optional folder for assets and other general files needed by your project and is not present by default. It is, however, symlinked to build/dev/<target>/<package> upon build.
external/ is an optional folder for external dependencies cloned locally as Git submodules (see "Overriding incompatible packages").

Additionally, some projects may opt into creating an output/ folder to cache build output for ease of use from Nix (see "Import a Gleam package in Nix").

Project Configuration

You can configure your project through a gleam.toml file. Its settings are mostly the same as Gleam's usual settings, available at https://gleam.run/writing-gleam/gleam-toml/. However, Glistix additionally defines a few extra settings:

There is a new [glistix.preview] section for temporary settings while Glistix is in beta. It is expected that those settings will change in the future.
Within it, you can define local-overrides = ["list", "of", "packages"]. For example:
```
[glistix.preview]
local-overrides = ["gleam_stdlib"]
```
This is used in case of conflicts between local dependencies. When the root package (the project) being compiled specifies some packages in that list, and the project itself depends on those packages, then the project's local dependencies are prioritized over the conflicting transitive local dependencies. (Motivation at "Overriding incompatible packages".)
There is also a new [glistix.preview.hex-patch] section, which is similar to [dependencies], however it only applies when publishing your package to Hex. This is a workaround so you can have local dependencies to Nix-compatible forks of packages, but still be able to publish a package to Hex. (More at "Overriding incompatible packages".)

For example:
```
[dependencies]
# When developing locally, use stdlib patch at that path.
gleam_stdlib = { path = "./external/stdlib" }

[glistix.preview.hex-patch]
# When published to Hex, depend on the normal stdlib instead.
# Downstream users will have to patch manually.
gleam_stdlib = ">= 0.34.0 and < 2.0.0"
```

Recipes

This chapter will provide some insights on tasks which often come up when using Glistix.

Import a Gleam package in Nix

Great, you've made an awesome Glistix project which you now want to use in Nix, maybe for some Nix derivation you're configuring in some other repository, or for your NixOS configuration, for example. Your main function under src/packagename.gleam is ready and returns some functions and values in an attribute set which you plan to use in those projects (again an example).

Now, how will your other Nix projects access your Gleam main function? (Or any other function!)

Luckily for us, glistix new automatically creates a flake.nix file for our new project, as well as default.nix. This makes it easy to import your Glistix project as part of another Nix project: everything you need is exported by these two Nix files (the former with builtins.getFlake or as a flake input of another flake, the latter with import).

This is because both files (with default.nix just mirroring flake.nix) export lib.loadGlistixPackage. This is a Nix function with the sole purpose of importing your Gleam code transpiled to Nix. You simply call it with lib.loadGlistixPackage { } and, by default, it will import your package's main module (with the same name as your package, which usually has the main function). You can pick the imported module with lib.loadGlistixPackage { module = "my/module"; }, for example.

It does that by invoking the Glistix compiler to build your project from scratch, and then importing the resulting build folder (moved to somewhere at /nix/store). However, since that process depends on the Glistix compiler, that might trigger a full compilation of Glistix itself, which is slow. Therefore, you might want to cache the built Nix files in your repository to speed up the process - more at the next section.

When in Nix's pure evaluation mode (e.g. when using Flakes without --impure), you will have to specify the current system in package.lib.loadGlistixPackage { system = "system name"; } (e.g. "x86_64-linux"), as the package would have to be built using a Glistix derivation (which depends on the system), unless that package caches build output as explained in the section below (in which case the system isn't necessary at all, as the Nix files are ready to be imported, so no build occurs).

As an example, let's assume your Glistix project is stored in a subfolder of your main Nix project. You can then import it like this:

let
  yourProject = import ./project/folder;
  yourPackage = yourProject.lib.loadGlistixPackage { };
  mainResult = yourPackage.main { };  # call main()
in { inherit mainResult; } # use for whatever you want!

With Flakes, you'd add your project as an input. For example:

{
  inputs = {
    yourProject.url = "github:your/repo";
    # ...
  };

  outputs = inputs@{ yourProject, somethingElse, ... }:
    let
      # You can import functions from other modules, not just main.
      # 'system' is needed here so the correct Glistix derivation
      # is used to build your package.
      yourModule = system: yourProject.lib.loadGlistixPackage {
        inherit system; module = "mod/name";
      };
      funcResult = system: (yourModule system).add 50 12;
    in
    {
      # Use it as you wish!
      # For example, within your NixOS configuration,
      # or as a parameter to a builder, or anything else, really.
      outputName.x86_64-linux.default = somethingElse (funcResult "x86_64-linux");
      outputName.x86_64-darwin.default = somethingElse (funcResult "x86_64-darwin");
    };
}

And that's it! You can now use your Gleam project within other Nix projects.

When invoking a Gleam function from Nix, make sure to use the correct representation of Gleam's types, as well as follow the conventions for calling Gleam functions. Read the "Nix Target" chapter for more information.

Caching your built Nix files

It is worth noting, however, that, by default, your project is built from scratch each time it is loaded, which requires Glistix. Since machines might not have Glistix installed, the first load might require building Glistix from source (as Glistix is not currently available on nixpkgs). That's also why the system parameter is needed when evaluating in pure mode (the default for flakes), as builtins.currentSystem (the default) fails.

To avoid that problem and have your Glistix project not depend on Glistix for Nix consumers, you can cache the built Nix files. To do so, trigger a Glistix build locally - for example, glistix build -t nix - and then copy the build results to an output/ directory, like so:

# Create destination
mkdir -p output/dev

# Copy just the resulting Nix files
# (IMPORTANT: -L flag used to deep copy symlinked folders in 'build')
cp -rL build/dev/nix -t output/dev

# Check it in so the flake can access and import from the folder
git add output

The -L flag is needed as the Glistix compiler (and the upstream Gleam compiler, for that matter) generates symlinks at the build directory to each package's priv/ folder (if it has one). This option will ensure the symlinks are properly resolved into actual files and folders (they are duplicated from the build directory).

Once that's in place, future calls to lib.loadGlistixPackage { } will automatically pick up your output/ folder, skipping the build process entirely for downstream Nix users, as long as it's properly set up (output/dev/nix is present and added to Git). Otherwise (if not properly set up), it will fallback to building from scratch. (You can force it to not do that by passing { derivation = null; } - the flake.nix has a user-friendly option at the top to enable that. Then it will just error instead.)

The major downside, of course, is having to keep the output/ folder up-to-date whenever you make relevant changes to your Gleam code. The tradeoffs should be considered.

Using `loadGlistixPackage`

The default lib.loadGlistixPackage function exported by packages, generated through glistix new, is basically a wrapper over the same function exported by the compiler itself. It takes the following additional named arguments (in { ... }):

system: Used to select the derivation of the Glistix compiler with which to build the package from scratch (if build output isn't cached). Defaults to builtins.currentSystem, if available.
glistix: Used to override the Glistix compiler derivation entirely, if desired.

The following arguments are inherited from the compiler's function:

package: Name of the package to take from the build output. Defaults to the top-level project's package (read from gleam.toml), if available, otherwise fails.
module: Gleam module to read from the build output, in the form a/b/c (no extension). Defaults to the value of package, where the main function is usually located.
output: The cached build output path. If it exists, it will be used over compiling the package from scratch. This defaults to (src)/output.
derivation: The derivation which builds the package from scratch. This is provided by the package itself usually, so this is only here to override it if you want to.
nixRoot: Defaults to dev/nix, indicates the root of the built Nix files.

Overriding incompatible packages

Many existing Gleam packages available over Hex will not work on Nix by default, as they rely on FFI with Gleam's typical targets (Erlang and JavaScript). Most importantly, this includes Gleam's standard library (gleam_stdlib).

The workaround is to create Nix-compatible forks of those packages. For instance, Glistix maintains its own fork of the standard library. To use them, however, we have the following challenges:

Currently, Gleam does not have a proper requirement overriding system (tracked at upstream issue #2899). As such, we have to use those forks as local or Git dependencies. That way, they override transitive Hex dependencies with the same name, thus ensuring the fork is used instead of the Hex version, even if the package appears as a transitive dependency (i.e. a dependency of a dependency, at any level).
However, Gleam does not currently support Git dependencies (tracked at upstream issue #1338). As such, we depend on local paths pointing to cloned Git submodules.
Finally, each fork needs to have the same name as the package it's patching (due to a compiler requirement, while there is no built-in patching yet) and, as such, cannot be published to Hex (as it can't replace the patched package itself, of course).

It is expected that a proper solution to these problems will be available in the future. For now, the best we have is cloning forks as Git submodules and using them as local dependencies.

Don't worry, though - glistix new automatically patches gleam_stdlib for you by setting up gleam.toml and cloning it as a submodule to external/stdlib. However, you will have to do that by yourself for any other package patches you might need (e.g. glistix/json).

Steps to override a package

The gleam_json package, for example, does not support the Nix target by default, while several Gleam packages depend on it, creating a problem if we want to use them. Luckily, the Glistix project maintains a Nix-compatible fork of this package at https://github.com/glistix/json. Here's how we can use it, ensuring we can use packages which depend on gleam_json (and also so we can depend on it ourselves):

Run the command below to add the repository as a Git submodule of your project. We add submodules to the external/ folder as a convention:
```
git submodule add --name json -- https://github.com/glistix/json external/json
```
Add gleam_json as a local dependency to that submodule at your project's gleam.toml:
```
[dependencies]
gleam_json = { path = "./external/json" }
```
To ensure your local override of gleam_json takes precedence over transitive local dependencies (see explanation at "Dealing with local package conflicts when patching"), you can add gleam_json to local-overrides at [glistix.preview], as below:
```
[glistix.preview]
# note that 'gleam_stdlib' is there by default
local-overrides = ["gleam_stdlib", "gleam_json"]
```
Additionally, if you intend on publishing your package to Hex, which doesn't support local dependencies, you will have to use the (temporary) workaround below to tell Hex you depend on gleam_json from Hex instead, as explained in the next section.
```
[glistix.preview.hex-patch]
gleam_json = ">= 1.0.0 and < 2.0.0" # for example
```

Finally, make sure to update your flake.nix file so that it will correctly clone the submodule when building your package through Nix. You can do so by adding the fork's repository as a Flake input, and then passing its downloaded source to the submodules list (which is later used as an argument to buildGlistixPackage), as below:

# At your flake.nix
{
  inputs = {
    # ...

    json = {  # <-- add this input
      url = "github:glistix/json";
      flake = false; # <-- get the source, not the flake
    };
  };

  outputs =
    inputs@{
      # ...
      json, # <-- add this argument
      # ...
    }:
    let
      # ... initial definitions here ...
      submodules = [
        {
          src = stdlib;
          dest = "external/stdlib";
        }
        {
          src = json;  # <-- add this here
          dest = "external/json";
        }
      ];
      # ...
    in
    {
      # ...
    };
}

Run nix flake lock to update your flake.lock to include the new submodule input.

Finally, make sure everything is working by running:

glistix build, to update the manifest.toml and ensure your package builds;
nix build, to ensure your package can still be built from the flake.

Publishing to Hex with patches

Normally, patching packages wouldn't let you publish your packages to Hex, since you cannot depend on local packages as a Hex package (you must depend on other Hex packages). Therefore, as a temporary workaround, Glistix added a [glistix.preview.hex-patch] setting to gleam.toml, where you can override your package's dependencies once published to Hex. This allows you to replace a local dependency with a Hex dependency, but only when publishing. For example:

# ...
[dependencies]
# While developing, depend on your local patch of the stdlib...
gleam_stdlib = { path = "./external/stdlib" }

[glistix.preview.hex-patch]
# However, once published to Hex, your package will depend
# on the regular stdlib instead, for compatibility with
# other packages.
gleam_stdlib = ">= 0.34.0 and < 2.0.0"

As such, users of your package will be responsible for patching by themselves, but we expect patching to only be truly necessary for core packages (e.g. gleam_stdlib), which most Glistix users will have to patch anyway (until we get proper requirement overriding).

Dealing with local package conflicts when patching

The procedure above works as local dependencies always override transitive Hex dependencies. However, they do not override other local dependencies with the same name by default.

For example, if you clone both glistix/stdlib and glistix/json repositories as submodules to external/stdlib and external/json respectively, and depend on them on gleam.toml via gleam_stdlib = { path = "./external/stdlib" } and gleam_json = { path = "./external/json" } respectively, you will get an error because gleam_json depends on gleam_stdlib at external/json/external/stdlib, while you depend on gleam_stdlib at external/stdlib. In other words, there are conflicting local dependencies.

To solve this, while a proper requirement overriding system isn't in place, you can, as a temporary workaround, add the setting below to the root package's gleam.toml (i.e. your project's gleam.toml) so that its version of gleam_stdlib is prioritized by the compiler:

[glistix.preview]
# Ensure the root package's local dependency
# on 'gleam_stdlib' is prioritized.
local-overrides = ["gleam_stdlib"]

That way, Glistix will know how to solve the conflict (use gleam_stdlib from external/stdlib, not from external/json/external/stdlib). Note that this setting is ignored for non-root packages (dependencies cannot apply local-overrides).

The Nix Target

This chapter will provide details on how Glistix translates your Gleam code to Nix, which is the Glistix compiler's main feature, and how you can use this information within your projects.

Types

Built-in types

The table below shows how Glistix translates most common Gleam types into Nix types.

Gleam Type	Nix Type	Support	Notes
Bool	Bool (`true`, `false`)	Full
Int	Int	Full*	Signed 64 bits (not unrestricted) "0b/0x/0o" literals parsed at runtime
Float	Float	Full	Excessively large float literals can cause a runtime error
String	String	Full	Unicode escapes parsed using TOML at runtime
Functions	Lambdas	Full*	Cannot compare functions for equality (always false) Functions with zero arguments take a single `{ }` argument Functions with one or more arguments take positional arguments
Tuples	Arrays (Nix Lists)	Full
Lists	Nested attribute sets When the tag is `Empty`, has no fields; when the tag is `NotEmpty`, has `head` (contained element) and `tail` (next item)	Full	No tail call optimization, so traversing large lists can cause a stack overflow
Records	Tagged attribute sets (see below)	Full
Bit Arrays	Attribute set with `buffer` field containing array of bytes (0-255 integers)	Partial	Must be byte-aligned (can't specify individual bits) Limited support for specifying segments (sized integers, UTF-8 strings and other bit arrays) Compared to the JS target, doesn't support float->bytes conversion Limited pattern matching support (equivalent to JS here)

Records

User-created types are translated to Nix as follows:

Types without constructors only exist in the Gleam type system. Therefore, it is not possible to create a type without a constructor unless you do it through FFI. This is useful to create Gleam representations of Nix types. For example, you can define an Array type which you can't construct through Gleam, but can through FFI:
```
// Only constructible via FFI
pub type Array(a)

/// Create a new Array
@external(nix, "./ffi.nix", "createArray")
pub fn new() -> Array(a)
```
Then, on the Nix side (./ffi.nix):
```
let
  createArray = { }: [ ];
in { inherit createArray; }
```
Otherwise, the Array type is not known to Nix at all (even if it understands the underlying representation of instances of Array).

Records with constructors are always represented by attribute sets. Those attribute sets contain, at least, their constructors' tags, as well as any fields. For example:

pub type Example {
  Constructor1
  Constructor2(Int, Float)
  Constructor3(field: Int, inherit: Int)  // Nix keyword? No problem
  Constructor4(Int, mixed: Int, Float, Int)
}

The module above compiles to

let
  # ! A record without fields is not a function !
  Constructor1 = { __gleamTag = "Constructor1"; };

  Constructor2 = x0: x1: { __gleamTag = "Constructor2"; _0 = x0; _1 = x1; };

  Constructor3 =
    field: inherit':
      { __gleamTag = "Constructor3"; inherit field; "inherit" = inherit'; };

  Constructor4 =
    x0: mixed: x2: x3:
      { __gleamTag = "Constructor4"; inherit mixed; _0 = x0; _2 = x2; _3 = x3; };
in
{ inherit Constructor1 Constructor2 Constructor3 Constructor4; }

Note that positional fields become _N fields, where N is the field's position. Named fields keep their names, even if they are Nix keywords.

You can construct these records in Nix by just calling their constructors. In this case, Constructor1 can be used directly; Constructor2(a, b) in Gleam would correspond to Constructor2 a b in Nix; and so on.

Modules

In Gleam, each module corresponds to a single .gleam file, and has its own exported names (pub), which includes functions, constants and type constructors.

When translated to Nix, each Gleam module becomes a single Nix file. For example, if your package is named hello, the file at src/my/module.gleam will generate a file build/dev/nix/hello/my/module.nix on build.

Each module file evaluates to an attribute set containing all exported names.

For example, the module below:

pub type Type {
  Constructor(a: Int, b: Int)
}

pub const constant: Int = 5

pub fn name() {
  "Hello World!"
}

transpiles to

let
  Constructor = a: b: { __gleamTag = "Constructor"; inherit a b; };

  name = { }: "Hello World!";

  constant = 5;
in
{ inherit Constructor name constant; }

Prelude

The prelude is always at build/dev/nix/prelude.nix by default, and contains functions which are automatically imported by the compiler as needed.

Functions

We follow the conventions below for functions. Please keep those in mind when calling Gleam functions from Nix.

Gleam functions with zero arguments are called with empty attribute sets. For example, a function such as below would be called with main { }, which would give you the Nix integer 5 (as per "Types").
```
pub fn main() {
  5
}
```
Gleam functions with one or more arguments take them positionally. For example, the function below would be called as add 1 2 and would return 3.
```
pub fn add(a: Int, b: Int) -> Int {
    a + b
}
```

Function bodies

Function bodies, just like blocks, are translated into let...in expressions. For example, the module below:

pub fn myfunc() -> Int {
  let x = 5
  let y = 10
  let z = x * y
  let w = x - z

  x + y * w
}

is translated to

let
  myfunc = { }: let x = 5; y = 10; z = x * y; w = x - z; in x + (y * w);
in
{ inherit myfunc; }

Standard Library

The Gleam standard library can be used within Glistix through the official Glistix stdlib port at https://github.com/Glistix/stdlib.

However, currently, it must be used as a local dependency to a Git submodule. See "Overriding incompatible packages" for instructions.

In the specific case of the standard library, however, this is already done automatically by glistix new.

Nix bindings library

The Glistix project maintains a library for fundamental bindings to Nix's built-in types and functions. It is rather small at the moment, but might expand further in the future. It contains bindings for Array (built-in Nix lists), AttrSet and a few other types, as well as some built-in functions.

It is available on Hex, so, to use it, just run the command below:

glistix add glistix_nix

Check its documentation at HexDocs.

Glistix Architecture

The compiler architecture inherited from the Gleam compiler is largely unchanged, more information of which can be found in the Gleam repository.

Therefore, this chapter intends to discuss a bit about the changes made in the Glistix code base on top of the base Gleam compiler. Those changes should be the main focus of contribution to the Glistix project.

General modifications

Glistix has applied several patches and additions to the base Gleam compiler. We intend to document most of them here.

The most important change is the addition of a Nix compilation target. This is reflected as a variant in the Target enumeration in compiler-core/src/build.rs. This lead to changes across many files in the compiler, mostly related to replicating code otherwise meant for other targets for Nix as well.
- We created a Nix codegen backend (which compiles Gleam code to Nix) for the Nix target at compiler-core/src/nix.rs and its submodules. There is some information about the Nix backend in the relevant section.
  - This required creating a Nix structure in compiler-core/src/codegen.rs, whose methods manage the Nix codegen process, compiling every module in a project.
  - This also required the creation of a prelude.nix file under compiler-core/templates to be consumed by the Nix backend.
- We updated the parser (at compiler-core/src/parser.rs) to add support for the @external(nix, ..., ...) attribute.
  - This introduced new external_nix-like fields in multiple structures across the compiler.
  - This required changes to analyse.rs in order to validate the paths and names of external Nix functions.
  - Similarly, format.rs was also changed so that @external(nix, ..., ...) is properly kept after formatting.
- This required a few changes in the compiler's Cap'n Proto schema (compiler-core/schema.capnp and the generated files at compiler-core/src/generated) in order to store information for the Nix target in cache (in particular, external Nix functions).
- Many compiler tests and test snapshots had to be updated as a consequence of that and other changes.
We have customized several bits of the compiler so that they display information relevant to Glistix instead of Gleam compiler. Those changes are mostly minor, and include changing error messages to point to the Glistix repository, for example.
In addition to the point above, the compiler's crates were renamed to glistix-* instead of gleam-* and their versions were changed to Glistix's own version scheme (with the initial version being v0.1.0) to better reflect Glistix as a separate project.
- As a consequence, the file compiler-core/src/version.rs was changed to reflect not the Glistix version (defined in Cargo.toml files), but the Gleam compiler version we are basing ourselves on. This means that this file must be updated each time Glistix is updated to a new Gleam version. That version is checked by packages to ensure they are being used with a compatible Gleam compiler, so using Glistix's own version would cause wrong results and/or incompatibilities when checking compiler version restrictions.
- This also required renaming all test snapshots as they are prefixed with the crate name they apply to.
We have customized several CLI commands to add Nix-specific functionality. This includes at least glistix new, glistix build, glistix run and glistix test. Please check the dedicated chapter for CLI more information.
We have added a [glistix] section to gleam.toml for Glistix-specific configuration. This was done at compiler-core/src/config.rs. Right now, it includes the [glistix.preview] section, which contains:
1. [glistix.preview.hex-patch]: allows specifying dependency metadata to override when publishing to Hex. For example, specify package = ">= 0.34.0 and < 2.0.0" under this section to tell Hex you depend on that package with that version (that will be your effective dependency for packages which depend on yours via Hex), whereas in [dependencies] you depend on the package to be available locally (only effective when developing the package locally or running tests etc.). This is a temporary workaround while we don't have proper dependency patching. It is necessary because the stdlib needs to be patched to include Nix support, which is done through local dependencies on Git submodules.
  - Relevant implementation is at compiler-cli/src/publish.rs.
2. local-overrides = [ ... ] (directly under [glistix.preview]): allows listing local dependencies which should take priority over transitive local dependencies with the same name. By default, Gleam (and thus Glistix) errors when you have two local dependencies with the same name pointing to different paths; this setting overrides that such that the root local dependency always wins. If the root package doesn't have this dependency or doesn't set local-overrides, the error can still occur. This is also a workaround while Gleam doesn't have proper dependency patching.
We have added Nix syntax highlighting support to packages' generated documentation (through glistix docs). The relevant highlight.js-compatible file is at compiler-core/templates/docs-js/highlightjs-nix.min.js.
It is worth noting that we have reutilized most of the Gleam compiler's GitHub Actions workflows for our own usage in CI and other tasks. However, noteworthy changes were made to better adapt them to Glistix's needs.
We have modified language tests (in test/language) to support the Nix target. These tests run on CI.

Nix backend

The Nix codegen backend is very much based on the JavaScript backend, and is implemented as follows in the compiler:

The entrypoint is at compiler-core/src/nix.rs. This Rust module implements the compilation of a Gleam module (file) to a Nix file. The module function is the entrypoint and invokes the compile method of nix::Generator, which is ultimately responsible for generating the module's corresponding Nix code.
- nix::module is invoked at compiler-core/src/codegen.rs whenever the project is being compiled with the Nix target.
- nix::Generator contains methods such as for generating a function definition, a record definition and module constants, as well as handling imports and exports, delegating to nix::import where necessary (see below).
The file compiler-core/src/nix/expression.rs is very important, as it implements the compilation of each kind of Gleam expression to Nix code. It contains a Generator struct which is initialized once for each function in a module, and it essentially traverses the Typed AST for each expression, converting each inner expression to Nix, recursively.
The file compiler-core/src/nix/import.rs handles imports, generating a series of import lines which appear at the top of the generated Nix file, as well as listing names to be exported at the bottom of the file (which are picked up by nix::Generator::compile).
The file compiler-core/src/nix/pattern.rs is responsible for traversing patterns (used in case clauses and let / let assert statements) and converting them into assignments and conditionals (in an abstract manner), which are then consumed by generators of case and let/let assert at nix/expression.rs and translated to if...then...else if statements where appropriate.
The file compiler-core/src/nix/syntax.rs contains multiple helpers to generate specific kinds of Nix expressions. For example, to generate a let...in expression, to generate an attribute set from a list of name/value pairs, and so on. These helpers are extensively used by other submodules of nix.
Tests are located in compiler-core/src/nix/tests.

This page is under construction.

CLI Modifications

We have changed the following parts of the CLI (crate compiler-cli):

new.rs (glistix new):
- Added Nix-relevant file templates (default.nix, shell.nix and flake.nix);
- Clone glistix/stdlib to externals/stdlib as a Git submodule by default;
- Changed default gleam.toml to include Glistix-specific options.
run.rs (glistix run, glistix test):
- Use nix-instantiate when calling glistix run or glistix test on the Nix target.
publish.rs (glistix publish):
- Implement [glistix.preview.hex-patch] by replacing dependencies with what's specified in hex-patch right before publishing.
dependency.rs (resolving versions for the manifest.toml):
- Implement local-overrides from [glistix.preview] by replacing provided (local/Git) dependencies with what the root package specified for them, if overridden by the root.
fs.rs: Added Git operations used by new.rs.

glistix build (build.rs) wasn't directly modified, but it now supports --target nix as well.

Changelog

Contains changes made for each Glistix release.

Glistix v0.1.0 (2024-04-27)

Base Gleam version: v1.1.0
Initial (beta) release of ✨ Glistix ❄️, a fork of the Gleam compiler which allows compiling Gleam to Nix.

Glistix v0.2.0 (2024-06-12)

Base Gleam version: v1.2.1
Updated Glistix to Gleam v1.2.1 (#1 and #3).
- This brings great improvements to the LSP, as well as many important bug fixes and improvements to diagnostics.
- Target aarch64-unknown-linux-gnu was added to CI.
Ported some fixes for the JavaScript target from Gleam v1.2.0 to the Nix target as well (#2).
- This fixes a miscompilation when using a module alias in a guard clause (see gleam-lang/gleam#3045).

Glistix v0.3.0 (2024-07-29)

Base Gleam version: v1.3.2
Updated Glistix to Gleam v1.3.2 (#8, #11 and #12).
- This release improves LSP autocomplete, adds arithmetic operation support to case clause guards, adds version specifier support to gleam add, and brings several other improvements and bug fixes to the compiler.
- A prebuilt Wasm binary, in order to use the compiler in the browser, has been added to releases.
Ported some fixes for the JavaScript target from Gleam 1.3 to the Nix target as well (#13).
- This fixes a miscompilation when using a record constructor alias in a constant (see gleam-lang/gleam#3294).
- A similar fix has been made to record constructor aliases in case clause guards as well (see gleam-lang/gleam#3447).
- This also adds a compile-time error when trying to use a non-byte-aligned bit array on the Nix target (which currently only supports byte-aligned bit arrays).
- Finally, this ensures gleam.nix (exported to the build folder and used to import the Nix prelude) isn't unnecessarily rewritten to avoid problems with watchers.
Added Nix target support to language tests in the compiler (#10).
- This change doesn't directly affect Glistix users, but adds some proper testing of Gleam's language features to the Nix target, improving the likelihood of bugs being caught in the Nix target implementation.

Glistix v0.4.0 (2024-09-07)

Base Gleam version: v1.4.1
Updated Glistix to Gleam v1.4.1 (#15 and #19).
- This release adds label punning syntax (writing function(data:) instead of function(data: data)), adds support for the <> (string concatenation) operator in const variables' definitions, as well as several language server improvements (listing document symbols, completion for record fields on access, signature help when calling functions, and so on).
- The JavaScript target also received support for endianness (little and big) and signedness (signed and unsigned integers) options on bit arrays, as well as sized float options (32-bit and 64-bit) and pattern matching on utf8 bit arrays. These options are not yet supported on the Nix target (but will be on a future Glistix release).
- The gleam docs (and thus the equivalent glistix docs) command received support for --target, allowing you to pick the target used to compile the project to retrieve docs for. This can be handy to generate Nix-specific docs with glistix docs --target nix.

The Glistix Handbook