Development environment setup
Fixie uses a few tools to streamline development:
Homebrewis a package manager for MacOS that also mostly works for Linux. If you're running Debian or Ubuntu Linux, you can alternatively get by with apt.
Poetrymanages our Python packages.
Bufmanages our protos.
Justsimplifies our shell workflows. It frequently functions as our interface to all the other tools.
npmpowers our frontend.
PostgreSQLpowers our databases.
Here's a quick guide to setting up your local development environment. It's written for MacOS (with Homebrew), but each tool's website has simple alternatives if you prefer
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
pyenvto create a Python environment with Python 3.11:
brew install xz
brew install pyenv
pyenv install 3.11
pyenv global 3.11
pyenv shell 3.11
brew install bufbuild/buf/buf
protoc-gen-openapiplugin, required by
brew install go
go install github.com/google/gnostic/cmd/protoc-gen-openapi
If you run into any issues here, make sure that
$HOME/go/binis in your PATH.
brew install postgresql@14
brew install pgvector
brew services start postgresql@14
aptinstead, see the Postgres docs for setting up the pgdg APT repository and then run
sudo apt install postgresql-15
sudo apt install postgresql-15-pgvector
Create your development databases:
psql postgres -c "CREATE EXTENSION vector;"
psql < postgres_setup.sql
You can re-run the command any time you'd like to drop all your data to get a fresh start.
brew install just
git lfsbecause all the images in the repo are hosted there :
brew install git-lfs
git lfs install
git lfs pull
brew install node
Set up the Public SDK:
pip install fixieai
In this directory, run
just install. If you have any issues with pip here try following these instructions https://pip.pypa.io/en/stable/installation/
If you get any module error messages, run
git submodule update --init. The error messages may look something like:
fixie/chain/sql/evaluation/spider/metric/spider_exact_match.py:11: error: Module "third_party.elementai.spider" has no attribute "evaluation" [attr-defined]
fixie/chain/sql/evaluation/spider/metric/spider_test_suite.py:11: error: Module "third_party.elementai.test_suite_sql_eval" has no attribute "evaluation" [attr-defined]
Found 2 errors in 2 files (checked 233 source files)
error: Recipe `install` failed on line 28 with exit code 1
If you're going to develop the frontend:
Google Cloud credentials
To access the relevant Google Cloud resources locally you will need allow Google SDKs to use your @fixie.ai Google Account.
- When it asks you which cloud project to use, choose
We use the Black formatter, and mostly adhere to the Google Python style guide.
just format to run
isort on all of the code.
just check to run formatting and type checks.
just test to run tests.
Running a local copy of the server
You need three running processes:
- beta.fixie.ai frontend (only needed if you're making changes in
- Generic Sidekick Frontend UI (GSF)
- The Default Agent Runtime For Fixie (DARFF)
Run these proccees with:
just run will produce a lot of output. You'll know the server is ready when the output includes:
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
just browse to interact with the local app.
Running the console UI
just console to run the console UI.
Adding new Python dependencies
Add them to
pyproject.toml and run
poetry update && poetry install.
When developing new code in this repository, please follow these steps:
- Create a branch named after your username, e.g.,
git checkout -b zach/new-feature
- Make the changes in your branch.
justto ensure that the code is properly formatted and checks and tests pass.
just runto test the server locally.
- When you're ready to request a code review, push your branch to GitHub:
- Create a pull request for your changes, and add the relevant people as reviewers. Note that doing so will cause your branch to be deployed automatically to Google Cloud Run in its own hermetic environment, with its own Docker images and databases. You can see the process in the Actions tab of GitHub, and the resulting deployment will be linked from the "Conversation" page of your PR.
- Once your changes are approved, submit the code to
main. This will cause the changes to be deployed to the main Fixie app URL.
Changing database models
When editing a database model (typically in fixie/frame/models.py), a database migration will need to
be created to ensure the production database can be updated in-place to know about the new field.
To create a database migration, simply run
just makemigrations, and you can update your local
db instance via
IMPORTANT: When changing a database model, the model edit and associated migration must be in their own PR (that is, separate from any other feature functionality). Reverting a migration can be problematic, and this approach allows us to ensure any database changes are low-risk.
Deployment on Google Cloud Run
Our production and staging environments are deployed using Google Cloud Run. This requires building a Docker image containing the local code, along with configuration to cause the deployed service to use Google Cloud SQL as the database for storing user data.
These Docker images are built and deployed automatically when new code is pushed
main branch, using GitHub Actions. The configuration for GitHub Actions
are in the
You can perform a manual build and push by following these steps:
just push-dockerto build and push the Docker images based on the code in your local branch.
./scripts/deploy-cloud-run.sh. This script takes two arguments.
--environment-nameis the name of the Cloud Run service to deploy, which is usually named after your local branch.
--image-tagis the tag for the Docker image to deploy, which you get when running
just push-dockerstep above.
Running Docker image locally
You can also build and run Docker images locally, which is a good way to test
the service in a hermetic environment without depending on your local machine
configuration. To do this, run
just run-docker. This will build the Docker
images and run them locally. Note that the local Docker image does not depend on
the Google Cloud SQL database; it uses a local SQLite database, so state is not
persisted outside of the container.
Generating new public/private key pair
$ openssl genpkey -algorithm ED25519 -out private.pem
$ openssl pkey -in private.pem -pubout -out public.pem
$ gcloud secrets versions add llamalabs-private-key --data-file private.pem
$ gcloud secrets versions add llamalabs-public-key --data-file public.pem
$ rm private.pem public.pem