Install Lumigator on Kubernetes
This guide will walk you through the process of installing Lumigator on a Kubernetes cluster. The official way to deploy the Lumigator application in cloud environments with Kubernetes is to use the Helm package manager.
At this moment this Helm chart deploys the Lumigator core REST API (backend) and the Lumigator frontend. Additionally, by default, it also deploys a Ray cluster with a postgres instance as a dependency of the backend.
If you want to use your existent relational database instance or Ray cluster, you will have to reconfigure the dependencies of the backend chart.
Prerequisites
To install Lumigator on a Kubernetes cluster, you need to have the following prerequisites:
A Kubernetes cluster running.
A S3-compatible storage bucket.
Helm installed.
Configuration
The Lumigator chart is composed of two sub-charts, backend and frontend. The main Lumigator one is designed to deploy everything you need with a single command, and only include in the values file those values that are required to make the sub charts work together (like the address of Ray).
By default, the backend chart also deploys a PostgreSQL instance, and a Ray cluster into Kubernetes, with a minimal configuration ready to work with Lumigator.
Note
If the Mistral and/or the OpenAI API is used, there are two ways to provide it to Lumigator:
Using an existing Secret, whose name will be specified in property
existingMistralAPISecret
and/orexistingOpenaiAPISecret
Using an explicit Mistral and/or OpenAI key in property
mistralAPIKey
and/oropenaiAPIKey
, which will be added in a new Secret.
In order to be able to use Mistral and/or the OpenAI API, you also have to add this configuration to your values file:
ray-cluster:
head:
containerEnv:
- name: MISTRAL_API_KEY
valueFrom: "name-of-the-mistral-secret"
secretKeyRef: "name-of-the-mistral-secret-key" # pragma: allowlist secret
- name: OPENAI_API_KEY
valueFrom: "name-of-the-openai-secret"
secretKeyRef: "name-of-the-openai-secret-key" # pragma: allowlist secret
Example values configurations
This is just a minimal example of the values that you have to set in the values files of the backend chart in order to make Lumigator work as expected:
Backend:
s3Bucket: "example-bucket-name" # Name of the S3 bucket you want to use.
AWSAccessKey: "EXAMPLE_AWS_ACCESS_KEY" # AWS access key.
AWSSecretKey: "EXAMPLE_AWS_SECRET_KEY" # AWS secret key. # pragma: allowlist secret
rayAddress: "example-ray-cluster-head-address" # Address of the Ray cluster head service. If you use the Ray cluster deployed with the backend chart, the value is the first word of your helm release name + -lumigator-kuberay-head-svc
ray-cluster:
head:
containerEnv:
- name: FSSPEC_S3_KEY
value: "example-s3-key" # S3 access key used by containers.
- name: FSSPEC_S3_SECRET
value: "example-s3-secret" # S3 secret key used by containers.
- name: FSSPEC_S3_ENDPOINT_URL
value: "https://example-s3-endpoint.com" # S3 endpoint URL used by containers.
Save this example as a values.yaml
file, customize it with your own values, and you can proceed to install Lumigator following the steps describe in the next section.
Installation
To install Lumigator on a Kubernetes cluster, follow these steps:
Clone the Lumigator repository:
user@host:~$ git clone git@github.com:mozilla-ai/lumigator.git
Install the Lumigator Helm chart:
user@host:~/lumigator$ helm install lumigator ./infra/helm/lumigator -f values.yaml
Next Steps
Congratulations! You have successfully installed Lumigator on your Kubernetes cluster. Now you can start using it to evaluate your language models. Head over to the user guides to learn how to interact with the Lumigator API to create new evaluation jobs.