# Welcome Welcome to the Neurox documentation! Whether you’re installing our platform for the first time, exploring features, or troubleshooting, you’ll find clear, concise guides to help you every step of the way. ### What is Neurox? Neurox helps monitor your AI workloads running on your Kubernetes GPU cluster. Purpose-built dashboards and reports combine metrics and live Kubernetes runtime state data to help admins, developers, researchers, and finance auditors surface relevant insights. Visit [our main website](https://neurox.com) for information. ### Get started
InstallationDeploy Neurox onto your Kubernetes GPU cluster/pages/aeq4mQr5OYcwntzHdIK1
Architecture OverviewHow Neurox is architected to support varying infrastructure constraints/pages/h0Bf8M0I8bRdFUtKr5kL
Configure Google as your IdPStep by step guide to creating an OAuth application/pages/pp0PVeujwhKSVx8q3R9U
Concepts & TerminologyLearn the basics of Neurox terminology/pages/Ml7JJsHDw9GvOzY0zvbz
# Install Neurox ## Quickstart Please follow the instructions on the [Neurox Install](https://app.neurox.com/install) page. It will guide you step-by-step to a working deployment on your Kubernetes GPU cluster. The instructions will install a free, self-hosted deployment of a single, combined Neurox Control + Workload cluster. It will contain both [Control](/architecture/overview) plane components as well as [Workload](/architecture/overview) management components. It will be fully ready-to-use with ingress, TLS certs configured and available at your Neurox Control Portal subdomain (). ## Install process The installation process will: * provision a subdomain (random-words.goneurox.com) to access your Control Portal * provision image registry credentials * help you to configure an IdP to authenticate your users * automatically request TLS certificates for your subdomain {% hint style="info" %} Neurox is self-hosted software, so you install and manage it yourself. We only communicate with our servers during install and for support and billing purposes. **We will not have remote access to your deployment or any workload data on it.** {% endhint %} The install process is primarily designed to help simplify manual creation of DNS records and requesting TLS certificates, typically associated with self-hosted software. Post install, the only data to ever leave your cluster will be to handle support and billing. Airgapped installs are also available to further eliminate all outbound traffic. Although Neurox is not open-source software, Neurox is free for monitoring up to 64 GPUs, which we believe should fit many use cases, including personal, academic and light commercial use. For more information, see our [pricing plans](https://neurox.com/pricing). We also offer alternative, source-available licensing options for enterprise customers. ## Prerequisites Prior to installing Neurox, you'll need an existing Kubernetes cluster with at least 1 GPU. Having GPU workloads already running on the cluster helps showcase Neurox's features and capabilities but is not strictly necessary if you just want to poke around. We only support NVIDIA GPUs at this time. ### Cluster requirements * Kubernetes and CLI 1.29+ * Helm CLI 3.8+ * 12 CPUs * 24 GB of RAM * 120 GB Persistent Volume Storage * At least 1 GPU node * Ingress reachable from Internet ### [Application Prerequisites](/installation/alternative-install-methods/install-prerequisites) At a minimum you will need both **cert-manager** and **ingress-nginx** to run the [Neurox control chart](https://github.com/neuroxhq/helm-chart-neurox-control) **Cert Manager** Required for automated provisioning of Neurox SSL/TLS certificates. Install with: {% code overflow="wrap" %} ```sh helm repo add jetstack https://charts.jetstack.io --force-update helm repo update helm install --create-namespace -n cert-manager cert-manager jetstack/cert-manager --version v1.17.0 --set crds.enabled=true ``` {% endcode %} For more information on how to configure cert-manager: #### Ingress Nginx Required to access the Neurox web portal. Install with: {% code overflow="wrap" %} ```sh helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx helm repo update helm install --create-namespace -n ingress-nginx ingress-nginx ingress-nginx/ingress-nginx --version 4.12.1 ``` {% endcode %} For more information on how to configure ingress-nginx: For the Neurox workload component, you will need the NVIDIA GPU operator and Kube Prometheus stack. By default, the install instructions bundle the [Neurox workload helm chart](https://github.com/neuroxhq/helm-chart-neurox-workload) with the Neurox control helm chart. You can change this behavior by simply omitting the `--set workload.enabled=true` parameter in the Neurox control helm chart install command. #### NVIDIA GPU Operator Required to run GPU workloads. Install with: {% code overflow="wrap" %} ```sh helm repo add nvidia https://helm.ngc.nvidia.com/nvidia helm repo update helm install --create-namespace -n gpu-operator gpu-operator nvidia/gpu-operator --version=v25.3.0 ``` {% endcode %} For more information on how to configure NVIDIA GPU operator: #### Kube Prometheus Stack Required to gather metrics. Install with: {% code overflow="wrap" %} ```sh helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm repo update helm install --create-namespace -n monitoring kube-prometheus-stack prometheus-community/kube-prometheus-stack --set alertmanager.enabled=false --set grafana.enabled=false --set prometheus.enabled=false ``` {% endcode %} For more information on how to configure kube-prometheus-stack: # Identity Providers Neurox comes built-in support for local users and groups as well as popular IdP's. Configuring an IdP automates provisioning and deprovisioning of users. We currently support the following IdP's: * [Google](/installation/identity-providers/google) (available on Free and Plus plan) * Microsoft (Coming soon) * SAML 2.0 (Contact us for instructions) * OpenID Connect (Contact us for instructions) * LDAP (Coming soon) # Google This guide shows how to configure Google as your identity provider. ## Prerequisites * You are an administrator of your Google Cloud Console * You have [enabled the Admin SDK API](https://developers.google.com/workspace/guides/enable-apis) (required for synchronizing Groups) ## Create the OAuth application 1. Log into the [Google Cloud Console](https://console.cloud.google.com/). 2. Select an existing Project or create a new one. 3. Navigate to the [Google Auth Platform](https://console.cloud.google.com/auth/overview) via the Menu ![](https://storage.googleapis.com/support-kms-prod/JxKYG9DqcsormHflJJ8Z8bHuyVI5YheC0lAp) bar on the left. 4. If Google Auth Platform is not yet configured, click **Get Started**![](/files/r9GtHR4ih0lpSqmYNBmx) 5. Enter a name for this application (e.g. **Neurox Web App**) and select a user support email. Click **Next**.\ .![](/files/8uN6bs2lAdtf6bUwoKCp) 6. Select your intended **Audience**. We recommend **Internal** unless you want users to log in with email domains outside of your Google Workspace.\ Click **Next**.\ ![](/files/ri9wcf9doIp9d3p2vGAa) 7. Enter an email address for Google to contact you. Click **Next**.\ ![](/files/jvzgHwsTWSJFruwv8duh) 8. Check the box to agree to Google's API Services policies. Click **Continue**.\ ![](/files/jurqDZs8Lq5piVLtkVuL) 9. Click the **Create** button to create your application.\ ![](/files/tDboaU9f6d5LbH9QE28r) ### Create the OAuth client 1. Click the **Create OAuth Client** button.\ ![](/files/CuXUrUadfBX79B1Jn8lR) 2. Under **Application type** select **Web application**\ ![](/files/uRQHCW3N2wV1MYLRcMv9)• 3. Enter a **Name** for your OAuth Client (e.g. **Neurox Web App**)\ ![](/files/y1uT0ALEJfnZdMF63hsZ) 4. Under **Authorized JavaScript origins** click **Add URI**\ ![](/files/eGSv2kDMw3DALmBarAJO) 5. In **URIs 1** enter your **Control Portal** URL (e.g. `random-words.goneurox.com`)\ ![](/files/LiFad2xFJzASw0U9mkyk) 6. Under **Authorized redirect URIs** click **Add URI**\ ![](/files/rU1qKiAjB29UTrwYHIvd) 7. Enter the following URIs (click **Add URI** to add more): 1. `https://random-words.goneurox.com/api/connect/google/callback` 2. `https://random-words.goneurox.com/idp/callback` 3. `https://random-words.goneurox.com/sso/auth`\ \ Be sure to replace `random-words.goneurox.com` with your actual subdomain.\ ![](/files/OtbWCy9vloFASbmHKfA1) 8. Click the **Create** button\ ![](/files/pNAvTJBVWyDUw33Inj0k) ### Copy & Apply your OAuth credentials 1. Click the **Download** icon under **Actions**\ ![](/files/b15KldbUTpgKU81pGYBH) 2. In the pop up modal, copy both the **Client ID** and **Client Secret** values 3. Run the following commands: {% code overflow="wrap" %} ```shell kubectl create secret generic -n neurox neurox-control-idp-google --from-literal=clientId= --from-literal=clientSecret= ``` {% endcode %} # Alternative Install Methods Generally, we recommend most users deploy a single, combined Neurox Control + Workload cluster for simplicity. See [Install Neurox](/installation/install-neurox). However, this default method requires that the Kubernetes cluster to have all 3: ingress, persistent disk, and GPU nodes. Sometimes clusters with GPU nodes may not have ingress or persistent disk, such as certain on-prem deployments. Neurox is architected in such a way that the Control plane components and the Workload management components can be deployed separately to different Kubernetes clusters due to infrastructure or networking constraints. See [Architecture Overview](/architecture/overview) for more information. ## Install Neurox Control plane standalone Neurox Control can be installed anywhere that has ingress and persistent disk and does not require GPU nodes (those can be joined as Workload clusters with GPU nodes). A common approach is to deploy Neurox Control onto vanilla EKS, GKE, AKS or IKS cluster. See [Neurox Control plane standalone](/installation/alternative-install-methods/neurox-control-plane-standalone) for instructions. # Neurox Control plane standalone Neurox Control can be installed anywhere that has ingress and persistent disk and does not require GPU nodes (those can be joined as Workload clusters with GPU nodes). A common approach is to deploy Neurox Control onto vanilla EKS, GKE, AKS or IKS cluster. ## Cluster requirements * Kubernetes and CLI 1.29+ * Helm CLI 3.8+ * 4 CPUs * 8 GB of RAM * 120 GB Persistent Volume Storage * Ingress reachable from Internet ## Prerequisites At a minimum you will need both **cert-manager** and **ingress-nginx** to run the Neurox control chart. **Cert Manager** Required for automated provisioning of Neurox SSL/TLS certificates. Install with: {% code overflow="wrap" %} ``` helm repo add jetstack https://charts.jetstack.io --force-update helm repo update helm install --create-namespace -n cert-manager cert-manager jetstack/cert-manager --version v1.17.0 --set crds.enabled=true ``` {% endcode %} For more information on how to configure cert-manager: **Ingress Nginx** Required to access the Neurox web portal. Install with: {% code overflow="wrap" %} ``` helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx helm repo update helm install --create-namespace -n ingress-nginx ingress-nginx ingress-nginx/ingress-nginx --version 4.12.1 ``` {% endcode %} For more information on how to configure ingress-nginx: [https://github.com/kubernetes/ingress-nginx/tree/main/charts/ingress-nginx](https://cert-manager.io/docs/installation/helm/) ## Credentials * Your Neurox subdomain * Your Neurox install key * Your Neurox registry username and password ## Install Once you have all of the above information, set the environment variables accordingly in the script below, then copy the script into your shell and run it. The install may take up to 5 minutes to complete. {% code overflow="wrap" %} ```shell NEUROX_DOMAIN=random-words.goneurox.com INSTALL_KEY=i_installkeyhere NEUROX_HELM_REGISTRY=oci://ghcr.io/neuroxhq/helm-charts NEUROX_IMAGE_REGISTRY=registry.neurox.com NEUROX_USERNAME=random-words-goneurox-com NEUROX_PASSWORD=yourregistrypassword kubectl create ns neurox kubectl create secret generic -n neurox neurox-control-license --from-literal=install-key=${INSTALL_KEY} kubectl create secret docker-registry -n neurox neurox-image-registry --docker-server=${NEUROX_IMAGE_REGISTRY} --docker-username=${NEUROX_USERNAME} --docker-password=${NEUROX_PASSWORD} helm install neurox-control ${NEUROX_HELM_REGISTRY}/neurox-control --namespace neurox --set global.domain=${NEUROX_DOMAIN} ``` {% endcode %} ### Configure IdP By default, Neurox supports local users and groups. If you want to [configure Google as your IdP](/installation/identity-providers/google), follow those instructions, set the secret, then include this argument at the end of the Helm install command above: ``` --set global.identityProviders.enabled={google} ``` # Neurox Workload cluster The Neurox Workload management cluster is where GPU workloads run on GPU nodes. When deployed standalone, it does not require ingress nor persistent disk. Typically, the Neurox Workload components are installed together with Neurox Control plane components in a [single combined](/installation/install-neurox) Kubernetes cluster. This page outline the requirements needed to deploy standalone Neurox Workload components into additional Kubernetes GPU clusters. Neurox Workload can autodetect many Cloud Service Provider (CSP) environments, automatically surfacing metadata such as region or availability zone, as well as identify models of GPUs attached. ## Multi-Cluster setup One of the best features of Neurox is monitoring multiple Neurox Workload clusters from a single Neurox Control plane. Common use cases include joining GPU clusters from various cloud providers or even on-prem clusters. Please see our [pricing plans](https://neurox.com/pricing) to determine how many Neurox Workload clusters may be joined into a Neurox Control cluster. ## Cluster requirements * Kubernetes and CLI 1.29+ * Helm CLI 3.8+ * 4 CPUs * 8 GB of RAM * At least 1 GPU node ## [Prerequisites](/installation/alternative-install-methods/install-prerequisites) You will need both **NVIDIA GPU Operator** and **Kube Prometheus Stack** to run the [Neurox workload chart](https://github.com/neuroxhq/helm-chart-neurox-workload). **NVIDIA GPU Operator** Required to run GPU workloads. Install with: {% code overflow="wrap" %} ``` helm repo add nvidia https://helm.ngc.nvidia.com/nvidia helm repo update helm install --create-namespace -n gpu-operator gpu-operator nvidia/gpu-operator --version=v25.3.0 ``` {% endcode %} For more information on how to configure NVIDIA GPU operator: **Kube Prometheus Stack** Required to gather metrics. Install with: {% code overflow="wrap" %} ``` helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm repo update # This is the minimum required configuration. Feel free to enable components if you need them. helm install --create-namespace -n monitoring kube-prometheus-stack prometheus-community/kube-prometheus-stack --set alertmanager.enabled=false --set grafana.enabled=false --set prometheus.enabled=false ``` {% endcode %} For more information on how to configure kube-prometheus-stack: ## Credentials * Your Neurox subdomain * Your Neurox Workload auth secret (provided by Neurox Control) * Your Neurox registry username and password ## Install To join a Neurox Workload cluster to an existing Neurox Control cluster, you can obtain the install script by going to your Neurox Control portal > Clusters > New Cluster button and a fully generated install script (with auth secret) will be available to copy/paste. The example below was based on the output of the generated install script: {% code overflow="wrap" %} ```shell CLUSTER_NAME=iks-mlops-us-south-dal12 # customize this NEUROX_DOMAIN=random-words.goneurox.com WORKLOAD_AUTH_SECRET=yourworkloadauthsecret NEUROX_HELM_REGISTRY=oci://ghcr.io/neuroxhq/helm-charts NEUROX_IMAGE_REGISTRY=registry.neurox.com NEUROX_USERNAME=random-words-goneurox-com NEUROX_PASSWORD=yourregistrypassword kubectl create ns neurox kubectl create secret generic -n neurox neurox-control-auth --from-literal=shared-secret=${WORKLOAD_AUTH_SECRET} kubectl create secret docker-registry -n neurox neurox-image-registry --docker-server=${NEUROX_IMAGE_REGISTRY} --docker-username=${NEUROX_USERNAME} --docker-password=${NEUROX_PASSWORD} helm install neurox-workload ${NEUROX_HELM_REGISTRY}/neurox-workload --namespace neurox --set global.workloadCluster.name=${CLUSTER_NAME} --set global.controlHost=${NEUROX_DOMAIN} ``` {% endcode %} # Install Prerequisites ## Cert Manager Instructions from: {% code overflow="wrap" %} ```sh helm repo add jetstack https://charts.jetstack.io --force-update helm repo update helm install --create-namespace -n cert-manager cert-manager jetstack/cert-manager --version v1.17.0 --set crds.enabled=true ``` {% endcode %} ## Ingress Controller Instructions from: {% code overflow="wrap" %} ```sh helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx helm repo update helm install --create-namespace -n ingress-nginx ingress-nginx ingress-nginx/ingress-nginx --version 4.12.1 ``` {% endcode %} ## NVIDIA GPU Operator Instructions from: {% code overflow="wrap" %} ```sh helm repo add nvidia https://helm.ngc.nvidia.com/nvidia helm repo update helm install --create-namespace -n gpu-operator gpu-operator nvidia/gpu-operator --version=v25.3.0 ``` {% endcode %} ## Prometheus Instructions from: {% code overflow="wrap" %} ```sh helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm repo update helm install --create-namespace -n monitoring kube-prometheus-stack prometheus-community/kube-prometheus-stack --set alertmanager.enabled=false --set grafana.enabled=false --set prometheus.enabled=false ``` {% endcode %} # Uninstall Neurox {% hint style="danger" %} This will delete your Neurox Control plane install and all data. Proceed with caution. {% endhint %} {% code overflow="wrap" %} ```sh helm uninstall -n neurox neurox-control kubectl delete ns neurox ``` {% endcode %} # Configure Features # Multi-Cluster One of the best features of Neurox is to monitor multiple Neurox Workload clusters from a single Neurox Control plane. Common use cases include joining GPU clusters from various cloud providers or even on-prem clusters. Please see our [pricing plans](https://neurox.com/pricing) to determine how many Neurox Workload clusters may be joined into a Neurox Control cluster. ## Install See instructions by going to your Neurox Control portal > Clusters > New Cluster button. For more information, see [Neurox Workload cluster](/installation/alternative-install-methods/workload-cluster-standalone). # SSO SSO instructions can be found at [Identity Providers](/installation/identity-providers). # Architecture Overview Neurox is architected in such a way that the Control plane components and the Workload management components can be deployed separately to different Kubernetes clusters due to infrastructure or networking constraints. ## Control plane cluster * Where GPU workloads are configured/managed * Communicates with all of its joined Workload clusters * Requires ingress and persistent disk * Hosts the endpoints for web access to the Neurox Control Portal ## Workload management cluster * Where GPU workloads run * Communicates with its control cluster * Does not require ingress * Does not require persistent disk * Directly interacts with the Kubernetes API ## Components These are the components of each type of cluster. ### Control plane cluster * *Portal*: Hosts the web app * *API*: Endpoint for web app requests * *Redis*: Persists Workload and configuration data * *Thanos*: Receives and aggregates metrics from all Workload clusters * *Relay Server*: Manages the secure connections from Workload clusters * *Authentication Server*: Manages user sessions * *Identity Provider Connector*: Manages user authentication ### Workload management cluster * *Workload Manager*: Directly manages GPU workloads running in Kubernetes * *Agent*: Collects GPU metrics * *Prometheus*: Forwards metrics to the Control cluster * *Relay Client*: Establishes a secure connection to the Control cluster # Architecture Diagram {% @mermaid/diagram content="graph TD %% Define Distinct Subgraph Backgrounds for Better Contrast style WorkloadCluster fill:#C7D5E0,stroke:#6D8299,stroke-width:2px,color:#333333 style Private fill:#D2DFE8,stroke:#6D8299,stroke-width:1px,color:#333333 style ControlCluster fill:#B3C2CF,stroke:#6D8299,stroke-width:2px,color:#333333 style BehindIngress fill:#C0CDD8,stroke:#6D8299,stroke-width:1px,color:#333333 ``` %% Define Node Styles with More Contrast classDef uniformNode fill:#A4B6C5,stroke:#495866,stroke-width:1px,color:#333333; classDef specialNode fill:#7C9CBF,stroke:#495866,stroke-width:1px,color:#333333; %% Define Arrow Color and Increased Stroke Width linkStyle default stroke:#333333,stroke-width:2.5px; %% Define Subgraphs subgraph WorkloadCluster ["Workload Cluster"] subgraph Private ["No Ingress"] A["Workload Agent
[DaemonSet]
Generates metrics
and runs diagnostics"]:::uniformNode C["Metrics Forwarder
[Prometheus Instance]
Collects & forwards metrics to Thanos"]:::uniformNode B["Workload Manager
[Deployment]
Relay between the
Control Cluster and K8s API"]:::uniformNode D["Relay Client
[Deployment]
Relays requests from Control Cluster"]:::uniformNode end end subgraph ControlCluster ["Control Cluster"] E["Ingress
[Ingress]
Routes external requests and terminates TLS"]:::specialNode subgraph BehindIngress ["Behind Ingress"] F["Redis
[StatefulSet]
Persistent Data Store"]:::uniformNode G["Control API
[Deployment]
Entrypoint for web and CLI"]:::uniformNode H["Relay Server
[Deployment]
Workload cluster relay"]:::uniformNode I["Metrics Aggregator
[Thanos Instance]
Stores and queries metrics"]:::uniformNode K["Control SSO
[Deployment]
Session Management"]:::uniformNode L["Control IdP
[Deployment]
User Authentication"]:::uniformNode J["Control Portal
[Deployment]
Cluster Dashboard"]:::uniformNode end end Z["Web Browser"]:::specialNode %% Connections with Enhanced Contrast for Arrows E -->|Session Validation| K K -->|Token Validation| L E -->|User Authentication| L A -->|Gathered Metrics| C F -->|Relay configuration| H G -->|Metric Queries| I G -->|Workload cluster requests| H E -->|Forwarded Metrics| I C -->|Forwarded Metrics
#91;HTTPS]| E B -->|Join Request
#91;HTTPS]| E E -->|Join Request| G E -->|API Requests| G D -->|Relayed Requests| B D -->|Relay Client
#91;TLS]| E E -->|Relay Client| H G -->|User & Cluster Data| F E -->|Web Portal| J Z -->|API Requests
#91;HTTPS]| E Z -->|Web Portal
#91;HTTPS]| E" %} ``` # Concepts & Terminology Neurox has a number of different concepts and terminology that are important to understand. ### Users * A **User** is associated with a unique email address * A **User** can belong to one or more **Groups** * **Users** can log in as local users or via their configured [Identity Provider (IdP)](/installation/identity-providers) ### Groups * A group associates **Users** with **Workloads** * Groups are automatically synced from your configured [Identity Provider (IdP)](/installation/identity-providers) * Your IdP also determines which **Users** are part of which **Groups** * In addition to users, groups also have an associated set of **Projects** and **Quotas** * **Projects** and **Quotas** are configured in Neurox and not in your IdP ### Projects * **Projects** are company initiatives * Every **Project** has an associated **Priority Limit** and **Priority Default** * Every P**roject** has at least one allowed **Node Pool** * A single P**roject** may be associated with multiple **Groups** ### Node Pools * A **Node Pool** is a set of GPU nodes within your [**Workload Clusters**](/architecture/overview) * **Node Pools** typically have similar GPU hardware across all nodes * The total number of GPUs in a **Node Pool** constrains the maximum value of its summed associated **Quotas** ### Priority Limits * A **Priority Limit** is a numeric value directly associated with **Projects** * **Priority Limits** determine the maximum **Priority Level** for a **Workload** ### Priority Default * A P**riority Default** is a numeric value directly associated with **Projects** * **Priority Defaults** determine the default **Priority Level** that is selected for a **Workload** ### Priority Levels * A **Priority Level** is a numeric value directly associated with **Workloads** * The value of a priority level can be set to a preset less than or equal to the **Workload's** **Project's** **Priority Limit** * During eviction, higher **Priority Levels** take precedence over lower ones ### Quotas * A **Quota** allocates a number GPUs within a single **Node Pool** to a specific **Group** * A **Quota** guarantees that number of GPU resources within a **Node Pool** for that **Group's Workloads** * The sum of of all **Quotas** associated with a given **Node Pool** must not exceed the number of GPUs within that **Node Pool** ### Workloads * A **Workload** is an application that requests GPU * Every **Workload** has an associated **Group**, **Project**, and **Priority Level** * By association, every W**orkload** also has a set of **Node Pools** and **Quotas** * The user may select the **Node Pools**, and override **Priority Level**