Docs / Overview

Kernpool: manage all your AI compute

Kernpool runs, manages, and scales AI workloads on any AI infrastructure — Kubernetes, Slurm, VMs, on-prem machines, and 20+ clouds, behind one interface.

Getting started

Overview

Kernpool runs, manages, and scales AI workloads on any AI infrastructure. AI teams get one interface for launching work wherever capacity exists. Infra teams get a single control plane over all of it, with scheduling, scaling, and orchestration built in.

It stays out of the way for the people using it, makes Kubernetes workable for AI, unifies clusters and clouds behind one interface, and keeps the fleet busy through autostop, binpacking, and a scheduler that places work where it is cheapest and actually available.

Installation

Install the CLI with the extras for the providers you use, then run a check to see what Kernpool can reach. Python 3.9 or newer is required.

$ pip install "kernpool[kubernetes,aws,gcp,azure]"$ kernpool check# Kernpool reuses whatever credentials you already have configured.

No separate service account is needed to get started. If you want Kernpool to run under a dedicated identity, see Cloud accounts and permissions.

Quickstart

A job is a short YAML file describing what it needs and what to run. Launch it and Kernpool finds a match, provisions it, syncs your code, and streams logs back.

resources:accelerators: H100:8any_of: [prod-us-west, burst-aws, gpucloud-eu]num_nodes: 64setup: pip install -r requirements.txtrun: torchrun --nnodes=$KP_NUM_NODES train.py

$ kernpool launch -c pretrain train.yaml$ kernpool queue$ kernpool ssh pretrain

Examples

Working recipes for distributed training, reinforcement learning, batch inference, and serving. Each one is a YAML file you can copy and adapt rather than a tutorial to read through.

• Distributed training with PyTorch, DeepSpeed, and Ray
• RL rollouts with heterogeneous trainer, sampler, and environment groups
• Large-scale batch inference over object storage
• Serving open-weight models behind an autoscaled endpoint

Concept: compute pooling

Compute pooling treats every cluster, reservation, and rented GPU as one addressable resource pool rather than a set of separate systems each with its own access path.

The practical consequence is that a job describes what it needs, not where it runs. Kernpool resolves that description against everything available and moves the job if conditions change.

Kernpool Platform

The open source project handles orchestration on infrastructure you operate yourself. Kernpool Platform is the managed control plane for teams running large fleets across several providers.

• High-availability managed control plane
• Multi-cluster fleet management with GPU health checks and auto-remediation
• Priority queueing, quotas, and workspaces
• SSO, RBAC, and policy enforcement
• Cost reporting and chargeback
• Dedicated support

Sandboxes

Sub-second sandboxes for agents and RL rollouts, running on your own infrastructure. Spawn them in bulk, tear them down instantly, and hold no standing capacity between runs.

$ kernpool sandbox create --count 240 --image agent-env:latest# Sandboxes are ephemeral; anything not persisted is discarded on exit.

Clusters

Start a development cluster

Bring up a machine with GPUs attached, then connect to it the way you normally work: SSH, an IDE remote session, or a synced working tree. Idle clusters stop themselves so nobody has to remember.

$ kernpool dev up --gpus H100:1$ kernpool ssh dev-01

Cluster jobs

Several jobs can share one cluster. Kernpool keeps a queue per cluster, runs what fits, and holds the rest until resources free up, so a cluster you already paid for stays busy.

Provisioning compute

When a job needs new capacity, Kernpool searches every enabled provider in cheapest-and-available order and fails over automatically when a region is out of stock or a request is rejected.

The search space is whatever you allow: a single Kubernetes namespace, a list of named clusters, or every provider you have credentials for.

Autostop and autodown

Autostop halts a cluster after a period with no running jobs and keeps the disk. Autodown tears it down completely. Both are set per cluster and are the single biggest lever on idle spend.

$ kernpool autostop pretrain --idle-minutes 30

Jobs

Managed jobs

A managed job is supervised for its whole life. If the underlying instance is preempted or fails, Kernpool re-provisions elsewhere and resumes, without you watching the terminal.

Checkpointing and recovery

Point Kernpool at your checkpoint directory and it syncs to durable storage as the run progresses. On recovery the job restarts from the last complete checkpoint rather than from zero.

Multi-node jobs

Multi-node jobs are gang scheduled: either every node is available and the job starts, or none are held. Rank, world size, and the coordinator address are exposed as environment variables.

$KP_NUM_NODES   # total nodes in this job$KP_NODE_RANK   # 0-indexed rank of this node$KP_HEAD_IP     # address of rank 0

Many parallel jobs

Hyperparameter sweeps and evaluation grids submit as one command. Kernpool spreads them across whatever is free and packs several small jobs onto a shared node when they fit.

Model training guide

Practical guidance for long runs: how to size checkpoint intervals against interruption rates, when to prefer reserved capacity over spot, and how to structure a run so a failed node costs minutes rather than hours.

Worker pools

A pool is a set of warm workers that stay up between tasks. Use one when startup cost dominates: short inference tasks, agent rollouts, or anything dispatched thousands of times.

Batch inference

Read from object storage, process in parallel, and write results back. Failed tasks retry on their own and preempted workers resume from where they stopped, so a batch survives losing part of its fleet.

Job groups for RL

RL workloads are several coupled processes with different hardware needs. A job group co-schedules trainers, samplers, and environments together, each on hardware suited to it, and treats the whole group as one lifecycle.

Model serving

Endpoints

Deploy a model as an HTTP endpoint on any cluster you own. Kernpool handles replica placement, health checking, and routing across clusters.

$ kernpool serve up -n chat serve.yaml$ kernpool serve status chat

Autoscaling

Replicas scale on request rate or queue depth, within bounds you set. Scale-to-zero is available for endpoints that are idle for long stretches.

Updating a service

Rolling updates replace replicas gradually and roll back automatically if the new version fails health checks. Canary releases send a fixed share of traffic to the new version first.

Authorization

Endpoints can require a bearer token or sit behind your own identity provider. Tokens are issued and revoked per consumer rather than shared across a team.

Serving on spot instances

A spot policy keeps a floor of on-demand replicas for availability and fills the rest with interruptible capacity. When spot replicas are reclaimed, traffic shifts before they disappear.

HTTPS encryption

Terminate TLS at the endpoint with your own certificate, or let Kernpool provision and renew one automatically.

High-availability controller

The serving controller can run replicated so that a controller failure does not take down the endpoints it manages. Recommended for anything in production.

Infra choices

Using Kubernetes

Kernpool makes an existing cluster AI-native rather than replacing it. Jobs and deployments are launched through an interface built for AI work; development against pods stops being painful.

• Gang scheduling, multi-node jobs, and a real queue instead of pods quietly stuck Pending
• Multi-cluster support brings every cluster under one control plane
• Topology-aware placement, priorities, and preemption
• SSH, code sync, and IDE attach against pods

Using Slurm

Point Kernpool at an existing Slurm cluster and it submits through Slurm rather than around it. Useful when an HPC cluster is the capacity you already have and cannot replace.

Using existing machines

A list of IP addresses you can SSH into is enough. On-prem racks and leftover reserved capacity join the same pool as everything else, with no agent to install first.

node_pool: rack-ahosts: [10.0.4.11, 10.0.4.12, 10.0.4.13]user: ubuntu

Using reservations

Committed capacity is used first and burst capacity only when the reservation is full. This is the default ordering, and it is what keeps a reservation from sitting idle while you pay for on-demand.

Using cloud VMs

Kernpool provisions VMs directly when no cluster is involved, including spot instances with automatic recovery. Quota increase requests are the usual first obstacle; the guide covers the request path per provider.

GPUs and accelerators

NVIDIA GPUs, AMD GPUs, Google TPUs, and plain CPU instances are all addressed the same way in a resources block. Kernpool matches by type and count and falls back through alternatives you allow.

accelerators: H100:8        # exactaccelerators: {H100:8, A100:8}  # eitheraccelerators: TPU-v5e-8

Data

Object storage buckets

Mount a bucket into the job filesystem, or sync it up front. S3, GCS, Azure Blob, Cloudflare R2, and VastData are supported, and a bucket in one provider can be read by a job running in another.

Volumes

Persistent volumes survive between jobs on the same cluster. Use them for datasets that are expensive to re-download and for checkpoint directories during a long run.

Syncing code, Git, and files

Your working directory is synced to the cluster at launch, respecting gitignore rules. You can also have Kernpool clone a specific commit so a run is reproducible from the record alone.

User guides

Recipes

Short, copyable configurations for common situations: burst to a second provider when the primary is full, pin a job to a reservation, or run the same evaluation across several model versions.

Migrating from Slurm

Most teams migrate incrementally: point Kernpool at the existing Slurm cluster, move new jobs over first, and retire the old submission scripts once nothing depends on them.

Asynchronous execution

Launch and return immediately instead of holding the terminal. Job state is queryable afterwards, which is what you want from CI or from an agent submitting work on your behalf.

Environment variables and secrets

Environment variables are declared in the job file. Secrets are referenced by name and resolved at launch from your secret store, so they never appear in the YAML or in logs.

Docker containers

Run inside your own image instead of letting Kernpool build the environment. Private registries are supported with the credentials already configured on the cluster.

Opening ports

Expose a port from a job to reach a training dashboard, a notebook, or a temporary service. Ports are closed by default and opened only for the jobs that declare them.

Lifecycle hooks

Run commands at defined points: after provisioning, before the job starts, and on teardown. Useful for mounting internal storage or registering a run with your own tracking system.

Frequently asked questions

Common questions about cost attribution, what happens during a provider outage, how Kernpool behaves when every provider is out of capacity, and how to reproduce a run months later.

Administrator guides

API server deployment

The API server is the shared control plane for a team. Deploy it with the Helm chart, back it with a managed database, and run it replicated for high availability.

$ helm repo add kernpool https://charts.kernpool.com$ helm install kernpool kernpool/api-server \ --set ha.enabled=true --set postgres.external=true

Authentication and RBAC

Users sign in through your identity provider. Roles control who can launch on which infrastructure, who can change quotas, and who can see cost data.

Workspaces: isolating teams

A workspace scopes clusters, quotas, and secrets to one team. Teams see their own resources only, while the fleet stays a single pool underneath.

Cloud accounts and permissions

Per-provider guidance on the minimum permissions Kernpool needs, and how to attach a dedicated role or service account instead of using an individual's credentials.

Admin policies

Policies are enforced at submission time: block regions, require a maximum spend per job, force autostop, or restrict which images may run. A rejected job explains which rule it violated.

External logging storage

Ship job logs to your own storage or observability stack so they outlive the cluster that produced them and stay within your retention rules.

Airgapped environments

Kernpool can run with no outbound internet access, using a mirrored package index and an internal container registry. Cluster and provider metadata are supplied from a local file.

Reference

Kernpool YAML

The full task specification: resources, node count, priority, setup and run commands, file mounts, storage, environment, and service configuration. Every field is optional.

CLI

Command reference for launching, inspecting, and cleaning up.

$ kernpool launch    # provision and run$ kernpool queue     # what is running and waiting$ kernpool logs      # stream output$ kernpool down      # tear down

Python SDK

The same operations as the CLI, callable from Python. Use it when jobs are submitted by another system rather than by a person at a terminal.

Advanced configuration

Global and per-project settings: which providers are allowed, default VPCs and subnets, instance tags for cost attribution, and controller sizing.

Internals

How the optimizer builds a search space, how provisioning and failover are sequenced, and what state the control plane keeps. Read this before extending Kernpool.

Contributing

How to set up a development environment, run the test suite, and propose a change. Adding a new provider follows a documented path.