Install with Helm
Install LLMariner with Helm.
Installation
Choose the guide that best suits your needs and platform.
LLMariner takes ControlPlane-Worker model. The control plane gets a request and gives instructions to the worker while the worker processes a task such as inference.
Both components can operate within a single cluster, but if you want to utilize GPU resources across multiple clusters, they can also be installed into separate clusters.
Set up a Playground on a GPU EC2 Instance
Set up the playground environment on an Amazon EC2 instance with GPUs.
Set up a Playground on a CPU-only Kind Cluster
Set up the playground environment on a local kind cluster (CPU-only).
Install in a Single EKS Cluster
Install LLMariner in an EKS cluster with the standalone mode.
Install in a Single On-premise Cluster
Install LLMariner in an on-premise Kubernetes cluster with the standalone mode.
Install across Multiple Clusters
Install LLMarinr across multiple Kubernetes clusters.
Hosted Control Plane
Install just the worker plane and use it with the hosted control plane.