Getting Started

Install operator

Don't have an existing Kubernetes cluster?

Run the following commands to create a new Kubernetes cluster with kind:

macOS

brew install --cask docker
brew install docker kind kubectl
wget https://raw.githubusercontent.com/nekomeowww/ollama-operator/main/hack/kind-config.yaml
kind create cluster --config kind-config.yaml

Windows

Invoke-WebRequest  -OutFile "./Docker Desktop Installer.exe"
Start-Process 'Docker Desktop Installer.exe' -Wait install
start /w "" "Docker Desktop Installer.exe" install

# If you use Scoop command line installer
scoop install docker kubectl go
# Alternatively, if you use Chocolatey as package manager
choco install docker-desktop kubernetes-cli golang

go install sigs.k8s.io/kind@latest
wget https://raw.githubusercontent.com/nekomeowww/ollama-operator/main/hack/kind-config.yaml
kind create cluster --config kind-config.yaml

Linux

# refer to Install Docker Engine on Debian | Docker Docs https://docs.docker.com/engine/install/debian/
# and Install and Set Up kubectl on Linux | Kubernetes https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/

1. Install operator.

kubectl apply \
  --server-side=true \
  -f https://raw.githubusercontent.com/nekomeowww/ollama-operator/v0.10.1/dist/install.yaml

2. Wait for the operator to be ready:

kubectl wait \
   -n ollama-operator-system \
  --for=jsonpath='{.status.readyReplicas}'=1 deployment/ollama-operator-controller-manager

Deploy the model

There are two major ways to get started with the Ollama Operator:

Deploy models through Kollama CLI

Dislike YAML?

Faster and better UX?

No worries, kollama here to rescue!

          # as regular binary cli
          $ kollama deploy phi
        

or

          # as kubectl plugin
          $ kubectl ollama deploy phi
        

Deploy models through CRD

Fine-grained control over parameters?

GitOps and CI/CD?

CRD is simple enough with 6 lines!

          apiVersion: ollama.ayaka.io/v1
          kind: Model
          metadata:
            name: phi
          spec:
            image: phi
        

1. kollama provides a simple way to deploy the Ollama model CRD to your Kubernetes cluster.
2. General Kubernetes CRD is available for advanced users who want to customize the Ollama model CRD.

Different ways provides different levels of customization and flexibility. Choose the one that best fits your needs.

Contributors

Neko Ayaka

Changelog

Last edited 9 months ago

View full history

b871251-docs: updated docs

466e51c-docs: added docs about kollama cli