Ollama Operator

Large language models, scaled, deployed

Yet another operator for running large language models on Kubernetes with ease. Powered by Ollama! 🐫

Launch and chat

Easy to use API, the spec is simple enough to just a few lines of YAML to deploy a model, and you can chat with it right away.

Cross-kubernetes

Extend the user experience of Ollama to any Kubernetes cluster, edge or any cloud infrastructure, with the same spec, and chat with it from anywhere.

OpenAI API compatible

Your familiar /v1/chat/completions endpoint is here, with the same request and response format. No need to change your code or switch to another API.

Langchain ready

Power on to function calling, agents, knowldgebase retrieving. Unleash all the power Langchain has out of box with Ollama Operator.

Getting started

Deploy models through Kollama CLI

Dislike YAML?

Faster and better UX?

No worries, kollama here to rescue!

          # as regular binary cli
          $ kollama deploy phi

          # as kubectl plugin
          $ kubectl ollama deploy phi

Deploy models through CRD

Fine-grained control over parameters?

GitOps and CI/CD?

CRD is simple enough with 6 lines!

          apiVersion: ollama.ayaka.io/v1
          kind: Model
          metadata:
            name: phi
          spec:
            image: phi

Ollama Operator

Launch and chat

Cross-kubernetes

OpenAI API compatible

Langchain ready

Getting started ​

See it in action ​

Getting started

See it in action