Deploy through kollama CLI

We have a CLI called kollama here to simplify the deployment process. It is a simple way to deploy Ollama models to your Kubernetes cluster.

Getting Started

1. Install the CLI:

go install github.com/nekomeowww/ollama-operator/cmd/kollama@latest

To learn about the supported commands, please refer to kollama.

2. Deploy a model:

kollama deploy phi --expose --node-port 30101

For more information about the deploy command, please refer to kollama deploy.

That's it.

3. Interact with the model:

OLLAMA_HOST=<Node ip>:30101 ollama run phi

or use the OpenAI API compatible endpoint:

curl http://<Node ip>:30101/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "phi",
  "messages": [
      {
          "role": "user",
          "content": "Hello!"
      }
  ]
}'

Contributors

Neko Ayaka

Changelog

Last edited 3 months agoView full history

v0.5.1

9a6a3-fix: incorrect labels

v0.4.0

466e5-docs: added docs about kollama cli