kollama expose

kolamma expose is a command to expose a Model as a service.

• Use cases
  • Expose a Model service
  • Expose a Model service in a specific namespace
  • Expose the service as a LoadBalancer type
• Flags
  • --namespace
  • --service-type
  • --service-type
  • --service-name
  • --node-port

Use cases

Expose a Model service

kollama expose phi

Expose a Model service in a specific namespace

kollama expose phi --namespace=production

Expose the service as a LoadBalancer type

kollama expose phi --service-type=LoadBalancer

Flags

--namespace

If present, the namespace scope for this CLI request.

--service-type

kollama deploy phi --expose --service-type=NodePort

Default: NodePort

Type of the Service to expose the model. Only valid when --expose is specified.

If not specified, the service will be exposed as NodePort.

To understand how many Services are associated to models...

kubectl get svc --selector ollama.ayaka.io/type=model

Use LoadBalancer to expose the service as LoadBalancer.

--service-type

kollama deploy phi --expose --service-type=NodePort

Default: NodePort

Type of the Service to expose the Model. Only valid when --expose is specified.

If not specified, the service will be exposed as NodePort.

To understand how many Services are associated to Model...

kubectl get svc --selector ollama.ayaka.io/type=model

Use LoadBalancer to expose the service as LoadBalancer.

--service-name

kollama deploy phi --expose --service-name=phi-svc-nodeport

Default: ollama-model-<model name>-<service type>

Name of the Service to expose the Model.

If not specified, the Model name will be used as the service name with -nodeport as the suffix for NodePort.

--node-port

kollama deploy phi --expose --service-type=NodePort --node-port=30000

Default: Random port

To understand what NodePort is used for the Model...

kubectl get svc --selector model.ollama.ayaka.io/name=<model name> -o json | jq ".spec.ports[0].nodePort"

You can't simply specify a port number!

There are several restrictions:

1. By default, 30000-32767 is the NodePort port range in the Kubernetes cluster. If you want to use ports outside this range, you need to configure the --service-node-port-range parameter for the cluster.
2. You can't use the port number already occupied by other services.

For more information about choosing your own port number, please refer to Chapter of Kubernetes Official Document about nodePort.

nodePort to expose the Model.

If not specified, a random port will be assigned. Only valid when --expose is specified, and --service-type is set to NodePort.

Contributors

Neko Ayaka

Changelog

Last edited over 1 year ago

View full history

b541d40-fix(ollama-operator): align labels