Initializing Operalta...

Operalta

CLI with Local Models

Use OpenAI-compatible local model servers while keeping tool boundaries clear

Audience
Developer
Truth
Mixed sources
Surface
CLI
Status
Active

Related documentation: Developer documentation

Local model mode removes the cloud LLM dependency for CLI orchestration. It does not make every tool offline: tools that read or write Operalta, integrations, or web resources still call their respective APIs.

What becomes local

  • The model reasoning loop can run against an OpenAI-compatible endpoint on your machine.
  • The model can decide which CLI tools to call, generate tool arguments, read tool results, and continue the turn without calling Anthropic.
  • Local-only tools can stay fully local when they only touch your machine: file reads, file edits, grep/glob search, and shell commands, subject to your CLI approval mode.

What is not offline

  • Operalta business tools still call Operalta APIs when they need company context, initiatives, pipeline records, metrics, artifacts, rooms, or account data.
  • Integration-backed tools still call their providers. Google Drive, Composio, GitHub, web search, Stripe, and similar services are not made local by switching the model.
  • MCP runtime tools supplied by the host environment are separate from the CLI model provider. Changing the CLI model does not replace host-provided memory, filesystem, connector, or approval behavior.
  • The useful mental model is: local brain, tool-specific data rails. The brain can be local; the data rail may still be cloud or networked.

Configure a local model

  • OPERALTA_LLM_PROVIDER=openai-compatible selects the OpenAI-compatible provider.
  • OPERALTA_LLM_BASE_URL points to the local server. Common defaults are Ollama http://localhost:11434/v1, LM Studio http://localhost:1234/v1, and vLLM http://localhost:8000/v1.
  • OPERALTA_LLM_MODEL must match the model name exposed by your local server.
  • OPERALTA_LLM_API_KEY is optional for local servers that do not require authentication.
OPERALTA_LLM_PROVIDER=openai-compatible \
OPERALTA_LLM_BASE_URL=http://localhost:11434/v1 \
OPERALTA_LLM_MODEL=qwen2.5:7b \
operalta

Examples

  • Read README.md and suggest a patch can be fully local if the CLI only reads or edits local files.
  • List my initiatives uses the local model for orchestration, then calls an Operalta API tool with your OPERALTA_API_KEY to fetch the data.
  • Sync this folder to a room is not a model-tool workflow today; use explicit Transporter or room upload operations for tenant-bound persistence.

When to use it

  • Use local model mode when you want to avoid cloud LLM calls for terminal reasoning, codebase exploration, or local document work.
  • Use direct provider keys such as Anthropic, Mistral, or Bedrock when you need stronger tool-call reliability, multimodal support, or larger context behavior than your local model provides.
  • For sensitive work, combine local model mode with restrictive CLI approval modes and path boundaries. Local model mode is not a substitute for reviewing tool permissions.