Documentation    npm    MIT License

@neongate-ai/orbz is a framework-agnostic, SSR-safe Web Component for AI voice interfaces. It gives a voice experience a visible identity through expressive motion, conversation states, configurable palettes, and provider-neutral voice APIs without requiring a framework-specific UI package.

Orbz renders the native <orb-z> element. Your application keeps ownership of the persona, transcript UI, authentication, backend, memory, tools, quotas, and product logic.

npm install @neongate-ai/orbz

Or:

pnpm add @neongate-ai/orbz

Register <orb-z> from browser-only code:

import '@neongate-ai/orbz/browser'

Then use it as a native element:

<orb-z
  role="img"
  aria-label="Voice assistant"
  preset="neongate"
  state="idle"
></orb-z>

For typed JavaScript access:

import type { OrbzElement } from '@neongate-ai/orbz'
import '@neongate-ai/orbz/browser'

const orb = document.querySelector<OrbzElement>('orb-z')!

orb.state = 'listening'
orb.size = '18rem'
orb.speed = 1.2

Do not combine an explicit preset with custom color attributes. When a preset is selected, the preset wins and custom color attributes are ignored.

The element exposes the same presentation controls plus the voice and conversation APIs that should not be serialized into HTML attributes.

Orbz keeps visual presence and voice transport separate from your application's persona and backend. You can use the built-in browser/TTS/Realtime integrations or provide your own voice engine.

Selecting a model is silent. Orbz does not automatically start playback or open a microphone session when voiceModel, speech, or voiceEngine changes.

import {
  type OrbzElement,
  WebSpeechAdapter
} from '@neongate-ai/orbz'
import '@neongate-ai/orbz/browser'

const orb = document.querySelector<OrbzElement>('orb-z')!

orb.speech = 'Welcome. How can I help?'
orb.voiceEngine = new WebSpeechAdapter({
  language: 'en-US',
  rate: 1,
  pitch: 1,
  volume: 1,
  preferredVoices: ['Google US English']
})

await orb.startTalking()

WebSpeechAdapter accepts:

Language selection controls voice matching and pronunciation. Orbz does not translate the supplied text.

Use an application-owned endpoint so the permanent OpenAI API key never reaches the browser:

import type { OrbzElement } from '@neongate-ai/orbz'
import '@neongate-ai/orbz/browser'

const orb = document.querySelector<OrbzElement>('orb-z')!

orb.speech = 'Your order is ready.'
orb.voiceModel = {
  provider: 'openai-speech',
  endpoint: '/api/voice/speech',
  model: 'gpt-4o-mini-tts',
  voice: 'marin',
  responseFormat: 'mp3'
}

await orb.startTalking()

The application endpoint returns audio. Public openai-speech options are:

The lower-level OpenAISpeechAdapter also supports application-endpoint fetch options and instructions. Do not use its headers option to expose a permanent provider key to browser code.

import type { OrbzElement } from '@neongate-ai/orbz'
import '@neongate-ai/orbz/browser'

const orb = document.querySelector<OrbzElement>('orb-z')!

orb.voiceModel = {
  provider: 'openai-realtime',
  model: 'gpt-realtime-2',
  voice: 'marin'
}

orb.realtimeSession = {
  endpoint: '/api/voice/session',
  credentials: 'same-origin'
}

await orb.startConversation()

Public Realtime model options are:

realtimeSession can be either an application endpoint object or an async authorizer callback.

Endpoint form:

orb.realtimeSession = {
  endpoint: '/api/voice/session',
  credentials: 'same-origin'
}

Callback form:

orb.realtimeSession = async ({ sdp, model, voice, signal }) => {
  const response = await fetch('/api/voice/session', {
    method: 'POST',
    body: JSON.stringify({ sdp, model, voice }),
    headers: { 'content-type': 'application/json' },
    signal
  })

  if (!response.ok) throw new Error('Unable to create voice session')
  return response.text()
}

For the endpoint form, Orbz posts JSON containing { sdp, model, voice } and expects the SDP answer as text. Your server authenticates the user, enforces the models/voices your product allows, talks to the provider with its server-side credential, and returns the answer.

After setup, microphone and assistant audio use the provider's Realtime browser transport. Your server remains responsible for session policy, instructions, tools, quotas, billing rules, sideband connections, and remote cleanup.

If you already have a speech provider or your own model gateway, implement the small OrbzVoiceEnginePort contract:

import type { OrbzVoiceEnginePort } from '@neongate-ai/orbz'

class MyVoiceEngine implements OrbzVoiceEnginePort {
  async speak(text: string): Promise<void> {
    // Send text to your provider and play the resulting audio.
  }

  stop(): void {
    // Cancel the active request/playback.
  }
}

orb.voiceEngine = new MyVoiceEngine()
orb.speech = 'Hello from my own voice stack.'
await orb.startTalking()

An explicitly assigned voiceEngine takes precedence over voiceModel. Set orb.voiceEngine = undefined to return control to the selected voiceModel. Starting output-only speech stops a live Realtime conversation, and starting a Realtime conversation stops output-only speech.

Orbz ships without a persona, greeting, fallback copy, or hidden conversation script. If your UI needs a small explicit talk flow, provide it yourself:

orb.talkFlow = [
  {
    id: 'welcome',
    kind: 'say',
    needsAuth: false,
    text: 'Hello. What is your name?'
  },
  {
    id: 'name',
    kind: 'ask',
    needsAuth: false,
    capture: 'fullName',
    text: 'I am listening.'
  }
]

await orb.startTalking()
await orb.receive('Ana')

Applications can also supply an intelligence object whose respond(input, context) method returns application-generated text. This is an integration boundary, not an embedded Orbz LLM or credential store.

Visual state controls the orb animation:

<orb-z state="thinking"></orb-z>

When Orbz speaks through its talk API, it temporarily switches to speaking and restores the previous visual state when speech completes or is cancelled.

Live Realtime sessions also expose the read-only conversationState property: idle, connecting, listening, thinking, speaking, or error.

The default preset is NeonGate (neongate).

<orb-z preset="peach" state="listening"></orb-z>

Or supply your own palette:

<orb-z
  color-primary="#4F46E5"
  color-secondary="#22D3EE"
  color-accent="#F472B6"
  color-highlight="#FEF3C7"
  color-background="#0F172A"
></orb-z>

Presentation controls can be combined independently:

<orb-z
  size="18rem"
  speed="1.2"
  reduced-motion="system"
  elevated
></orb-z>

Use play(), pause(), and restart() for imperative motion control.

Orbz dispatches native CustomEvent instances from the host element.

orb.addEventListener('orbz-transcript', (event) => {
  const { role, text, final } = (
    event as CustomEvent<{
      role: 'user' | 'assistant'
      text: string
      final: boolean
    }>
  ).detail

  console.log({ role, text, final })
})

Orbz does not store transcript history, conversation memory, or audio history. Your application decides whether and where those events are persisted.

Orbz remains a native custom element. React projects can add JSX typing without using a wrapper component:

import '@neongate-ai/orbz/react-types'
import '@neongate-ai/orbz/browser'

Then:

<orb-z
  state="listening"
  preset="neongate"
  size="18rem"
  reduced-motion="system"
  aria-label="Voice assistant"
/>

voiceModel and realtimeSession are also typed properties for React usage. className is intentionally excluded because a host class cannot style the closed shadow tree. Use Orbz appearance APIs for the component itself and wrap the element when you need page-layout styling.

The core package is safe to import when HTMLElement and customElements are not available:

import type { OrbzElement } from '@neongate-ai/orbz'

Register the element only inside a browser/client boundary:

await import('@neongate-ai/orbz/browser')

If you prefer explicit registration instead of the browser side-effect entry:

import { defineOrbz } from '@neongate-ai/orbz'

defineOrbz()

defineOrbz() defines <orb-z> once and safely returns without registering in a non-browser environment.

Orbz accepts public model configuration and an application authorization boundary. It is not a credential vault.

Keep permanent provider API keys on your server. A JavaScript property is not an HTML attribute, but anything delivered to browser memory can still be read by compromised client code.

For Realtime, the endpoint form accepts only the public endpoint, Fetch cookie policy, and an optional consumer-owned fetch implementation. The application server authenticates and authorizes the user and uses its own provider key.

For application-hosted TTS, send requests to your own endpoint and let that endpoint call the provider. Do not embed permanent provider keys in URLs, attributes, model objects, headers shipped to the client, or custom-element properties.

The animated shadow content is visual and hidden from assistive technology. The host element gets its meaning from your application.

• For a meaningful visual identity, provide an appropriate role and accessible name.
• For a decorative orb, hide the host from assistive technology.
• Keep spoken content available as visible text, captions, or a transcript.
• Announce listening, thinking, speaking, and errors through application-owned status text or live regions.
• Keep reduced-motion="system" unless your product has an explicit user preference.
• Do not use animation or palette changes as the only way to communicate meaning.

The README is intentionally focused on using Orbz in an application. Deeper integration guides and API documentation are available at orbz.site.

Read the documentation →

MIT © gojhonny