SIP • OPENAI REALTIME • C#

Build an On-Premises SIP-to-OpenAI Voice Gateway in C#

Use the VaxVoIP Agent AI SDK to build an on-premises C# gateway that runs on Windows and connects your SIP softphone or PBX to OpenAI Realtime voice models. The SDK handles the connection, so you can get started without setting up webhooks or managing WebSocket connections.

The walkthrough uses the demo application included with the C# sample.

How the gateway connects SIP calls to OpenAI

SIP softphone or PBX ↔ C# gateway (built with the VaxAgent SDK) ↔ OpenAI Realtime voice model

The phone system sends SIP calls to the gateway on your network. The VaxVoIP SDK uses WebRTC to exchange audio with the selected voice model. Callers speak and hear the model’s replies on the same SIP call.

Your C# application decides which calls to accept, sets the voice model’s instructions, and makes application functions available for data lookups.

Set up the C# sample and place a test call

You’ll need a Windows PC with Visual Studio, the VaxVoIP Agent AI SDK for Windows (a 30-day trial is available), an OpenAI API key, and a SIP softphone or PBX.

  1. Download the C# sample project, extract it, and follow ReadMe.txt for SDK and project setup. Watch the demo video included in the ZIP file, then open the project in Visual Studio.
  2. Configure the demo application with your OpenAI API key. Keep the key out of source control by loading it from an environment variable or secret store.
  3. Run the sample. Register your softphone with the gateway using the SIP settings in the sample. If you’re testing through a PBX, configure a test extension to route calls to the gateway.
  4. Dial the extension configured for the demo and ask whether laptops are available. Listen for the spoken reply.

The incoming-call handler accepts the call after the application configures the voice model, prompt, and tools.

base.AcceptCallSession(30, KeyOpenAI, ModelPrompt, Model, "coral", 1.0);

When the model requests a product lookup, the application checks the catalog and returns the result for the spoken reply.

Watch the C# gateway demo for a walkthrough.

What you can build

  • AI receptionist: Look up departments by name and route callers to the right team.
  • Appointment booking agent: Check open slots and confirm a booking on the call.
  • Order-taking agent: Extend the product lookup to collect quantities and confirm the order details.
  • Internal help desk: Look up support hours or let callers check the status of an existing ticket.