How to Build a Real-Time AI Voice Chat Room with MCP and LiveKit

In a previous post, we discussed how to build an MCP-powered voice agent with real-time capabilities. This allowed you to use your own data, applications, and services to create an AI assistant for yourself or your customers, all using LiveKit in a local environment with OpenAI and Deepgram.

In this tutorial, we'll take it a step further. We will create a virtual chat room where you and an AI agent can join and have a live conversation.

By the end of this guide, you should have a chat room open in your browser where you can talk with an agent. If you equip it with the right APIs and data, your agent could:

Do research for you
Check your emails
Organize your day
Or simply have a friendly chat
...and much more.

Let’s get started.

Prerequisites

What we need to accomplish this is almost the same as in the last tutorial. The main difference is that this time, we will use the Cloud Livekit environment.

Deepgram: You will need a Deepgram API key.
OpenAI: You will need an OpenAI API key.
LiveKit Cloud: You will need a LiveKit Cloud account to get your LIVEKIT_URL, LIVEKIT_API_KEY, and LIVEKIT_API_SECRET.
(Optional) Smithery.ai: To use a pre-built MCP server, you can get a Smithery API key.

Overview of the Setup

The setup process might seem complex, but it's quite straightforward and very similar to the previous one, with just a few additions. Here's the plan:

Create a playground environment in the LiveKit Cloud with a few clicks.
Prepare the agent script (you can copy and paste the code below).
Connect the agent to the playground.