Building Big Mama with ElevenLabs: Voice Persona as Product Design

Hey guys, I’m Chris Watkins, also known as Bingo Codes. I’m a security engineer transitioning into voice-first AI engineering while building Djembe AI and Big Mama — a culturally grounded voice-first agentic AI platform designed to help Black communities discover businesses, preserve culture, and help SMBs grow through intelligent AI systems.

The voice of Big Mama is not just audio output. It is part of the product’s trust layer. If the voice feels cold, fake, or disrespectful, the product fails before the agent even gets useful. Today I’m looking at how ElevenLabs can help me prototype the voice layer while thinking carefully about consent, culture, and trust.

When you are building a product that relies on voice interaction, the stakes are incredibly high. Voice is intimate. It is personal. It carries cultural markers, emotional weight, and subtle cues that tell the listener whether they are in a safe space or a transactional one. For Big Mama, the goal is to create an environment where users feel understood and supported. This means the voice we choose, or design, must reflect the community it serves. It cannot be an afterthought. It must be a deliberate design choice, grounded in respect and utility.

What Text-to-Speech Does

Text-to-speech, or TTS, converts written text into spoken audio. In a voice-agent pipeline, the model may generate a text response, and the TTS layer turns that response into voice output. In a realtime system, this needs to happen quickly enough to feel conversational.

ElevenLabs describes its platform as AI voice infrastructure that includes text-to-speech, speech-to-text, voice cloning, conversational agents, and generative audio through APIs and official Python and TypeScript SDKs.1

For Big Mama, TTS is where the reasoning layer becomes something the user can hear. It is the bridge between the complex algorithms running in the background and the human experience happening in the foreground. When a user asks Big Mama for a recommendation for a local Black-owned restaurant, the reasoning engine processes the request, searches the database, and formulates a response. But it is the TTS layer that delivers that response. If the delivery is robotic, hesitant, or culturally tone-deaf, the value of the recommendation is diminished. The user might get the information they need, but they won’t feel the connection that Big Mama is designed to foster.

This is why selecting the right TTS infrastructure is so critical. It is not just about finding a system that can read text aloud. It is about finding a system that can convey nuance, emotion, and cultural context. ElevenLabs offers a robust set of tools for this, allowing developers to fine-tune voices, adjust pacing, and even clone voices with permission. But with these powerful tools comes a significant responsibility to use them ethically and effectively.

Voice Persona Is Product Design

A voice persona includes more than accent or pitch. It includes pacing, warmth, clarity, emotional range, pronunciation, and how the assistant handles uncertainty. Big Mama’s voice should feel supportive and grounded without becoming a stereotype.

“Culturally grounded does not mean caricature. It means respectful, useful, familiar, and built with care.”

This is where community feedback matters. A voice that sounds good in a lab may not feel right to real users. I plan to test multiple voice directions with trusted listeners before treating any voice as final.

Designing a voice persona is akin to casting an actor for a crucial role in a film. The actor must embody the character’s traits, understand their motivations, and deliver their lines with authenticity. For Big Mama, the “character” is a knowledgeable, supportive, and culturally attuned assistant. The voice must convey authority without being authoritarian, warmth without being overly familiar, and competence without being clinical.

Consider how Big Mama might handle a situation where it doesn’t know the answer to a user’s question. A poorly designed voice persona might respond with a blunt, robotic “I do not understand.” A well-designed persona, on the other hand, might say, “I’m not sure about that, but let me see what I can find out for you,” delivered with a tone of helpful curiosity. The difference in user experience is profound. One response shuts the user down; the other invites them to continue the interaction.

Furthermore, pronunciation is a critical component of a culturally grounded voice persona. Big Mama needs to be able to correctly pronounce the names of local businesses, cultural figures, and community landmarks. A voice that stumbles over these names will quickly lose credibility. This requires careful testing and tuning, ensuring that the TTS engine is equipped with the necessary phonetic knowledge to handle diverse linguistic inputs.

Choosing a Voice Strategy

There are three broad approaches to Big Mama’s voice.

Strategy	Description	Advantage	Risk
Stock voice	Use a high-quality existing voice from a provider.	Fastest path to prototype.	May feel generic or off-brand.
Designed synthetic voice	Create or tune a voice persona around brand needs.	More distinctive and intentional.	Requires careful testing and iteration.
Voice clone	Clone a real person’s voice with permission.	Familiarity and uniqueness.	Consent, legal, ethical, and misuse concerns.

For early prototypes, I should prioritize quality, consent, and speed over chasing the perfect persona. The goal is to validate the conversation experience first.

Starting with a stock voice allows me to focus on the underlying architecture—the reasoning engine, the data retrieval, the real-time streaming—without getting bogged down in the complexities of voice design. Once the core functionality is solid, I can begin to experiment with designed synthetic voices or even voice cloning, always keeping the community’s needs and ethical considerations at the forefront.

The transition from a stock voice to a custom voice will be a significant milestone for Big Mama. It will mark the point where the product truly begins to embody its brand identity. But this transition must be handled carefully. A custom voice that misses the mark can be more damaging than a generic stock voice. It requires a deep understanding of the target audience, a clear vision for the brand, and a willingness to iterate based on feedback.

Voice cloning is powerful because it captures characteristics such as timbre, cadence, accent, and pronunciation and applies them to new speech generation. That power creates obvious ethical issues. A person’s voice is part of their identity.

“If a system can create speech that sounds like a real person, then consent, access control, disclosure, and abuse prevention are not optional features.”

Big Mama should not imitate real people without explicit consent. If synthetic voices are used, users should understand that they are speaking with AI. Voice assets should be protected. Internal tools should limit who can generate speech, what can be generated, and how logs are reviewed.

As a security engineer, I view voice cloning through the lens of threat modeling. What are the potential abuse cases? How could a bad actor misuse this technology? The risks are significant. Voice cloning could be used for social engineering, fraud, or harassment. It could be used to create deepfakes that damage a person’s reputation or spread misinformation.

To mitigate these risks, Big Mama must implement strict security controls around voice cloning. This includes robust authentication and authorization mechanisms to ensure that only authorized users can create or use voice clones. It also includes clear disclosure policies, ensuring that users are always aware when they are interacting with a synthetic voice. Furthermore, we must establish clear guidelines for what types of content can be generated using voice clones, prohibiting the creation of harmful or deceptive speech.

Latency and Model Choice

Voice quality and latency often involve tradeoffs. A highly expressive model may sound better but respond more slowly. A faster model may be better for real-time conversation. ElevenLabs documents models that differ by quality, latency, and language coverage, including fast models aimed at real-time use cases.1

“For Big Mama, the best voice is not only the prettiest voice. It is the voice that sounds good, responds fast, and works reliably in the actual product.”

This is a crucial engineering point. The prototype should measure first-audio latency, total response latency, audio quality, pronunciation accuracy, and user comfort.

In a real-time voice conversation, latency is the enemy of natural interaction. If there is a noticeable delay between the user’s input and the agent’s response, the conversation will feel stilted and awkward. Users will start talking over the agent, or they will assume the system has crashed. To avoid this, we must carefully balance voice quality with response speed.

This involves selecting the right TTS model, optimizing the network infrastructure, and fine-tuning the reasoning engine to generate responses as quickly as possible. It also involves implementing techniques like streaming audio, where the TTS engine begins playing the audio before the entire response has been generated. This can significantly reduce perceived latency and make the conversation feel more fluid.

Building the First Big Mama Voice Prototype

Here is what the first prototype might include. I do not need to overpromise a finished product. The goal is to describe the build path.

Prototype Step	Goal
Select two or three candidate voices.	Compare tone, warmth, clarity, and cultural fit.
Generate sample responses.	Test typical Big Mama use cases.
Measure response speed.	Understand latency tradeoffs.
Test pronunciation.	Check names, places, Black-owned business terms, and cultural references.
Gather feedback.	Ask trusted users which voice feels useful and respectful.
Log issues.	Build a voice QA list for iteration.

The prototyping phase is all about learning and iteration. It is an opportunity to test assumptions, identify bottlenecks, and gather feedback from real users. By starting small and focusing on the core conversation experience, we can build a solid foundation for Big Mama’s voice layer.

One of the most important steps in this process is testing pronunciation. Big Mama must be able to correctly pronounce the names of local businesses, cultural figures, and community landmarks. This requires creating a comprehensive test suite of challenging words and phrases, and evaluating how well the candidate voices handle them. If a voice struggles with certain pronunciations, we may need to provide phonetic spellings or custom dictionaries to improve its accuracy.

Security Engineer Lens

The voice layer introduces risk. Attackers could try prompt injection through spoken input. Users could request harmful synthetic speech. Private business data could be spoken aloud in the wrong context. Audio logs could contain sensitive information.

“Any time a system can listen and speak, I have to think about privacy, consent, storage, and misuse.”

Controls might include clear user consent, careful retention policies, abuse detection, content rules, redaction, voice asset permissions, and admin logging.

As a security engineer, my job is to anticipate how things might break or be broken. When you add a voice interface to an AI agent, you expand the attack surface significantly. Prompt injection, for example, becomes a much more complex problem when the input is spoken rather than typed. An attacker might try to use subtle variations in tone or pacing to trick the reasoning engine into executing unauthorized commands.

Furthermore, the TTS layer itself can be a target for abuse. If an attacker can gain access to the TTS API, they could generate harmful or deceptive speech using Big Mama’s voice. This could be used to spread misinformation, impersonate the brand, or harass users. To prevent this, we must implement strict access controls and monitor the TTS API for suspicious activity.

Big Mama Build Connection

Big Mama’s first voice should communicate three things: warmth, clarity, and competence. It should be welcoming enough for community discovery, professional enough for SMB workflows, and fast enough for real conversation.

“I want Big Mama to feel like help arrived, not like software started talking.”

This is the ultimate goal of the voice layer. It is not just about converting text to speech. It is about creating a connection with the user. It is about building trust. When a user interacts with Big Mama, they should feel like they are talking to a knowledgeable and supportive assistant who understands their needs and respects their culture.

Achieving this requires a deep understanding of the target audience, a clear vision for the brand, and a commitment to ethical and secure engineering practices. It requires careful selection of TTS infrastructure, thoughtful design of the voice persona, and rigorous testing and iteration. It is a complex and challenging process, but it is essential for building a voice-first AI product that truly serves its community.

Practical Takeaway

When building a voice-first AI product, treat the voice persona as a core product feature, not an afterthought. Start with a high-quality stock voice to validate the conversation flow, then iterate based on user feedback. Always prioritize consent, privacy, and security when handling voice data.

Remember that the voice is the primary interface between the user and the AI. It is the first thing they experience, and it sets the tone for the entire interaction. A well-designed voice persona can build trust, foster engagement, and enhance the overall user experience. A poorly designed voice persona can alienate users, damage the brand, and undermine the value of the product.

Closing

Next episode, I’m moving from the voice itself to the real-time infrastructure. We are going to look at LiveKit and how voice gets streamed between a user, an AI agent, and the application in real time.

If you are building in AI, security, voice infrastructure, or community-centered technology, follow along. This series is my public proof-of-work as I learn, build, and ship Djembe AI and Big Mama in public. Drop a comment with what you want me to build or explain next, and I’ll see you in the next episode.