Direct voice audio access
How to access voice audio.
Read time 1 minuteLast updated 8 months ago
Direct access to the audio buffers on the capture client or render client allows developers to:
- Add an audio effect that modifies voice (for example, a robot voice, radio distortion, or static)
- Use a third-party audio engine (such as Wwise) for voice render instead of an automatic render provided by Vivox
- Record audio
- Perform phoneme analysis
Note
The Vivox SDK automatically handles the capture and render of voice data without requiring audio buffer access. Unless your implementation requires the situations detailed in the preceding list, you most likely do not need audio buffer access.
The Vivox Core SDK provides four hooks for optional callback functions that allow access to the audio buffers:
-
Capture side
- After capture, but before Vivox audio processing
- After Vivox audio processing, but before being sent to the Vivox server
-
Render side
- After being received from the Vivox server, but before audio processing and mixdown to a single audio stream
- After Vivox audio processing and mixdown, but before render
Note
Callbacks only occur when in a channel session and the audio data there is interleaved and in 16-bit integer format.
Vivox Core users have access to these hooks directly as displayed in the following sections.
Note
Accessing or modifying the audio buffers incorrectly can have a substantial negative impact on the user experience. The callbacks covered in these topics occur on Vivox’s real-time audio thread so take precautions when using them. Limit memory allocation on this thread to avoid negatively affecting the audio quality, especially on low-end devices.
Topic | Description |
|---|---|
| Callback APIs | How to set the callback pointers. |
| Callback usage | Detailed information on each callback and it's usage. |
| Read and alter audio data for each participant | Managing individual participant audio data. |