我使用Java Sound API开发了一个Java应用程序。它的作用是捕获来自麦克风的数据，并通过UDP将其发送到其他计算机，以便在那里播放。现在，我遇到了音量，质量和速度问题。我无法找出问题的根源，所以我需要帮助来找出程序的问题。
<小时 />更新
速度问题似乎是由于Java Sound API很慢。我尝试了没有UDP套接字的程序，并且有同样的延迟，因此UDP不会在LAN中引入额外的延迟。当程序与耳机一起使用时，回声问题消失了。整体声音质量还不错。唯一剩下的问题是音量。
以下是发件人：
import javax.sound.sampled.*;
import java.io.ByteArrayOutputStream;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
public class VoipApp
{
public static void main(String[]args) throws Exception
{
AudioFormat.Encoding encoding = AudioFormat.Encoding.PCM_SIGNED;
float rate = 44100.0f;
int sampleSize = 16;
int channels = 2;
int frameSize = 4;
boolean bigEndian = true;
AudioFormat format = new AudioFormat(encoding, rate, sampleSize, channels, (sampleSize / 8)
* channels, rate, bigEndian);
TargetDataLine line;
DataLine.Info info = new DataLine.Info(TargetDataLine.class, format);
if(!AudioSystem.isLineSupported(info)){
System.out.println("Not Supported");
System.exit(1);
}
DatagramSocket socket = new DatagramSocket(8081);
//InetAddress IPAddress = InetAddress.getLocalHost();
InetAddress IPAddress = InetAddress.getByName("192.168.0.14");
try
{
line = (TargetDataLine) AudioSystem.getLine(info);
line.open(format);
//ByteArrayOutputStream out = new ByteArrayOutputStream();
int numBytesRead;
byte[] data = new byte [line.getBufferSize() / 5];
int totalBytesRead = 0;
line.start();
while(true){
numBytesRead = line.read(data,0, data.length);
DatagramPacket sendPacket = new DatagramPacket(data, data.length, IPAddress, 8080);
// totalBytesRead += numBytesRead;
socket.send(sendPacket);
//out.write(data, 0, numBytesRead);
// System.out.println("Debug");
}
}
catch(LineUnavailableException e)
{
e.printStackTrace();
}
}
}
下面是接收器：
import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.DataLine;
import javax.sound.sampled.SourceDataLine;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
public class VoipAppTwo
{
public static void main(String[]args) throws Exception
{
AudioFormat.Encoding encoding = AudioFormat.Encoding.PCM_SIGNED;
float rate = 44100.0f;
int sampleSize = 16;
int channels = 2;
int frameSize = 4;
boolean bigEndian = true;
AudioFormat format = new AudioFormat(encoding, rate, sampleSize, channels, (sampleSize / 8)
* channels, rate, bigEndian);
SourceDataLine speakers;
DataLine.Info dataLineInfo = new DataLine.Info(SourceDataLine.class, format);
speakers = (SourceDataLine) AudioSystem.getLine(dataLineInfo);
speakers.open(format);
DatagramSocket socket = new DatagramSocket(8080);
byte[] data = new byte[speakers.getBufferSize() / 5];
speakers.start();
while(true)
{
DatagramPacket receivePacket = new DatagramPacket(data, data.length);
socket.receive(receivePacket);
speakers.write(data, 0, data.length);
}
}
}
一些想法，需要注意的是，我没有尝试使用 UDP 的直接经验。
在我看来，必须"处理"丢失和无序的数据包(假设使用 UDP(，否则预期的不连续性将不断产生破坏性的响亮点击。但是IDK通常是如何完成的。过滤 器？缓冲？将数据包数据封装到窗口(Hann 或 Hamming？(帧中以桥接数据包不连续性？
javax.sound.sampled非常接近原生声音。这是一篇很好的文章，可以参考与基于 Java 的实时、低延迟音频相关的注意事项。
最后，我能够在这里回答我自己的问题。我使用ALSA API(Linux(，PulseAudio(Linux(和JackAudio(Linux(使程序几乎实时(小于1秒延迟(。PulseAudio 的阻塞 API 比 ALSA 和 JackAudio 好用得多，但我遇到了与 Java Sound API 相同的延迟。杰克音频 是用于专业音频工作的低延迟音频服务器。我看到这个例子：https://github.com/jackaudio/example-clients/blob/master/simple_client.c，很快意识到这是用于单声道输出的。 我稍微修改了代码以使其立体声，然后添加了UDP套接字和第二个回调函数。回调函数的工作方式就像它们是单独的线程一样，因此无需引入 POSIX 线程。
代码并不完美，可能不是使用 JackAudio 的好方法，并且其中包含大量错误。但是，我相信对于那些对这些东西感兴趣的人来说，这是一个很好的起点。要使用它，请首先通过发行版的包管理器安装JackAudio服务器和开发文件。
让我知道是否有更好的方法可以使用任何可用的工具集实现相同的功能。
杰克音频
杰克音频解决方案：
#include <stdio.h>
#include <errno.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <jack/jack.h>
#define PORT 8080
int create_UDP_socket_send(struct sockaddr_in * server, const char * ip)
{
int sock;
if((sock = socket(AF_INET, SOCK_DGRAM, 0)) < 0)
{
perror("Socket");
exit(EXIT_FAILURE);
}
memset(server, 0, sizeof(*server));
server->sin_family = AF_INET;
server->sin_port = htons(PORT);
server->sin_addr.s_addr = inet_addr(ip);
return sock;
}
int create_UDP_socket_receive()
{
struct sockaddr_in server;
int sock;
if((sock = socket(AF_INET, SOCK_DGRAM, 0)) < 0)
{
perror("Socket");
exit(EXIT_FAILURE);
}
memset(&server, 0, sizeof(server));
server.sin_family = AF_INET;
server.sin_port = htons(PORT);
server.sin_addr.s_addr = INADDR_ANY;
if (bind(sock, (struct sockaddr *)&server, sizeof(server)) < 0)
{
perror("bind()");
exit(EXIT_FAILURE);
}
return sock;
}
typedef struct{
int socket;
struct sockaddr_in server;
} __Info__;
jack_port_t *input_port;
jack_port_t *output_port1;
jack_port_t *output_port2;
jack_client_t *client1;
jack_client_t *client2;
int processFirst (jack_nframes_t nframes, void *arg)
{
jack_default_audio_sample_t *in, *out;
__Info__ * info = (__Info__ *) arg;
in = jack_port_get_buffer (input_port, nframes);
sendto(info->socket, in, sizeof(jack_default_audio_sample_t) * nframes, 0, (struct sockaddr *)&(info->server), sizeof(info->server));
return 0;
}
int processSecond (jack_nframes_t nframes, void *arg)
{
jack_default_audio_sample_t *out1, *out2;
int * socket = (int *)arg;
out1 = jack_port_get_buffer (output_port1, nframes);
out2 = jack_port_get_buffer (output_port2, nframes);
unsigned char buffer[sizeof(jack_default_audio_sample_t) * nframes];
recvfrom(*socket, buffer, sizeof(jack_default_audio_sample_t) * nframes, 0, NULL, NULL);
memcpy(out1, buffer, sizeof(jack_default_audio_sample_t) * nframes);
memcpy(out2, buffer, sizeof(jack_default_audio_sample_t) * nframes);
return 0;
}
void jack_shutdown (void *arg)
{
exit (1);
}
int main (int argc, char *argv[])
{
const char **ports;
const char *client_name1 = "simple1";
const char *client_name2 = "simple2";
const char *server_name = NULL;
jack_options_t options = JackNullOption;
jack_status_t status;
client1 = jack_client_open (client_name1, options, &status, server_name);
if (client1 == NULL) {
fprintf (stderr, "jack_client_open() failed, "
"status = 0x%2.0xn", status);
if (status & JackServerFailed) {
fprintf (stderr, "Unable to connect to JACK servern");
}
exit (1);
}
client2 = jack_client_open (client_name2, options, &status, server_name);
if (client2 == NULL) {
fprintf (stderr, "jack_client_open() failed, "
"status = 0x%2.0xn", status);
if (status & JackServerFailed) {
fprintf (stderr, "Unable to connect to JACK servern");
}
exit (1);
}
if (status & JackServerStarted) {
fprintf (stderr, "JACK server startedn");
}
if (status & JackNameNotUnique) {
client_name1 = jack_get_client_name(client1);
fprintf (stderr, "unique name `%s' assignedn", client_name1);
}
struct sockaddr_in server;
int socket = create_UDP_socket_send(&server, argv[1]);
__Info__ info = {
socket,
server
};
int socket2 = create_UDP_socket_receive();
jack_set_process_callback (client1, processFirst, &info);
jack_set_process_callback (client2, processSecond, &socket2);
jack_on_shutdown (client1, jack_shutdown, 0);
jack_on_shutdown (client2, jack_shutdown, 0);
printf ("engine sample rate: %" PRIu32 "n", jack_get_sample_rate (client1));
printf ("engine sample rate: %" PRIu32 "n", jack_get_sample_rate (client2));
input_port = jack_port_register (client1, "input", JACK_DEFAULT_AUDIO_TYPE, JackPortIsInput, 0);
output_port1 = jack_port_register (client2, "output1", JACK_DEFAULT_AUDIO_TYPE, JackPortIsOutput, 0);
output_port2 = jack_port_register (client2, "output2", JACK_DEFAULT_AUDIO_TYPE, JackPortIsOutput, 1);
if ((input_port == NULL) || (output_port1 == NULL) || (output_port2 == NULL)) {
fprintf(stderr, "no more JACK ports availablen");
exit (1);
}
if (jack_activate (client1)) {
fprintf (stderr, "cannot activate client");
exit (1);
}
if (jack_activate (client2)) {
fprintf (stderr, "cannot activate client2");
exit (1);
}
ports = jack_get_ports (client1, NULL, NULL, JackPortIsPhysical | JackPortIsOutput);
if (ports == NULL) {
fprintf(stderr, "no physical capture portsn");
exit (1);
}
if (jack_connect (client1, ports[0], jack_port_name (input_port))) {
fprintf (stderr, "cannot connect input portsn");
}
free (ports);
ports = jack_get_ports (client2, NULL, NULL, JackPortIsPhysical | JackPortIsInput);
if (ports == NULL) {
fprintf(stderr, "no physical playback portsn");
exit (1);
}
if (jack_connect (client2, jack_port_name (output_port1), ports[0])) {
fprintf (stderr, "cannot connect output portsn");
}
if (jack_connect (client2, jack_port_name (output_port2), ports[1])) {
fprintf (stderr, "cannot connect output portsn");
}
sleep (-1);
jack_client_close (client1);
jack_client_close (client2);
exit (0);
}
阿尔萨原料药
我使用了以下教程中的代码：https://www.linuxjournal.com/article/6735?page=0,2 有时，接收器端恰好存在欠载。
ALSA API 发送器：
#define ALSA_PCM_NEW_HW_PARAMS_API
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <errno.h>
#include <alsa/asoundlib.h>
#define PORT 8080
int create_UDP_socket_send(struct sockaddr_in * server, const char * address)
{
int sock;
if((sock = socket(AF_INET, SOCK_DGRAM, 0)) < 0)
{
perror("Socket");
exit(EXIT_FAILURE);
}
memset(server, 0, sizeof(*server));
server->sin_family = AF_INET;
server->sin_port = htons(PORT);
server->sin_addr.s_addr = inet_addr((const char *)address);
return sock;
}
int main() {
long loops;
int rc;
int size;
snd_pcm_t *handle;
snd_pcm_hw_params_t *params;
unsigned int val;
int dir;
snd_pcm_uframes_t frames;
//char *buffer;
/* Open PCM device for recording (capture). */
rc = snd_pcm_open(&handle, "default", SND_PCM_STREAM_CAPTURE, 0);
if (rc < 0) {
fprintf(stderr, "unable to open pcm device: %sn", snd_strerror(rc));
exit(1);
}
/* Allocate a hardware parameters object. */
snd_pcm_hw_params_alloca(¶ms);
/* Fill it in with default values. */
snd_pcm_hw_params_any(handle, params);
/* Set the desired hardware parameters. */
/* Interleaved mode */
snd_pcm_hw_params_set_access(handle, params, SND_PCM_ACCESS_RW_INTERLEAVED);
/* Signed 16-bit little-endian format */
snd_pcm_hw_params_set_format(handle, params, SND_PCM_FORMAT_S16_LE);
/* Two channels (stereo) */
snd_pcm_hw_params_set_channels(handle, params, 2);
/* 44100 bits/second sampling rate (CD quality) */
val = 44100;
snd_pcm_hw_params_set_rate_near(handle, params, &val, &dir);
/* Set period size to 32 frames. */
frames = 32;
snd_pcm_hw_params_set_period_size_near(handle, params, &frames, &dir);
/* Write the parameters to the driver */
rc = snd_pcm_hw_params(handle, params);
if (rc < 0) {
fprintf(stderr, "unable to set hw parameters: %sn", snd_strerror(rc));
exit(1);
}
/* Use a buffer large enough to hold one period */
snd_pcm_hw_params_get_period_size(params, &frames, &dir);
size = frames * 4; /* 2 bytes/sample, 2 channels */
// buffer = (char *) malloc(size);
char buffer[size];
int nsent;
struct sockaddr_in server;
int socket = create_UDP_socket_send(&server, "127.0.0.1");
snd_pcm_hw_params_get_period_time(params, &val, &dir);
while (1) {
rc = snd_pcm_readi(handle, buffer, frames);
if (rc == -EPIPE) {
/* EPIPE means overrun */
fprintf(stderr, "overrun occurredn");
snd_pcm_prepare(handle);
} else if (rc < 0) {
fprintf(stderr, "error from read: %sn", snd_strerror(rc));
} else if (rc != (int)frames) {
fprintf(stderr, "short read, read %d framesn", rc);
}
// rc = write(1, buffer, size);
if((nsent = sendto(socket, buffer, sizeof(buffer), 0, (struct sockaddr *)&server, sizeof(server))) < 0)
{
perror("sendto()");
exit(EXIT_FAILURE);
}
}
snd_pcm_drain(handle);
snd_pcm_close(handle);
return 0;
}
ALSA API 接收器：
#define ALSA_PCM_NEW_HW_PARAMS_API
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <errno.h>
#include <alsa/asoundlib.h>
#define PORT 8080
int create_UDP_socket_receive()
{
struct sockaddr_in server;
int sock;
if((sock = socket(AF_INET, SOCK_DGRAM, 0)) < 0)
{
perror("Socket");
exit(EXIT_FAILURE);
}
memset(&server, 0, sizeof(server));
server.sin_family = AF_INET;
server.sin_port = htons(PORT);
server.sin_addr.s_addr = INADDR_ANY;
if (bind(sock, (struct sockaddr *)&server, sizeof(server)) < 0)
{
perror("bind()");
exit(EXIT_FAILURE);
}
return sock;
}
int main() {
long loops;
int rc;
int size;
snd_pcm_t *handle;
snd_pcm_hw_params_t *params;
unsigned int val;
int dir;
snd_pcm_uframes_t frames;
/* Open PCM device for playback. */
rc = snd_pcm_open(&handle, "default", SND_PCM_STREAM_PLAYBACK, 0);
if (rc < 0) {
fprintf(stderr, "unable to open pcm device: %sn", snd_strerror(rc));
exit(1);
}
/* Allocate a hardware parameters object. */
snd_pcm_hw_params_alloca(¶ms);
/* Fill it in with default values. */
snd_pcm_hw_params_any(handle, params);
/* Set the desired hardware parameters. */
/* Interleaved mode */
snd_pcm_hw_params_set_access(handle, params, SND_PCM_ACCESS_RW_INTERLEAVED);
/* Signed 16-bit little-endian format */
snd_pcm_hw_params_set_format(handle, params, SND_PCM_FORMAT_S16_LE);
/* Two channels (stereo) */
snd_pcm_hw_params_set_channels(handle, params, 2);
/* 44100 bits/second sampling rate (CD quality) */
val = 44100;
snd_pcm_hw_params_set_rate_near(handle, params, &val, &dir);
/* Set period size to 32 frames. */
frames = 32;
snd_pcm_hw_params_set_period_size_near(handle, params, &frames, &dir);
/* Write the parameters to the driver */
rc = snd_pcm_hw_params(handle, params);
if (rc < 0) {
fprintf(stderr, "unable to set hw parameters: %sn", snd_strerror(rc));
exit(1);
}
/* Use a buffer large enough to hold one period */
snd_pcm_hw_params_get_period_size(params, &frames, &dir);
size = frames * 4; /* 2 bytes/sample, 2 channels */
char buffer[size];
int nread;
int socket = create_UDP_socket_receive();
snd_pcm_hw_params_get_period_time(params, &val, &dir);
while (1) {
if((nread = recvfrom(socket, buffer, sizeof(buffer), 0, NULL, NULL)) < 0)
{
perror("recvfrom()");
exit(EXIT_FAILURE);
}
// write(1, buffer, sizeof(buffer));
rc = snd_pcm_writei(handle, buffer, frames);
if (rc == -EPIPE) {
/* EPIPE means underrun */
fprintf(stderr, "underrun occurredn");
snd_pcm_prepare(handle);
} else if (rc < 0) {
fprintf(stderr, "error from writei: %sn", snd_strerror(rc));
} else if (rc != (int)frames) {
fprintf(stderr, "short write, write %d framesn", rc);
}
}
snd_pcm_drain(handle);
snd_pcm_close(handle);
free(buffer);
return 0;
}
网络语音是一个复杂的主题，有很多棘手的问题。
我认为您的问题来自网络层的不规则性。
如果您的网络不是 100% 恒定的(在您的计算机/盒子/等的缓冲区中受到其他应用程序的干扰非常困难。 你的声音会有点机器人。
避免这种情况的一种方法是添加一个小缓冲区，这将延迟接收方和发送方之间的通信。如果延迟太低，性能问题影响将继续发生。如果它太长了，那就太长了;)...
可能发生的另一个问题(最有可能在WAN网络中(是数据包丢失，可能会给通信带来麻烦。要解决此问题..我没有好的解决方案，这取决于网络或/和用例。