位图创建中的优化

Question

我正在编写一个应用程序，该应用程序将实时从远程TCP连接实时收到的图片序列呈现为ImageView元素。 stream 由以PGM格式编码的单帧组成，并以9Hz发送，我想使用的是非常低的帧速率，应该使用将完全解码的位图发送到我的MainActivity的背景Service轻松处理。<<<<<<<<<<<<<<<<<<<<<<<</p>

这是我的VideoService(我认为这是唯一的兴趣之一，所以我只发布run()方法(：

    public void run() {
        InetAddress serverAddr = null;
        try {
            serverAddr = InetAddress.getByName(VIDEO_SERVER_ADDR);
        } catch (UnknownHostException e) {
            Log.e(getClass().getName(), e.getMessage());
            e.printStackTrace();
            return;
        }
        Socket socket = null;
        BufferedReader reader = null;
        do {
            try {
                socket = new Socket(serverAddr, VIDEO_SERVER_PORT);
                reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
                boolean frameStart = false;
                LinkedList<String> frameList = new LinkedList<>();
                while (keepRunning) {
                    final String message = reader.readLine();
                    if (!frameStart && message.startsWith("F"))
                        frameStart = true;
                    else if (frameStart && message.startsWith("EF")) {
                        frameStart = false;
                        final Bitmap bitmap = Bitmap.createBitmap(IR_FRAME_WIDTH, IR_FRAME_HEIGHT, Bitmap.Config.ARGB_8888);
                        final Canvas canvas = new Canvas(bitmap);
                        final String[] data = frameList.toArray(new String[frameList.size()]);
                        canvas.drawBitmap(bitmap, 0, 0, null);
                        //Log.d(this.getClass().getName(), "IR FRAME COLLECTED");
                        if ((data.length - 6) == IR_FRAME_HEIGHT) {
                            float grayScaleRatio = Float.parseFloat(data[2].trim()) / 255.0f;
                            for (int y = 0; y < IR_FRAME_HEIGHT; y++) {
                                final String line = data[y + 3];
                                final String[] points = line.split("\s+");
                                if (points.length == IR_FRAME_WIDTH) {
                                    for (int x = 0; x < IR_FRAME_WIDTH; x++) {
                                        final float grayLevel = Float.parseFloat(points[x]) / grayScaleRatio;
                                        Paint paint = new Paint();
                                        paint.setStyle(Paint.Style.FILL);
                                        final int level = (int)grayLevel;
                                        paint.setColor(Color.rgb(level, level, level));
                                        canvas.drawPoint(x, y, paint);
                                    }
                                } else
                                    Log.d(this.getClass().getName(), "Malformed line");
                            }
                            final Intent messageIntent = new Intent();
                            messageIntent.setAction(VIDEO_BROADCAST_KEY);
                            ByteArrayOutputStream stream = new ByteArrayOutputStream();
                            bitmap.compress(Bitmap.CompressFormat.PNG, 100, stream);
                            bitmap.recycle();
                            messageIntent.putExtra(VIDEO_MESSAGE_KEY, stream.toByteArray());
                            stream.close();
                            sendBroadcast(messageIntent);
                        } else
                            Log.d(this.getClass().getName(), "Malformed data");
                        frameList.clear();
                    } else if (frameStart)
                        frameList.add(message);
                }
                Thread.sleep(VIDEO_SERVER_RESPAWN);
            } catch (Throwable e) {
                Log.e(getClass().getName(), e.getMessage());
                e.printStackTrace();
            }
        } while (keepRunning);
        if (socket != null) {
            try {
                socket.close();
            } catch (Throwable e) {
                Log.e(getClass().getName(), e.getMessage());
                e.printStackTrace();
            }
        }
    }

message是以下文本的一行：

F
P2
160 120
1226
193 141 158 152 193 186 171 177 186 160 195 182 ... (160 times)
                         .
                         . (120 lines)
                         .
278 248 253 261 257 284 310 304 304 272 227 208 ... (160 times)

EF

在MainActivity中，我处理此代码：

class VideoReceiver extends BroadcastReceiver {
    final public Queue<Bitmap> imagesQueue = new LinkedList<>();
    @Override
    public void onReceive(Context context, Intent intent) {
        try {
            //Log.d(getClass().getName(), "onReceive() called");
            final byte[] data = intent.getByteArrayExtra(VideoService.VIDEO_MESSAGE_KEY);
            final Bitmap bitmap = BitmapFactory.decodeByteArray(data,0,data.length);
            imagesQueue.add(bitmap);
            runOnUiThread(updateVideoTask);
        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }
}

updateVideoTask任务是这样定义的：

    updateVideoTask = new Runnable() {
        public void run() {
            if (videoReceiver == null) return;
            if (!videoReceiver.imagesQueue.isEmpty())
            {
                final Bitmap image = videoReceiver.imagesQueue.poll();
                if (image == null) return;
                videoView.setImageBitmap(image);
                Log.d(this.getClass().getName(), "Images to spool: " + videoReceiver.imagesQueue.size());
            }
        }
    };

运行应用程序时，我会发现帧速率非常低，延迟非常大。我不能争论发生了什么。我从 logcat 获得的唯一提示是这些行：

2019-05-20 16:37:08.817 29566-29580/it.tux.gcs I/art: Background sticky concurrent mark sweep GC freed 88152(3MB) AllocSpace objects, 3(52KB) LOS objects, 22% free, 7MB/10MB, paused 3.937ms total 111.782ms
2019-05-20 16:37:08.832 29566-29587/it.tux.gcs D/skia: Encode PNG Singlethread :      13003 us, width=160, height=120

即使以所有这些延迟的总和(140毫秒(的总和，该应用程序在达到0.25hz甚至更糟时的帧速率也应超过5Hz。

经过一些调查，我发现移动：

Paint paint = new Paint();
paint.setStyle(Paint.Style.FILL);

从嵌套环中出来防止GC如此频繁地调用，我发现了这一行延迟的另一个主要来源：

final String[] points = line.split("\s+");

它每次燃烧2毫秒，所以我决定去做一些不太聪明但更快的东西：

final String[] points = line.split(" ");

无论如何它仍然不够。.

之间的代码

canvas.drawBitmap(bitmap, 0, 0, null);

和

sendBroadcast(messageIntent);

仍然消耗超过200ms ...我该怎么做比这更好？

我很确定，有一种更有效的方法可以从TCP服务器收集此大小和速率的框架并在ImageView上显示。

当然，这可能是软件体系结构不仅优化此代码本身的问题。除了本机代码以外，我对任何新方法都开放(我不熟悉它(。

更新(03/11/2019(：

活动方面：

public class MainActivity extends AppCompatActivity implements FrameReadyCallBack {
    private Intent videoServiceIntent;
    private VideoService videoService;
    private boolean bound = false;
    private ImageView surfaceView_video = null;
    private String videoPort = "5002";
    private String videoServerAddr = "192.168.10.107";
    private ServiceConnection serviceConnection = null;
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        surfaceView_video = findViewById(R.id.surfaceView_video);
        serviceConnection = new ServiceConnection() {
            @Override
            public void onServiceConnected(ComponentName className, IBinder service) {
                VideoService.VideoServiceBinder binder = (VideoService.VideoServiceBinder) service;
                videoService = binder.getService();
                bound = true;
                videoService.registerCallBack(MainActivity.this); // register
            }
            @Override
            public void onServiceDisconnected(ComponentName arg0) {
                bound = false;
            }
        };
        startVideoService();
    }
    @Override
    public void frameReady(byte[] image_data) {
        //TODO: create image and update surfaceView_video
    }
    public void startVideoService()
    {
        videoServiceIntent = new Intent(this, VideoService.class);
        videoServiceIntent.putExtra(VideoService.LOCAL_PORT_KEY, videoPort);
        videoServiceIntent.putExtra(VideoService.LOCAL_VIDEOSERVER_ADDR_KEY, videoServerAddr);
        startService(videoServiceIntent);
    }
    @Override
    protected void onStart() {
        super.onStart();
        bindService();
    }
    @Override
    protected void onStop() {
        super.onStop();
        unbindService();
    }
    private void bindService() {
        bindService(videoServiceIntent, serviceConnection, Context.BIND_AUTO_CREATE);
    }
    private void unbindService(){
        if (bound) {
            videoService.registerCallBack(null); // unregister
            unbindService(serviceConnection);
            bound = false;
        }
    }
}

服务方面：

public class VideoService extends Service {
    public static final String LOCAL_PORT_KEY = "video_port";
    public static final String LOCAL_VIDEOSERVER_ADDR_KEY = "video_server_addr";
    private static final int DEFAULT_VIDEO_PORT = 5002;
    private static final int VIDEO_SERVER_RESPAWN = 2000;
    private volatile FrameReadyCallBack frameReadyCallBack = null;
    private VideoReceiver videoReceiver = null;
    private IBinder videoServiceBinder = new VideoServiceBinder();
    @Nullable
    @Override
    public IBinder onBind(Intent intent) {
        return videoServiceBinder ;
    }
    @Override
    public boolean onUnbind(Intent intent) {
        videoReceiver.kill();
        return super.onUnbind(intent);
    }
    @Override
    public int onStartCommand(Intent intent, int flags, int startId) {
        final int localVideoPort = intent.getIntExtra(LOCAL_PORT_KEY, DEFAULT_VIDEO_PORT);
        final String videoServerAddr = intent.getStringExtra(LOCAL_VIDEOSERVER_ADDR_KEY);
        videoReceiver = new VideoReceiver(videoServerAddr, localVideoPort);
        videoReceiver.start();
        return Service.START_NOT_STICKY;
    }
    public void registerCallBack(FrameReadyCallBack frameReadyCallBack) {
        this.frameReadyCallBack = frameReadyCallBack;
    }
    public class VideoServiceBinder extends Binder {
        public VideoService getService() {
            return VideoService.this;
        }
    }
    private class VideoReceiver extends Thread {
        private boolean keepRunning = true;
        private int VIDEO_SERVER_PORT;
        private String VIDEO_SERVER_ADDR;
        private int bad_frames;
        private int frames;
        private int link_respawn;
        private FrameDecodingStatus status;
        public VideoReceiver(String addr, int listen_port) {
            VIDEO_SERVER_PORT = listen_port;
            VIDEO_SERVER_ADDR = addr;
        }
        public void run() {
            InetAddress serverAddr;
            link_respawn = 0;
            try {
                serverAddr = InetAddress.getByName(VIDEO_SERVER_ADDR);
            } catch (UnknownHostException e) {
                Log.e(getClass().getName(), e.getMessage());
                e.printStackTrace();
                return;
            }
            Socket socket = null;
            DataInputStream stream;
            do {
                bad_frames = 0;
                frames = 0;
                status = FrameDecodingStatus.Idle;
                try {
                    socket = new Socket(serverAddr, VIDEO_SERVER_PORT);
                    stream = new DataInputStream(new BufferedInputStream(socket.getInputStream()));
                    final byte[] _data = new byte[PACKET_SIZE];
                    final byte[] _image_data = new byte[IMAGE_SIZE];
                    int _data_index = 0;
                    while (keepRunning) {
                        if (stream.read(_data, 0, _data.length) == 0)
                            continue;
                        for (byte _byte : _data) {
                            if (status == FrameDecodingStatus.Idle) {
                               //Wait SoM
                            } else if (status == FrameDecodingStatus.Data) {
                               //Collect data
                            } else {
                                    frameReadyCallBack.frameReady(_image_data);
                                    status = FrameDecodingStatus.Idle;
                            }
                        }
                    }
                    link_respawn++;
                    Thread.sleep(VIDEO_SERVER_RESPAWN);
                    Log.d(getClass().getName(), "Link respawn: " + link_respawn);
                } catch (Throwable e) {
                    Log.e(getClass().getName(), e.getMessage());
                    e.printStackTrace();
                }
            } while (keepRunning);
            if (socket != null) {
                try {
                    socket.close();
                } catch (Throwable e) {
                    Log.e(getClass().getName(), e.getMessage());
                    e.printStackTrace();
                }
            }
        }
        public void kill() {
            keepRunning = false;
        }
    }
}

Answer 1

首先，出于某种原因，您正在通过BroadcastReceiver提交带有新图像更改的结果。您可以显着提高整体速度，但可以消除这种逻辑。并通过绑定功能替换Service的通信。

    // Bind to LocalService
    Intent intent = new Intent(this, LocalService.class);
    bindService(intent, connection, Context.BIND_AUTO_CREATE);

然后接收连接。

/** Defines callbacks for service binding, passed to bindService() */
private ServiceConnection connection = new ServiceConnection() {
    @Override
    public void onServiceConnected(ComponentName className,
            IBinder service) {
        // We've bound to LocalService, cast the IBinder and get LocalService instance
        LocalBinder binder = (LocalBinder) service;
        mService = binder.getService();
        mBound = true;
    }
    @Override
    public void onServiceDisconnected(ComponentName arg0) {
        mBound = false;
    }
};

然后，使用Service粘合剂实例订阅Activity，然后在Service中使用回调来发布新的数据字节。