ljn
Posts: 9
Joined: Wed Oct 14, 2015 1:39 pm

[SOLVED] h264_mmal failing at run time on pi zero

Wed Jun 09, 2021 2:11 pm

First, apologies for being a little off topic but:
1 I can't find a more appropriate forum;
2 this forum seems to attract viewers with the right areas of expertise;
3 moderator 6by9 has stated (on 2020-08-04 at 10:08h) that 'OpenMAX has been a dead since around 2012', so perhaps there is a case for renaming the forum to something like 'OpenMAX and successors'.
Anyway, down to business!

I have successfully run scrcpy (version 1.16) using the h264_mmal decoder on a pi 2B (model v1.1 with BCM 2836 chip) running raspiOS Buster by replacing the following two lines in scrcpy's 'stream.c'

Code: Select all

    AVCodec *codec = avcodec_find_decoder(AV_CODEC_ID_H264);
    stream->parser = av_parser_init(AV_CODEC_ID_H264);

by those below (lines numbers 188 and 217).

Code: Select all

    AVCodec *codec = avcodec_find_decoder_by_name("h264_mmal");
    stream->parser = av_parser_init(codec->id);

I have also added debug code to output the duration of the decoding and texturing/rendering processes for each frame, which indicates that decoding with h264_mmal is around seven times faster than with the standard h264 decoder.

I have built the same scrcpy code on a pi zero but found errors when running it. Enabling debugging output on scrcpy and redirecting that output to a file shows the following two lines (presumably generated by the mmal decoder) occurring after the first two incoming video frames are reportedly decoded (though the zero duration of decoding in the former case may indicate that decoding did not really take place for that frame).
mmal: mmal_port_event_get: vc.ril.video_decode(1:0) port 0xa9402380, no event buffer left for ERRO
mmalipc: mmal_vc_handle_event_msg: no event buffer available to receive event (2)

Decoding of input packets apparently continues, however, until around the time that the renderer starts, when the following two error messages are generated by scrcpy, which then terminates without having displayed any video frames.
ERROR: Could not send video packet: -1313558101
ERROR: Could not process frame

The former of these messages is generated in scrcpy's 'decoder.c' at line 68 by the code

Code: Select all

    if ((ret = avcodec_send_packet(decoder->codec_ctx, packet)) < 0) {
        LOGE("Could not send video packet: %d", ret);
        return false;
    }

which indicates that the error is generated in avcodec_send_packet (presumably part of h264_mmal). The error code of -1313558101 is the value 0x4E4B4E55 which is ASCII 'UNKN', which presumably stands for 'Unknown'.

The complete debug output is given below. Note that the lines containing 'LJN' are generated by my debugging code.
INFO: scrcpy 1.16 <https://github.com/Genymobile/scrcpy>
DEBUG: Using server: /usr/local/share/scrcpy/scrcpy-server
DEBUG: Screensaver enabled
DEBUG: Starting stream thread
DEBUG: Starting controller thread
DEBUG: Starting receiver thread
DEBUG: LJN 0ms decoder start
DEBUG: LJN 0ms decoder duration
DEBUG: LJN 0ms decoder start
DEBUG: LJN -900 timestamp offset
DEBUG: LJN 0 packet->pts
DEBUG: LJN 7ms decoder duration
mmal: mmal_port_event_get: vc.ril.video_decode(1:0) port 0xa9402380, no event buffer left for ERRO
mmalipc: mmal_vc_handle_event_msg: no event buffer available to receive event (2)
DEBUG: LJN 0ms decoder start
DEBUG: LJN -1800 timestamp offset
DEBUG: LJN 99100 packet->pts
DEBUG: LJN 1ms decoder duration
DEBUG: LJN 0ms decoder start
DEBUG: LJN -2700 timestamp offset
DEBUG: LJN 198200 packet->pts
DEBUG: LJN 8ms decoder duration
DEBUG: LJN 0ms decoder start
DEBUG: LJN -3600 timestamp offset
DEBUG: LJN 270619 packet->pts
DEBUG: LJN 1ms decoder duration
DEBUG: LJN 0ms decoder start
DEBUG: LJN -4500 timestamp offset
DEBUG: LJN 309484 packet->pts
DEBUG: LJN 0ms decoder duration
DEBUG: LJN 0ms decoder start
DEBUG: LJN -5400 timestamp offset
DEBUG: LJN 359065 packet->pts
DEBUG: LJN 2ms decoder duration
DEBUG: LJN 0ms decoder start
DEBUG: LJN -6300 timestamp offset
DEBUG: LJN 387155 packet->pts
DEBUG: LJN 1ms decoder duration
DEBUG: LJN 0ms decoder start
DEBUG: LJN -7200 timestamp offset
DEBUG: LJN 431338 packet->pts
DEBUG: LJN 10ms decoder duration
DEBUG: LJN 0ms decoder start
DEBUG: LJN -8100 timestamp offset
DEBUG: LJN 459415 packet->pts
DEBUG: LJN 1ms decoder duration
DEBUG: LJN 0ms decoder start
DEBUG: LJN -9000 timestamp offset
DEBUG: LJN 476098 packet->pts
DEBUG: LJN 1ms decoder duration
DEBUG: LJN 0ms decoder start
DEBUG: LJN -9900 timestamp offset
DEBUG: LJN 498763 packet->pts
DEBUG: LJN 0ms decoder duration
DEBUG: LJN 0ms decoder start
DEBUG: LJN -10800 timestamp offset
DEBUG: LJN 521468 packet->pts
DEBUG: LJN 0ms decoder duration
DEBUG: LJN 0ms decoder start
DEBUG: LJN -11700 timestamp offset
DEBUG: LJN 545644 packet->pts
DEBUG: LJN 0ms decoder duration
DEBUG: LJN 0ms decoder start
DEBUG: LJN -12600 timestamp offset
DEBUG: LJN 568365 packet->pts
DEBUG: LJN 1ms decoder duration
DEBUG: LJN 0ms decoder start
DEBUG: LJN -13500 timestamp offset
DEBUG: LJN 589698 packet->pts
DEBUG: LJN 0ms decoder duration
DEBUG: LJN 0ms decoder start
DEBUG: LJN -14400 timestamp offset
DEBUG: LJN 613513 packet->pts
DEBUG: LJN 6ms decoder duration
DEBUG: LJN 0ms decoder start
DEBUG: LJN -15300 timestamp offset
DEBUG: LJN 637907 packet->pts
ERROR: Could not send video packet: -1313558101
ERROR: Could not process frame
DEBUG: End of frames
INFO: Renderer: opengl
INFO: OpenGL version: 3.1 Mesa 19.3.2
INFO: Trilinear filtering enabled
INFO: Initial texture: 720x1280
DEBUG: Video stream stopped
WARN: Device disconnected
DEBUG: quit...
DEBUG: Receiver stopped
DEBUG: Server terminated

Has anyone any ideas as to what is going wrong and any suggestions as to how to fix it?
Last edited by ljn on Sat Jun 12, 2021 4:11 pm, edited 1 time in total.

ljn
Posts: 9
Joined: Wed Oct 14, 2015 1:39 pm

Re: h264_mmal failing at run time on pi zero

Sat Jun 12, 2021 4:00 pm

Well, after further thought and testing, I think I have found the explanation and solution to my problem.

On my pi zero, the GPU memory size was set to the default value of 64MB. Setting this to 256MB, in 'Raspberry menu>Settings>Raspberry Pi Configuration>Performance' led to correct running of scrcpy with no errors. On my pi 2, I had GPU memory set to 508MB.

Although scrcpy now runs correctly on the pi zero with mmal decoding, it is still unusably slow with only a single frame displayed every ten seconds or so and a 10-20s delay from the Android phone display to the pi's screen. By contrast, on the pi 2 every frame is displayed and the delay is around half a second (possibly attributable to the in-built USB hub). These figures are for a 1280x720 test video.

Return to “OpenMAX”