Tests: use h264 encoder in mp4 tests with newer ffmpeg.

[defanator]

Proposed changes

Recent ffmpeg versions (e.g. 7.1.2 on Fedora) do not have libx264 encoder available, but h264 does the same thing. This PR proposes dynamic selection of an encoder based on what's available in the environment.

Checklist

Before creating a PR, run through this checklist and mark each as complete:

• I have read the contributing guidelines.
• I have signed the F5 Contributor License Agreement (CLA).
• If applicable, I have added tests that prove my fix is effective or that my feature works.
• If applicable, I have checked that any relevant tests pass after adding my changes.
• I have updated any relevant documentation (README.md and/or CHANGELOG.md).

[pluknet]

@defanator
It's not about ffmpeg version but how H264 support was compiled.
In particular, alpine and/or other distributions we have in pipelines are known to have ffmpeg 7.1.2 or above, and do not experience these mp4 tests failure due to that, ref: https://repology.org/project/ffmpeg/versions

Specifically, ffmpeg has different options to include H264 support:

• libx264, included with --enable-libx264
• libopenh264, included with --enable-libopenh264

The latter seems started to be used instead at least in Fedora as seen in ffmpeg -version:

ffmpeg version 7.1.2 Copyright (c) 2000-2025 the FFmpeg developers
...
configuration: ... --enable-libopenh264

Same can be gained with FreeBSD ports:

# make showconfig | grep 264
     OPENH264=on: H.264 video codec support via OpenH264
     X264=off: H.264 video codec support via x264
# ffmpeg -version
ffmpeg version 8.0.1 Copyright (c) 2000-2025 the FFmpeg developers
...
configuration: ... --enable-libopenh264 ... --disable-libx264

The difference is that libopenh264 permits only -c h264, while libx264 supports both clauses -c h264 and -c libx264.

The tests are not sensitive enough to detect the difference, and since both codecs support acting as "h264",
it makes sense to change this to h264 unconditionally, or at least check libx264 support by a different command ffmpeg -loglevel quiet -encoders:

$ ffmpeg -loglevel quiet -encoders | grep libx264
 V..... libx264              libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (codec h264)
 V..... libx264rgb           libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 RGB (codec h264)

This won't match if it's built with openh264, such as ffmpeg on FreeBSD configured above:

$ ffmpeg -loglevel quiet -encoders | grep 264
 V....D libopenh264          OpenH264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (codec h264)

[pluknet]

I prefer this version instead, if you don't mind.