It has been a bit more than a year since the last official 1.22.0 release of the GStreamer project, and since then a lot of improvements have been made to GStreamer, our favorite multimedia framework.

This release is thanks to the more than ~300 contributors who have worked on it, which is up 33% (!!!) since 1.22. They made almost 5,000 commits to the the core GStreamer repositories (there were 4,000 in 1.22). A full description of all the improvements in this important release can be found here

During this cycle, the multimedia team at Igalia worked really hard, with almost 500 commits in this cycle. We are proud to present you our essential contributions.

Hardware accelerated codecs from VA to Vulkan

One of the main contributions from our team is the support for Vulkan Video, especially the Vulkan Video decoders H.264 and H.265 which allows decoding media content on modern hardware since the release of Vulkan Video in the Vulkan SDK. Major hardware vendors support this new feature, so it was very important to us to bring it to GStreamer.

To tackle this challenge, a new object offers a reliable mechanism to achieve queue operation with modern synchronization such as the Synchronization2 extension. This work also includes support for timeline semaphores, in addition to multiple improvements in the Vulkan GST stack.

A lot of attention has also been provided to VA library support in GStreamer. With 1.24, VA plugins are now first-class citizens of the framework with a rank set to PRIMARY+1. GStreamer-VAAPI has been ranked None to progressively deprecate those elements. GstVA should always be preferred.

We actively participated in the design of the negotiation of DMABuf-based video streams, and implemented it in the OpenGL uploader and in VA-API encoders, decoders, and filters. This mechanism would allow zero-copy across video chains in the pipeline graph.

GStreamer Editing Services

On the road to full hardware acceleration

Another step toward enabling full hardware acceleration support in GES based pipelines has been done through the reimplementation of the autovideoconvert element and the implementation of the autodeinterlace and autovideoflip elements. The next step is to enable full hardware acceleration support, and a merge request to make that happen has already been opened. Compositing with hardware-based compositors is now fully supported in GStreamer 1.24.

Serializing information about media file asset enhanced

For GStreamer 1.24, we implemented a new GESDiscovererManager which allows applications to use GStreamer to serialize information about their media asset and retrieve them when reloading a project. Thanks to that upgrade, reloading projects is much faster. This allows the usage of media information in third-party applications such as web services using dedicated databases.

Give more flexibility for effects

In GES, any GStreamer filtering element can be used as an Effect, but until now, scaling input clips was done after all effects were applied and right before the compositing. By contrast, some applications want to apply their effects on already-scaled video frames. To allow that we implemented a gesvideoscale element which can be used as an effect and be applied wherever the user wants in the chain of elements. In GES, clip scaling and positioning information are set and contained on a specific GstMeta, the GESFrameCompositionMeta, which is now exposed publicly so effects that require the information can make use of it.

Other key contributions

As an important part of our expertise, the WPE plugin got a lot of attention to fix some bugs and do some maintenance.

Improvements have been made to the support of adaptive content such as DASH. The dashsink element can now support properly MP4 fragment using the fmp4mux element, and dashdemux now supports container-specific-track-id as a unique-id. This change is also available for demuxers such as qtdemux or matroskademux.

In addition, the ISO MP4 demuxer in GStreamer has been improved to properly support Webkit Media Source Extensions and Encrypted Media Extensions using GStreamer.

The decodebin3 and playbin3 elements are now considered stable thanks to a joint effort with the community to address the remaining pitfalls and make it ready to replace the old elements, which can get a deserved retirement.

As you may notice, Rust became a very important source of contributions to GStreamer with more than 25% of the commits written with this language. A lot of those were related to WebRTC and provide a better stability. A new Rust element has been implemented that allows streaming to a Janus WebRTC server using GStreamer. This comes along with various bugfixes in support of the WebRTC API, such as in webrtcbin.

A large number of bug fixes and stability reinforcements have been achieved in the GstValidate framework, which increases the reliability of GStreamer.

The meson GST build system can now generate a full static build for both Windows and Linux platform, allowing an application to embed GStreamer completely. A new plugin named fullstaticfeatures has been created to embed all the selected features.

If you have any questions about these or other new features, please don’t hesitate to contact us.