GStreamer consultancy

GStreamer is the framework of choice for Linux-based multimedia. It is deployed on a wide range of hardware products and environments. Our multimedia team has accumulated years of experience in the development of the GStreamer core framework and its plugins. Thanks to its flexible architecture design, the framework can be used to implement software applications targeting many use-cases such as media players, multimedia-aware web engines, video editors, transcoders and streaming servers.

Web engines integration

Web engines provide an attractive approach to designing user-interface in an increasing number of products. Moreover the Web platform provides multiple specifications related to Multimedia support, such as HTML5 video playback, WebAudio, WebRTC and adaptive streaming (Media Source Extensions).

We are the main developers and maintainers of the GStreamer-based multimedia back-ends of WebKit that are used by several of its ports, including WebkitGTK and WPE. Since 2009 we have led the development of the GStreamer WebKit back-ends, always making sure they integrate and perform well in the web.

Leveraging our deep knowledge of the GStreamer WebKit back-ends, we also worked with Mozilla, adding a GStreamer backend for hardware-accelerated multimedia playback to their Servo web engine.

Web augmented video production

Live video events commonly display infographics overlays on top of the video content. For instance a live sports broadcast event would often display a score-board overlay in a corner of the screen. Another example are news tickers, auxiliary infographics scrolling over news broadcasts.

Nowadays the Web platform allows for rich and visually compelling content animations, relying on CSS animations, styling and optionally JavaScript.

By combining the rendering potential of WPEWebKit with the GStreamer multimedia framework, HTML-based graphics overlays can be mixed and composed with any live video source. We are the maintainers of the GstWPE GStreamer plugin, which was designed exactly for this use-case. Any GStreamer-based application can now leverage the full potential of Web-based graphics overlays in the context of live video production.

Our multimedia engineers can offer advise on the design and implementation of GStreamer applications covering live video mixing use-cases, with a dedicated focus on embedded graphics overlays.

Hardware accelerated multimedia processing

Hardware accelerated multimedia is a popular strategy to offload media processing (decoding, encoding, post-processing), which are compute-intensive, from the general purpose processor to custom hardware such as GPUs, DSPs, and so on. Platform vendors usually offer kernel or user-space interfaces to access that external hardware, and interchange encoded/decoded video frames, for example.

We have experience bringing hardware accelerated multimedia to GStreamer, writing plugins and the core functionality required to efficiently use these resources. We are the maintainers of GStreamer-VAAPI, a plugin which provides hardware-accelerated video encoding, decoding, post-processing and rendering for most Intel-based platforms, AMD-based GPUs and some ARM SoCs, through VA-API, a widely adopted user-space interface for hardware accelerated multimedia. We also have experience and contributions in GStreamer-Video4Linux plugin, which uses the kernel V4L2 interface, offered for most ARM SoCs vendors. We have contributed to OpenMAX GStreamer plugins too, which used to support platforms such as the Raspberry Pi, and a few other custom and vendor-specific interfaces. We are currently working with the new Vulkan extension for video processing to use it with GStreamer-Vulkan plugin.

Post production

Non-linear video editing tools are a key component of the post-production pipeline and require a solid and mature multimedia framework. Igalia’s contributions to GStreamer aim to position the project as the sustainable alternative to proprietary and monolithic tools in this particular field of the multimedia industry.

For example, as part of the movie production process, editors often need to script their project timeline creation in order to automate otherwise repetitive tasks. Unfortunately the existing tools don’t naturally fit these new workflows.

GStreamer Editing Services (GES) provides a high level API to build video editing tools in a simple and efficient way, making it possible to reuse the amazing set of GStreamer elements out there. By leveraging its API, new tools can be developed in order support a new generation of editing workflows. For instance, GES allows easy scripting of the timeline editing process from the Python programming language. GES is maintained as a Free Software (FOSS) project, its developers are constantly adding new features in order to make it a good alternative to proprietary frameworks and tools.

Building upon the GES foundations, our Multimedia team members are core contributors and maintainers of the Open-Source Pitivi video editing application. Pitivi is a simple, flexible and powerful application with an easy-to-use user interface. By relying on the GES API, Pitivi provides all the features commonly needed for video post-production editing. Additionally, it can easily be adapted for custom editing workflows, our engineers can provide guidance towards achieving that goal.

Igalia has years of experience developing key technologies for video content creation. We lead the development of the GStreamer Editing Services framework. As maintainers and core developers of the library we are able to guide and advise our customers in the development of custom editing software tools aimed to optimize their workflow.

Other frameworks

GStreamer integrates with third-party multimedia frameworks, such as FFmpeg, and as example, our multimedia engineers have developed GStreamer plugins using Intel’s VA-API and MediaSDK or Google’s libwebrtc.

Thus, our multimedia engineers also have deep expertise in third-party frameworks such as the before mentioned ones. Either by working on the core of these libraries or by making use of them in Multimedia applications, we can help our customers reach their technical milestones.