Virtual Match Sports Video Compositing

Highlights

• Replaced manual use of Adobe After Effects with custom solution using multi-threading and Core Image, providing an elapsed time performance improvement from 20 times real-time processing to 2 times real-time, compositing around 250,000 image frames for all variations of a 15 video second clip.
• Composited multiple image layers with masking using a matrix of team combinations to efficiently generate over a hundred variations from the source renders of a single clip.
• Abstracted the core composite engine to support the layer combinations rendered for multiple sport types.

Background

Video clips for the football highlight streaming project were rendered out as five separate images for each video frame; background, branding, defending team, attacking team, and a mask used to separate the players, officials and goalkeepers.  By rendering these out separately, the branding on hoardings and the different player team colours could be rendered without having to process all of the other elements for every one of the different combinations (112 videos with 8 team colours).  This was a huge saving on rendering time, without which the project would not have been possible.  However it placed a heavy burden on the process of compositing of those image layers into video clips.

The compositing was initially performed using scripts in Adobe After Effects that would run for between 24 to 48 hours for each clip after which the next clip would have to be started manually.  The project consisted of over one hundred clips.  To reduce the elapsed time and automate the processing multiple clips a custom compositing engine was developed.

Responsibilities

• Analysis, design, development, unit testing.
• Lead developer of the whole solution.

Solution

Technologies: Objective-C, Xcode, OS X/macOS

Achievements

• Delivered solution for football rendered layers.
• Delivered solution for tennis rendered layers, reusing the core compositing engine.