Industry Insights: Machine learning, AI and software-first workflows coming to production switchers

Production switchers are at the heart of broadcast production.

In this installment of our Industry Insights roundtable, our experts from the field of production control and switchers discuss customer feedback, look ahead at future workflows, examine software-first production and the potential for AI and machine learning in broadcast.

Industry Insights: Pressing forward with production switchers

How did the past year change your outlook?

“2020 was the catalyst that accelerated many trends already coming to the fore, such as remote production and cloud enabled workflows. Whether it was sports news and highlights, singing competitions, or late-night comedy, Grass Valley remote switching models allowed production teams to provide live content from the safety of a socially distanced environment. As a business, the forward thinking that made us ready to adapt last year is continuing to drive innovation in 2021,” said Grass Valley’s VP of production switchers, Greg Huttie.

“Naturally, the requirement to work remotely during the pandemic has added focus to alternative workflows and processes. Fortunately, we had already built distributable control systems for our production solutions via our DashBoard and RossTalk protocol, which has simplified what might have been complex control and management setups. What was perhaps a little surprising was the way that producers have embraced compromise—signal compressions schemes and software apps that they would not have considered pre-pandemic. This was, of course, driven by necessity but it has allowed the broadcast community to get the job done in critical times,” Ross Video’s Nigel Spratling, VP of production switchers & video servers.

“Coming into 2020, we introduced an IP/IT switcher platform. Despite our initial concerns, we saw the market was picking up its pace to switch to all IP mid-year, so we’re following that trend as we develop new approaches to our technology. Also in light of the growing number of remote workforces who are suffering from “Zoom fatigue,” we are working on ways to improve the production value of remote meetings,” said Mike Bergeron, senior category owner – advanced technology, video production at Panasonic North America.

“The COVID-19 pandemic accelerated our integration of remote production with our video switchers, specifically the use of remote control GUIs and physical control panels. With the shift to at-home and other remote production models, switchers need to allow operators to work outside of the traditional control room using public internet,” said Satoshi Kanemura, president of FOR-A Corporation of America.

“The last year changed how we do business and interact with our customers, especially as they became more reliant upon technology and solutions to adapt to all of the changes. At the beginning of the year new technologies like IP were being discussed and looked at, but the pandemic accelerated the process, and we all had to adjust. There were hurdles to overcome but important lessons were learned, and in the end, our outlook remains positive. Television production was pushed into a new reality, the future is now, and that is exciting,” Scott McQuaid, a senior sales support engineer for Sony Electronics, told us.

Advertisement

“The inability for anyone to move around definitely increased the demand for video products overall. NewTek’s outlook, however, did not change much. We have always believed the future of video production involves software, computers, and networks. The result of 2020 is that we have sped down that path much quicker than anticipated,” said Barbara Spicek, president of NewTek Worldwide.

“As a company, we had already embarked on a strategy of networking, remote control and streaming in 2019, so the events of 2020 accelerated that strategy. With the explosion in videoconferencing and streaming events, we have focused on useability and incorporating automation to help a non-broadcast-trained user improve their production values. Produce better looking, more compelling content easier,” said Graham Sharp, CEO of Broadcast Pix.

What workflow enhancements do you see coming to switchers in the near future?

“We are seeing a rapid adoption of IP for distributed production. IP connections enable efficient production models where remote studios with switcher control panels connect back to centralized processing, and is the enabling technology when producing in premium formats such as UHD and HDR. In addition to IP, the transition to cloud-based workflows rocketed last year. For production teams spread across the world, switching in the cloud is a major advantage,” Huttie noted.

“Continuous evaluation of production workflows is necessary to ensure that product development roadmaps are regularly updated to reflect industry needs. One of the big desires for local or mobile production systems has been the need for better control integration and a reduced demand on power, cooling, and rack space – while adding the ability to manage UHD formats and IP transport schemes seamlessly. Ross has taken some big steps to accommodate these desires with the recent introduction of Hyper-Converged production systems. These systems consolidate multiple functions into compact and programable hardware that offers tight control integration, while freeing the operator to focus on the production,” said Spratling.

“You’re going to see more adoption of various IP formats across the board. Remote production is not going away, so you need to mix the reliability of hardware-based switchers with the ability to take sources from remote locations. Also, look for more comprehensive operation in order to reduce key strokes, operation failures, and manpower while increasing creativity,” Kanemura indicated.

“With resource sharing, instead of individual stand-alone control rooms, we can have the switcher resources in a main central hub with just the control panels in the separate control rooms, and then allocate the resources needed to each room. This offers more flexibility in production and in moving resources and personnel where they are most needed. We have taken the first steps already, creating two logical switchers from one processor. We can send the logical switchers’ resources to control panels in the facility and to multiple control panels,” said McQuaid.

“Production switchers spent decades locked in a control room so they could have direct access to the local sources they were switching. NewTek’s vision has always been that video production can be made available to everyone and we are seeing that realization happening today. IP-based video production removes the requirement for all gear and, as a result, the requirement for operators to always be located in the same room or down the hall. That fundamental change gives storytellers a whole new dimension when thinking about how to realize their content through video,” Spicek said.

“The industry appears to be chasing more and bigger: panels, formats, number of mix effects, etc. We are taking a different approach and are focused on creating easy-to-use UIs and automated macros to enable non-trained operators to create compelling content with high production values,” Sharp noted.

How do you envision the transition to software-first production related to switchers?

“Switchers are the beating heart of live production; it’s key to building any show and creating immersive consumer experiences. We anticipate a hybrid environment for some time to come, where the switcher may be part of an SDI production one day and a cloud-based production the next. The format will depend on the needs of the production. For technical directors (TDs) operating our switcher technology, much of the switching is done by muscle memory. They know the panel so well that their conscious attention can focus on the incoming video sources and cues to build a show, and the physical actions simply flow. Regardless of the production format, the TD punching the show has to have the immediacy of real-time response regardless of the engine that sits behind the control panel,” said Huttie.

“This is a hot topic and one that has been elevated due to the work-from-home requirements. If the production needs are relatively simple, software-based systems are effective. However, as the production tasks grow, it becomes very difficult to provide the features and connectivity of large hardware-based systems due to CPU/GPU and connectivity limitations. The practical solution may be to take advantage of different technologies when they are most appropriate,” said Spratling.

“This is natural to NewTek. Our TriCaster line of products are based around software switching and video manipulation as their core function. What is amazing about being software-first is that we can address fast trends in the industry as they develop. For example, when productions needed access to remote guests, we were able to incorporate video conferencing applications natively in the switcher. Additionally, the software-first capabilities of our live production systems allowed operational control to extend away from the control room via LivePanel,” Spicek explained.

“For us we are already software, so it’s not an issue. The real question is where you run that software. The next step is to run it natively in the cloud. This involves a lot of work because it is not a case of just renting a server for instance, it needs to scale up and down with demand, releasing spare processing capacity and storage when it can. This is the only way that such solutions become cost-effective and it involves quite a lot of work to develop the management software,” Sharp said.

How will new technologies like AI and machine learning impact production switchers in the future?

“I think the market acceptance of the intense computing power required for useful machine learning applied to production switchers will slow the adoption of AI — the cost/benefit tradeoff isn’t favorable right now. Producing compelling media is not just about applying technology, it requires creativity and the ability to tell a story. Those aren’t things that AI and machine learning do well. So I don’t see AI replacing a TD to produce a show. AI does do well at recognizing patterns and quickly processing data, so when compute costs come down – and they eventually will – there are many tasks that AI could assist with so that the TD can focus more on the creative aspects of the show,” Huttie said.

Advertisement

“Automation is opening doors to new markets by making it easier to staff live production feed control rooms. We’re seeing multi-camera, multi-sourced and switched live production proliferating into a broader market for all communications. With AI and machine learning, it’s feasible to give production switchers an “auto” button in predictable venues like a conference room to simplify operations,” said Bergeron.

“Some production tasks can be very complex but are often highly repetitive. Sources and timings are typically unique, but the layering and switching requirements are often the same. AI has the potential to allow those complex tasks to be operationally simplified by learning how an operator works,” Spratling explained.

“When you limit human intervention in live production, you limit creativity. Automation certainly has its place, and FOR-A has our share of products that simplify certain tasks in the production chain. But when it comes to switching live productions, we’re more focused on helping our customers be creative rather than being replaced by a machine,” Kanemura said.

“Technologies like AI, machine learning, and even volumetric video are exciting and make for great topics of discussion. At NewTek, our focus is on developing the tools for storytellers to get their stories to their viewers through video. We have great product and development teams at NewTek, and I would argue the best in the world when it comes to software-based video production. They strive to make sure the customer’s needs drive these new technologies,” said Spicek. 

“I am not sure they will. I guess you could learn what an operator does and automatically create macros, or perhaps use scene detection to create switching algorithms, but the programming required to do this type of machine learning would cost much more than just manually creating a few macros to automate common tasks,” Sharp suggested.

What are common concerns or questions you’re hearing from customers?

“As consumer demand for more content, across numerous devices and available at any time rises, in parallel, broadcasters and content owners’ budgets have either shrunk or remained static. For our customers, this means the ability to do more with less is vital for business success. Switchers that can scale from SDI to IP and handle SD, HD and UHD HDR format requirements are crucial for customers looking for greater flexibility and agility,” Huttie explained.

“At the network level, businesses are concerned about revenue. In the past these businesses have typically run CapEx models but today many are looking to shift to OpEx. As a result, the interest in software as a service and cloud-based systems have become major discussion points. In mobile production, there is a need for greater equipment flexibility while remaining cost efficient. This includes the ability to connect to any signal format and transport scheme while reducing the weight and power of systems. The desire for smaller, lighter, and easy to configure systems is high,” Spratling said.

“Customers frequently ask us how to prepare for the future. The answer is: Pick a future. If you are planning for a 4K upgrade down the road, for example, we can recommend solutions that can handle today’s workflows and provide an appropriate upgrade path,” said Kanemura.

“Customers want to understand the transition from SDI to IP, which is the first step to opening up new technologies and workflows. Flexibility in production workflows is also important. When the world can change in a second, customers want to have flexible solutions so that the impact on production is minimal. Customers want to understand how to better utilize the resources they have and have the flexibility to go from standard production to remote production, and back, without interruption,” McQuaid pointed out.

“The biggest concern by any user, regardless of size or industry, is how to keep their productions relevant to what viewers expect while also generating more content and ever-increasing rates. They must do this without increasing staffing or blowing up the budget. Fortunately, we can deliver solutions here,” Spicek said.

“Most of the questions that we receive are around networking and streaming. As I said, the pandemic accelerated existing trends in the industry – particularly the move to networking and streaming and we see many of our customers struggle to come to grips with the technology. Video over IP is not complex, but it is different to video over SDI and there are some real ‘gotchas,’ particularly around firewalls and opening ports etc. We have responded by creating a networking training course for our resellers, which ultimately, we will repurpose and provide to our customers,” said Sharp.

Participants

Greg Huttie, Grass Valley
Nigel Spratling, Ross Video
Satoshi Kanemura, FOR-A Corporation of America
Scott McQuaid, Sony Electronics
Barbara Spicek, NewTek Worldwide
Graham Sharp, Broadcast Pix
Mike Bergeron, Panasonic North America