Column: Best practices for remote monitoring and operations for business continuity

By John Wastcoat, Zixi

Featured Column

View more columns from NewscastStudio contributors

The time for implementing remote workflows has been forced on us, whether we were ready for it or not. Zixi recently conducted a poll of major media industry executives on their remote working readiness in the face of the coronavirus pandemic. With much of the media industry based in large cities like New York and LA, many companies were suddenly forced to operate with 95% of their workforce working remotely and most of them found out that they weren’t prepared for it. The survey revealed that 66% of respondents believe that this is not a single one-time event and 81% now believe that their organizations require permanent remote workforce capabilities for future possible events. Moving forward, remote monitoring and operations will be crucial for business continuity. So how does an organization adopt this critical capability, and what are the requirements needed to be successful?

What are the requirements?

To get started with remote operations, make things simple by utilizing a single digitized control plane to deploy, configure, monitor and manage content over any IP to and from anywhere in the world. One interface provides process efficiency by creating a simplified and agile way to manage large IP implementations remotely. Cloud virtualization is required to administer these remote workflows at scale because it provides the ultimate flexibility and agility. Getting into and across the cloud can be a challenge, but when IP can be combined it makes transport a lot easier to manage.

Next, the IP workflow architecture should be visible across the organization from engineering to operations, and executive teams. You need to be able to get live studio contribution and signal acquisition, provision live streams and allow the participation of multiple team members including directors, producers, editors and talent. Engineers need to be able to launch and configure these new workflows from anywhere. Organizations have to be able to extend the reach of remote digital NOCs and network operating centers globally while managing source and delivery targets. At the same time, they need to monitor those remote workflows with telemetry that allows visibility to the network, TR10-1 analytics, audio issues and video metrics. It is critical to know, are packets being recovered? Are we experiencing jitter? Is our system health operating adequately? Can we be proactively alerted and notified?

Once errors are visible, finding out where they originated and how to resolve them becomes mandatory, as does the need to be able to determine Root Cause Analysis across the media supply chain, to know what, where and when an error happened from source to partner to affiliates. This can be a very costly process per programming channel and becomes much more important as the extension to remote operation mode is made.

Advertisement

To be effective, using a robust and reliable protocol that can traverse congested IP networks while maintaining flawless QoS is fundamental. The protocol will enable telemetry by providing network, audio, and video health data on each stream. It is essential to enable transport between public and private cloud, and ease VPN congestion while acquiring feeds securely with 99.999+% reliability. All of this, especially in a live newsgathering environment, means that latency needs to be optimized and tuned so that video can be transported at low latency regionally and globally based on bandwidth.

Lastly, it is critical to be able to contribute on mobile when other options aren’t available, and an on-air application allows for remote access the same way it is possible to do so from a professional camera.

Now that I’m set up, what best practices should I employ?

When 95% of staff who previously went into work start working from home, previously employed best practices need to change but should not be a drastic departure from what has been effective. The best approach to implementing remote operations for business continuity is broken into two phases.

In phase one an initial baseline needs to be established with a focus on the result of achieving the programming output desired. Workflows will evolve and they may need to be done in more complex ways, but the end goal is to make sure that the programming, ad insertion and closed captioning, etc, exist.

It is important to leverage existing infrastructure from hybrid IP and fiber networks cloud-based workflows, but it is most important to put trust in the open Internet. There have been many people who have been hesitant to adopt the open Internet for delivery, but over the last several years we are experiencing it being successfully deployed on a global basis. Adoption will continue to rapidly accelerate as it is extremely difficult to scale without leveraging the internet and a public cloud base. AWS, Microsoft and Google are excellent infrastructure providers to be able to provide ingress and egress across cloud touchpoints.

For workflows with the most optimal flexibility, embracing a software-defined video solution like Zixi is important because it allows interoperability that is crucial for remote operations using a wide range of systems and platforms. A great benefit of a software-defined video platform is the understanding of the range and limitations of an organization’s architecture within a matter of days instead of weeks. When working remotely, text or e-mail alert-based operations for video and confidence monitoring is vital and instantly permits fast resolution to the problem. The telemetry of that data is critical to recognize what the networks, edge devices from encode to decode, entry points into CDNs and other partners are looking like from end to end, always confirming that audio and video are the low latency, broadcast-quality consumers mandate.

Phase two brings an evolution of this process. With confidence in the underlying infrastructure, workflow templates to streamline the process can be established. Templated sets of workflows that allow a studio or at home acquisition of a signal ensure that the relevant directors, producers, executive and graphic operators can react with agility and speed across a wide range of programming and content creation. It is fundamental to build a portfolio of toolsets that address the range of use cases and to operationally standardize responses to each. A remote-ready future means that many vendors who have been sitting disparate from each other will soon be standardized and homogenized into a common area of cloud and IP compatibility, and operations and engineering teams will be able to leverage the full spectrum of the vendor ecosystems and integrate accordingly.

Remote readiness is essential for business continuity now and in the future

The blueprints for remote workflows that had long been sitting dusty have now been mobilized, and from here there is no turning back. These remote virtual workflows will continue to expand beyond the current pace, extending our existing workflows and remain long term. In this landscape full API access between all workflows is mission critical because it allows legacy systems and new remote systems to be optimized in terms of control, data share and true seamless interoperability and workflow.

A control plane to monitor these APIs and remote workflows is imperative for operational and technical scalability. For example, Zixi’s ZEN Master control plane can provide grids for monitoring systems and stream components at scale, complete workflow visualization with thumbnail monitoring, and system monitoring of all components along with a broad range of analytics. It enables visibility across the partner supply chain from source, vendors, affiliates, and endpoints, all from one centralized dashboard that can be drilled down into, allowing for the configuration and management of everything remotely, from uptime of channels to QoS, ensuring the ability to handle outages and version controls, scheduling and automation, and provide reporting metrics and data to ensure meeting the QoE that consumers demand. And it is all API accessible with Zixi technical support.

The time for remote workflows has been forced on the industry, and it is very likely going to become a permanent capability for major media companies moving forward. But with the right tools and outlook, broadcast continuity for business operations can be achieved with the scale and speed that the consumer quality of experience demands.

John Wastcoat, ZixiJohn Wastcoat has more than 20 years’ experience in senior management with industry-leading media and communications technology companies. Before joining Zixi John successfully ran the Aspera Business Development group, acquired by IBM in 2014. He was responsible for scaling the business across numerous channels around the world, and previously held the positions of VP of Global Business Development at Dexrex and VP Sales at Cantata Technology, which was acquired by Dialogic. At Zixi John is responsible for the OEM, technology alliance and reseller ecosystem as well as overall corporate messaging. John holds a Bachelor of Science Degree in Political Science from the University of Montana as well as Japanese language from Nichibei Kaiwa Gakuin in Tokyo.

Author Avatar