This action might not be possible to undo. Are you sure you want to continue?
How IP Network Connections Change the Time‐To‐Air Process
Tom Ohanian, Chief Strategy Officer, Signiant
Making Media Move Introduction Can the use of the Internet redefine the notion of “in‐field” broadcasting? Can anyone with a cell phone or wireless card become a broadcaster? If the amount of people who can potentially cover a news‐worthy event were to be able to coordinate their activities, what would that mean for event coverage? As everyone who has ever watched compelling news footage can attest, it is no longer pristine image quality that takes precedence over content as to whether or not the material will air. Today, more than ever, the race is on to get timely content ingested, reviewed, and distributed as quickly as possible. From Broadcast News to Breaking News There is an iconic moment in the 1987 film “Broadcast News” where actress Joan Cusack must deliver the ¾” videotape master of a just‐edited news story. What ensues is one of the funniest (and truest!) moments that is recognizable to anyone who has been in the broadcasting business. Cusack, undaunted, dodges everything from crowded halls to stationary water coolers as she rushes to get the tape into master control for airing. Finally, bruised and limping, she delivers the tape, makes the deadline, and then confidently strides away. Broadcast news indeed! Breaking news, as we have come to think of it, has typically been characterized by an interruption of regular programming in order to cover a live, unfolding event. Camera trucks are sent and live microwave shots are established. Naturally, there is some time delay involved between dispatching the trucks until the video feed becomes available. Of course, all that is now changing… Capturing and Sending Images from the Field Today, the time element for putting images on the air is being dramatically affected by all forms of technology. The combination of portable cameras and still‐image and video‐capable cell phones has created the possibility of a virtual extension of the broadcasting staff. The use of the open, public Internet along with these portable capture devices is rapidly augmenting and, ultimately, will redefine the definition of “in‐field” broadcasting. Today, anyone with a cell phone or a wireless card in a laptop can become a broadcaster. Field‐Generated versus User‐Generated Content There are usually two terms that are used, somewhat interchangeably, but for the purposes of this paper, it is helpful to refer to refer to field‐generated content (FGC) as being content that is created by professional staff members of an organization. User‐generated content (UGC) will be used to describe content that is created by non‐staff members—typically citizen journalists or
it is important to provide certain features and functions specific to each group. and locations that people are viewing content on and at are undergoing rapid change. you are all sharing a common desire to bring the most relevant content in the timeliest fashion to your audience. those distribution channels must be served fast. or on a mobile handset. It is not that quality has been forsaken or that it is not valued. Gone. Timeliness vs. are the days of being sticklers for quality. an earthquake in Pakistan. Security. forever. You’re scheduled in the morning after it’s decided what major stories are going to be covered. size. from a system design aspect. when the event is not scheduled. covering pre‐determined events are an important aspect of any broadcast organization. Further. Each of these distribution and consumption channels must be serviced. and Acceleration How will field‐generated content be submitted? The most sensible approach is to provide a method for electronic submission. journalist. While the distinction may seem unnecessary. frame rate. viewers have demonstrated that they will go where the content is and will watch it whether it is provided to them on television. on a computer screen.Making Media Move public persons who have happened upon news and event worthy occurrences. composition or how steady or shaky. Getting Footage in From the Field If you are a News Director. Further. but the competition for viewers has never been fiercer. And. Naturally. when—indeed—it is something that is rapidly unfolding. This is especially important whether we are talking about a 16‐station group or a network comprised of 220 affiliates. or videographer. it is important to distinguish between the two groups. the nature of the content and how compelling it is will win out over any technical consideration. Quality Regardless of the quality of the images. This can be done by creating a website which can be specifically branded for each organization in order to maintain station identity. What are some of the most relevant issues that arise in trying to ingest. With so many potential people able to capture breaking news. These distinctions will be outlined within this paper. broadcasters must develop strategies to handle this in‐field generated content. Whether it is a hurricane in Louisiana. or a citizen journalist that has captured footage of an accident. the screen type. review. reporter. But. 3 . the staff gets dispatched to cover it. and distribute in‐field generated content and how is that impacted when it’s not your staff but a pool of citizen journalists equipped with camcorders and cell phones? Content Submission.
Making Media Move Next.. Figure. West Palm Beach. and to check the required legal release. Local Stations in Local Touch Even if we limit our scope to the U. 1: A web page that is branded for a specific organization and prompts the user for metadata and provides for file transfer security and file transfer acceleration. Figure 1 is an example of a submission page along with prompts for metadata and file transfer acceleration. contact telephone number. Corpus Christi. Florida). Texas) to 60 stations (Ion Media Networks. we find that there are still 90 station groups. this practice now encompasses 4 . Users are then prompted to choose the file and then upload it. While this process began with still photos sent via email by viewers. it is likely that you will want to prompt the submitter for some metadata such as a name. who range from 1 station (GH Broadcasting. Many of these station groups have already begun to create methods by which their journalists and Videographers as well as their viewers can upload content.S.
security takes on a greater role than ever before. cutting together stories on laptops. Today. First. the integrity of the file has been hampered—either through defects or due to tampering. a secure and verifiable electronic “affidavit” and “certified delivery” equivalents are necessary. Security Takes On An Even Greater Role In a file‐based workflow centric world. it may be required that the integrity of the data is protected—that is—the actual bits must be verified as exactly the same from sender to receiver. one of the critical enablers of using field‐generated content is mobilizing it over the public Internet. Further. protecting the data in flight by securing the connection (the “wire”) via encrypted datagram is easily accomplished. This is also one of the challenges that must be addressed. Faced with the challenges of remaining local. the payload—in this case. One of the most common is AES at either 64. today. In this way. This level of security can appear to be. If the hash value has been changed. the data—can be encrypted using any number of encryption methodologies. two methods can be employed. These techniques provide for data security and data integrity in transferring data over TCP/IP networks. perhaps. and going to Internet cafes to upload content. First. or 256‐bit key strength. reporters are shooting in the field. over‐designed for the topic being discussed. Also. a secure transmission is highly desired and should be established between the page in the web browser and the web server. In this scenario. blocks of data that are sent can be compared on the receiving end. there are two important issues: security concerns and TCP/IP‐based transfers. Regardless of whether the data is a credit card number or data that represents a video file. if it is necessary to provide additional data protection or to prove that the data has not been tampered with. 5 . One way of accomplishing this is to utilize a Secure Hash Algorithm (SHA) that essentially assigns a fix‐sized bit string to random blocks of data. it is not enough to be a major network affiliate—it is critical to be locally relevant to the viewing community served. This can easily be accomplished by using Transport Layer Security (TLS) or Secure Sockets Layer (SSL). but imagine a news director receiving a video interview of a witness of a tragic international incident or of a lawyer receiving a video testimony from a witness who cannot travel to a courtroom: it will often be necessary to prove that the literal bits of the file have not been tampered with. As the Media and Entertainment (M&E) industry increasingly adopts and transforms to a completely file‐based workflow. how can a news organization receive content that is especially timely but without having a news van on site? Whether it is a member of the public or a staff member. Local television stations are clearly in pursuit of their greatest asset—being local. For example.128.Making Media Move getting video content from the field in a fast and efficient process.
of course. FTP (file transfer protocol) over TCP/IP and its positive acknowledgment requirement places a limit on how quickly content can be sent. Authentication. Figure 2. This means that content is in the clear and that there is also some level of communal sharing of the total available network bandwidth. WAN acceleration technology also favorably assists whenever considerable amounts of latency are introduced at the network level. 6 . SHA‐1 hashing. By clicking on a button in the web page that engages this WAN acceleration technology. and media encryption techniques used to establish data confidentiality and integrity. One such implementation is in the form of a software‐based WAN (wide area network) accelerator. It is also important to consider that use of the public Internet is an open and unsecured network that everyone is sharing. The submission of content using the public Internet also brings with it both file transfer and security issues. several methods to accelerate the movement of content. Figure 3 shows the effects of latency on file transmission times.Making Media Move Figure 2 shows the various security components and methods that can be employed in the transfer of digital files between sender and receiver. There are. content can be uploaded approximately 60% faster over tradition methods such as FTP.
As network latency increases. and can then move on to the next story and location. Payload. Often these are not taken into consideration. For example. there are simple and practical issues that must be understood. While it may appear that any network and protocol will work. file send times using FTP increases dramatically. it is necessary to understand the practical requirements with respect to the bandwidth that is available to the person attempting to upload content. These transfers are typically accomplished using HTTP 7 . These are important factors that must be taken into account in order for any submission system to actually operate. The field reporter then definitively knows that the content arrived. and Latency In order to create a reliable system to receive FGC and UGC. It is. quite common to simply attach a file to an email message. Bandwidth.Making Media Move Figure 3. it is a misconception that any IP network and transmission methodology will suffice in order to send content from the field into a facility. Let’s explore some of the most common methods. of course. Including both security and WAN acceleration technologies into the web‐based upload process can also benefit from providing notification to both submitter and receiver that the content was uploaded and received. that the network connection can be closed. the payload (the amount of data) and the latency of the IP network.
Further.g. one must take into account the following: • • • • Amount of data to be sent Network link speed Network latency Transport protocol The cumulative effects of these four items will indicate how long it will take to receive files from the field.Making Media Move (hypertext transfer protocol). sent data is at risk unless there is some form of protection in place. Checkpointing enables systems to stay in synchronization. a common checkpointing interval is 32 KBytes (kilobytes). must operate according to TCP/IP constructs and is therefore susceptible to the same delay in the request/response process. Network connectivity is a robust 500 mbits/sec. we are attempting to send two hours of DV 25 (Digital Video at 25 megabits/sec [mbits/sec]) material. the amount of data that can be sent decreases over time. 8 . in practical implementations there are simply minimal speeds that must be in place for a viable FGC/UGC solution to provide acceptable and timely results. the person uploading the content) sends data to the server. typically using time intervals or data transferred intervals. as the latency of the network increases. Without this. Let us examine the real effects of latency on a practical set of data. if a file is being sent via FTP and a loss of network connectivity occurs. These settings are completely arbitrary. Because of this request/response process. because. File Transfer Protocol (FTP) is a very typical method of transferring files between systems and it. As latency increases. A common approach of protecting sent data is via checkpointing. Specifically. The resulting file size is 20. However. the entire file must be re‐sent.9 gigabytes (GB). too. How Much Network is Really Required? One of the most important factors to understand in architecting and implementing a solution for transferring FGC and UGC content is to understand all the variables that will potentially affect the user experience. while that may be conceptually true. In Figure 4. a common misconception is that any network connection will suffice. An examination of the graphs and accompanying tables clearly indicates the beneficial aspects of WAN acceleration and latency compensation technology. the ability to send data is directly related to the time that it takes for sent data to be acknowledged. HTTP is the method by which most email‐based transfers occur. This transfer mechanism uses a request‐response method wherein the client (e. for example. The maximum amount of data lost would therefore be equated to the set time interval or set data amount between the systems. While not limited to TCP/IP.
In Figure 5. However. 28 hours are needed to send the file versus six minutes and 34 seconds using WAN acceleration. With zero latency. However.9 GB file will take six minutes to reach its destination. At 264 milliseconds (ms) latency—typically for an intercontinental link—only an additional 34 seconds are required. With a robust network of 500 mbits/sec. note that we have decreased the amount of DV 25 content to be sent to only five minutes for a total of 894 MB. the 20. there is minimal effect on completion times when using WAN acceleration. note the increase in delivery time when latency increases between 40‐ 72 ms (typical for a NY to LA link). At zero latency. shown in the lower table. For example.Making Media Move Figure 4. note that as latency increases. let us now examine the results on transfers when network connectivity is lessened. Contrast this to using TCP as the transport protocol. The reason for this is that the network link is simply not large enough to 9 . WAN acceleration demonstrates clear benefits over TCP transport. Further. delivery times via WAN acceleration or TCP are the same. at six minutes. At 264 ms. note that the effect of network latency is no longer a key factor.
Making Media Move accommodate an increased amount of data packets that could be sent using WAN acceleration techniques. Reducing the amount of data to be sent has no effect on transfer times with either WAN acceleration or standard TCP protocols because network connectivity is so low. 10 . Figure 5.
The effects of latency are negligible with T1 connectivity. but we have increased the network link to 1. what occurs in the real world is often very different. Again. The Tale of the Hotel Room Continuing to refer to Figure 6. note that there is no difference in the delivery times. The following example is indicative of what you are very likely to experience when using software‐ based WAN acceleration. the file size remains constant at 894 MB. while theoretically the delivery times are equal between WAN acceleration and TCP. Enticingly. Figure 6. this example is titled “the tale of the hotel room” and is based on a true story.5 mbits/sec.Making Media Move In the next example (Figure 6). 11 . note that.
This true tale clearly indicates that slower networks (e. 1 mbit/sec) can be of use when latency is a factor. 12 . He tries to upload to a server in the United Kingdom using both FTP and Signiant’s Media Exchange application which includes software‐based WAN acceleration technology. the file uploads at just 60 kilobits / second. or 15 times faster. indeed. this time using Media Exchange which includes WAN acceleration and uploads at 900 kilobits / second. possible and achievable. In this case. despite the 1 megabit / second capability of the network. The hotel WI‐FI connection supports a 1 (one) megabit/second upload and download speed. we begin to see the positive effects of WAN acceleration as latency reaches 72 ms and higher (Figure 7). Now. The file is then re‐sent.Making Media Move A person just finishes capturing an event and goes into a hotel lobby in Oslo.g. theoretical speeds did not matter and a key factor in being able to take advantage of even limited bandwidth (in this case 1 mbit / second) with WAN acceleration is. Norway. There was over 150 ms of latency from Oslo to London over this network link. Using FTP. as we increase the capacity of the network link.
we begin to see the increase in delivery times using TCP transport at the 72 ms level.Making Media Move Figure 7. 13 . As the capacity of the network increases to 100 mbits/sec. notable improvements occur at even 8 ms latency (Figure 8). As both network capacity and latency increase.
5 mbits/sec (OC3). 14 . WAN acceleration delivers consistent results while we begin to see increasing delivery times using TCP transport at the 8 ms level. we can clearly see the benefits of WAN acceleration (Figure 9).Making Media Move Figure 8. With network connectivity at 100 mbits/sec. Finally. as the network capacity increases to 155.
the ability to utilize a network link to its complete capacity reaches a limit. it is critical that some form of checkpointing be implemented. As network capacity decreases and even as latency increases. Here we see the classic “flat lining” of a consistent delivery time using WAN acceleration and the rapid reduction in sustained data transfer using TCP transport. Otherwise. Under these circumstances.Making Media Move Figure 9. Getting The File versus Not Getting It At All The preceding examples indicate that there is a level at which being able to upload content from the field is not affected by WAN acceleration techniques. a loss of network connectivity—regardless of how momentary and as is often the case with data traffic using the Internet—will result in the need to resend the entire 15 .
In these cases. cases where FGC and UGC cannot utilize WAN acceleration. it is important to consider that there have. for that matter receive) content in an accelerated fashion. However. If an organization has a dedicated link to the Internet which provides a guaranteed amount of mbits/sec. However. this is counterproductive to getting any content that is perceived will be in high demand. indeed. cable modem. it may be necessary to offer the ability to automatically handle different protocol requirements. 16 . then that link can be apportioned according to the FGC / UGC needs. checkpointing. Protocol Failover Depending upon how content is being sent to your organization. been instances where an ISP may: • • • Provide the agreed‐upon bandwidth for the length of the connection. It is therefore important for one to be cognizant of these network provider issues. limit or regulate the transfer speed over a period of time. and resumption of a transfer from the point of interruption become critical items to have implemented in the solution. This often occurs as a result of a corporate policy wherein UDP‐based protocols are not allowed to function on the corporate network. it may not be possible to send (or.Making Media Move file. let’s say that a person uploading FGC is doing so using a corporate provided network link. This means that the person uploading the content has been granted the ability to use UDP (User Datagram Protocol) WAN acceleration. it may only be possible to send content via TCP or via HTTP protocols. there are. This could be in many formats. For example. the importance of robust auto‐restart of an interrupted transfer. that is. consider what could occur when content is being sent by either a professional staff member or by a citizen journalist over the open and insecure public Internet and utilizing connectivity to the Internet via an Internet Service Provider (ISP). Instead. naturally—perhaps mobile. The third possibility—stepping down the transfer speed—can have particularly damaging effects to the goal of getting content in from the field. Provide less than the agree‐upon bandwidth for the length of the connection. indeed. It may also occur due to the presence of a firewall or web proxy server between the uploader and the server processing the incoming data. satellite—and so forth. when implementing a solution for FGC and UGC. Obviously. Regulating of Upload and Download Speeds Further impacting the ability to upload (or download) content from the field is the extent to which the sender will have a sustained amount of bandwidth to utilize during the course of the transfer. Step‐down. If this occurs. However.
In this example. does it need to be edited and packaged in some way? Should the packaged piece of content then go to the six US New England affiliates. a transfer could be determined to begin as a UDP transfer and then automatically failover to TCP or to HTTP. By building a web‐based submission and review system. before it airs. the 13 mid‐Atlantic US affiliates or to every affiliate? How is this file movement coordinated? In Figure 1. you will note that in the content reviewer page a menu item shows how content can be “routed” to very specific locations. decisions must be made. one should employ the ability to automatically failover among transport protocols. Is the content relevant? If yes. This can be an email that arrives at the desktop or on a mobile device. to whom? What needs to happen to the content? Can it be sent as raw video? Or. An example of a protocol set‐up screen which indicates automatic failover from UDP to TCP to HTTP transport methods. 17 .Making Media Move In designing a system for processing FGC and UGC. Shown in Figure 10 is a setup profile that indicates what automatic action should be taken as FGC and UGC transfers are processed. In this manner. transfers are determined to begin as UDP‐based and then will automatically failover to TCP and then to HTTP protocols. the key item is that the system should automatically provide this failover technology without human intervention. a workflow automation routine has been designed to carry out these content movement orders. Again. Embedding a link to the media into the email notification allows the reviewer to review footage by invoking the proper media player. Reviewing the Footage and Deciding What Happens Next Now imagine the situation at the news desk when content is electronically submitted either by citizen journalists or by staff members. Figure 10. Next. the news director can be electronically notified that content has arrived and needs to be examined. Behind the scenes.
This content preparation process—intelligent platform‐ specific publishing. public Internet? Further. content type (format) and content naming conventions (what the file was labeled by the content uploader versus what the facility naming convention requires) are two common areas that often need to be addressed before the content can be utilized. Resizing images. All of the ensuing questions and decisions that occur the moment that the news director has reviewed a piece of field‐generated content and determined that it is newsworthy can benefit from workflow automation functionality. If the content needs to be routed to specific sets of affiliates. The Importance of Workflow Automation Getting content in from the field is only one component of an entire workflow that must be accomplished. an affiliate’s playback server? What is the distribution system to facilitate this? Is it FTP? Do we need to accelerate these file transfers? Are we employing less expensive network connections and rather than using expensive satellite time. it is important to interrogate any data that is being brought into the facility as a result of FGC and UGC endeavors. once content has been moved into either the newsroom or the production / post‐production facility. we mustn’t forget that multiple distribution channels are critical and that the content will be needed by the development group in charge of placing the content on the web page and also must service the mobile partners. 18 . can it be done so automatically? Can the content be routed to a drop box or directly into. if you will—can all be automated. encoding. what ensues are multiple uses of the content that must be facilitated in the most efficient manner possible. This may be as straightforward as temporarily quarantining the incoming content for an amount of time that it can be checked for viruses prior to being brought into the main network attached storage (NAS) in the facility. In this example. In fact.Making Media Move Viruses and Content Checking As anyone responsible for data integrity and corporate IT security will attest. Figure 11 is an example of how a series of scripts can be linked together to form an automated workflow. In addition. at a specific timing interval a SOAP (Simple Object Access Protocol) message is generated which initiates a file transfer which then deposits the file(s) into a specific folder destination and then initiates and delivers an email notification of the event. and distribution can all be part of a workflow automation set of routines. are we using the open. say.
The reverse of Aggregate.Making Media Move Figure 11. etc. 19 . of course would be an automatic Distribute mechanism. mobile.g date.) benefits greatly from tasks that can be sequenced and then can be automated. time. These workflows range from relatively simple tasks such as outlined in Figure 11 to routine tasks that fall into easily identifiable categories. etc. Aggregate refers to the need to aggregate content from one or from many sources (as would be the case with FGC or UGC).) and then content becomes available to the recipient. “Transport” is self‐explanatory. The creation of workflow: linking together scripts that perform specific tasks that can be sequenced together to create an automated process. but an imposed embargo of the content is applied until specific conditions are met (e. Modeling Routine and Complex Workflows The ingest of content generated in the field and distributed to any number of venues and screen types (terrestrial. Embargo is an interesting capability as it refers to the distribution of content to one or more locations. satellite. The “Transport S3” workflow that is shown refers to the movement of content to an Amazon S3 (Simple Storage Service) cloud. Shown in Figure 12 are examples of categories that routinely need to be accomplished when media transactions occur.
20 . Figure 12. By using a sophisticated workflow modeling engine (WME). content oriented tasks can be sequenced and be invoked to automatically and programmatically move content according to pre‐set rules and policies that dictate when content should be moved. and within an organization. relatively simple categories of workflows can be combined in order to create more complex workflows. Various categories of workflows that represent routine tasks for an enterprise and which observe the rules and policies by which content flows to. from.Making Media Move Finally. the “Apple FCP Transfer to iTunes” shows how content can be automatically delivered from an Apple Final Cut editing application directly to the Apple iTunes servers for content ingest. and so forth. Further. From Aggregation to Delivery Workflow modeling systems must be flexible enough in order to accommodate and service the ever‐changing business models for today’s media and entertainment industry. what bandwidth utilization should be applied.
An automated workflow framework to allow for flexibility of file‐based workflows. 21 . network bandwidth utilization. A secure. auditing and reporting functionality. we would accomplish an end‐to‐end Aggregation‐to‐Distribution result. Editorial. and resolved to the target editing system. As part of this aggregation cycle. In this case. and If. the workflow calls for automatic email notifications upon successful task completion. The resulting files are then delivered via Signiant agent technology. Once the editing is complete. ultimately. Else. A more complex workflow created using Signiant’s Workflow Modeling Engine. and Exception actions. a Signiant transport call is made and the newly edited content is then delivered to a multi‐stage series of processes. content database and multiple transport delivery protocols are all involved in this automated workflow. 2). applications. the content is then delivered to a folder for ingest into an editing system. a question of business priorities. A centrally managed distribution system that provides routing. and distribution system to support field‐generated content should take into consideration three important areas: 1). transcoding. Evaluating Needs and Implementation A digital ingest. notifications if exceptions are encountered. and database updating. FTP. scheduling. How much of your company bandwidth to use for any given transfer or set of transfers is. review. Finally. if we use the “Aggregate” workflow shown in Figure 12 and combine it with the multi‐step and multi‐process workflow shown in Figure 13. WAN‐accelerated web‐based upload and download capability. Figure 13. These processes are contained in the foundational group which indicates two transcode operations and a transfer to a geographically separated editing system. Shown are linked workflows with multiple processes.Making Media Move For example. our Aggregate template would be used to process ingest of FGC or UGC. 3). Having proper controls to determine this is important.
Figure 14. Conclusion In today’s competitive broadcast and media environment. 22 . He is an Academy Award® and two‐time Emmy® award recipient. packaging and distribution methodology. and a network bandwidth management tools. But there is a wide range of activities that may occur from the moment that an event is captured to the time and places that it is viewed. it is now a reality that anyone with a camera. review. locations of site transfers. A dashboard view that shows digital media transfers. The technologies to accomplish these tasks and to implement them have never been timelier. a phone. and an Internet connection can be a broadcaster. Getting breaking news “on the air” means getting it on the air over many device and screen types and demands the implementation of a coordinated file ingest. Signiant Content Distribution Management (CDM) software addresses these needs.Making Media Move Figure 14 is an example of a dashboard view that provides real‐time feedback of digital media transfers and bandwidth control functionality. Tom Ohanian is Chief Strategy Officer at Signiant. processing.
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue listening from where you left off, or restart the preview.