Several years ago, it was unheard of to deliver media content to end users across large distances and at high speeds, while still maintaining quality. But modern technologies and telecommunication facilities have progressed to the point where we can now achieve such goals. For example, the quality of audio and video streaming across the Internet has greatly improved. But despite recent developments in the telecommunication environment, some problems still persist. A major problem with media applications is interactivity and response time to user actions. Media application developers are increasingly interested in infrastructure optimization solutions that help users run such applications more efficiently—especially high-load applications.
Microsoft IIS Media Services seeks to solve these and other problems with media applications on the Internet. IIS Media Services is essentially a set of extensions for Microsoft Internet Information Services (IIS), which is an application server included in Windows Server 2008. Both IIS and IIS Media Services are free products that greatly improve media content delivery to end users. This article explores the development of media applications that use IIS Media Services.
Delivering Media Content to End Users
Before we delve into a discussion of IIS Media Services, let’s look at current approaches to media content delivery. Existing technologies for delivering media content include traditional streaming and progressive download.
Traditional streaming. Traditional streaming is based on constant transfer of an information stream from a server to a client. After a client connects to a source, the server constantly broadcasts a stream for the client. If it’s necessary for the client to control the playing process, the client sends commands to the server to specify particular actions. For example, the client can send the server a command to seek and play for a particular number of seconds, or a command to stop and resume playing. The server receives these commands and alters the stream that is transferring to the client.
This approach undoubtedly has advantages. First, using a traditional data streaming transfer channel is quite effective, because only the portion of the media content that should be played immediately is loaded. Media content that will play later isn’t loaded. Another advantage of this approach is the ability to construct a responsive user interface. Because the data transfer channel is used only to transfer currently playing content, seek or resume operations can be executed in a short period of time. In addition, this approach lets you trace user activity on the server side. As the client sends seek, stop, and resume commands, the server can collect this information for future analysis. Such information can be useful for media content owners.
Traditional streaming also has disadvantages. The main disadvantage of this approach is its complex infrastructure construction. Because each client maintains a connection with the server during the entire video viewing session, the server’s infrastructure organization has certain imposed restrictions. The client/server connection is a limited resource that you must consider separately. Thus, you can’t organize your infrastructure based on the classic web server approach. Moreover, traditional streaming doesn’t let you use existing caching mechanisms.
Progressive download. Progressive download uses an existing web server infrastructure. The progressive download of media content is similar to downloading a file from a web server. However, a client can start playing a file before it's downloaded. The player actually loads part of a file, which begins playing on the client, while the rest of the file is downloading.
Progressive download has several advantages over traditional streaming. The main advantage is the ability to use an existing infrastructure to place content. That is, you can place media content on the Internet using an existing web server via the HTTP protocol. In addition, you can scale a similar solution using already existing mechanisms (for example, HTTP caching).
Nevertheless, progressive download also has a number of disadvantages. Because this approach is similar to a typical file download, buffering comes into play. When a network is fast enough to keep up with playback, buffering is not necessary; however, where factors such as line status, sudden changes in a server's load, or simultaneous downloads on the client side may lead to an unstable bandwidth, buffering becomes absolutely necessary because any moderate drop in connection speed under the amount required by the stream could interrupt the user's viewing experience.
With progressive download it's also impossible to receive the analytical information about a media content viewing. As the player downloads contents as an ordinal file, a server cannot trace what part of the file has been watched. A more serious problem posed by progressive download is excessive traffic consumption. This problem is more common for users with a high-speed Internet connection. With a high-speed connection, a user can download and begin watching a video very quickly. However, if the user stops watching the video after a few seconds, the full download of the file is wasted and the data transfer channel is used inefficiently.
Despite its disadvantages, progressive download is the most popular media transfer technology. The cost of deploying and maintaining a server infrastructure is lower for progressive download than for traditional streaming.
IIS Media Services
IIS Media Services are extensions for IIS web server that allow you to organize the delivery of media content to an end user more effectively. These extensions include tools for more effective delivery of media content using progressive download, and for an alternative approach to delivering media content: adaptive streaming, which lets the client "adjust" how the video is played.
To work with IIS Media Services, you need to download and install the free extensions. The most convenient way to do this is to use the Web Platform Installer (Figure 1).
Let's look at the two approaches to video streaming that IIS Media Services supports: progressive download and adaptive streaming.
Progressive Download
IIS Media Services offers improvements to progressive download via Bit Rate Throttling and Web Playlists.
Bit Rate Throttling - Bit Rate Throttling, which addresses the problem of inefficient use of traffic during video viewing, lets a client define a download speed. No speed restriction is set initially; thus, the client can download quickly at first and begin playing a video. After the initial download, the download will be limited by some fixed speed. If a user stops viewing the video after a few seconds, the server does not download the rest of the file.
You can set Bit Rate Throttling based on a media file's size (in kilobytes) or length (in seconds); for example, you can specify that the first 200KB of a file or the first 20 seconds be downloaded. You also can specify download speed in absolute and relative values; for example, you might specify an absolute value of 300 Kbps or a relative value based on the quality of the media file (bit rate). Figure 2 shows sample settings for Bit Rate Throttling.
You also can specify different settings for different file types, as shown in Figure 3. Because you can control how quickly a file is downloaded, Bit Rate Throttling lets you use the data transfer channel more effectively.
Web Playlists. One way media content providers generate revenue is through sale of advertising. Web Playlists offer powerful customization features that make it possible to monetize media delivery scenarios by inserting in-stream or pre-roll video ads and dynamically determining the content to be delivered based on the session history and configurable server-side rules. Web Playlists are generated on the server side and a unique URL is transferred to the client. This unique URL is used by an end user to watch all media files included in this playlist. You activate Web Playlists as you would other IIS Media Services extensions in Figure 4.
To set up playlists (Figure 5), you specify the playlist file name (to generate the final URL), the title of the playlist, the description of the playlist, and the list of media files included in the playlist.
When you add a media file to a playlist (Figure 6) you need to specify the file's address (you can use a relative URL, an absolute URL, or a physical path). You also can choose options to disable skip forward or seek, which guarantee that an advertisement will be watched.
After you set up a playlist, an .isx file containing the description of the playlist is createdin Figure 7. This description, created as XML, contains information about all elements of the playlist.
Adaptive Streaming
The most important capability of IIS Media Services is support for adaptive streaming, a technology that adapts image quality to the current client environment (e.g., channel throughput and CPU load). This capability mitigates what is known as the "last mile challenge"—a scenario that occurs when a client has a slow Internet connection, bandwidth congestion, or overburdened CPU. Rather than slowing the video download, in each case a lower-quality video image will be downloaded; once the Internet connection, bandwidth congestion, or CPU load improves, a higher-quality image is downloaded. As a video plays, the player continues to adapt to the current client environment, switching from low to high quality as needed.
Smooth Streaming
Smooth streaming attempts to combine the best of progressive download and traditional streaming to provide a powerful platform for video streaming. With smooth streaming the player makes decisions about the quality of a stream to play based on client heuristics. These heuristics take into account the speed of a stream transfer and the CPU load. If the client environment changes while the stream is transferring, the player will switch transfer to a lower-quality video automatically. When the current environment improves, the player switches to a more high-quality video.
Let's look in detail at how smooth streaming works. You use IIS web server as the infrastructure and HTTP as the protocol. To set up adaptive streaming on a server, you need versions of differing quality for each video. You can define these versions when a video is encoded. With different quality versions to choose from, a player accesses a server via HTTP and requests a small fragment of a media file, usually two seconds in length (you can set different values for this parameter during media content encoding).
Initially, a low-quality video stream is downloaded. This fragment is quickly delivered to the client and the player begins playing the video. The subsequent requests for video-content quality are based on the transfer speed (time of delivery of the last fragment) and the CPU load. These characteristics are defined by heuristic algorithms built into a player on the client side. The next client request downloads a small fragment of media content and plays it. This process is repeated until video playing won't be stopped. Thus, during various moments of time video content of various qualities can be played, depending on the current user’s environment.
Because all interactions occur through HTTP the data transferred from a server to a client are typical HTTP objects. In other words, transferring media content is no different from transferring other data via HTTP. Therefore, there are no restrictions with firewalls and other network mechanisms to limit delivery. In addition, the data can be written in a cache on intermediate servers that support HTTP caching. For example, such caching server can located at the ISP or in a corporate network. When a user is watching media content, fragments of the content can be cached on a local server. Thus, other network users can receive data and watch the video much more faster.
During video content encoding, a set of video streams is prepared in advance in various extensions (.ismv), as well as a manifest file (.ism). The manifest contains information about the media content, as well as a list of encoded video streams. With this information, a client can request media content in various qualities. The file manifest is used as a starting point to get access to media content. Certain rules of access apply. For example, to receive a media resource description a client must get access to a .ism file with a «Manifest» key: «http://../media.ism/Manifest». In this case, the client will get a description of the media resource in XML format in Figure 9.
At the beginning of the manifest is a set of rules for URL generation to access a media resource. Typically, a standard template—«QualityLevels(\\{bitrate\\})/Fragments(video=\\{start time\\})»--is used. In this template, instead of \\{bitrate\\} the demanded quality of a required fragment is substituted and in \\{start time\\} the time shift of a demanded fragment is set. This template is used by a player to receive requested media content. Following this template, the manifest describes accessible video qualities.
While playing media content the client and server stay in touch constantly. It follows from the principle of adaptive streaming that it's then possible to trace user actions and build various analytics.
Encoding
As noted, smooth streaming requires that you prepare, in advance, videos of varying quality. Microsoft Expression Encoder, a Windows-based program for encoding digital video for Silverlight-based web videos, helps you prepare and publish media content in Figure 10.
You can use Expression Encoder to create video compressed as VC-1, as well as H.264, for distribution via Silverlight. Expression Encoder features a WPF-based GUI as well as a command-line interface. It supports Silverlight player controls and Silverlight templates.
During encoding you first must define the quantity of video streams that will be encoded in various qualities. For example, you define and set options for seven video streamsin Figure 11. You also can define how video streams will be stored—each stream in a separate file or all streams in the same file. These two scenarios are useful in different situations.
As described earlier, after making a request to a server a player downloads a small fragment of media content. A typical fragment is two seconds long, but you can redefine this parameter and set other values in Figure 12.
After encoding an initial file in smooth streaming format, you can publish it to a web server. Expression Encoder creates everything needed to play a file, including a player and a HTML code. Files of the manifesto, options, and all files of video streams are published in various qualities. Thus, you can specify a template according to which all media files will be named. By default, this template is defined as «\\{Original file name\\}.\\{Default extension\\}».
Finally, you can choose a player template for publishing media content. Expression Encoder contains a considerable quantity of the built-in styles that can be used in applications in Figure 14. Undoubtedly, in the applications it is possible to redefine style of this player and set it according to the user interface of the application.
Smooth Streaming Wins Out
Smooth streaming, an implementation of adaptive streaming, has become a leading concept among technologies for delivering media over the Internet.
At the beginning of this article we considered other approaches to delivering media content to the end user: traditional streaming and progressive download. Both of these approaches have advantages and disadvantages. Traditional streaming allows a user to work more conveniently with video, to collect analytics, and to effectively use the data transfer channel. But traditional streaming has a major disadvantage: the cost of an infrastructure organization for streaming. On the other hand, progressive download, which uses a typical web server infrastructure, solves this problem. Progressive download has other problems, though. Most significant are the unfriendly user interface, inefficient use of the data transfer channel, and no possibility of collecting data for analysis.
Smooth streaming technology attempts to integrate positive aspects of traditional streaming and progressive download to bring media content streaming on the Internet to a new level of quality. Smooth streaming also has the ability to scale. Thanks to effective methods of HTTP caching, it's possible to organize the transfer of media content in large amounts. And smooth streaming technology lets you obtain analytical data about media content viewing.
