Using gzip compression when making HTTP requests in Windows Phone apps

Bandwidth and data consumption on mobile devices remain one of the most critical resources an app consumes on any platform. A majority of users have limited data plans and actually pay more if they use more data on their mobile devices. This makes it critical for us, as app developers, to ensure our apps consume as less data as possible. And when apps download less data, they perform better and become more responsive too.

I overlooked this myself earlier but fixed it as soon as I noticed. In fact, a lot of apps is store are not using compression including some heavily used apps like famous YouTube clients. The response size for 10 entries in most popular list from YouTube is ~50KB by default and ~7.4KB when compression is used, that’s 85% less data downlaoded. It varies for each request but the savings are somewhere from 30% to 85%.

Overview

We all use compressed archives on PCs in one format or the other like zip, rar, 7z and others. For the web, what it really means is that the data being transferred over the network is compressed and hence uses less bandwidth. This is defined in http spec as a standard. When a client, like your app, sends a request, it can tell the server that it’s capable of handling compressed response in Accept-Encoding request header. The server then sends the response in compressed format (if it’s capable of doing so) and tells the client using Content-Encoding response header. Obviously, the server and client both not only need to support this but also send consent that this capability is used. This is how http works for everything so no surprises there.

Good news is that you don’t have to implement compression to take advantage of this as the HttpClient library you should be using added supports for this over a year ago. Unfortunately, this wasn’t the case in early days of Windows Phone but workarounds existed. Ideally, the API should use compression by default but it doesn’t.

Clarification

The HttpClient I mentioned above is for Windows Phone 7.x and 8.x apps using Silverlight not WinRT. The .NET stack has had multiple APIs for making web request overtime but HttpClient is the one everyone should be using now which is wrapper/replacement for all other HTTP APIs in .NET framework.

Starting with WinRT which is used for developing “universal” apps, there is a separate HttpClient API. This seems to be using compression by default but please verify that it is. Here is a good starting point for HttpClient in WinRT and samples. If this all seems very confusing, you are not alone. The HTTP API in Microsoft world is a mystery wonderland of confusion so ask if you have questions.

Implementation

Using compression in your app is extremely simple and quick as explained by the framework team. When you create an instance of HttpClient class, send an instance of HttpClientHandler in constructor with appropriate setup and you are done. All http requests sent using this instance will now send correct request header and automatically un-compress the response.

var handler = new HttpClientHandler();
if (handler.SupportsAutomaticDecompression)
{
    handler.AutomaticDecompression = DecompressionMethods.GZip |
                                     DecompressionMethods.Deflate;
}

var client = new HttpClient(handler);

 

Performance Tip

While you are at it, here is another performance tip. When you are downloading data from internet and deserializing it, instead of first storing response as a string and then parsing it, parse data directly from the response stream. This works seamlessly for JSON as well as XML content.

For JSON, JSON.NET supports deserializing data from stream. It’s not as obvious as deserializing from a string though. Using the HttpClient instance we created above, here is a code snippet to deserialize response stream.

try
{
    using (var stream = await httpClient.GetStreamAsync(url))
    {
        using (var streamReader = new StreamReader(stream))
        {
            using (var textReader = new JsonTextReader(streamReader))
            {
                var serializer = new JsonSerializer();
                var data = serializer.Deserialize<List<Entry>>(textReader);
            }
        }
    }
}
catch (ObjectDisposedException ex)
{
    // ignore this exception as stream gets closed somehow and "using" block tries to dispose it again
    Debug.WriteLine(ex);
}

For XML response, XDocument supports parsing streams directly.

using (var stream = await httpClient.GetStreamAsync(url))
{
    XDocument.Load(stream);
}

For using stream with JSON.NET, I came across an issue though. As I am using “using” constructs to ensure the resources get cleaned up, the stream gets closed twice which causes an exception. To workaround that, ignore ObjectDisposedException exception as I have done above.

One drawback of using compression is that the phone will have to spend some CPU cycles to uncompress data. With today’s phones though, this is negligible. There is no reason to not use compression in your apps.

Hope this helps.

image credit: wpcentral

Notifying users when app update is available in Windows Phone store

Ensuring that users of your app are on latest version is critically important. I have seen users reporting issues and requesting features which were implemented a few updates ago. With Windows Phone 8.1, users can enable automatic app updates installation but for users who are still on Windows 8 or who do not enable automatic updates, the responsibility is on you, as a developer.

Unfortunately, there is no support in Windows Phone SDK to find out if there is a new version of the current app in store or what version is available in store. For the converge app, I was looking to solve this issue so I traced store app’s network traffic using Fiddler to understand the available APIs. Looking at the network traffic, the API used by store app is fairly obvious. Here is the API request Windows Phone 8 store app sends for app details.

GET http://marketplaceedgeservice.windowsphone.com/v8/catalog/apps/b658425e-ba4c-4478-9af3-791fd0f1abfe?os=8.0.10521.0&cc=US&lang=en-US&hw=520208901&dm=RM-940_nam_att_200&oemId=NOKIA&moId=att-us&cf=99-1
Accept: */*
Accept-Encoding: gzip
User-Agent: ZDM/4.0; Windows Mobile 8.0
X-WP-Client-Config-Version: 107
X-WP-MO-Config-Version: 1224
X-WP-Device-ID: ****
X-WP-ImpressionId: ****:API.BDIGeneric
X-WP-ImpressionK: 5

You don’t need to send all headers or parameters in requests. The response to this call has complete details for your application in the target country/language in xml format. From here on, you can use LINQ to XML to get the data you need to know what version is available in store. Here is slightly modified version of the code I use in the converge app. Feel free to reuse.

//This code has dependency on Microsoft.Net.Http NuGet package.
public async Task CheckUpdate()
{
    const string storeAppDetailsUri = "http://marketplaceedgeservice.windowsphone.com/v8/catalog/apps/b658425e-ba4c-4478-9af3-791fd0f1abfe?os={0}&cc={1}&lang={2}";

    var updatedAvailable = false;

    try
    {
        var osVersion = Environment.OSVersion.Version.ToString(4);
        var lang = CultureInfo.CurrentCulture.Name;
        var countryCode = lang.Length == 5 ? lang.Substring(3) : "US";

        using (var message = new HttpRequestMessage(HttpMethod.Get, string.Format(storeAppDetailsUri, osVersion, countryCode, lang)))
        {
            message.Headers.Add("User-Agent", "Windows Mobile 8.0");

            using (var client = new HttpClient())
            {
                var response = await client.SendAsync(message);
                if (response.StatusCode != HttpStatusCode.OK) return;

                using (var stream = await response.Content.ReadAsStreamAsync())
                {
                    XNamespace atom = "http://www.w3.org/2005/Atom";
                    XNamespace apps = "http://schemas.zune.net/catalog/apps/2008/02";

                    var doc = XDocument.Load(stream);
                    if (doc.Document == null) return;

                    var entry = doc.Document.Descendants(atom + "feed")
                        .Descendants(atom + "entry")
                        .FirstOrDefault();

                    if (entry == null) return;

                    var versionElement = entry.Elements(apps + "version").FirstOrDefault();
                    if (versionElement == null) return;

                    Version storeVersion;

                    if (Version.TryParse(versionElement.Value, out storeVersion))
                    {
                        var currentVersion = new AssemblyName(Assembly.GetExecutingAssembly().FullName).Version;

                        updatedAvailable = storeVersion > currentVersion;
                    }
                }
            }
        }
    }
    catch (Exception ex)
    {
        // HANDLE ERROR HERE. THERE IS NO POINT IN SHOWING USER A MESSAGE. GOOD PLACE TO SEND SILENT ERROR REPORT OR JUST SWALLOW THE EXCEPTION.
        Debug.WriteLine(ex);
    }

    if (updatedAvailable)
    {
        // APP UPDATE IS AVAILABLE. SHOW FIREWORKS IN APP OR JUST OPEN STORE APP TO UPDATE THE APP AFTER USER'S CONSENT.
    }
}

This is definitely not a documented and supported API  but considering there are millions of devices using this API, I doubt this will break in foreseeable future. I just don’t understand why Windows Phone team doesn’t make this API public and have folks build stuff on top of it. They have struggled to produce a good store experience since Windows Phone launched anyways. For Windows 8, the APIs are as useless as the store app is so don’t bother trying to do something similar there.

A side note: regardless of how good you think the updates are, don’t update the app too frequently (multiple times in a week)  if you have this notification implemented in the app. This becomes annoying and users start reporting it in reviews. This happened when I had to release multiple updates in a week to resolve content infringement complaints submitted by The Verge.

Hope this help.

The Verge app for Windows Phone

If you are part of the technology world, there is a good chance you read The Verge for getting your fix of news, reviews and their fantastic long reads not only related to technology but also science, art and culture. If want to do that on a Windows Phone though, there are no good options. Until now.

The Verge

Over the last few weekends, I have built a new app for Windows Phone for consuming everything The Verge has to offer. Here are the features it currently supports.

  • Super smooth experience for browsing and reading all the latest news, reviews, features, and exclusives.
  • Read comments posted by The Verge readers.
  • Watch “On The Verge,” “The Vergecast,” “Small Empires,” and more without ever leaving the app.
  • Share news and stories with your friends via Facebook, Twitter, and email.

More features like forums and live tiles are not implemented in the app just yet but I plan to add these over the coming weeks. Feel free to suggest or vote on feature requests on the feedback site. Here is some feedback from the community so far.

Great. The best by far. Great design. Better and more good looking than the android app.

As much as most of The Verge crew don’t like WP, this is a great app to prove how good it can be.

I suddenly feel like we don’t need an official app anymore. Awesome.

Obviously, the app is neither sponsored nor endorsed by The Verge or Vox Media.

Khan Academy on Windows 8: Now Open Source

Khan Academy app launched with Windows 8 last October. A few of us worked together to get the first version developed just in time for Windows 8 launch. Since then, there have been a few minor updates and a big overhaul which was published in store last month. This week is another great milestone: the app is now open source and ready for anyone to contribute.

KhanAcademyInStore

How We Got Here

I developed the initial version with a lot of help from folks over at Khan Academy and an internal Microsoft team. It was really a port of Learning Circle phone app to Windows 8 so naturally, this was developed using C# and XAML.

Later last year, a few more folks from Microsfot and BGC3 stepped up to help and we got some funding. That helped to get Pixel Lab team involved with development and after working on it for a few months,  the next version of the app was published in store last month. This new version was developed in HTML5/JavaScript and this is what we open sourced this week.

Get Involved

If you have experience in developing Windows 8 apps using HTML5 and JavaScript, come join us in making a great Khan Academy app for Windows 8. In addition to HTML5 and JavaScript, having good understanding of WinJS, WinRT, TypeScript and web in general would be great. If you don’t have development experience, you can still help with testing.

Let’s Go!

Using and Customizing Player Framework in Windows 8 Xaml Apps

If you are creating media apps targeting Xaml frameworks i.e. apps which playback media especially videos, you start with MediaElement control. As you start using it, you’d probably search for properties to enable playback controls (play, pause, seek bar et al) to learn that there are none. That’s right, MediaElement just plays video in a rectangle for an assigned source and you are supposed to build everything else yourself.

Using MediaPlayer from Player Framework

Depending on your requirements, building a video player can be difficult and time consuming. But before you start building one yourself, you should look at Player Framework on codeplex, a set of great player controls for Windows 8, HTML5, Silverlight and Windows Phone. The project is managed by Microsoft and being continuously improved upon. The player controls in this framework have rich set of features and you probably won’t have to build one yourself. To get started, check out the documentation and samples on codeplex to learn more about all the features it supports including IIS smooth streaming, closed captioning, ads and more.

MediaPlayer control in the player framework comes with inbuilt playback controls which are good to start with. Refer the getting started documentation for using player framework in your app. Here is the experience provided by the default controls.

DefaultPlayerControls

Using Windows 8 Style Playback Controls

For Windows 8 apps, you’d want to use playback controls with a modern Windows 8 user experience with full screen overlay controls similar to the the default Video app or Khan Academy app. The MediaPlayer control ships with an entertainment theme for getting better playback controls. To enable entertainment theme, all we need to do is add entertainment theme to our page where we are using MediaPlayer control using Xaml markup as shown below. Checkout the documentation on Applying Theme for more customization options.

<Page.Resources>
    <ResourceDictionary>
        <ResourceDictionary.MergedDictionaries>
            <ResourceDictionary Source="ms-appx:///Microsoft.PlayerFramework/themes/EntertainmentTheme.xaml"/>
        </ResourceDictionary.MergedDictionaries>
    </ResourceDictionary>
</Page.Resources>

With the entertainment theme, you get solid UI improvement over default playback controls.
EntertainmentThemeControls

Customizing Template of Playback Controls

Now that we have good understanding of using MediaPlayer control and it’s out of the box entertainment theme, let’s look at the options to customize the playback control UI. The MediaPlayer control is designed using MVVM pattern which means that the views are completely separate from the behavior and the views are template driven. This makes the control extremely flexible for customizations as you can define a completely new template for playback controls and replace the default template. In fact, that’s what we did above by replacing the default template with the one in entertainment theme. Let’s customize the entertainment theme playback controls further and use them in our app. To start, we’ll make a copy of the entertainment theme in our app and customize the template. As an example here, we will use dark red color for the slider pushpin rather than the default color.

  • First, locate the EntertainmentTheme.xaml which is installed with player framework and also gets copied to your app’s output directory during compilation at <approot>\bin\Debug\AppX\Microsoft.PlayerFramework\Themes folder. You can also get it from installation folder located at <drive>:\Program Files (x86)\Microsoft SDKs\Windows\v8.0\ExtensionSDKs\Microsoft.PlayerFramework.Xaml\<version>\ Redist\CommonConfiguration\neutral\Microsoft.PlayerFramework\Themes.
  • Once you have located the file, add it you project in visual studio. If you don’t know how, use either of these options
    • from Visual Studio, use “Add Existing Item” item from context menu in solution explorer, or
    • copy the file to you app folder in file explorer, enable “Show All Files” from the toolbar in solution explorer and use “Include In Project” item from context menu for the file you need to add.
  • Update the Xaml markup of the player page shown above to use the EntertainmentTheme.xaml from project instead of player framework dll.
    <Page.Resources>
        <ResourceDictionary>
            <ResourceDictionary.MergedDictionaries>
                <ResourceDictionary Source="Common/EntertainmentTheme.xaml"/>
            </ResourceDictionary.MergedDictionaries>
        </ResourceDictionary>
    </Page.Resources>
    
  • At this point, the control panel UI is rendered using template defined in this file in the project. You can make any changes here to customize the control panel and the changes would reflect in UI. Let’s change the color of the pushpin for our example.
    <Grid>
        <Ellipse Stroke="{StaticResource ButtonForegroundThemeBrush}" Fill="DarkRed" StrokeThickness="2" Width="35" Height="35"/>
        <Path Fill="{StaticResource ButtonForegroundThemeBrush}" Margin="0,0,0,-7" VerticalAlignment="Bottom" HorizontalAlignment="Center" Data="M 0,0 8,0 4,8Z"/>
    </Grid>
    

We changed the Fill value for the Ellipse to dark red in this example. This demonstrates how to make changes in the template.

Let’s take customization to the next level. The behavior and visibility of playback controls are driven by a ViewModel. In the entertainment theme template we changed above, you must have noticed that bindings are used to control visibility. The control panel uses InteractiveViewModel class in player framework which has a number of properties to control visibility of all the elements in playback controls. I highly recommend going through the type definition for InteractiveViewModel to understand the available properties to use in your custom template. Refer API documentation.

If you have full screen video player in your app, you might want to add a Back button to the player screen. This can be added separately to you page but we can take advantage of MediaPlayer’s extensibility. Using this approach, we can also control the visibility of Back button as per playback controls visibility on user interactions. To achieve this, we’ll have to add the Back button to the ControlPanel template in ExtertainmentTheme.xaml we added to the project above.

Another essential requirement might be to show video title for the currently playing video. Again, we can achieve this by putting a TextBlock on the page but then we will have to handle the visibility of the TextBlock ourselves. Instead of doing that, we’ll take advantage of MediaPlayer’s extensibility and add the TextBlock to the template. This has an issue though as we don’t have a property in InteractiveViewModel which can be bound to the Text property of the TextBlock. We definitely don’t want to hard code this. To resolve this, we will create a subclass of InteractiveViewModel and add the additional property in that. In the new ViewModel, we’ll also add a command for binding it to Back button. Once we have the ViewModel, all we’ll have to do is to wire up the custom ViewModel to the MediaPlayer control.

First, let’s go back to the ControlPanel template and add the title TextBlock and Back button to the playback controls template.

<Grid>
    <Grid.RowDefinitions>
        <RowDefinition Height="Auto"/>
        <RowDefinition Height="*"/>
        <RowDefinition Height="Auto"/>
    </Grid.RowDefinitions>

    <Grid x:Name="HeaderContainer" Background="{StaticResource ItemBackgroundThemeBrush}" Visibility="{Binding IsTimelineVisible, RelativeSource={RelativeSource TemplatedParent}, Converter={StaticResource VisibleIfConverter}}">
        <Grid.ColumnDefinitions>
            <ColumnDefinition Width="Auto" />
            <ColumnDefinition Width="*" />
        </Grid.ColumnDefinitions>
        <Button Command="{Binding GoBackCommand}" Style="{StaticResource BackButtonStyle}" Margin="36,20" VerticalAlignment="Center" />
        <TextBlock Grid.Column="1" TextWrapping="NoWrap" Text="{Binding VideoTitle}" Style="{StaticResource SubheaderTextStyle}"
            Foreground="{StaticResource ApplicationForegroundThemeBrush}" VerticalAlignment="Center" FontSize="40"/>
    </Grid>

    <StackPanel Grid.Row="1" Orientation="Horizontal" VerticalAlignment="Center" HorizontalAlignment="Center">

Notice we are using bindings to set the Text for TextBlock and Command for Button. Next, let’s create the custom ViewModel with properties we can use for binding.

public class CustomInteractiveViewModel : InteractiveViewModel
{
    public CustomInteractiveViewModel(Action goBack, MediaPlayer player)
        : base(player)
    {
        _goBackCommand = new DelegateCommand(goBack);
    }

    private string _videoTitle;
    public string VideoTitle
    {
        get
        {
            return _videoTitle;
        }
        set
        {
            if (_videoTitle != value)
            {
                _videoTitle = value;
                base.OnPropertyChanged("VideoTitle");
            }
        }
    }

    DelegateCommand _goBackCommand;
    public DelegateCommand GoBackCommand
    {
        get
        {
            return _goBackCommand;
        }
    }
}

In the player page, let’s add the MediaPlayer control to the Xaml markup and assign an instance of the CustomInteractiveViewModel to InteractiveViewModel property of the MediaPlayer control to wire up the custom ViewModel to the MediaPlayer.

<Page x:Class="CustomizingPlayerDemo.PlayerPage"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="using:CustomizingPlayerDemo"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    xmlns:mmppf="using:Microsoft.PlayerFramework" mc:Ignorable="d">

    <Page.Resources>
        <ResourceDictionary>
            <ResourceDictionary.MergedDictionaries>
                <ResourceDictionary Source="Common/EntertainmentTheme.xaml"/>
            </ResourceDictionary.MergedDictionaries>
        </ResourceDictionary>
    </Page.Resources>
    <Grid Background="{StaticResource ApplicationPageBackgroundThemeBrush}">
        <mmppf:MediaPlayer x:Name="player" Source="http://download.ted.com/talks/BillGates_2010-480p.mp4?apikey=TEDDOWNLOAD"/>
    </Grid>
</Page>

 

using CustomizingPlayerDemo.Common;
using Windows.UI.Xaml.Controls;

namespace CustomizingPlayerDemo
{
    public sealed partial class PlayerPage : Page
    {
        public PlayerPage()
        {
            this.InitializeComponent();

            player.InteractiveViewModel = new CustomInteractiveViewModel(() => this.Frame.GoBack(), player)
            {
                VideoTitle = "Bill Gates on energy: Innovating to zero!"
            };
        }
    }
}

This gives us a fully customized playback controls UI for player framework MediaPlayer control. This is the same approach I have used in Khan Academy app as well.

CustomizedPlayerControls

The controls we have added to the ControlPanel template are part of the player so we get visibility transitions for these without any extra work. You can download the source code of this sample to see how it works. Please don’t forget to install the player framework using links above before you try to run the sample.

Update:

Hope this helps.