Khan Academy on Windows 8: Now Open Source

Khan Academy app launched with Windows 8 last October. A few of us worked together to get the first version developed just in time for Windows 8 launch. Since then, there have been a few minor updates and a big overhaul which was published in store last month. This week is another great milestone: the app is now open source and ready for anyone to contribute.

KhanAcademyInStore

How We Got Here

I developed the initial version with a lot of help from folks over at Khan Academy and an internal Microsoft team. It was really a port of Learning Circle phone app to Windows 8 so naturally, this was developed using C# and XAML.

Later last year, a few more folks from Microsfot and BGC3 stepped up to help and we got some funding. That helped to get Pixel Lab team involved with development and after working on it for a few months,  the next version of the app was published in store last month. This new version was developed in HTML5/JavaScript and this is what we open sourced this week.

Get Involved

If you have experience in developing Windows 8 apps using HTML5 and JavaScript, come join us in making a great Khan Academy app for Windows 8. In addition to HTML5 and JavaScript, having good understanding of WinJS, WinRT, TypeScript and web in general would be great. If you don’t have development experience, you can still help with testing.

Let’s Go!

Using and Customizing Player Framework in Windows 8 Xaml Apps

If you are creating media apps targeting Xaml frameworks i.e. apps which playback media especially videos, you start with MediaElement control. As you start using it, you’d probably search for properties to enable playback controls (play, pause, seek bar et al) to learn that there are none. That’s right, MediaElement just plays video in a rectangle for an assigned source and you are supposed to build everything else yourself.

Using MediaPlayer from Player Framework

Depending on your requirements, building a video player can be difficult and time consuming. But before you start building one yourself, you should look at Player Framework on codeplex, a set of great player controls for Windows 8, HTML5, Silverlight and Windows Phone. The project is managed by Microsoft and being continuously improved upon. The player controls in this framework have rich set of features and you probably won’t have to build one yourself. To get started, check out the documentation and samples on codeplex to learn more about all the features it supports including IIS smooth streaming, closed captioning, ads and more.

MediaPlayer control in the player framework comes with inbuilt playback controls which are good to start with. Refer the getting started documentation for using player framework in your app. Here is the experience provided by the default controls.

DefaultPlayerControls

Using Windows 8 Style Playback Controls

For Windows 8 apps, you’d want to use playback controls with a modern Windows 8 user experience with full screen overlay controls similar to the the default Video app or Khan Academy app. The MediaPlayer control ships with an entertainment theme for getting better playback controls. To enable entertainment theme, all we need to do is add entertainment theme to our page where we are using MediaPlayer control using Xaml markup as shown below. Checkout the documentation on Applying Theme for more customization options.

<Page.Resources>
    <ResourceDictionary>
        <ResourceDictionary.MergedDictionaries>
            <ResourceDictionary Source="ms-appx:///Microsoft.PlayerFramework/themes/EntertainmentTheme.xaml"/>
        </ResourceDictionary.MergedDictionaries>
    </ResourceDictionary>
</Page.Resources>

With the entertainment theme, you get solid UI improvement over default playback controls.
EntertainmentThemeControls

Customizing Template of Playback Controls

Now that we have good understanding of using MediaPlayer control and it’s out of the box entertainment theme, let’s look at the options to customize the playback control UI. The MediaPlayer control is designed using MVVM pattern which means that the views are completely separate from the behavior and the views are template driven. This makes the control extremely flexible for customizations as you can define a completely new template for playback controls and replace the default template. In fact, that’s what we did above by replacing the default template with the one in entertainment theme. Let’s customize the entertainment theme playback controls further and use them in our app. To start, we’ll make a copy of the entertainment theme in our app and customize the template. As an example here, we will use dark red color for the slider pushpin rather than the default color.

  • First, locate the EntertainmentTheme.xaml which is installed with player framework and also gets copied to your app’s output directory during compilation at <approot>\bin\Debug\AppX\Microsoft.PlayerFramework\Themes folder. You can also get it from installation folder located at <drive>:\Program Files (x86)\Microsoft SDKs\Windows\v8.0\ExtensionSDKs\Microsoft.PlayerFramework.Xaml\<version>\ Redist\CommonConfiguration\neutral\Microsoft.PlayerFramework\Themes.
  • Once you have located the file, add it you project in visual studio. If you don’t know how, use either of these options
    • from Visual Studio, use “Add Existing Item” item from context menu in solution explorer, or
    • copy the file to you app folder in file explorer, enable “Show All Files” from the toolbar in solution explorer and use “Include In Project” item from context menu for the file you need to add.
  • Update the Xaml markup of the player page shown above to use the EntertainmentTheme.xaml from project instead of player framework dll.
    <Page.Resources>
        <ResourceDictionary>
            <ResourceDictionary.MergedDictionaries>
                <ResourceDictionary Source="Common/EntertainmentTheme.xaml"/>
            </ResourceDictionary.MergedDictionaries>
        </ResourceDictionary>
    </Page.Resources>
    
  • At this point, the control panel UI is rendered using template defined in this file in the project. You can make any changes here to customize the control panel and the changes would reflect in UI. Let’s change the color of the pushpin for our example.
    <Grid>
        <Ellipse Stroke="{StaticResource ButtonForegroundThemeBrush}" Fill="DarkRed" StrokeThickness="2" Width="35" Height="35"/>
        <Path Fill="{StaticResource ButtonForegroundThemeBrush}" Margin="0,0,0,-7" VerticalAlignment="Bottom" HorizontalAlignment="Center" Data="M 0,0 8,0 4,8Z"/>
    </Grid>
    

We changed the Fill value for the Ellipse to dark red in this example. This demonstrates how to make changes in the template.

Let’s take customization to the next level. The behavior and visibility of playback controls are driven by a ViewModel. In the entertainment theme template we changed above, you must have noticed that bindings are used to control visibility. The control panel uses InteractiveViewModel class in player framework which has a number of properties to control visibility of all the elements in playback controls. I highly recommend going through the type definition for InteractiveViewModel to understand the available properties to use in your custom template. Refer API documentation.

If you have full screen video player in your app, you might want to add a Back button to the player screen. This can be added separately to you page but we can take advantage of MediaPlayer’s extensibility. Using this approach, we can also control the visibility of Back button as per playback controls visibility on user interactions. To achieve this, we’ll have to add the Back button to the ControlPanel template in ExtertainmentTheme.xaml we added to the project above.

Another essential requirement might be to show video title for the currently playing video. Again, we can achieve this by putting a TextBlock on the page but then we will have to handle the visibility of the TextBlock ourselves. Instead of doing that, we’ll take advantage of MediaPlayer’s extensibility and add the TextBlock to the template. This has an issue though as we don’t have a property in InteractiveViewModel which can be bound to the Text property of the TextBlock. We definitely don’t want to hard code this. To resolve this, we will create a subclass of InteractiveViewModel and add the additional property in that. In the new ViewModel, we’ll also add a command for binding it to Back button. Once we have the ViewModel, all we’ll have to do is to wire up the custom ViewModel to the MediaPlayer control.

First, let’s go back to the ControlPanel template and add the title TextBlock and Back button to the playback controls template.

<Grid>
    <Grid.RowDefinitions>
        <RowDefinition Height="Auto"/>
        <RowDefinition Height="*"/>
        <RowDefinition Height="Auto"/>
    </Grid.RowDefinitions>

    <Grid x:Name="HeaderContainer" Background="{StaticResource ItemBackgroundThemeBrush}" Visibility="{Binding IsTimelineVisible, RelativeSource={RelativeSource TemplatedParent}, Converter={StaticResource VisibleIfConverter}}">
        <Grid.ColumnDefinitions>
            <ColumnDefinition Width="Auto" />
            <ColumnDefinition Width="*" />
        </Grid.ColumnDefinitions>
        <Button Command="{Binding GoBackCommand}" Style="{StaticResource BackButtonStyle}" Margin="36,20" VerticalAlignment="Center" />
        <TextBlock Grid.Column="1" TextWrapping="NoWrap" Text="{Binding VideoTitle}" Style="{StaticResource SubheaderTextStyle}"
            Foreground="{StaticResource ApplicationForegroundThemeBrush}" VerticalAlignment="Center" FontSize="40"/>
    </Grid>

    <StackPanel Grid.Row="1" Orientation="Horizontal" VerticalAlignment="Center" HorizontalAlignment="Center">

Notice we are using bindings to set the Text for TextBlock and Command for Button. Next, let’s create the custom ViewModel with properties we can use for binding.

public class CustomInteractiveViewModel : InteractiveViewModel
{
    public CustomInteractiveViewModel(Action goBack, MediaPlayer player)
        : base(player)
    {
        _goBackCommand = new DelegateCommand(goBack);
    }

    private string _videoTitle;
    public string VideoTitle
    {
        get
        {
            return _videoTitle;
        }
        set
        {
            if (_videoTitle != value)
            {
                _videoTitle = value;
                base.OnPropertyChanged("VideoTitle");
            }
        }
    }

    DelegateCommand _goBackCommand;
    public DelegateCommand GoBackCommand
    {
        get
        {
            return _goBackCommand;
        }
    }
}

In the player page, let’s add the MediaPlayer control to the Xaml markup and assign an instance of the CustomInteractiveViewModel to InteractiveViewModel property of the MediaPlayer control to wire up the custom ViewModel to the MediaPlayer.

<Page x:Class="CustomizingPlayerDemo.PlayerPage"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="using:CustomizingPlayerDemo"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    xmlns:mmppf="using:Microsoft.PlayerFramework" mc:Ignorable="d">

    <Page.Resources>
        <ResourceDictionary>
            <ResourceDictionary.MergedDictionaries>
                <ResourceDictionary Source="Common/EntertainmentTheme.xaml"/>
            </ResourceDictionary.MergedDictionaries>
        </ResourceDictionary>
    </Page.Resources>
    <Grid Background="{StaticResource ApplicationPageBackgroundThemeBrush}">
        <mmppf:MediaPlayer x:Name="player" Source="http://download.ted.com/talks/BillGates_2010-480p.mp4?apikey=TEDDOWNLOAD"/>
    </Grid>
</Page>

 

using CustomizingPlayerDemo.Common;
using Windows.UI.Xaml.Controls;

namespace CustomizingPlayerDemo
{
    public sealed partial class PlayerPage : Page
    {
        public PlayerPage()
        {
            this.InitializeComponent();

            player.InteractiveViewModel = new CustomInteractiveViewModel(() => this.Frame.GoBack(), player)
            {
                VideoTitle = "Bill Gates on energy: Innovating to zero!"
            };
        }
    }
}

This gives us a fully customized playback controls UI for player framework MediaPlayer control. This is the same approach I have used in Khan Academy app as well.

CustomizedPlayerControls

The controls we have added to the ControlPanel template are part of the player so we get visibility transitions for these without any extra work. You can download the source code of this sample to see how it works. Please don’t forget to install the player framework using links above before you try to run the sample.

Update:

Hope this helps.

Using Frames to Improve Navigation and Performance in Win 8 Xaml Apps

While working on the Khan Academy app, I realized early in the development that using the navigation pattern provided by the default project templates was not optimal. If you haven’t noticed, the code in default templates which bootstraps your application and creates the frame where your pages are hosted is by default in App.xaml.cs. When the app launches, it checks if the root content is a Frame and creates a new one if not.

protected override async void OnLaunched(LaunchActivatedEventArgs args)
{
	Frame rootFrame = Window.Current.Content as Frame;

	// Do not repeat app initialization when the Window already has content,
	// just ensure that the window is active

	if (rootFrame == null)
	{
		// Create a Frame to act as the navigation context and navigate to the first page
		rootFrame = new Frame();

		//code to restore navigation state
		//this will cause navigation for &quot;a&quot; page if there was anything on navigation stack

		// Place the frame in the current Window
		Window.Current.Content = rootFrame;
	}
	if (rootFrame.Content == null)
	{
		// When the navigation stack isn't restored navigate to the first page,
		// configuring the new page by passing required information as a navigation
		// parameter
		if (!rootFrame.Navigate(typeof(GroupedItemsPage), &quot;AllGroups&quot;))
		{
			throw new Exception(&quot;Failed to create initial page&quot;);
		}
	}
}

Once the frame is created, it navigates to the first page. If you are implementing an extended splash screen, this code moves there but we still have a frame and all navigation happens in that frame. Here are the reasons why I think that the default navigation is not optimal. Your mileage may vary.

1. Entrance Animations for Pages: If we want to use entrance animations for pages, that doesn’t come by default. It’s doable with the default frame created in bootstrap code as you can just add theme transitions to the frame and it would work as you’d expect.

//
if (rootFrame.ContentTransitions == null) rootFrame.ContentTransitions = new TransitionCollection();
rootFrame.ContentTransitions.Clear();
rootFrame.ContentTransitions.Add(new EdgeUIThemeTransition { Edge = EdgeTransitionLocation.Right });
//

If you want to use different entrance animations for different pages, that’s when it gets ugly. Let’s say you are using one consistent entrance animation for all pages but for one special page, you want a unique entrance animation. It’s doable but you’ll have to keep changing frame content transitions in the code every time you need different transitions from what is assigned to frame right now. That’s a big “no”. A different approach is to use separate frames as we will see later in this post. I used this approach in Khan Academy app to slide in the player from bottom of the screen whereas all other pages use standard animation.

2. Maintaining User Actions Across Page Navigations: Every time you navigate away from a page, the instance of the page gets disposed. Navigating to that page again will recreate the instance and have the root frame navigate to it. If the main page of the app is complex and has lot of content with options for users to perform different actions, maintaining the state where user was when he/she navigated away and restoring that state on navigating back to the main page is tricky. For example, let’s say user performs following actions in Khan Academy app.

  • Launch the app
  • Open the subject popup
  • Navigate within the subject popup
  • Tap on a video to play

At this point, if user navigates to player screen on the root frame, navigating back to main page will cause recreation of main page and we’d lose the state for user interactions. This is because I am showing subject details in popup rather than navigating to a separate page due to deep content hierarchy. If I’d do a full page navigation every time user wants to open subject content, this problem wouldn’t exist but the experience would be annoying as the users will have to keep going back and forth all the time.

This can be solved by enabling NavigationCacheMode on the page which makes Xaml runtime use one instance of the page across navigations. This comes with additional state maintenance though. Alternatively, you can use separate frames as will see below.

3. Performance due to Page Navigations: As the page instances are getting destroyed every time you navigate from a page and getting recreated when you navigate to it, this obviously comes with performance overhead. And it’s very noticeable on Windows RT devices which use low powered ARM chips. You can use NavigationCacheMode to cache pages but that comes with additional handling to refresh them when needed. This still doesn’t avoid the work needed for “navigating” to the cached page. In some cases, this might be good enough, you should know your options and pick what works for you.

Using multiple Frames to mitigate these issues

First, we would replace the root frame which was provided by the template with a page with multiple frames in it, lets call it HostPage. So, instead of creating a root frame and assigning that to current window content, we create an instance of HostPage and assign that to window content. At this point, HostPage becomes our one stop shop for all things navigation. For Khan Academy app, here is how the HostPage Xaml looks like.

<Grid x:Name="rootGrid" Style="{StaticResource LayoutRootStyle}">
	<Image Source="../Assets/Images/footer-leaves.png" VerticalAlignment="Bottom" HorizontalAlignment="Center" RenderTransformOrigin="0.5,0.5" >
		<Image.RenderTransform>
			<CompositeTransform ScaleY="-1"/>
		</Image.RenderTransform>
	</Image>

	<Frame x:Name="hostFrame" Foreground="{x:Null}">
		<Frame.ContentTransitions>
			<TransitionCollection>
				<ContentThemeTransition HorizontalOffset="300" />
			</TransitionCollection>
		</Frame.ContentTransitions>
	</Frame>

	<Grid Visibility="{Binding IsOffline, Converter={StaticResource VisibilityConverter}}"
		  HorizontalAlignment="Center" VerticalAlignment="Top" Margin="0,0,0,10">
		<Border Background="{StaticResource ThemeContrastColorBrush}" CornerRadius="5" Margin="-10" />
		<TextBlock Text="Offline Mode" Style="{StaticResource TitleTextStyle}" Foreground="{StaticResource LightForegroundThemeBrush}" />
	</Grid>

	<Frame x:Name="playerFrame" Foreground="{x:Null}">
		<Frame.ContentTransitions>
			<TransitionCollection>
				<PaneThemeTransition Edge="Bottom" />
			</TransitionCollection>
		</Frame.ContentTransitions>
	</Frame>

	<ProgressBar x:Name="busyProgressBar" HorizontalAlignment="Stretch" Height="10" Margin="0,5" VerticalAlignment="Top"
				 IsIndeterminate="{Binding IsLoading}" Visibility="{Binding IsLoading, Converter={StaticResource VisibilityConverter}}"/>
</Grid>

It has two frames, one for hosting all pages and a separate one for player. When user plays a video (from featured videos, subject popup, search or download page), the player page is loaded in player frame with a unique sliding entrance animation from bottom of the screen. When the player is closed, user gets back to where he/she was without causing navigation as we just remove player page from player frame . The order in which you put the frames is important.

An additional benefit of this approach is that we can put additional content in HostPage which we need to display regardless of which page user is on. For Khan Academy app, I have added a background image, a global progress indicator and a offline indicator to show status about network activity and availability. I use a separate class to monitor long running operations and network connectivity changes to control visibility for these items.

As the app can be launched due to user starting the app or initiating search, the host page would have to be smart enough to handle that. This is something we do for root frame in default implementation which moves to HostPage. In my case, I created a few properties to handle this which HostPage used for navigation. Here is the code for HostPage.

    public sealed partial class HostPage : Page
    {
        static HostPage _instance;

        public bool LoadNavigationState { get; set; }
        public Type StartupScreen { get; set; }
        public string Parameter { get; set; }
        public static HostPage Current
        {
            get
            {
                return _instance;
            }
        }

        public HostPage()
        {
            _instance = this;

            this.InitializeComponent();
            this.Loaded += OnHostPageLoaded;
            this.Unloaded += HostPageUnloaded;

            this.DataContext = GlobalStatus.Current;
        }

        async void OnHostPageLoaded(object sender, RoutedEventArgs e)
        {
            //Create a Frame to act as the navigation context and associate it with a SuspensionManager key
            SuspensionManager.RegisterFrame(hostFrame, &quot;AppFrame&quot;);

            if (LoadNavigationState)
            {
                // Restore the saved session state only when appropriate
                await SuspensionManager.RestoreAsync();
            }

            if (hostFrame.Content == null)
            {
                // When the navigation stack isn't restored navigate to the main page,
                if (!hostFrame.Navigate(StartupScreen, Parameter))
                {
                    throw new Exception(&quot;Failed to create initial page&quot;);
                }
            }
        }

        #region navigation methods
        public async Task Navigate(Type sourcePageType)
        {
            //navigate on the host frame
        }

        public async Task Navigate(Type sourcePageType, object parameter)
        {
            //navigate on the host frame
        }

        public Page GetCurrentPage()
        {
            return hostFrame.Content as Page;
        }
        #endregion

        #region player methods
        // Launches player for the passed video id with first playlist which contains the video. Used from search suggestion selection in search charm
        public async Task ShowPlayer(string youTubeId)
        {
            //navigate on the player frame
        }

        // Launches player for passed playlist and video ids. Used from search and downloads screens
        public async Task ShowPlayer(string playlistId, string youTubeId)
        {
            //navigate on the host frame
        }
        #endregion
    }

To wire this up, we assign a HostPage instance to current window content instead of a frame which we saw in first code snippet above. Notice that we also handle different scenarios in which app can be activated.

	Type pageType = null; string parameter = null;

	if (_args.Kind == ActivationKind.Launch)
	{
		pageType = typeof(MainPage);
	}
	else if (_args.Kind == ActivationKind.Search)
	{
		pageType = typeof(SearchPage);

		var searchArgs = _args as SearchActivatedEventArgs;

		if (searchArgs != null) parameter = searchArgs.QueryText;
	}

	Window.Current.Content = new HostPage
	{
		LoadNavigationState = _args.PreviousExecutionState == ApplicationExecutionState.Terminated,
		StartupScreen = pageType,
		Parameter = parameter
	};

And that should be it. With this approach, I avoided a number of navigations and customized the ones which I kept to distinguish the app behavior. This approach helped with improving navigation experience and performance in Khan Academy app. Hope this helps you in developing great apps. Chime away in comments if you have any feedback or questions.