Working with Images and Animation on the iPhone and iPad with .NET and C#: Part 2

This article resumes where we left off in "Working with Images and Animation on the iPhone and iPad with .NET and C#: Part 1," which looked at multimedia app development on the iPhone and iPad, including how to display simple images, take pictures with the device's camera, and perform some simple animation with the images. In this article, part 2, we'll look at how to perform more complicated animation using the NSTimer and UIView classes, how to display and record video, and how to edit the video.

Animation Based on NSTimer

Setting the UIImageView to perform animation works well for a defined set of images. But what happens when we want to move something around on the screen? To answer that question, let's look at a more complicated example. In this example, we will move a golf ball around the screen and bounce it off of the edges of the screen. We'll also use a slider control. As we move the value of the slider control to the left, the golf ball will move around faster. As we move the value of the slider control to the right, the golf ball moves around more slowly. The following steps lay out the sequence of these operations; Figure 1 shows the accompanying code.

NSTimer timer;
float ballRadius;
PointF delta;

public override void ViewDidLoad ()
{
	base.ViewDidLoad ();
	Title = "NSTimer";
	UIImage u = UIImage.FromFile("./Images/images.jpg");
	uiBall.Image = u;
	ballRadius = uiBall.Bounds.Size.Width / 2;
	delta = new PointF(12.0f, 4.0f); 
	slider.MaxValue = 100;
	slider.MinValue = .01f;
	slider.Value = 50;
	slider.ValueChanged += delegate(object sender, EventArgs e) {
		timer.Dispose();
		timer = null;
		timer = NSTimer.CreateRepeatingScheduledTimer(new TimeSpan(0, 0, 0, 0, (int)slider.Value), delegate {
		HandleTimerUp ();});
	} ;		
}

public override void ViewDidAppear (bool animated)
{
	base.ViewDidAppear (animated);
	timer = NSTimer.CreateRepeatingScheduledTimer(new TimeSpan(0, 0, 0, 0, (int)slider.Value), delegate {
		HandleTimerUp ();
	} );

}
public override void ViewDidDisappear (bool animated)
{
	base.ViewDidDisappear (animated);
	timer.Dispose ();
	timer = null;
}
private void HandleTimerUp()
{
	uiBall.Center = new PointF(uiBall.Center.X + delta.X,
           uiBall.Center.Y + delta.Y);
	if ( uiBall.Center.X > this.View.Bounds.Size.Width - ballRadius ||
	    uiBall.Center.X < ballRadius )
		delta.X = - delta.X;
	if ( uiBall.Center.Y > this.View.Bounds.Size.Height - ballRadius ||
	    uiBall.Center.Y < ballRadius )
		delta.Y = - delta.Y;
}

The first thing that we will do is create these items:

a UIImageView and put it in the center of the screen
a Slider and put it at the bottom of the view

Along with these controls, we need to create outlets for them.

We'll perform various setup operations, such as setting up the slider values and how to handle the changes to the slider.

We will create an NSTimer. One of the things to note about the NSTimer is that it performs the same type of operation as the Timer classes that we are used to in .NET. However, NSTimer has some differences to be aware of. Here are the two biggest differences:

Once an NSTimer object is created, we will need to delete the object and start with a new object instead of changing the original object.
An NSTimer can be created several ways. In this example, the .CreateRepeatingScheduledTimer method is created to run on the NSRunLoop. This allows us to run operations on the current UI thread and not have to worry about whether or not we are running on the correct thread.

When our view is no longer visible to the user, in the ViewDidDisappear method, we are quitting the timer. By doing this, we are being a good citizen on the device and are no longer performing operations that the user can't see anyway.

Once this view is loaded, the golf ball begins bouncing around the screen.

UIView Animations

In our NSTimer example, as the slider goes to the right and the animation slows down, did you notice how the animation is a little bit jerky? We'd like to code a way to clean that up so that the golf ball moves a little more smoothly around the screen. This can be accomplished via the static .Animate() method on the UIView, as shown in the code in Figure 2. In this situation, we do nearly the exact same thing as the previous example. The biggest difference is that instead of directly moving the image, we will allow the .Animate() method to perform this action. The result is a very smooth animation effect at specified update intervals of the NSTimer.

UIImage u = UIImage.FromFile("./Images/images.jpg");
uiv.Image = u;
ballRadius = uiv.Bounds.Size.Width / 2;
delta = new PointF(12.0f, 4.0f); 
 
timer = NSTimer.CreateRepeatingScheduledTimer(new TimeSpan(0, 0, 0, 0, 100), delegate {
	AnimateGolfBall ();
});

private void AnimateGolfBall(){
	UIView.Animate(.1, 0, 
	UIViewAnimationOptions.CurveLinear | 
UIViewAnimationOptions.AllowUserInteraction, 
	delegate() {
		HandleTimerUp();
	} , null);
}

Playing Video

Who doesn't like video? Video tells a great story. But to add video to an iOS app, there are several issues that we need to understand:

Video doesn't typically exist within an application. We can't guarantee that any specific video actually does exist. We'll need to get the video from the gallery.
Video will be often loaded off the Internet. Video is larger than single images, so we will want to make sure that the video we create is optimized for loading on a mobile device.
Wireless cell networks are more often than not somewhat unreliable. Wireless networks are optimized for voice, not data. If an application locks the UI thread for too long, iOS will kill our application. We need to use something to load and play video off of the main thread in an asynchronous format. Thankfully, the movie player in the iPhone handles the asynchronous part for you. It will even allow us to register for events (notifications in iOS-speak) when the network stream to the movie player changes.

Here are the steps to play back video from an Internet source; Figure 3 shows the code to do this.

Create an instance of the MPMoviePlayerController. In the constructor of the class, we will pass in an NSUrl. As you can guess, an NSUrl is a native iOS object that is associated with URLs.
The next step is to begin to set up some events. As .NET C# developers, we are used to events being created using the "+=" syntax. In this case, we'll be dealing with something that is an iPhone-ism. We'll set up a notification, so that our application is made aware of changes -- here, changes to the media player.
Once the media player has gone through its preload process, the movie player will begin playing within the event.
Once the movie is finished, some cleanup of the media player is performed.
When the user clicks the navigation button that will navigate away from this screen, additional cleanup of the movie player occurs. This cleanup is here as well because I wanted to guarantee that the objects were cleaned up. It seems that the finished notification is not the correct spot to do this since it is not called until the video is finished.

public override void ViewDidAppear (bool b)
{
	base.ViewDidAppear (b);
	
	// Perform any additional setup after loading the view, typically from a nib.
	var videoUrl = NSUrl.FromString(
	"http://www.aspnetpodcast.com/videofiles/iphoneandroid.mp4"); 
	moviePlayer = new MPMoviePlayerController(videoUrl);
	var center = NSNotificationCenter.DefaultCenter;
	var preloadFinish = new NSString("MPMoviePlayerContentPreloadDidFinishNotification"); 
	didFinishPreload = center.AddObserver(preloadFinish,
		(notify) => {
			Console.WriteLine ("Start playing movie"); 
			moviePlayer.Play();
		} );
	var playbackFinished = new NSString(
		"MPMoviePlayerPlaybackDidFinishNotification"); 
	didFinishPlayback = center.AddObserver(playbackFinished,
		(notify) => {
			Console.WriteLine ("Movie finished, Clean up");
			center.RemoveObserver(didFinishPreload); 						center.RemoveObserver(didFinishPlayback);
			moviePlayer.Dispose();
			moviePlayer = null; 
		} );
	var _someView = new UIView();
	_someView.Frame = View.Bounds;   
	moviePlayer.View.Frame = _someView.Frame;
	_someView.AddSubview(moviePlayer.View);
	View.AddSubview(_someView);
	moviePlayer.Play();
}

public override void ViewDidDisappear (bool animated)
{
	base.ViewDidDisappear (animated);
	if ( ( moviePlayer != null ) && (
		moviePlayer.PlaybackState != MPMoviePlaybackState.Stopped))
	{	
		moviePlayer.Stop();
		moviePlayer = null;
	}
}

The video captured in the screenshot in Figure 4 is the result of running the code in Figure 3. (I hope that the speakers on your iPhone/iPad have been turned down.)

Figure 4: Results of running the code in Figure 3

I bet that you are now thinking, "OK, Wally, but about this notification thingy, how do I find out more about that? What are the possible notifications that I might get?" That's a fair question. To find the answer, I dug into Apple's developer site and found the notifications listed below. (Since we don't have IntelliSense support for these notifications because of their text format, I thought it would be a good idea to list them here.)

MPMovieDurationAvailableNotification. This is posted when the duration of the movie has been determined. The duration can be accessed via the movie player's .Duration property.
MPMovieMediaTypesAvailableNotification. This notification occurs when the available movie types are determined. The types are available via the movie player's .MovieMediaTypes property.
MPMovieNaturalSizeAvailableNotification. This notification occurs when the natural frame size of the movie is determined or changes. This is available via the movie player's .NaturalSize property.
MPMoviePlayerContentPreloadDidFinishNotification. This notification happens when the movie is in memory and ready to play.
MPMoviePlayerDidEnterFullscreenNotification. This occurs when the movie player has entered full-screen mode. This notification can be triggered programmatically or via user interaction with the movie player on screen. For this to be triggered programmatically, the movie player method SetFullscreen can be called.
MPMoviePlayerDidExitFullscreenNotification. This occurs when the movie player has exited full-screen mode. This notification can be triggered programmatically via the .SetFullscreen() method or via user interaction.
MPMoviePlayerIsAirPlayVideoActiveDidChangeNotification. This notification occurs when a movie player has started or ended playing a movie via airplay. This is available via the .AirPlayVideoActive property.
MPMoviePlayerLoadStateDidChangeNotification. This notification occurs when a movie player's network buffering state has changed. This is available via the .LoadState property.
MPMoviePlayerNOwPlayingMovieDidChangeNotification. This notification occurs when the currently playing movie has changed. The URL for the currently playing video is available via the .ContentURL property.
MPMoviePlayerPlaybackDidFinishNotification. This notification occurs when a movie is finished playing. The UserInfo dictionary of this notification contains the key MPMoviePlayerPlaybackDidFinishReasonUserInfoKey, which indications the reason the playback finished.
MPMoviePlayerPlaybackStateDidChangeNotification. This notification occurs when the movie player's playback state changes. This property is available via the .PlaybackState.
MPMoviePlayerScalingModeDidChangeNotification. This notification happens when the scale mode of the movie player changes. This can be done programmatically via the .ScalingMode porperty or via the user.
MPMoviePlayerThumbnailImageRequestDidFinishNofication. This notification occurs when a request to capture a thumbnail has finished.
MPMoviePlayerWillEnterFullscreenNotification. This notification occurs when a movie player is about to enter full-screen mode.
MPMoviePlayerWillExitFullscreenNotification. This notification occurs when a movie player is about to leave full-screen mode.
MPMovieSourcetypeAvailableNotification. This notification occurs when the source type of a movie player was previously unknown and becomes available. This is available via the .MovieSourceType property.

Recording Video

iOS has had the ability to record video since version 3. Recording video is simple and is based on the UIImagePickerController (yes, even though it says "Image" in the name). Following are the basic steps to record video; Figure 5 shows the accompanying code.

Verify that the device supports the camera.
Create the UIImagePickerController.
Set the SourceType to that of the camera.
Set the MediaTypes to create a movie. This is done via a string array.
Once this is completed, we should get the standard camera interface. We get this because we have passed in PresentModalViewController an instance of our UIImagePickerController.

public override void ViewDidLoad ()
{
	base.ViewDidLoad ();
	this.Title = "Record Vid";
	var cameraType = UIImagePickerControllerSourceType.Camera;
	if(HasVideoSupport())
	{
		UIImagePickerController picker = new UIImagePickerController();
		picker.SourceType = cameraType;
		picker.MediaTypes = new []{"public.movie"} ;
		picker.Delegate = new VideoImagePickerDelegate(this);
		this.PresentModalViewController(picker, true);
	}
	else
	{
	   using (var alert = new UIAlertView("Whoops", 
			"No video support found", null, "Ok!", null))
	   {
	  alert.Show();
	   }
	}
}

private bool HasVideoSupport()
{
	var cameraType = UIImagePickerControllerSourceType.Camera;
	var cameraSupport = UIImagePickerController.IsSourceTypeAvailable(cameraType);
	return (!cameraSupport) ? false :  
		UIImagePickerController.AvailableMediaTypes(cameraType).Contains("public.movie");
}

Now that we've recorded our video, let's look at our delegate, which will handle the callbacks from the camera. In this instance, as shown in Figure 6, we have inherited from the UIImagePickerControllerDelegate. With this object, we have overridden the Canceled and FinishedPickingMedia methods so that we can act when a user either cancels or selects a media file.

public class VideoImagePickerDelegate : UIImagePickerControllerDelegate
{
   Video _vi;
   public VideoImagePickerDelegate(Video vi)
   {
  _vi = vi;	
   }

   public override void Canceled (UIImagePickerController picker)
   {
	  _vi.DismissModalViewControllerAnimated(true);
  Console.WriteLine ("Cancelled picking an image");
   }

	public override void FinishedPickingMedia (UIImagePickerController picker, NSDictionary info)
	{
		var mediaUrlKey = new NSString("UIImagePickerControllerMediaURL");
		var mediaPath = (NSUrl) info.ObjectForKey(mediaUrlKey);
		if(mediaPath != null)
		{
			if(UIVideo.IsCompatibleWithSavedPhotosAlbum(mediaPath.Path))
			{
				UIVideo.SaveToPhotosAlbum(mediaPath.Path, delegate (string path, NSError errors)
				{
					using (var alert = new UIAlertView("Success", "Video saved!",
		             null, "Ok!", null))
					{
		alert.Show();
					}
				} );
			}
			else
			{
				using (var alert = new UIAlertView("Whoops", "We can't save this video",
		             null, "Ok!", null))
				{
					alert.Show();
				}
			}
		}
		_vi.DismissModalViewControllerAnimated(true);
		Console.WriteLine ("Did finish picking media");
	}
}

A word of warning with the FinishedPickingMedia method: I used this method to place the media within the gallery of media in our device. I found it to be a little problematic to get the actual path for the selected file. One would think that calling info[key] would give us the appropriate path; however, it did not. I had to use ((NSUrl)info.ObjectForKey(…….)).Path to get the appropriate path to the file. Once I did this, I was able to save the file to the photo album appropriately. Many thanks to Martin Bowling for pointing out this tip to me.

Editing Video

Within a phone, we have the ability to edit video. Now, realistically, the phone is not going to provide a great experience for editing video. However, iOS includes support for some basic editing, using the UIVideoEditorController object. This object manages a system-supplied UI for removing the video frames from the beginning and ending of a previously recorded movie. In addition, the UIVideoEditorController object can be used to re-encode a movie to a lower quality.

The UIVideoEditorController class manages user interactions and provides a file system object pointing to the edited movie. Although the UIImagePickerController class enables trimming from the beginning and end of a movie, its primary role is to save pictures and movies as well as capture new images and movies. The UIVideoEditorController class provides a better experience for this. Note that the UIVideoEditorController class supports portrait mode only and is not designed to be inherited from.

In the sequence of events for our video-editing example, the first step is to create a UIImagePickerController for movies. In this case, we need to send an array of media types that we would like to display. Specifically, we will need to send the "pubic.movie" media type, as shown in the following code snippet:

picker = new UIImagePickerController();
picker.MediaTypes = new string[] { "public.movie" };
picker.Delegate = new VideoGalleryPickerDelegate(this);
picker.SourceType = UIImagePickerControllerSourceType.PhotoLibrary;
this.PresentModalViewController(picker, true);
btnEditVideo.TouchUpInside += HandleBtnEditVideoTouchUpInside;

Figure 7 shows the result of creating the UIImagePickerController.

Figure 7: Screen displayed after creating UIImagePickerController

The next step is to process the FinishedPickingMedia method. The key sticking point is getting the path of the video to edit, which is the next step. The confusing thing that I did was to get the URI of the movie. The key is to get the NSObject, convert that into an NSUrl, and then return the .Path property of the movie file.

var mediaUrlKey = new NSString("UIImagePickerControllerMediaURL");
var path = ((NSUrl) info.ObjectForKey(mediaUrlKey)).Path;

The next step is to start the video editing. There are several steps to this, as shown in the code in Figure 8. The first is to test to see whether the UIVideoEditorController supports the video type and can perform the edit operation by calling .CanEditVideoAtPath. The next step is to create UIVideoEditorController and add some properties to the object. The final step is to display the class via the PresentModalViewController method.

if ( UIVideoEditorController.CanEditVideoAtPath(moviePath) )
{
	var videoEditor = new UIVideoEditorController();
	videoEditor.VideoPath = moviePath;
	videoEditor.Saved += delegate(object s, UIPathEventArgs EventArgs) {
		videoEditor.DismissModalViewControllerAnimated(true);
	} ;
	videoEditor.Failed += delegate(object s, NSErrorEventArgs EventArgs) {
		videoEditor.DismissModalViewControllerAnimated(true);
	} ;
	videoEditor.UserCancelled += delegate(object s, EventArgs EventArgs) {
		videoEditor.DismissModalViewControllerAnimated(true);
	} ;
	this.PresentModalViewController(videoEditor, true);
}
else
{
	using (var alert = new UIAlertView("Whoops", "Video type not supported.", null, "Ok!", null))
	{
		alert.Show();
	}
}

Figure 9 shows a screenshot of editing a short video I took when I woke up my family's cat Wells, and he decided to yawn right after I started recording (his timing was purrrfect).

In this image I've circled several items:

a Cancel button in the top left corner
a Save button in the top right corner
the yellow start and stop areas at the top of the screen near the center to allow a user to trim the movie
I'm not quite sure how visible it is in this image, but it is possible to display the movie along the timeline with a small slider type of control.

Various Devices

You may be thinking, this is great, but I have an older iPhone 3GS that only has one camera -- how is this going to work for me? How can I support these older devices? How can I test to make sure that the device supports a feature we want to use instead of putting logic in our application that checks based on the device?

Fortunately, we can perform feature detection in iOS to determine whether a device has a camera. To do this, we use the following code:

var cameraType = UIImagePickerControllerSourceType.Camera;
if(UIImagePickerController.IsSourceTypeAvailable(cameraType))
{
// code to use the camera.
}

Expand Your iOS Apps

This concludes my two-part tutorial on displaying and creating pictures and video in iOS. In these articles, I've demonstrated how you can start working with images and animation in iOS apps using the MonoTouch C# development platform. The iPhone and iPad offer plenty of opportunity for developers to take advantage of multimedia in their apps, and with MonoTouch, .NET C# developers have an easier path into iOS app development.

Wallace B. "Wally" McClure is an ASPInsider, member of the national INETA Speakers Bureau, author of seven programming books, and a partner in Scalable Development.

[sidebar]

References and Comments

This code was written and verified with iOS 5.01, MonoTouch 5.2.x, and MonoDevelop 2.8.6.4.

Chris Hardy wrote the Multimedia Chapter in our book, Professional iPhone Programming with MonoTouch and .NET/C# (Wrox). He has been a great friend and resource over the years.

Martin Bowling was helpful regarding a few of the finer points of the UIImagePickerController.

Two additional books, by Mike Bluestein and Bryan Costanich, respectively, are also worthwhile MonoTouch references. There is no competition amongst authors in my mind (we're fighting over too few peanuts). Mike and Bryan have been great friends and resources over the past few years as I have jumped head first into the iPhone and Android.

More information on topics discussed in this article:

More Dev Pro articles on iOS development: