Navigating the Data Avalanche

Last weekend, I migrated my wife's PC from the aging Pentium III 866 processor she'd been using to my old Pentium 4 1.8GHz system, which I had recently abandoned for a newer 2.8GHz model. The job required some planning, and naturally I relied on some of the data synchronization and backup strategies we've been discussing in Connected Home EXPRESS the past several weeks. I'm happy to report that I got all my wife's applications and data successfully migrated with nary a hitch.

But before I start patting myself on the back, I should make an admission: My wife's computing needs are almost archaically simplistic. Her entire My Documents folder structure consumes less than 300MB and most of that is photos and music files I copied over to her system last year (files she probably never accesses). For applications, she uses only Microsoft Word, Outlook, Internet Explorer (IE), and Money, along with a few small utilities. Overall, the job was pretty simple.

For my system, the migration process was decidedly more painful. I can't even go on a road trip without making several DVD-based backups, and the My Documents folder structure on my system, which doesn't include any digital-media files, is larger than 19GB and growing daily. I also have literally hundreds of gigabytes of music, photo, and movie files on my system.

I suspect most people fall in a more typical medium between my wife and me when it comes to storage needs. But regardless of how many files you have to deal with or the total storage space allotted to those files, the process of finding the files you need is growing increasingly complex. With my particularly addled-brain system, I often know that I have a particular file somewhere but can't remember the name, let alone where I stored it--and that's just standard data files, such as text and Word documents. What happens when you throw digital-media files into the mix? On my Apple Computer iMac, Apple iTunes tells me I have more than 4560 songs that occupy more than 23GB of space; it would take 14.1 days to play through this entire collection, back to back, the program tells me. Add photos and video the mix, and my head starts spinning. How do I keep track of all this stuff?

That question is one that companies such as Apple and Microsoft are working on. Today, the current file-management system on Windows is almost untenable. On the Macintosh, it's a bit better. But if the approaches these companies are taking are any indication, finding information on our PCs is going to be a lot simpler in the future.

First, let's look at Windows. Today, Windows forces users to have a fairly intimate understanding of its dated, drive-letter-based file system. To make life a bit easier, Microsoft developed a My Documents-based file structure that includes special subfolders for photos and other digital images (My Pictures), digital audio and music (My Music), and digital movies (My Videos). If you want to stay organized, you work with documents and digital media files in these folders and only in these folders. But what happens when you save a file in IE or Word to a nonstandard location? You have to search, and this is where the Windows Search tool makes life just a bit bearable because it can search by content, not just filename. Nice.

On the Mac, Apple has worked to steer users away from the technical file-system approach and has created applications such as iTunes and Apple iPhoto that abstract the physical location in which files are stored. This approach is much simpler than Windows' approach. If you want to work with digital photos, just start iPhoto. Digital music? Run iTunes. But these applications don't just hide the file system. They also provide what I call a views-based interface to your digital media, and they both work the same way. Consider iTunes. In the default view, you see your entire music collection, sorted by some attribute (typically Artist, but you can choose others, such as Album, Genre, or My Rating). iTunes also supports something called playlists (called albums in iPhoto), which are filtered views of the wider library. For example, you can manually create playlists of your favorite songs, or iTunes can automatically create smart playlists populated by categories such as rating, artist name, or genre. iPhoto works in a similar manner.

This structure is important. Playlists and albums don't contain duplicates of the original source material; instead, they contain shortcuts to the originals, so no data duplication exists. What you're really doing is something database experts have been doing for years: You're working with one master list of information but filtering the list based on certain criteria. Furthermore, you can perform simple ad hoc queries on this data in iTunes by using the search box. So you might display a playlist of all Rock songs in iTunes, then type Van Halen in the search box to see only rock songs by Van Halen; this action doesn't permanently alter your playlists, any of your music, or the songs themselves. But you can now manage and listen to just those particular songs. Suddenly, the unmanageable is workable.

Going forward, Windows will adopt this views-based approach right in the OS and will drop drive letters, so files' physical locations won't matter. The next Windows release, code-named Longhorn and due in 2005, will include a new shell structure called a Library that will resemble a folder, but will instead automatically collect lists of particular file types from all over your system and give you a logical view of those files that abstracts their true locations. So the new Pictures & Videos Library will give you iTunes-like views of your multimedia data right in the file system, without forcing you to worry about physical file locations. And, yes, Longhorn will have a Filter box, right in the window, for performing ad hoc queries on the current view. So you'll be able to buy a new hard disk, start putting documents and digital media files on it, and Windows will find them no matter where they are, physically.

Of course, we're still 2 years out from Longhorn. In the meantime, people with large digital-media collections or massive amounts of documents have had to become experts in the Windows file system or risk losing data forever. One day we'll look back on this task with contempt, the same way that none of us could imagine going back to rotary telephones wired directly to the wall. I can't wait.

Hide comments


  • Allowed HTML tags: <em> <strong> <blockquote> <br> <p>

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.