Edward Ropple
ed.ropple@gmail.com
A Windows-Based Application Packaging System

Position Sponsor: WinLibre

One major factor that is keeping open-source adoption back is a lack of a seamless, painless install process on the market-dominating Windows operating system. To facilitate open-source adoption, a standardized system of application and library packaging is necessary. This proposal provides a possible solution to this obstacle.

More information is available on the WPM page.


INTRODUCTION

Every time I have to switch back from Linux to Windows, I am struck by the little niceties that are often lacking in one or the other. While Linux often lacks certain user-interface features and usability tweaks, Windows in comparison lacks features that I consider core functionality.

An example of this lacking functionality is the severe lack of any sort of package management software. Services such as Valve Software's Steam exist, but they are for closed-source software and are game-based to boot; they do not help the adoption of open-source software. I'd like to propose a system that will help us get open-source software into the hands of everyday, nontechnical users. In addition, the server portion of the project could be repurposed to create a Macintosh-based packaging system for F/OSS as well.

I will first discuss thoughts of how to implement a windows package manager on the client side, and then the server.

CLIENT SYSTEMS

[Throughout this it is important to keep in mind that the Microsoft Installer (*.MSI, *.MSP) does a lot of what dpkg on a Debian system does in the areas of dependency checking and library versioning, and perhaps the format could be leveraged in our favor. As I am not sure as to the ramifications of using the closed source MSIs in an open-source packaging mechanism, I am choosing to ignore this possibility as of the moment. It is well worth checking into, though.]

First, thoughts on packaged programs themselves: I would suggest that any packaged F/OSS software adhere as closely as possible to the same set of rules that is used for portable USB-key applications. I know of portable OpenOffice, portable Firefox, and portable Gaim; I am sure there are hundreds of other programs out there that run just fine on a USB key. These rules ensure that the applications stay in their own directories and do not bother other programs, whether they are installed via WPM or MSI. This assists in easy uninstallation and updates, though it does increase the risk of security problems due to "floating" DLL files--a solution to this would be an indexed set of WPM-installed DLL files in the Windows directory.

I would recommend both a command-line and graphical interface for WPM. The command-line interface should mimic either apt or yum as best as possible, and my primary focus would be to get installation (and thus remote package retrieval), upgrade, and removal systems working as soon as possible. Making it work as similarly to the Linux package management systems has the benefits of easing usage for Linux users who must use Windows and making it easier to transition Windows users to Linux.

The graphical WPM client is of special importance--it must work flawlessly and be as easy to use as possible, otherwise it might as well not even be worth the hassle of creating it. I believe the Synaptic interface to be a good starting point, but there are ways it can be improved. A tagging feature similar to KDE's Adept APT manager would ease searching, as would a method for users to add helpful links and comments in a wiki-like fashion to individual packages. Search needs to be improved as well--I should not see something like "Linux base utilities" on the list when I am searching for "CD write". I believe that, unlike APT, WPM would be better off maintaining a list of repositories on a central server and downloading that list of repositories for presentation to the end user. Testing/debug repositories could very well exist and be added in some sort of advanced mode, but adding repositories manually is a task I feel is a bit out of the league of many end users--it fails the "Aunt Tillie" test--and so functionality to accomodate these users must be considered.

I would suggest keeping any WPM-managed applications out of the system registry. This has the unfortunate side effect of making them invisible to Add/Remove Programs, but a .CPL applet in the Control Panel can alleviate this to some degree, as could adding hooks to Add/Remove Programs that call the WPM package management system when a program needs to be uninstalled.

The Mono framework could be very useful to this project. My current language of choice is C#, and while I personally see no problem with using the .NET Framework for the client, others may disagree, and the Mono framework is there just waiting to be used. (This could also benefit open-source applications written to work with .NET, encouraging them to port to Mono.)

SERVER INTERFACES

[Point of clarity: Below I use the term "server" in a logical sense, not necessarily a physical one--load-balancing and clustering may of course occur along the line.]

Though the proposal for the client-facing software, written above, is largely based on the Debian model and adapted to Windows, I think that a slightly different approach is necessary from the server side of things. (It should almost go without saying that server functionality should be implemented on non-Windows computers!)

At the highest level is the Coordination Server. This server keeps a list of all registered repository servers and returns the list when queried. For cost-effectiveness, this (fairly simple) functionality could easily be run on the same computer as the main repositories, but I for one would suggest coupling it with the commentary functionality mentioned above and elaborated upon below. This server should be at a web address that will hopefully remain fixed as permanently as possible, but the protocol to access it should include backup servers and ways to point to updated servers for future-proofing.

The next level down are the repository servers. These servers, like the Debian repositories, use simple HTTP and FTP requests to pull WPM packages to the clients.

The final part of the primary functionality of the WPM services is the commentary/wiki system. It shall encompass the functions mentioned above--commentary, suggestions, helpful links, and the like--but will provide some benefit to developers as well: should a maintainer or developer be interested in participating, the commentary services will forward messages marked "Suggestion To Developers" to their e-mail.

CONCLUSION

In closing, I would first like very much to thank you for the opportunity to apply to work with you. This is a project that has interested me for some time, and I was considering the idea for the summer anyway--but the prospect of working with experienced F/OSS developers and advocates is far too compelling to ignore. I believe it is well within my capabilities and that I would bring an eagerness and zeal to this project that could make it the best it can be.

Thank you very much for your time.

Edward Ropple
March 14, 2007



More Ideas Since Submission

It's not like I stop thinking once I hit "Submit"!
Anything below this point is pure musing, thought,
and discussion, and is likely a bit disjointed.
Be warned.

Somewhat Disfocused Thoughts on WPM Packaging

Windows has a foot in the bucket compared to Linux, because of its lack of built-in programming tools. Kubuntu is an exception among Linux distros as it doesn't really come with much in the way of development tools--but they're all just an apt-get away. Other distros come loaded out of the box with perl, gcc, sometimes tcl and even Python. That's not even counting bash itself.

This presents a somewhat nontrivial problem for anyone who wants to build a Windows package manager, because it means that it becomes very, very difficult to get anywhere with installing programs; you're either forced to rely on some sort of manifest to decide what goes where and how your software gets installed, or bring a programming language with you. For WPM, I think that the latter approach is best.

If we're bringing our tools with us, the question becomes what to bring. (One must bring the right clothes for the pah-tay, after all.) I stated in my project submission that I believe that including the Mono framework by default is a good way to both immediately promote WPM as a solution for open-source .NET/Mono developers, and I believe that fully. But for installation, the Mono framework is less than ideal. There are a number of ways that we could go about this.

For best compatibility with Windows environments, I would personally wish for something akin to the Windows PowerShell, but to the best of my knowledge there is no F/OSS equivalent. (I say Windows PowerShell would be an optimal solution mostly because it seems that PowerShell is eventually going to be a standard on Windows machines, and it'd be nice to be able to say "yeah, we've got that, come try our stuff" when that day comes. But unless we use WPS itself, that doesn't seem likely. The most immediate alternative I can think of is bash, probably via MSYS (I know others like Cygwin, but I've had more experience and better luck with MSYS). That has its own downsides, such as being a completely foreign language for Windows programmers. At that point we're running ourselves in circles, because we're really targeting this at two separate groups of people.

The first group of people is mostly made up of *NIX enthusiasts and developers who are used to the quirks of *NIX. They'll like bash, because it's what they're used to. Some no doubt don't even use Windows, but are looking at it to try to roll a package together. Giving them the tools they're used to should be a priority, and that means bash, perl, and the like.

On the other hand, we've got Windows users and developers who are looking at open-source and going "hmm." They aren't used to bash, and most probably haven't written a batch file in years anyway. (I know I haven't--if you put a gun to my head I probably couldn't remember how to do a conditional!) Their tools are mostly compiled, such as C++, Java, and the .NET languages.

If this isn't enough, there's one more trick card to throw into the deck, one more mine in the Minesweeper playing field. I think that the most reasonable design for WPM packages is to keep as much logic out of the WPM client as possible. Once the package is opened, the client should hand off execution to the package in some way, and that's the end of that. So, my proposition for basic package structure is below.

A .wpm package is a .zip archive. Inside, at minimum, are three files (or two files and a directory):

When the package is downloaded, the client finds and executes the initialization command or commands in manifest.xml. While it is conceivable that the package would be installed solely through batch scripting within the manifest file, I think it is much more likely that it would be used to initiate the installer in a real language.

This method provides the most flexibility I can think of; it allows package maintainers the ability to remain in their comfort zone with the languages they use. A programmer could go so far as to include a .cs C# file in the package, compile it on the fly, and execute it to install his software, gaining access to the many useful functions of the Mono framework. A UNIX programmer would be free to use bash or another *NIX language to install his package.

I said above in my Summer of Code submission that the packaged software should be drawn from the USB-drive application pool. Upon further thought, this might not be entirely wise, as those applications do not take advantage of user settings and directories. There is no harm in a WPM-installed version of Firefox going ahead and dropping its temporary data into the user's Temp folder, nor saving its settings in the Local Settings folder. The only harm comes from registry changes, but even that could be avoided with manifest.xml fields that track registry changes. So long as the registry changes are done by the WPM client and not the program itself, they can easily be rolled back later. And since most cross-platform programs wisely avoid the registry like the plague, this becomes even less of a factor.



This document is copyrighted as of 2007 by Edward Ropple. Permission to redistribute it is granted under the Creative Commons Attribution-ShareAlike (by-sa) license, viewable here, but the author requests to be informed by any distributors as a courtesy.