World of Warcraft: Toward a better addon infrastructure

[ note: this proposal has been updated in order to improve tone and add in some additional material that resulted from conversations with WoW addon developers and users ]

It used to be that there were a fairly large number of sites that hosted WoW addons. It also used to be that there were 4-5 reasonably well-maintained programs for updating them. Some, like WUU, worked well with all manner of sites. Some, such as Curse, provided download access only for their own service, others were open to all services.

That's changed of late. Consolidation was followed by exclusion of external download clients which was followed by reduction of non-pay functionality in the proprietary download clients. Some of this happened because of abusive third parties which I don't want to discuss here. Many addon authors are fine with this. Many users are fine with this. I don't question their wisdom, but I do see an opportunity for a better way, and because I have considered distributing my own addons someday, I'd like to make there be more options and flexibility for developers while improving my lot as a user.

Here is the proposal:

Developers of addons sign up with the hosting service of their choice. Services that are not purely WoW-related, and which have more liberal download policies than existing WoW hosting services include:

• http://github.com/plans
• http://code.google.com/opensource/
• http://sourceforge.net/
• http://www.codeplex.com/

Each service will have to be analyzed to make sure that the intended patterns of access won't violate their terms of service, but I'm pretty sure that services like Google simply won't notice the impact while Sourceforge might. We shall see. Some mod authors already use some of these services, so there's nothing terribly new, here, so far.

Anyway, returning to infrastructure. The next phase is to build a map of these addons. This is not an end-user consumable item, and shouldn't be confused with existing attempts to make lists of addons. Instead, it is a structured document (in JSON, XML, whatever) that represents a method for developers to communicate with their users and inform them of what they provide and how they wish to provide it.

A short list of items to be included in this structured document include:

• Name of addon
• Official URL
• Bug reporting URL
• General public release direct download URL
• One or more pre-release direct download URLs
• Logo direct download URL for display in download clients
• List of addons enclosed (for UI bundles and addons in several parts)
• Licensing type
  

There need to be two points of access for this file: one is a read-only point that allows addon download clients (such as Curse or WUU) to pull down the entire list. The second must allow read-write access in a secure way that allows addon authors to update their hosting and versioning information. A simple RESTful interface should be sufficient to provide both interactions, and could easily be hosted by a small GAE or EC2 service. Someone might even donate the resources to run such an app, but the cost should be nominal even if it has to be hosted on donations because it's not providing access to individual addons.

[ side note: previous versions of this proposal suggested that the document/database should be licensed in such a way that one valid use was to create a third-party Web site based on it. I now believe that this would be a mistake, and that it should have a license more like the IMDB database: allowing free access to it, but not use in creating Web sites. This is to improve communications between users and developers and avoid hurdles in users providing feedback and bug reports. ]

For the service, we must establish authorship in a secure way. There are many ways to do this, but I think we'll have to start by placing all existing addons into the database as dummy entries, but establishing that a given authorship claim is valid can be complex. Most likely this will have to be accomplished in much the same way as search engines validate ownership of Web sites: by providing a "cookie" that the owner must place within the addon. This means that only addons which have a working direct download link will be "claimable". All others will remain dummy records, preventing other users from claiming authorship of a new addon of the same name.

The last component would be modifications to popular downloader clients such as WUU to enable access to the central data and developer repositories. This would need to include the ability to play nice with each hosting provider's means of access (perhaps as simple as downloading via HTTP or as complex as cloning a repository and extracting a given version, depending on the service). Along with this, we would also need a simple command-line tool (preferably that works under Windows, Mac and Linux) that allows authors to submit their updates without having to cobble together their own queries.

I think the benefits of such a system would be massive. For starters, end-users wouldn't have to go to several different sites in order to update their installed addons. For users with 50+ addons installed, this is a great boon. For developers it provides a way to initially select whatever hosting service best suits their needs (note that existing developers may not, at this time, see any direct benefit, nor particularly care; that's ok). For hosting sites, the situation is more complex. Of course, they would prefer not to have more competition from sites that have been doing project hosting for years. However, I believe that in the face of strong and open competition, these services will be made better and stronger. In the long run, I think it will help them as much as developers and users.

The linchpin is the centralized repository. [ note: can even the repository be distributed... sure, but I think that's version 2, as it introduces lots of complexity ] It needs to be rock-solid and very, very simple (complexity breeds downtime). I intend to start working on a prototype system, running from my home network for now. That prototype will be released to a few friends. If you're an addon author or download site maintainer, please let me know if you want to get involved by posting to this article on essays.ajs.com (I'm going to mirror this article in a few places).