The International Working Conference on Mining Software Repositories (MSR) has hosted a mining challenge since 2006. With this challenge we call upon everyone interested to apply their tools to bring research and industry closer together by analyzing a common data set. The challenge is for researchers and practitioners who bravely put their mining tools and approaches on a dare. 

This year the main theme of the challenge is on comparison of projects. We selected four open source projects, and you should use your brain, tools, computational power, and magic to compare them and uncover interesting similarities and differences. The projects are Eclipse and Netbeans, two popular IDEs written in Java (Group 1) and Firefox and Chrome, two web browsers written in C/C++ (Group 2). We encourage you to analyze more than one project, ideally in the same group, but you can also choose to analyze a single project.

How to Participate in the Challenge

To participate in the challenge, follow the following four steps:

1. Select one or more projects from the list below. Ideally you would analyze two projects within a group.
2. Download the data you want to analyze. We provide you with links on how to get data. Feel free to use additional data.
3. Discover interesting similarities or differences and summarize them in a four-page challenge report (see below for contents + format)
4. Submit your report to EasyChair before Sunday, February 20th, 2011.

For inspiration and ideas on what makes good contributions to the challenge, have a look at past MSR publications and the Hall of Fame of the winners of the past mining challenges.

Group 1: Eclipse and Netbeans (IDEs)

Eclipse and Netbeans are two top of the line Java development environments. Both provide support for a wide variety of development tasks such as refactoring and debugging. Both IDEs are programmed in Java and use a Bugzilla issue tracker to track reported bugs and feature requests. Can you assess which plug-in platform is better? Which code is more stable or less failure-prone? Or how easy is it to migrate a Netbeans plug-in to Eclipse? Just come up with a question and investigate these two IDEs to find an answer.

DATA
Eclipse: CVS.zip (6.1 GB) - bugzilla.mysql.zip (424 MB)
Netbeans: MERCURIAL.tar.gz (2.1 GB) - bugzilla.mysql.zip (4.6 GB)

The CVS.zip and MERCURLIAL.tar.gz contain the compressed CVS and MERCURIAL repositories respectively. The bugzilla.mysql.zip contain dumps of the bugzilla databases for the projects.

Group 2: Firefox and Chrome (Web Browsers)

Firefox and Chrome are two very popular web browsers, which implement a similar set of features using C/C++. Firefox uses a Bugzilla data base to track bug reports whereas Chrome uses their own Google issue reporting system. Being two very sophisticated web browsers, can you say which one is better from a developer point of view? Which one is easier to debug and which can run without crashing? Come up with a question and investigate these two browsers to find an answer.

DATA
Mozilla: CVS.zip (968 MB), MERCURIAL.tar.gz (372 MB) - bugs.html.zip (12 GB) part 1, 2, 3, 4, 5
Chrome: SVN.zip (1.6 GB) - bugs.html.zip (241 MB)

The CVS.zip, MERCURIAL.tar.gz and the SVN.zip contain the compressed CVS, MERCURIAL and SVN repositories. Note that for the Mozilla project the CVS.zip contains the source code for Mozilla until version 1.9.x whereas the MERCURIAL.tar.gz contains the Mozilla versions from there on. The files bugs.html.zip contain html files containing the bug information.

Details on the Challenge Report

The challenge report should describe the results of your work and cover the following aspects: questions addressed, input data, approach and tools used, derived results and interpretation of them, and conclusions. Keep in mind that the report will be evaluated by a jury. Make sure your contributions, purpose, scope, results and importance or relevance of your work is highlighted within your report. 

Challenge reports must be at most 4 pages long and must conform at time of submission to the ICSE (and MSR) 2011 Format and Submission Guidelines.

Submission Details

Submit your challenge report (at most four pages) to EasyChair before the submission deadline on Sunday, February 20th, 2011. Please submit your challenge reports to the "MSR 2011 Challenge Track". For the dates of author notification and camera-ready, please check the Important Dates.

Papers submitted for consideration should not have been published elsewhere and should not be under review or submitted for review elsewhere during the duration of consideration. ACM plagiarism policies and procedures shall be followed for cases of double submission.

Upon notification of acceptance, all authors of accepted papers will be asked to complete an ACM Copyright form and will receive further instructions for preparing their camera ready versions. At least one author of each paper is expected to present the results at the MSR 2011 conference. All accepted contributions will be published in the conference electronic proceedings.

Prize

Program Committee