You are on page 1of 2

An Open Proposal Encouraging Code Sharing at ISMIR 04 February 2013

As a young, evolving research community, every year presents us with the opportunity to improve: to push the boundaries of human knowledge, to involve more talented, new faces, to rene our practices and standards. The rst is an outcome of our dedicated efforts and the second a natural consequence of this progress, but the third can often require organization and collective action that is more difcult to achieve organically. Not to be deterred from such inherent challenges, I would like to propose a slight modication to the ISMIR Conference that could signicantly benet us and our research: simply give authors the option of publishing source code as part of the proceedings. Why is this worth considering? In addition to the observation that other research communities have been doing this for some time (to publish in Science, for example, one must make source code publicly available [1]), the virtues of open scientic software are myriad [2]. In the interest of brevity though, there are three worth mentioning here: Source code serves as a starting point for future work. There are several positive impacts of having access to the actual code on which research is based, from enabling reproducible results to fair comparisons with new algorithms. Additionally, novel methods and ideas are far more sustainable if it's easier for others to build upon and continue them. Simply writing code with the idea that another human might actually read it inherently makes it, and you, better. If you need proof, just try it. Your code will invariably be more stable, readable, and maintainable. Everyone understands that research code is not production code, surely, but nor is it an excuse to write spaghetti. This makes it easier to continue the work youve already done, on your own or by others. Its not always possible to exactly reproduce research or results from a publication. Sometimes, the only text that can do that is the code itself. Page limitations constrain the level of detail used to describe a system or algorithm, slight implementation differences can drastically change performance, and equations, as well as their explanations, are certainly not immune to typos. Couple this with the challenge of expressing complex systems in clear, concise language, and it's not hard to see why source code might be really helpful.

That said, encouraging and supporting the sharing of source code at the next ISMIR is a relatively easy change to embrace, and now, months ahead of the submission deadline, is the perfect time to consider it. What would have to happen to make this a reality?

1. Plan ahead. Code sharing is substantially more pleasant for everyone involved if you think you might from the start. 2. Give authors the option to submit source code with a paper. It wouldn't be mandatory, nor would it negatively affect the review process; the intent is only to supplement the writing, and obviously wont be relevant to all topics or submissions. 3. Acknowledge published source code. For example, papers with corresponding code could be marked in the proceedings with a "source code star" next to the title. It's all about good karma, folks. 4. Make source code available as part of the proceedings. Think of it like a research snapshot, to be kept all in one place, online and in the electronic proceedings. We don't version control the papers (yet), so we don't need to version control the code (yet). Baby steps.

As a nal comment, it is important to remember that published results and observations ultimately depend on the code that produced them; therefore, if a paper is good enough to go to print, then so is the code it's based on. Following these four steps, we could collectively alter the course of music informatics research in great ways. Moving forward, I would leave it to the conference organizers to decide on whether or not to incorporate the change (as participation would be optional). We all might benet from the discussion though, both on and off the mailing list, and I'm looking forward to any and all feedback from the group.

Eric J. Humphrey Ph.D. Candidate, MARL@NYU

[1] Brown. Journal pubs -- data and software release policies. Posted 14 July 2012. [2] Prli A, Procter JB (2012) Ten Simple Rules for the Open Development of Scientic Software. PLoS Comput Biol 8(12)

You might also like