An “Open” Proposal – Encouraging Code Sharing at ISMIR 04 February 2013

As a young, evolving research community, every year presents us with the opportunity to improve: to push the boundaries of human knowledge, to involve more talented, new faces, to refine our practices and standards. The first is an outcome of our dedicated efforts and the second a natural consequence of this progress, but the third can often require organization and collective action that is more difficult to achieve organically. Not to be deterred from such inherent challenges, I would like to propose a slight modification to the ISMIR Conference that could significantly benefit us and our research: simply give authors the option of publishing source code as part of the proceedings. Why is this worth considering? In addition to the observation that other research communities have been doing this for some time (to publish in Science, for example, one must make source code publicly available [1]), the virtues of open scientific software are myriad [2]. In the interest of brevity though, there are three worth mentioning here: ⁃ Source code serves as a starting point for future work. There are several positive impacts of having access to the actual code on which research is based, from enabling reproducible results to fair comparisons with new algorithms. Additionally, novel methods and ideas are far more sustainable if it's easier for others to build upon and continue them. ⁃ Simply writing code with the idea that another human might actually read it inherently makes it, and you, better. If you need proof, just try it. Your code will invariably be more stable, readable, and maintainable. Everyone understands that research code is not production code, surely, but nor is it an excuse to write spaghetti. This makes it easier to continue the work you’ve already done, on your own or by others. ⁃ It’s not always possible to exactly reproduce research or results from a publication. Sometimes, the only text that can do that is the code itself. Page limitations constrain the level of detail used to describe a system or algorithm, slight implementation differences can drastically change performance, and equations, as well as their explanations, are certainly not immune to typos. Couple this with the challenge of expressing complex systems in clear, concise language, and it's not hard to see why source code might be really helpful.

That said, encouraging and supporting the sharing of source code at the next ISMIR is a relatively easy change to embrace, and now, months ahead of the submission deadline, is the perfect time to consider it. What would have to happen to make this a reality?

1. Plan ahead. Code sharing is substantially more pleasant for everyone involved if you think you might from the start. 2. Give authors the option to submit source code with a paper. It wouldn't be mandatory, nor would it negatively affect the review process; the intent is only to supplement the writing, and obviously won’t be relevant to all topics or submissions. 3. Acknowledge published source code. For example, papers with corresponding code could be marked in the proceedings with a "source code star" next to the title. It's all about good karma, folks. 4. Make source code available as part of the proceedings. Think of it like a research snapshot, to be kept all in one place, online and in the electronic proceedings. We don't version control the papers (yet), so we don't need to version control the code (yet). Baby steps.

As a final comment, it is important to remember that published results and observations ultimately depend on the code that produced them; therefore, if a paper is good enough to go to print, then so is the code it's based on. Following these four steps, we could collectively alter the course of music informatics research in great ways. Moving forward, I would leave it to the conference organizers to decide on whether or not to incorporate the change (as participation would be optional). We all might benefit from the discussion though, both on and off the mailing list, and I'm looking forward to any and all feedback from the group.

Eric J. Humphrey Ph.D. Candidate, MARL@NYU

[1] Brown. Journal pubs -- data and software release policies. Posted 14 July 2012. [2] Prlić A, Procter JB (2012) Ten Simple Rules for the Open Development of Scientific Software. PLoS Comput Biol 8(12)