jonathan@researchcomputingteams.org
Parent categories: Technical Leadership
Other tags: | Technical Leadership: Software Development |
Quantifying Independently Reproducible Machine Learning - Edward Raff, writing at The Gradient We worry a lot about about replication and reproducibility in research computing. In this article, the author — who attempted to independently replicate the results and basic methods in 255 (!!!) ML papers. Crucial here is independent replication; it’s not enough to just run the code, but to implement independently. He was successful 162 times. That’s enough papers to do some quantitative analysis, and it’s interesting what aspects of the work were not...
Continue...Asymptotics of Reproducibility - Roger Peng A reminder that reproducibility/repeatability is not an immutable property of some computational work — it decays over time, requires maintenance, and that maintenance has to be done by someone.
Continue...