Submit to our NeurIPS 2020 Workshop!!

News : We have published the report of NeurIPS 2019 Retrospectives workshop!

Talk about flaws in your paper's methodology "We didn't actually optimize the performance of the baseline very much. We thought using the default parameters from previous work would be fine, but on our dataset it leads to instability."
Reveal limitations of your approach "We also tested our model on CIFAR-100, but it didn't work that well, so we didn't include it in the paper."
Discuss new perspectives on your work "When we wrote the paper, we thought our method would be most applicable to hierarchical RL. But now I think it'd work really well in multi-agent RL, with centralized learning and decentralized execution."

Start writing your retrospective here! See some published retrospectives from our NeurIPS 2019 workshop.


What is a retrospective?

Think about how you talk about your research when writing a paper.


(Pause for a second.)

Now, think about how you’d talk about your research to a good friend who wanted to build on it.


Feel the difference? For most of us, writing papers is like a performance: we’re trying to make our results look as good as possible so that reviewers will like it. There’s nothing wrong with that! But often, this performance hides some important parts about our research: tricks that were crucial to get a model to work, or limitations of the approach on different datasets. Sometimes, we don’t realize until later that there were flaws in the original paper. The problem is that researchers aren’t really incentivized to go back and fix them, or to talk about them at all.

We think that being open and transparent about your past research is really important. That’s why we’re introducing ML Retrospectives as a venue to incentivize this form of scholarship. ML Retrospectives hosts submissions of retrospectives, which are basically blog posts where you talk honestly about your past papers (see: how it works). Retrospectives submitted before May 18 will be considered for publication at the ICML 2020 Workshop!

The ultimate goal of retrospectives is to make research more human.

That means getting researchers to write papers like how they’d talk to a friend. It means making research transparent and accessible for everyone. And it means creating an environment in the machine learning community where it’s okay to make mistakes. After all, in the scientific endeavor, we’re all on the same team.

Have a previous paper where, looking back, you think: “Huh, there’s a lot more I could say about this now”? Consider writing a retrospective! Here’s how to do it. Know a researcher you think could write a great retrospective? Let them know on social media!

Questions? Contact ml.retrospectives@gmail.com.