Epistemic Status:Describing the general case suggested from my exposure to a finite set of specific events.
I am an academic in Computer Science, and I hope it will be a surprise to no one when I say that the vast majority of knowledge in papers depends on knowledge nowhere to be found in papers. I would say that across computer science this is less true than in many other fields, because many different communities require direct reproducibility of results (within some tolerance for different conditions) via code released by the authors of a paper. This is not nearly as much of a guarantee as it would sound like for three reasons:
(1) Even if you can run the code and get similar results, you don’t know what the code is actually doing, and often neither do the authors.
(2) What the evaluation metrics / scenario which you are evaluating…
View original post 1,436 more words