This is not a “replication crisis”. Running the paper gets you the same results as the author; it’s uniquely replicable. The results not being useful in a product is not the same as a fundamental failure in our use of the scientific process.
>Replicability means that the results hold for replication outside the specific circumstances of one study.
If by "hold for replication outside the specific circumstances of one study" you mean "useful for real world problems" as implied by your previous comment then I don't think you are correct.
From a quick search it seems there are multiple definitions of Reproducibility and Replicability with some using the words interchangeably but the most favorable one I found to what you are saying is this definition:
>Replicability is obtaining consistent results across studies aimed at answering the same scientific question, each of which has obtained its own data.
>[...]
>In general, whenever new data are obtained that constitute the results of a study aimed at answering the same scientific question as another study, the degree of consistency of the results from the two studies constitutes their degree of replication.[0]
However I think this holds true for a lot of ML research going on. The issue is not that the solutions do not generalize. It's that the solution itself is not useful for most real world applications. I don't see what replicability has to do with it. you can train a given model with a different but similar dataset and you will get the same quality non-useful results. I'm not sure exactly what definition of replicability you are using though if there is one I missed please point it out.
