- Citation
- Reproducible Computational Research
- OpenData
- OpenAccess
- OpenScience
- ORGANISATION
- Quality
- Scientific Research
- Resources
- Infrastructure
- VIDEO
- NEWS
- https://www.mediawiki.org/wiki/Citoid
- 2016, Martin Fenner, et.al., A Data Citation Roadmap for Scholarly Data Repositories, DOI: https://doi.org/10.1101/097196, http://biorxiv.org/content/early/2016/12/28/097196
- Katz, D.S. & Smith, A.M., (2015). Transitive Credit and JSON-LD. Journal of Open Research Software. 3(1), p.e7. DOI: http://doi.org/10.5334/jors.by, http://openresearchsoftware.metajnl.com/articles/10.5334/jors.by/
- https://en.wikipedia.org/wiki/Comparison_of_reference_management_software
- https://en.wikipedia.org/wiki/Category:Reference_management_software
- Andreas Rauber, Ari Asmi, Dieter van Uytvanck, Stefan Pröll, Identification of Reproducible Subsets for Data Citation, Sharing and Re-Use: https://www.rd-alliance.org/system/files/documents/TCDL-RDA-Guidelines_160411.pdf
- Comp Reproducibility dialog: contributors incl @NPRougier @khinsen @GaelVaroquaux @todd_gureckis @czzpr & more see: https://twitter.com/o_guest/status/832532349988589568
- Daniel S. Katz, "Is software reproducibility possible and practical?", https://danielskatzblog.wordpress.com/2017/02/07/is-software-reproducibility-possible-and-practical/, Talk Slides.
- Prof. Peng, "Reproducible Research Needs Some Limiting Principles", http://simplystatistics.org/2017/02/01/reproducible-research-limits/
- Git for Data Analysis – why version control is essential for collaboration and for gaining public trust.
- CERN Analysis Preservation: https://github.com/cernanalysispreservation
- Kitzes, J., Turek, D., & Deniz, F. (Eds.). (2017). The Practice of Reproducible Research: Case Studies and Lessons from the Data-Intensive Sciences. Oakland, CA: University of California Press.
- Titus Brown says "How I learned to stop worrying and love the coming archivability crisis in scientific software".
- 2017Jan10, Munafò M.R., et.al., A manifesto for reproducible science, Nature Human Behaviour 1, Article number: 0021 (2017), doi:10.1038/s41562-016-0021, http://www.nature.com/articles/s41562-016-0021
- Introduction to Computational Reproducibility
- The American Statistical Association Announces A New Reproducibility Policy: https://replicationnetwork.com/2017/01/07/the-american-statistical-association-announces-a-new-reproducibility-policy/
- Jo Barratt - Continuous data validation in scientific research - #scidata16.
- “It’s not reproducible if it only runs on your laptop”: Jon Zelner’s tips for a reproducible workflow in R and Stan
- Advancing Open Research: Q&A with Bastian Greshake
- Video-1 (6-part series): Improving the Reproducibility of Our Research Practices Using Open Science Framework, https://www.youtube.com/watch?list=PLMOU-iLiJIc0amNVabGXJ0liKwIwxqkO8&v=kzUtpDBo8wk
- Barbagroup Reproducibility Syllabus/, http://lorenabarba.com/blog/barbagroup-reproducibility-syllabus/
- https://twitter.com/search?q=%23reproducibility&src=typd
- Repeatability in Computer Science: http://reproducibility.cs.arizona.edu/
- 2016, Victoria Stodden Et al, Enhancing reproducibility for computational methods, http://science.sciencemag.org/content/354/6317/1240.full, Science, 09Dec2016, Vol. 354, Issue 6317, pp. 1240-1241, DOI: 10.1126/science.aah6168
- Reproducibility Project: Cancer Biology seeks participants to help evaluate reproducibility: http://blogs.biomedcentral.com/bmcblog/2016/06/03/recruiting-participants-evaluate-reproducibility/
- Blockchain : http://blogs.lse.ac.uk/impactofsocialsciences/2016/07/21/could-blockchain-provide-the-technical-fix-to-solve-sciences-reproducibility-crisis/
- http://www.nature.com/news/dutch-agency-launches-first-grants-programme-dedicated-to-replication-1.20287
- http://www.nature.com/nature/journal/v534/n7607/full/534326d.html
- https://github.com/Reproducible-Science-Curriculum
- A course on https://www.coursera.org/learn/reproducible-research
- Gael's blog posts :
- Prof. (Dr.) Titus Brown:
- https://danielskatzblog.wordpress.com/2015/04/20/scientific-software-quality-versus-purpose/
- http://www.qualitynano.eu/home/about-qnano.html
- Running an open experiment - transparency and reproducibility in soil and ecosystem science :: http://iopscience.iop.org/article/10.1088/1748-9326/11/8/084004/meta
- Research integrity: Don't let transparency damage science :: http://www.nature.com/news/research-integrity-don-t-let-transparency-damage-science-1.19219
- High-impact journals: where newsworthiness trumps methodology, 'Sexy Science Vs. Reproducibility' http://blogs.lse.ac.uk/impactofsocialsciences/2013/03/15/high-impact-journals-where-newsworthiness-trumps-methodology/
- 2016, Prasad Patil, Roger D. Peng, Jeffrey T. Leek, A statistical definition for reproducibility and replicability
- The TOMS Initiative and Policies for Replicated Computational Results (RCR): http://toms.acm.org/replicated-computational-results.cfm
- Muddled meanings hamper efforts to fix reproducibility crisis: http://www.nature.com/news/muddled-meanings-hamper-efforts-to-fix-reproducibility-crisis-1.20076
- https://news.ycombinator.com/item?id=12185964
- https://lwn.net/Articles/674573/
- Monya Baker: http://www.nature.com/search?journal=am%2Capp_notes%2Cavianflu%2Cclimate%2Cnature%2Cnaturejobs%2Cnbt%2Cncb%2Cnchem%2Cnchembio%2Cnchina%2Cnclimate%2Cncomms%2Cneuro%2Cng%2Cngeo%2Cni%2Cnindia%2Cnm%2Cnmat%2Cnmeth%2Cnmiddleeast%2Cnnano%2Cnphoton%2Cnphys%2Cnprot%2Cnsmb%2Cscientificamerican%2Cscientificamericanmind%2Csrep%2Cstemcells%2Cblogs&order=date_desc&pag-start=1&q=%22Monya%20Baker%22&q_match=all&sp-m=1
- http://metrics.stanford.edu/research/reproducibility
- http://www.sciencemag.org/news/2017/01/rigorous-replication-effort-succeeds-just-two-five-cancer-papers
- https://www.theatlantic.com/science/archive/2017/01/what-proportion-of-cancer-studies-are-reliable/513485/
- https://www.pbs.org/wgbh/nova/next/body/reproducibility-explainer/
- at #OpenDataDay, a group of us started to look at the reproducibility of Jupyter notebooks published along with papers in PubMed Central. While we did not get as far as we had hoped we would, the exercise has provided some valuable insights about the mechanics of reproducibility: https://markwoodbridge.com/2017/03/05/jupyter-reproducible-science.html Comments, suggestions and contributions are welcome via: sparcopen/open-research-doathon#25
- 2017Jan19: Hanne Watkins, System Justification in Social Psychology? A Survey of Responses to the Replicability Debate in Psychology, OSF.io, https://osf.io/preprints/psyarxiv/nn67y/
- 2010, Zeeya Merali, Computational science: ...Error …why scientific programming does not compute, Nature 467, 775-777 (2010), http://www.nature.com/news/2010/101013/full/467775a.html, DOI:10.1038/467775a.
- Phaf R. Hans, Replication Requires Psychological Rather than Statistical Hypotheses: The Case of Eye Movements Enhancing Word Recollection, Front. Psychol., 26 December 2016, DOI:https://doi.org/10.3389/fpsyg.2016.02023, http://journal.frontiersin.org/article/10.3389/fpsyg.2016.02023/full
- http://db-reproducibility.seas.harvard.edu/papers/#SIGMOD2015
- Ronald F. Boisvert, ACM, Incentivizing Reproducibility : http://cacm.acm.org/magazines/2016/10/207757-incentivizing-reproducibility/fulltext
- Psychology: http://andrewgelman.com/2016/09/21/what-has-happened-down-here-is-the-winds-have-changed/
- Reproducibility in Chemical Research : http://onlinelibrary.wiley.com/doi/10.1002/anie.201606591/pdf
- http://www.nature.com/news/reality-check-on-reproducibility-1.19961
- http://www.nature.com/nature/journal/v532/n7598/full/532177d.html
- http://www.nature.com/news/announcement-reducing-our-irreproducibility-1.12852
- https://systems.soe.ucsc.edu/sites/default/files/images/jimenez-srls15-presentation.pdf
- Chris Drummond, Replicability is not Reproducibility: Nor is it Good Science, 2009, http://cogprints.org/7691/7/ICMLws09.pdf
- 2016Sep28: Monya Baker, Reproducibility: Seek out stronger science, Nature, 537, 703-704 (2016), DOI:10.1038/nj7622-703a. Talks about statistics.
- A report states that F1000Research launched Faculty of 1000 for “Preclinical Reproducibility and Robustness”.
- Amgen researchers on Drug development: Raise standards for preclinical cancer research.
- Contra view: If a study fails to be replicated it does not automatically mean that its results are to be considered false, http://www.nature.com/news/biotech-giant-publishes-failures-to-confirm-high-profile-science-1.19269
- 2015, Florian Markowetz, Five selfish reasons to work reproducibly, DOI: 10.1186/s13059-015-0850-7
- 2016Oct28, Bertram Ludaescher, Kyle Chard, Niall Gaffney, Matthew B. Jones, Jaroslaw Nabrzyski, Victoria Stodden, Matthew Turk, Capturing the "Whole Tale" of Computational Research: Reproducibility in Computing Environments, https://arxiv.org/abs/1610.09958
- Video: Sile Lane talks about The hidden side of clinical trials as does the hashtag #AllTrials on OpenData.
- http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003285
- Rule 1: For Every Result, Keep Track of How It Was Produced
- Rule 2: Avoid Manual Data Manipulation Steps
- Rule 3: Archive the Exact Versions of All External Programs Used
- Rule 4: Version Control All Custom Scripts
- Rule 5: Record All Intermediate Results, When Possible in Standardized Formats
- Rule 6: For Analyses That Include Randomness, Note Underlying Random Seeds
- Rule 7: Always Store Raw Data behind Plots
- Rule 8: Generate Hierarchical Analysis Output, Allowing Layers of Increasing Detail to Be Inspected
- Rule 9: Connect Textual Statements to Underlying Results
- Rule 10: Provide Public Access to Scripts, Runs, and Results
- JORS: The Journal of Open Research Software
- JOSS: The Journal of Open Source Software, http://joss.theoj.org
- https://lgatto.github.io/software-data-papers/
- Björn on Why haven’t we already canceled all subscriptions? http://bjoern.brembs.net/2016/05/why-havent-we-already-canceled-all-subscriptions/
- An OpenAccess Journal of Society for Neuroscience
- A peer-reviewed publication path for software that is not necessarily scientifically novel.
- In statistics there is the Journal of Statistical Software.
- In neuroscience/imaging there are things like the Insight Journal and the "Frontiers in..." imprints which have software-specific tracks.
- Computer Physics Communications publishes a lot of articles and papers that are more intended to document free/open-source implementations than they are to describe novel algorithms. This is quite useful as a way to give people something to cite when they use your code, and also to obtain more traditional academic credit for free-software work.
- Stata and R have their own journals: Stata Journal and R Journal.
- The ActivePapers project and paper: http://f1000research.com/articles/3-289/v3
- For-Profit Journals Must End :: http://read.hipporeads.com/for-profit-journals-must-end/
- Easy steps towards open scholarship :: http://blogs.lse.ac.uk/impactofsocialsciences/2013/05/24/easy-steps-towards-open-scholarship
- Beware! Academics are getting reeled in by scam journals: http://www.universityaffairs.ca/features/feature-article/beware-academics-getting-reeled-scam-journals/
- 2017: Masuzzo P, Martens L. (2017) Do you speak open science? Resources and tips to learn the language. PeerJ Preprints 5:e2689v1, https://doi.org/10.7287/peerj.preprints.2689v1, https://peerj.com/preprints/2689v1/
- https://github.com/CenterForOpenScience/osf.io
- https://home.scienceopen.com/
- https://mitpress.mit.edu/sites/default/files/titles/content/openaccess/Suber_08_chap1.html
- Publisher copyright policies & self-archiving : http://www.sherpa.ac.uk/romeo/index.php
- Why scientists must share their research code: 'Reproducibility editor' Victoria Stodden explains the growing movement to make code and data available to others.
- http://retractionwatch.com/2016/09/24/weekend-reads-worlds-most-prolific-peer-reviewer-replication-backlash-carries-on-controversial-pace-study-re-analyzed/
- A counterview to Prof. Susan T. Fiske post about attacks on OpenScience/OpenData/OpenAccess sans peer-review: https://www.dropbox.com/s/9zubbn9fyi1xjcu/Fiske%20presidential%20guest%20column_APS%20Observer_copy-edited.pdf
- Why scientists must share their research code, https://cyberlaw.stanford.edu/press/why-scientists-must-share-their-research-code
- A study of institutional spending on open access publication fees in Germany: https://peerj.com/articles/2323/
- Benefits of Open Data: https://www.europeandataportal.eu/en/using-data/benefits-of-open-data
- Building a European Data Economy: https://ec.europa.eu/digital-single-market/en/building-european-data-economy
- Report on the Research conducted by the European Data Portal: Creating Value through Open Data
- Open Data Goldbook
- 2016Dec12, DANS-EUDAT-OpenAIRE webinar on "FAIR Data in Trustworthy Data Repositories".
- Defining Open Data: http://blog.okfn.org/2013/10/03/defining-open-data/
- http://www.opendatainception.io/
- Research impact of paywalled versus open access papers :: http://www.1science.com/oanumbr.html
- Hetionet in Neo4j, https://neo4j.het.io/browser/
- http://www.pasteur4oa.eu/sites/pasteur4oa/files/resource/Scholarly%20Platforms%20Briefing%20Paper_FINAL.pdf
- https://www.liberquarterly.eu/articles/10.18352/lq.10137/
- What Drives Academic Data Sharing?, Benedikt Fecher, Sascha Friesike, Marcel Hebing, PLOS. Published: February 25, 2015. DOI: 10.1371/journal.pone.0118053
- http://www.nature.com/sdata/
- Feb.2015, A Reputation Economy: Results from an Empirical Survey on Academic Data Sharing : http://www.ratswd.de/dl/RatSWD_WP_246.pdf
- Examining Data Sharing and Data Reuse in the DataONE Environment, https://www.asis.org/asist2014/proceedings/submissions/posters/335poster.pdf
- OECD 2015, “Making Open Science a Reality”, OECD Science, Technology and Industry Policy Papers, No. 25, OECD Publishing, Paris.
- Shared Responsibilities in Sharing Research Data: Policies and Partnerships Report of an ESF-DFG workshop, Padua, 21 September 2007 : http://www.esf.org/fileadmin/Public_documents/Publications/SharingData_01.pdf
- Reproducible Researchfor Scientific Computing:Tools and Strategiesfor Changing the Culture : http://www.bitss.org/wp-content/uploads/2014/06/reproducible-research-for-scientific-computing-tools-and-strategies-for-changing-the-culture3.pdf
- [Data Sharing in Research : Four Key Concerns](https://erc.europa.eu/sites/default/files/content/pages/pdf/2.1 Sabina Leonelli.pdf)
- Workshop on Research Data Management and Sharing, September 2014, ERC Executive Agency, Brussels, Abstracts of the presentations : https://erc.europa.eu/sites/default/files/content/pages/pdf/ERC_WS_on_RDMS_abstracts.pdf
- DOI: 10.1056/NEJMe1516564, Data Sharing, Dan L. Longo, M.D., and Jeffrey M. Drazen, M.D., N Engl J Med 2016; 374:276-277, 2016-January-21.
- Roger D. Peng, Reproducible research and Biostatistics, 2009.
- http://www.thelancet.com/series/research
- http://www.nejm.org/doi/full/10.1056/NEJMe1515172
- Reproducible Workflow: http://datasci.kitzes.com/lessons/python/reproducible_workflow.html
- The Harvard Dataverse Network: http://library.harvard.edu/gdc
- The Importance and the Complexities of Data Sharing, http://www.nejm.org/doi/full/10.1056/NEJMe1611027#pq=v6nEwl
- European countries are reaping the benefits of Open Data : https://ec.europa.eu/digital-single-market/en/blog/european-countries-are-reaping-benefits-open-data
- How is Open Data being re-used in Europe? https://www.europeandataportal.eu/en/content/how-open-data-being-re-used-europe
- The natural selection of bad science, https://arxiv.org/pdf/1605.09511v1.pdf
- Do That Again: A new initiative offers gold stars to researchers willing to have their studies replicated by other labs, but will it fix science’s growing irreproducibility problem?
- A reproducibility crisis?, October 2015.
- Estimating the reproducibility of psychological science, Science, 28 Aug 2015, DOI: 10.1126/science.aac4716
- Comment on “Estimating the reproducibility of psychological science”, by Daniel T. Gilbert1, Gary King, Stephen Pettigrew, Timothy D. Wilson, Science 04 Mar 2016, DOI: 10.1126/science.aad7243
- Scandals, http://www.the-scientist.com/?articles.view/articleNo/33695/title/Top-Science-Scandals-of-2012/
- Drug development: Raise standards for preclinical cancer research, http://www.nature.com/nature/journal/v483/n7391/full/483531a.html
- Bayer paper: http://www.nature.com/nrd/journal/v10/n9/full/nrd3439-c1.html
- https://www.rd-alliance.org/group/repository-platforms-research-data-ig/outcomes/matrix-use-cases-and-functional-requirements
- https://www.coar-repositories.org/activities/webinar-and-discussion/webinar-next-generation-repositories/
- Data Management Plan for Moore Investigator in Data Driven Discovery Grant: http://riojournal.com/articles.php?journal_name=rio&id=10708
- Managing small to medium-sized research software projects and on github
- https://en.wikipedia.org/wiki/Data_management_plan
- Guidelines on FAIR Data Management in Horizon 2020 : http://ec.europa.eu/research/participants/data/ref/h2020/grants_manual/hi/oa_pilot/h2020-hi-oa-data-mgt_en.pdf
- IFLA special issue on data services in libraries: http://www.ifla.org/files/assets/hq/publications/ifla-journal/ifla-journal-42-4_2016.pdf
- Christian T. Jacobs and Alexandros Avdis: Git-RDM: A research data management plugin for the Git version control system. http://joss.theoj.org/papers/10.21105/joss.00029
- Contemporary Challenges for Data-Intensive Scientific Workflow Management Systems, http://www.switchproject.eu/wp-content/uploads/2016/04/uva-2.pdf
- Five Things You Need to Know About RDM and the Law: DCC Checklist on Legal Aspects of RDM.
- #scidata16 keynote highlights: “Research data management for early career researchers”, http://blogs.nature.com/naturejobs/2016/12/01/scidata16-keynote-highlights-research-data-management-for-early-career-researchers/
- Data Re-purpose toolkit: http://www.ands.org.au/partners-and-communities/23-research-data-things/toolkit
- http://www.software.ac.uk/online-sustainability-evaluation
- On the Failings of Qualitative Inquiry: https://au.sagepub.com/sites/default/files/upm-binaries/24799_02_Hammersley_Ch_01.pdf
- How to Manage and Share Your Open Data: https://cos.io/blog/how-manage-and-share-your-open-data/
- Gene name errors are widespread in the scientific literature : http://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-1044-7
- The image detective who roots out manuscript flaws. Jana Christopher discusses her job as a data integrity analyst at EMBO Press: http://www.nature.com/news/the-image-detective-who-roots-out-manuscript-flaws-1.17749
- Paper retraction by @DanielBolnick due to R-lang statistical errors as a blog post "Wrong a lot?" and the tweet reference: https://twitter.com/BrianNosek/status/817708812593405952
- Scientific computing: Code alert, Monya Baker.
- http://k4webcast.mediasite.com/Mediasite/Play/2bc15b283f324784a945d79d9a06c76c1d
- 2016, Baker-Nature, http://doi.org/10.1038/533452a, http://www.nature.com/polopoly_fs/1.19970!/menu/main/topColumns/topLeftColumn/pdf/533452a.pdf
- 2014, Wilson et. al., Best Practices for Scientific Computing.
- https://pure.uvt.nl/portal/files/3053560/SP_Brandt_open_PoPS_2012.pdf
- https://en.wikipedia.org/wiki/Reproducibility_Project
- http://www.rug.nl/research/portal/files/13617856/derksenpresentation.pdf
- Prof. Yolanda Gil, http://www.isi.edu/~gil/
- http://semantic-cora.org
- http://semantic-cora.org/index.php/Publications_and_Talks
- https://people.stanford.edu/gentzkow/sites/default/files/codeanddata.pdf
- 2016, Marc A. Edwards and Siddhartha Roy, Academic Research in the 21st Century: Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition, http://online.liebertpub.com/doi/pdf/10.1089/ees.2016.0223
- Rules of good scientific practice, procedures, and penalties in the event of malpractice: https://www.humboldt-foundation.de/web/docs/F25752/rules_scientific_practice.pdf
- https://www.google.de/search?q=Principles+of+good+Scientific+practice&ie=utf-8&oe=utf-8&gws_rd=cr&ei=KeM6V4zBJ6WfgAbu4LbQCQ
- LibGen, http://93.174.95.27 and Sci-Hub, http://80.82.78.170
- http://cstheory.stackexchange.com/questions/1168/what-papers-should-everyone-read
- https://github.com/firaswehbe/data-programming-resources/blob/master/openscience.md
- https://erc.europa.eu/sites/default/files/content/pages/pdf/2.1%20Sabina%20Leonelli.pdf
- statistical analysis, https://zedshaw.com/archive/a-rubric-for-it-analysis-papers/
- https://www.google.de/search?q=infrastructure+for+analytical+reproducibility&ie=utf-8&oe=utf-8&gws_rd=cr&ei=GNp6V7e-CKqWgAatkYXYCQ
- http://www.kdd.org/exploration_files/p1V11n1.pdf
- https://scholar.google.de/scholar?oe=utf-8&gws_rd=cr&um=1&ie=UTF-8&lr&cites=11895311725021913820
- http://www.nature.com/news/statisticians-issue-warning-over-misuse-of-p-values-1.19503
- http://www.treesmapsandtheorems.com/
- https://en.wikipedia.org/wiki/PechaKucha
- http://blogs.lse.ac.uk/impactofsocialsciences/2016/09/12/how-proximity-and-trust-are-key-factors-in-getting-research-to-feed-into-policymaking/
- Should we cite blog posts during peer review? https://medium.com/@tpoi/should-we-cite-blog-posts-during-peer-review-20c86a6378a5#.o6deuigq1
- http://blogs.lse.ac.uk/impactofsocialsciences/2017/02/01/disentangling-the-academic-web-what-might-have-been-learnt-from-discogs-and-imdb/
- Code Ocean:: A cloud-based executable research platform that provides researchers and developers with an easy way to share, discover and run code published in academic journals and conferences.
- ActivePapers :: A tool for working with executable papers, which combine data, code, and documentation in single-file packages, suitable for publication as supplementary material or on sites such as figshare.
- REANA: high-energy-physics data analysis - preservation and reuse. Open Repositories, 2017 presentation: http://simko.home.cern.ch/simko//tmp/or2017-reana-2017-06-28.pdf
- Databrary :: A video data library for developmental science. Share videos, audio files, and related metadata. The source codes on github.
+ http://bjoern.brembs.net/2016/12/so-your-institute-went-cold-turkey-on-publisher-x-what-now/