Resources for PhD candidates

Books

• “Guide to Advanced Empirical Software Engineering” by Forrest Shull, Janice Singer, and Dag I. K. Sjøberg.
• “How To Write A Better Thesis” by David Evans, Justin Zobel, and Paul Gruba.
• “Writing for Computer Science” by Justin Zobel.
• “Contemporary Empirical Methods in Software Engineering” by Michael Felderer and Guilherme Horta Travassos.

How to do research?

• “PhD Completion Milestones” by Laurie Williams.
• “How to write a great research paper” by Simon Peyton Jones.
• “How to do good research” by Frédo Durand. Part of Resources for Students & Scholars.
• “Writing Research Papers” by Aaron Hertzmann. Look at “Advice for Graduate Students” and “Graduate Skills Seminars, 2010-2011” as well.
• “How to do research” by William T. Freeman.
• Researcher guidelines for Linux
• “Writing a scientific article: A step-by-step guide for beginners”
• I **ing hate Science

Writing

Articles

• “Writing Well” by Julian Shapiro.
• “Writing a book: is it worth it?” by Martin Kleppmann.
• “How to write a scientific abstract in six easy steps”.
• “Three Sins of Authors in Computer Science and Math” by Jonathan Shewchuk.
• APA Style Blog.
• “Writing Tips for IEEE Software” by IEEE.
• “Mathematical Writing” by Donald E. Knuth, Tracy Larrabee, and Paul M. Roberts. The PDF version is here. All the lectures are also on YouTube.
• “How (and How Not) to Write a Good Systems Paper” by Roy Levin and David D. Redell. A second copy is here.
• “How to Get a Paper Accepted at OOPSLA” by Kent Beck.
• “The Science of Scientific Writing” by George D. Gopen and Judith A. Swan, In American Scientist, Vol. 78, No. 6 (Nov–Dec, 1990), pp. 550–558.
• “On writing” by Terence Tao.
• The Grammar of Mathematics: Writing About Variables
• How to write the introduction

Books

• “The Elements of Style” by William Strunk Jr. and E. B. White.
• Either “The Dictionary of Concise Writing: 10,000 Alternatives to Wordy Phrases” or “To the Point: A Dictionary of Concise Writing” by Robert Hartwell Fiske.
• “Bugs in Writing: A Guide to Debugging Your Prose” by Lyn Dupre.

LaTeX

• “Advice for writing LaTeX documents”.
• “style-check for LaTex. Good references for more concise writing.
• Common errors in bibliographies
• Keeping tables/figures close to where they are mentioned
• Basic diagram examples
• LaTeX/Presentations
• LaTeX Books and Articles
• How to Write a Minimalistic CV in LaTeX
• Self-publishing your (LaTeX) thesis
• LaTeX book with examples, open-source eBook

Talking

• Oral Presentation Advice

Reviewing papers

• “Hints for Reviewing Empirical Work in Software Engineering” by Walter F. Tichy. Direct PDF.
• “Reviewing as an Essential Component of a Researcher’s Life” by Massimiliano Di Penta.
• “Have You Seen This Review?” by Dirk Riehle.

How to write rebuttals?

• “Writing an effective response to a manuscript review”.
• It’s just going through the motions … “The Toxic Culture of Rejection in Computer Science” by Edward Lee.

Completing your thesis

• IEEE Post-Publication Policies
• IEEE: Can I reuse my published article in my thesis?
• ACM Author Rights
• Can I include Figures and Texts from IEEE/ACM publications in my PhD thesis?

Mathematical statistics

General

• NIST/SEMATECH e-Handbook of Statistical Methods
• “Handbook of Biological Statistics” by John H. McDonald. https://www.youtube.com/@datatab/videos
• “DATAtab YouTube channel”
• “StatQuest with Josh Starmer YouTube channel”
• “Biostatistics YouTube channel”
• “Biostatistics Book Series” by Frans Rodenburg.
• Cochran–Armitage test for trend
• A Look at Precision, Recall, and F1-Score by Teemu Kanstrén.
• How should one interpret the comparison of means from different sample sizes?
• 30 Samples. Standard, Suggestion, or Superstition?
• Science and Cookies

Descriptive statistics, normality, correlations

• Normality Tests for Statistical Analysis: A Guide for Non-Statisticians
• Correlation
• Correlation by Pavan Akula.
• User’s guide to correlation coefficients by Haldun Akoglu.
• Correlation Coefficients
• Pearson Product-Moment Correlation
• Types of Variables, Descriptive Statistics, and Sample Size
• Everything You Need To Know About Correlation
• What does recall mean in Machine Learning?
• Power Analysis
• Power Analysis in Experimental Design
• Simulation for Power Analysis
• Power to the People: Power, Negative Results and Sample Size by Brianna N Gaskill and Joseph P Garner.
• Kruskal–Wallis Test
• Rigorous definition of an outlier?
• Dunn’s test
• Descriptive Statistics and Normality Tests for Statistical Data
• Multiple comparisons with many groups
• Review of Taleb, Nassim Nicholas. The Black Swan: The impact of the highly improbable.
• What references should be cited to support using 30 as a large enough sample size?
• Precision or Recall: Which Should You Use?
• The Chi-square test of independence by Mary L. McHugh.

The madness and controversy of p-values

• “The ASA Statement on p-Values: Context, Process, and Purpose” by Ronald L. Wasserstein and Nicole A. Lazar.
• Bad statistical practice in pharmacology (and other basic biomedical disciplines): you probably don’t know P by Michael J. Lew.
• Statistical Inference in the 21st Century: A World Beyond p < 0.05. The American Statistician, Volume 73, Issue sup1 (2019).
• “Moving to a World Beyond “p < 0.05” by Ronald L. Wasserstein, Allen L. Schirm, and Nicole A. Lazar.
• “An investigation of the false discovery rate and the misinterpretation of p-values” by David Colquhoun.
• “Do multiple outcome measures require p-value adjustment?” by Ronald J. Feise.
• “Understanding p-values and the Controversy Surrounding Them” by Jessica Utts.
• “P values and statistical practice” by Andrew Gelman.
• “Odds Are, It’s Wrong” by Tom Siegfried.
• “Scientists rise up against statistical significance” by Valentin Amrhein, Sander Greenland, and Blake McShane.
• “Things I Have Learned (So Far)” by Jacob Cohen. Direct PDF.
• “Statistical significance: p value, 0.05 threshold, and applications to radiomics—reasons for a conservative approach” by Giovanni Di Leo and Francesco Sardanelli.
• “Statistics Done Wrong. The woefully complete guide” by Alex Reinhart.
• “P – Value, a True Test of Statistical Significance? A Cautionary Note” by Tukur Dahiru.
• “On reporting and interpreting statistical significance and p values in medical research” by Herman Aguinis, Matt Vassar, and Cole Wayant.
• “p < 0.05, < 0.01, < 0.001, < 0.0001, < 0.00001, < 0.000001, or < 0.0000001 …” by Weimo Zhu.
• “Understanding results: P-values, confidence intervals, and number need to treat” by Lawrence Flechner and Timothy Y. Tseng.
• “How to control confounding effects by statistical analysis” by Mohamad Amin Pourhoseingholi, Ahmad Reza Baghestani, and Mohsen Vahedi.
• “A practical guide to methods controlling false discoveries in computational biology” by Keegan Korthauer et al.
• “Using false discovery rates for multiple comparisons in ecology and evolution” by Nathan Pike.

Mann-Whitney U test

• Mann-Whitney U-Test
• “Good explanation about Mann-Whitney U Test.”
• “Two-sample Mann–Whitney U Test” by Salvatore S. Mangiafico.
• “Confidence intervals for an effect size measure based on the Mann-Whitney statistic. Part 1: general issues and tail-area-based methods” by Robert G. Newcombe.
• “Confidence Intervals of the Mann-Whitney Parameter that are Compatible with the Wilcoxon-Mann-Whitney Test” by Michael P. Fay1 and Yaakov Malinovsky.
• Tests for Ordinal Categorical Data
• Mann Whitney U Test (Wilcoxon Rank Sum Test)

People who write interesting stuff about research and real life

• Andreas Zeller
• Arie van Deursen
• Austin Z. Henley
• Ayushi Rastogi
• Christian Bird
• Christopher J Parnin
• Dan Luu
• Diomidis Spinellis
• Eric Lippert
• Gerard J. Holzmann. One of the few researchers whose papers indicate that he has actually worked on real-world software. Just read everything he has published.
• Jaideep Ganguly
• Lorin Hochstein
• Massimiliano Di Penta
• Nachiappan (Nachi) Nagappan
• Ted Unangst
• Terence Tao
• Thomas Zimmermann

Journals

• Empirical Software Engineering. Look at Publish Open Access in EMSE
• IEEE Transactions on Software Engineering
• Journal of Systems and Software
• The long list of SE conferences and journals. Direct link to Google Sheets file.
• Springer Nature and their OA agreement for the Netherlands. And more details.

Random useful stuff

• Latest Software Engineering papers from arXiv.
• “Getting a Computer Science PhD in the USA”.
• “PhD Completion Milestones”.
• “The illustrated guide to a Ph.D.”. It’s worth reading all the other content from Matt Might as well.
• “A Reading List for PhD Students (and their Supervisors)”
• “Open Science Policies” by Daniel Graziotin.
• “The Ph.D. Grind. A Ph.D. Student Memoir” by Philip J. Guo. See if you can find a PDF copy somewhere. All the existing links are dead.
• Where to publish?
• Mars Code: building robust software

Random tools

• https://titlecaseconverter.com/
• https://capitalizemytitle.com/
• https://zbib.org/
• https://www.doi2bib.org/
• arxiv-latex-cleaner
• https://www.scopus.com/
• ISBN to BibTeX converter

Selling your work

• Free images: https://unsplash.com/
• Free images: https://pixabay.com/
• Presentation templates: https://www.canva.com/
• “Design 101 lecture” by Michele Lanza.
• “How to Get Your Research Adopted” by Emery D. Berger.

Negotiating your value in real life

• “Salary Negotiation: Make More Money, Be More Valued” by Patrick McKenzie .
• “Ten Rules for Negotiating a Job Offer” by Haseeb Qureshi.

Dealing with the madness of survey analysis

• “Questionnaires and Surveys: Analyses with R” by Martin Schweinberger.
• “Survey Data Analysis with R”
• “Using R to Analyze & Evaluate Survey Data” by Demetrius K. Green.
• “Survey analysis” from The Epidemiologist R Handbook.
• “On Likert Scales In R” by Jake Chanenson.
• “Do not use averages with Likert scale data” by Dwight Barry.
• “Data Vis for Likert Questions” by Laura Mudge.
• “Using likert on summary results” by M. Devlin.
• “Qualitative Analysis”.
• “Worse Than Spam: Issues In Sampling Software Developers” by Sebastian Baltes and Stephan Diehl.
• Is a Likert-type scale ordinal or interval data?
• can likert scale data ever be continuous? by Karen Grace-Martin.
• Analyzing ordinal data with metric models: What could possibly go wrong? by Torrin M. Liddell, John K. Kruschke.
• Introduction to Likert Data

Random articles about R

• JABSTB: Statistical Design and Analysis of Experiments with R
• “Summary and Analysis of Extension Program Evaluation in R” by Salvatore S. Mangiafico.
• “R for Data Science” by Hadley Wickham and Garrett Grolemund.
• “Wilcoxon test in R” by Antoine Soetewey.
• “Chi-square test of independence in R” by Antoine Soetewey.
• Chi-squared Test of Independence
• Multiple linear regression made simple by Antoine Soetewey.
• “Data Exploration: The Titanic”
• “How to Perform Correlation Analysis in Time Series data using R?” by Luciano Oliveira Batista.
• “Testing for time trend” by Ken.
• “R Cookbook, 2nd Edition” by James (JD) Long and Paul Teetor.
• Contingency Tables
• R - QQPlot: how to see whether data are normally distributed
• Why is message() a better choice than print() in R for writing a package?
• Aggregating and analyzing data with dplyr
• Scatterplots
• Bar Plots
• ggplot2 axis ticks : A guide to customize tick marks and labels
• GGPlot Axis Limits and Scales
• ggsave : Save a ggplot - R software and data visualization
• Normality Test in R
• Outliers detection in R
• Compute Summary Statistics in R
• Group by one or more variables
• R - Scatterplots
• How to Detect,Impute or Remove Outliers from a Dataset using Percentile Capping Method in R
• Descending order
• Logarithmic scale in R with ggplot2
• Basic Plotting With ggplot2
• ANOVA in R
• How to Filter in R: A Detailed Introduction to the dplyr Filter Function
• Data visualization with ggplot2
• GGPlot Log Scale Transformation
• Annotation: log tick marks
• How to Remove Outliers in R
• Correlation Test Between Two Variables in R
• Using Dates and Times in R
• ggplot2 scatter plots : Quick start guide - R software and data visualization
• Machine Learning with R: A Complete Guide to Decision Trees
• Chi-Square Test of Independence in R
• Decision Tree in R: Classification Tree with Example
• Spearman Rank Correlation
• Correlation tests Using R
• Statistical Analysis
• Visualizing data with ggplot2