Start here: the data science process

Every data project goes through the same seven stages, and every lab on this site practices some of them. For each stage: what it is, the mistake I see most, the concept video, the Python videos, the labs that practice it, and one outside resource worth your time. Use it to place what you are doing this week, and later to plan a project of your own.

Stage 1

Question

Write one sentence that says what you want to know and who will act on the answer. Decide early whether the job is prediction (what will happen) or inference (what causes what), because the two are built and checked differently. The mistake I see most is opening the data first and letting whatever happens to be in it choose the question.

Concept video: Regression for prediction and regression for inferencecoming·Random assignment as an estimatorcoming·Statistical powercoming
Python videos: Everywhere (the pandas you use at every stage; start here if you are new)
Labs that practice it: Correlation & Causality — Which Story Made This Scatterplot?·A/B Testing — Designing the Data-Generating Process·Experiment Design Audit — Power, Selection, and Weighting·Predictive Modeling — The Same Machinery, A New Scorecard
One outside resource: The Effect: An Introduction to Research Design and Causality (starts from the research design and the causal diagram, before any estimator)

Stage 2

Data

Find out where the rows came from, who is in them, who is missing, and what one row means. A large sample is no protection when the people in it chose to be there. Students tend to trust a tidy-looking file; ask how it was made before you ask what it says.

Concept video: Sampling biascoming
Python videos: Setup & load (Colab, imports, a CSV from a link, data that is not a CSV)
Labs that practice it: Pipeline Health Check — Finding Corruption Through EDA·SQL for Economists — The Same Operations, A Different Grammar·Hypothesis Testing — Was the Job Training Worth It?·Experiment Design Audit — Power, Selection, and Weighting
One outside resource: Python for Data Analysis, 3rd edition (the loading and cleaning chapters, by the author of pandas; free online)

Stage 3

Clean

Fix the types and the dates, decide what to do about missing values, join tables without doubling rows, and treat an outlier as a decision rather than a nuisance. Most of the hours of a project go here, and so do most of the quiet errors. The mistake is cleaning with the test set in view, so that information from the data you will score on leaks into the data you train on.

Concept video: Train-test split and leakagecoming
Python videos: Clean & reshape (groupby, dates, missing values, joins, text into numbers)
Labs that practice it: Pipeline Health Check — Finding Corruption Through EDA·Robust Statistics — When One Row Moves the Answer·Robust Statistics — Three Bugs in an Outlier Pipeline·SQL for Economists — The Same Operations, A Different Grammar
One outside resource: pandas user guide (the exact options for merge, pivot, resample and missing data, with runnable examples)

Stage 4

Explore

Look before you model: summary numbers, a histogram of everything, scatterplots of the pairs you care about, and a check of the sample against what you know about the population. Four datasets with the same means and the same correlation can look nothing alike. The mistake is reading a causal story off a scatterplot that three different stories could have made.

Concept video: Confounding and causal diagramscoming
Python videos: Explore & visualize (a first chart, more charts, summary numbers and spread)
Labs that practice it: Good, Bad and Ugly — Honest vs Misleading Visualizations·Correlation & Causality — Which Story Made This Scatterplot?·Robust Statistics — When One Row Moves the Answer·OLS Diagnostics — When the Standard Error Is Lying
One outside resource: Kaggle Learn (the short pandas and data visualization courses, with exercises that check themselves)

Stage 5

Model

Pick the simplest method that answers the question, fit it, and know what each number it prints means. For an effect, that is a design that deals with confounding; for a prediction, a model whose complexity matches how much data you have. The mistake is adding predictors or controls because the fit improves, when some of them create the very bias you were trying to remove.

Concept video: Difference-in-differencescoming·Regularization: ridge and lassocoming·Logistic regressioncoming
Python videos: Simulate & estimate·Regression & time series·Machine learning (seeds and resampling, statsmodels, scikit-learn's build-fit-predict)
Labs that practice it: Multivariate OLS — What “Controlling For” Actually Does·Predictive Modeling — The Same Machinery, A New Scorecard·Regularization — Buying Bias to Sell Variance·Probability & Simulation — When Intuition Is Wrong·A/B Testing — Diagnosing Experimental Failures
One outside resource: An Introduction to Statistical Learning with Applications in Python (regression, classification and regularization with the statistics behind each; free PDF)

Stage 6

Evaluate

Ask how wrong you could be: a standard error or an interval for an estimate, a score on held-out data for a prediction. The in-sample fit always flatters you. Students report an R-squared and a p-value and stop; report the uncertainty, the test on data the model never saw, and the result that would have changed your mind.

Concept video: The p-valuecoming·The bias-variance trade-offcoming·Precision, recall and the ROC curvecoming
Python videos: Evaluate & report (RMSE and R-squared, cross-validation, a results table)
Labs that practice it: Hypothesis Testing — Was the Job Training Worth It?·Estimation & Bootstrapping — How Uncertain Is the Median?·The Bias-Variance Tradeoff — Watching a Model Overfit·OLS Diagnostics — When the Standard Error Is Lying·Hypothesis Testing — Diagnosing Underpowered Studies and P-Hacking
One outside resource: Computational and Inferential Thinking (the p-value and the bootstrap interval built by simulation, so you see where each comes from)

Stage 7

Communicate

Turn the result into something another person can act on: one chart that does not mislead, one number with its interval, one paragraph in plain words, and code that runs again from the top. The mistake is a notebook that only works in the order you happened to click it, and a chart whose axis does the arguing for you.

Concept video: Bootstrap confidence intervalscoming
Python videos: Evaluate & report·Graduate extras (a results table that lines up; then functions with signatures, your own module, and code that checks itself)
Labs that practice it: Good, Bad and Ugly — Honest vs Misleading Visualizations·A/B Testing — Designing the Data-Generating Process·OLS Diagnostics — When the Standard Error Is Lying·Estimation & Bootstrapping — Diagnosing Broken Confidence Intervals
One outside resource: QuantEcon lectures (the standard to copy: every number in the text is produced by the notebook beside it)