Choosing a machine learning model for your thesis.

Before anything else, name what your model outputs. This single question removes most of the field.

This is the step students skip, and it is the one that most often decides whether a model generalizes or merely memorizes. Model capacity has to be matched to how much data you have.

A rough guide for tabular problems. Under a few hundred rows, stay with simple, well-regularized methods — linear and logistic regression, small decision trees. A few hundred to a few thousand rows is where tree ensembles like random forests and gradient boosting are typically strongest, and on tabular data they frequently outperform neural networks at this scale. Beyond several thousand rows, a neural network becomes a reasonable candidate rather than an aspiration.

The number of input variables matters alongside the row count. Many variables on few rows is the classic overfitting setup: the model finds patterns in noise, reports excellent training performance, and fails on anything new.

This is why our geotextile project used a smaller model deliberately. On a few hundred rows, that is the defensible choice, and the thesis argued it explicitly rather than apologizing for it.

This is the single most valuable habit in an academic machine learning project, and it costs almost nothing.

Before your intended model, train the simplest reasonable thing: linear or logistic regression, or even a rule that always predicts the majority class. Record its performance. That number is your floor.

It does two things. It tells you whether your sophisticated model is actually contributing anything — sometimes it is not, which is a genuine finding worth reporting. And it hands you a complete answer to the why-this-method question: because I compared it against a simpler alternative and it performed measurably better on held-out data.

Occasionally the baseline wins. Report that. A thesis showing that a simple model performs as well as a complex one on this problem is a real result, and defending it honestly is far stronger than quietly discarding it.

In some fields, knowing which factors drove a prediction matters as much as the prediction. In medicine, education and policy research, a model that cannot tell you why is often not useful even when it is accurate.

Linear and logistic regression are the most interpretable: each coefficient has a direct reading. Decision trees can be inspected as a set of rules. Tree ensembles are partly interpretable through feature-importance measures. Neural networks are the hardest to interpret directly, and the tooling for explaining them adds work.

If your research question is partly about which factors matter, that pushes you toward the interpretable end. Do not treat this as a compromise; for a great many research questions the interpretable model is the correct scientific choice, not the fallback.

Your justification should rest on evidence you generated, not on the literature alone. A defensible answer has four parts, and you should be able to give it in under a minute.

What the problem type is and what that ruled out. What your dataset size and dimensionality implied about model capacity. What baselines you compared against and how they performed. Why the chosen model's advantage was worth its added complexity — including on interpretability if that matters in your field.

The failure mode to avoid is a justification that appeals only to novelty or popularity. “Deep learning is the current state of the art” invites the obvious follow-up: state of the art on what data, at what scale, and is that your situation? On a few hundred rows it is usually not.

Prepare for the limitations question too, because it is coming. Know what your model does badly, what happens with inputs outside its training range, and how much more data you would need to do better. Answering that clearly reads as competence, not weakness.