Footfall and sales are simultaneously determined. Rainfall as an instrument for footfall variation breaks the simultaneity and identifies the true causal impact of in-store traffic on revenue.
Every retail analyst has, at some point, stared at a scatter plot of daily footfall against daily sales and drawn the obvious conclusion: more people in the store means more revenue. The correlation is strong, the regression line slopes upward, and the stakeholder presentation practically writes itself. But there is a problem buried inside that clean story, and it is the kind of problem that can quietly steer a business toward the wrong decisions for years.
The problem is simultaneity. Footfall does not just cause sales. Sales, or more precisely the commercial actions designed to generate sales, also cause footfall. A store running a weekend promotion simultaneously inflates both traffic and revenue. A new product launch draws crowds and rings up transactions at the same time. When the cause and the effect move together because a third force is pushing both, any naive regression of
sales on footfall produces a biased estimate. You end up overstating, sometimes dramatically, how much an additional visitor is worth.
This article walks through a practical solution drawn from econometrics: using weather data, specifically rainfall, as an instrumental variable to isolate the true causal impact of footfall on retail sales. We will cover why the naive approach fails, what makes a valid instrument, why rain is a surprisingly good candidate, and how to implement the entire workflow in Python with simulated data that mirrors real world retail patterns. If you have ever wondered whether your “revenue per visitor” metric is trustworthy, this piece is for you.
The Simultaneity Trap in Retail
Let us set up the situation formally. Suppose you are an analyst at a mid sized grocery chain. You have daily data for each store: total transactions (your proxy for sales), footfall counts from ceiling mounted sensors, and a handful of controls like day of week and whether a promotion was active. You estimate the following model using ordinary least squares (OLS):
Sales(t) = β₀ + β₁ · Footfall(t) + γ · X(t) + ε(t)
where X(t) is your vector of controls. OLS gives you a coefficient β̂₁ and you interpret it as the marginal effect of one additional visitor on daily sales. The number looks reasonable, so you hand it to the merchandising team and they use it to evaluate the ROI of a new loyalty campaign that is expected to boost foot traffic by 8%.
Here is the issue. OLS assumes that the error term ε(t) is uncorrelated with the regressor Footfall(t). But think about what sits inside that error term: unobserved promotional intensity, seasonal demand shocks, competitor activity, local events, even the mood of the store manager who decides to put the premium display at the entrance on a given Tuesday. Many of these unobserved factors drive both footfall and sales simultaneously. The moment Cov(Footfall(t), ε(t)) ≠ 0, the OLS estimate is biased and inconsistent. No amount of additional data will fix it.
In practice, the bias almost always runs in one direction: upward. The omitted variables that push people into the store (promotions, events, holidays) also push them to spend more once inside. So your estimated “value of a visitor” is inflated by all the demand side energy you cannot observe. The loyalty campaign gets a green light based on numbers that were never real to begin with.
This is not a theoretical curiosity. Research in empirical economics has shown that
simultaneity bias in demand estimation can inflate coefficients by 30% to 50% or more, depending on the setting [1]. In retail, where margins are thin and marketing budgets are scrutinized quarterly, that kind of error translates directly into misallocated capital.
Instrumental Variables: The Idea in Plain Language
The instrumental variables (IV) approach solves this problem by finding a source of variation in footfall that is unrelated to the unobserved factors contaminating the error term. Instead of asking “what happens to sales when footfall goes up?” which conflates all reasons footfall might rise, the IV approach asks a more surgical question: “what happens to sales when footfall goes up for reasons that have nothing to do with demand side shocks?”
An instrument Z must satisfy two conditions:
Relevance. The instrument must actually affect footfall. Formally, Cov(Z(t), Footfall(t)) ≠ 0. If the instrument has no relationship with your endogenous variable, it is useless.
Exclusion restriction. The instrument must affect sales only through its effect on footfall, not through any other channel. Formally, Cov(Z(t), ε(t)) = 0. This is the harder condition to argue and the one that requires domain knowledge rather than statistical tests.
When both conditions hold, you can use the instrument to extract the “clean” variation in footfall, the part driven by the instrument alone, and use only that variation to estimate the effect on sales. The technique is called Two Stage Least Squares (2SLS) because it proceeds in two stages:
Stage 1. Regress footfall on the instrument (and controls) to get predicted footfall, Footfall̂(t). This predicted value contains only the variation in footfall that is driven by the instrument.
Stage 2. Regress sales on the predicted footfall from Stage 1 (and the same controls). The resulting coefficient is your IV estimate of the causal effect.
The intuition is elegant. By replacing actual footfall with its instrument driven prediction, you strip away all the contaminated variation, the promotions, the events, the unobservable demand shocks, and keep only the exogenous movement. If the instrument is valid, the resulting estimate is consistent even in the presence of simultaneity.
Why Rainfall Works as an Instrument
Now we need a variable that shifts footfall around without directly affecting how much people spend once they are inside a store. Rainfall is a natural candidate, and it is one that has been used successfully in a variety of applied economics settings, from estimating the effect of protests on policy outcomes to measuring the impact of outdoor dining on restaurant revenue.
The argument for relevance is straightforward and empirically verifiable. When it rains, fewer people leave their homes for discretionary shopping trips. Footfall at physical retail locations drops measurably on rainy days. Any retailer with a few months of traffic data and a local weather feed can confirm this in about ten minutes.
The argument for the exclusion restriction is more subtle and requires careful thinking. The claim is that rainfall affects retail sales only through its effect on footfall, not through any other channel. Is this plausible?
Consider the alternatives. Could rain directly change how much a person spends once they are already inside the store? For a grocery store or a general merchandise retailer, the answer is mostly no. Your basket composition might shift slightly toward comfort food on a rainy day, but the aggregate spend per visitor is unlikely to move in a systematic way that correlates with rainfall intensity. The primary channel through which rain affects total daily sales is by keeping some shoppers at home, which is exactly the footfall channel.
There are scenarios where the exclusion restriction becomes weaker. A garden center, for instance, might see rain suppress demand for outdoor products independent of traffic. An apparel retailer might see rain boost umbrella and raincoat sales in a way that is not fully mediated by footfall. These are legitimate concerns, and they remind us that the validity of an instrument is always context dependent. For the broad category of general retail, however, rainfall passes the exclusion restriction test with reasonable confidence, especially when you control for seasonality and day of week effects that might correlate with both weather patterns and spending behavior.
One additional strength of rainfall as an instrument is that it is truly exogenous. No store manager, marketing team, or competitor can influence whether it rains on a given day. This eliminates a whole class of concerns about reverse causality or strategic behavior that plague many candidate instruments in business settings.
Implementation: A Complete Python Walkthrough
Let us move from theory to practice. Below is a complete implementation using Python, demonstrating how to run the IV regression using simulated data that mirrors the structure of a real retail dataset. In practice, you would replace the simulated data with your actual footfall counts, sales figures, and local weather station records.
A note on the data: The dataset used throughout this section is entirely synthetic, generated using NumPy to illustrate the mechanics of simultaneity bias and the IV correction. No real customer, store, or weather data is used. The simulation is designed so that the true causal parameter is known (β₁ = 2.0), which allows us to directly observe how OLS overestimates the effect and how IV recovers the correct value.
import numpy as np
import pandas as pd
from statsmodels.sandbox.regression.gmm import IV2SLS
import statsmodels.api as sm
np.random.seed(42)
n = 1000
rainfall = np.random.exponential(scale=3.0, size=n)
demand_shock = np.random.normal(0, 1, size=n)
footfall = 500 - 8 * rainfall + 40 * demand_shock +
np.random.normal(0, 20, size=n)
true_beta = 2.0
sales = 200 + true_beta * footfall + 30 * demand_shock + np.random.normal(0, 50, size=n)
data = pd.DataFrame({
"sales": sales,
"footfall": footfall,
"rainfall": rainfall
})
This is exactly the simultaneity we are worried about. The true causal effect of footfall on sales is 2.0, but OLS will overestimate it because it cannot separate the footfall variation caused by demand shocks from the footfall variation caused by rain.
X_ols = sm.add_constant(data["footfall"])
ols_model = sm.OLS(data["sales"], X_ols).fit()
print("OLS Estimate of Footfall Effect:",
round(ols_model.params["footfall"], 4))
Running this code will typically produce an estimate well above 2.0, often in the range of 2.5 to 2.8. That is the upward bias from simultaneity at work. The OLS model is picking up the fact that high demand shock days produce both more visitors and more spending, and it is incorrectly attributing all of that co movement to the causal effect of footfall.
X_first = sm.add_constant(data["rainfall"])
first_stage = sm.OLS(data["footfall"], X_first).fit()
print("First Stage F-statistic:", round(first_stage.fvalue, 2)) print("Rainfall coefficient:", round(first_stage.params["rainfall"], 4))
endog = data["sales"]
exog = sm.add_constant(data["footfall"]) # endogenous regressor instruments = sm.add_constant(data["rainfall"]) # instrument set
iv_model = IV2SLS(endog, exog, instruments).fit()
print("IV Estimate of Footfall Effect:",
round(iv_model.params["footfall"], 4))
Interpreting the Results for Business Decisions
The gap between the OLS estimate and the IV estimate is not a statistical curiosity. It has direct implications for how the business allocates resources.
Suppose the OLS model tells you that each additional visitor generates $2.70 in revenue, while the IV model says the true number is $2.00. If the marketing team is evaluating a campaign expected to drive 10,000 additional store visits at a cost of $22,000, the OLS based analysis shows a positive ROI ($27,000 in incremental revenue versus $22,000 in cost), while the IV based analysis shows a negative one ($20,000 versus $22,000). The campaign that looked profitable under naive analysis is
actually a money loser once you account for the bias.
This kind of correction matters most in three scenarios:
Campaign evaluation. Any time you estimate the revenue impact of a traffic driving initiative, using the IV estimate prevents you from over crediting the campaign with revenue that would have occurred anyway due to underlying demand.
Location strategy. Retailers comparing store performance often use revenue per visitor as a key metric. If that metric is inflated by simultaneity bias, you might undervalue stores in low promotional intensity markets and overvalue stores that happen to run heavy promotional calendars.
Staffing models. Many retailers use footfall forecasts to set labor schedules. If your model overstates the revenue impact of each visitor, you will systematically overstaff, eroding the margin gains you thought you were capturing.
Practical Considerations and Limitations
No method is without caveats, and intellectual honesty about limitations is what separates good analysis from misleading analysis.
Granularity of weather data. The instrument works best when your weather data is geographically specific. Using a city wide rainfall average for a store located 30 miles from the weather station introduces measurement error in the instrument, which attenuates the first stage relationship and weakens the IV estimate. Ideally, you want hyperlocal weather data from the nearest station or a gridded weather product.
Seasonal confounding. Rainfall is not uniformly distributed across the year. If your sales also have strong seasonality (and in retail, they always do), you need to control for time effects, whether through month fixed effects, day of week indicators, or a more flexible time trend, to ensure that rainfall is not proxying for “it is winter” which independently affects spending patterns.
The exclusion restriction is not testable. This is the uncomfortable truth about IV estimation. You can test whether the instrument is relevant (the first stage F test), but you cannot statistically verify that it satisfies the exclusion restriction. The argument must be made on economic and domain grounds. For general retail, the case is strong. For specialty retail categories where weather directly shifts product demand, it is
weaker, and you should think carefully before proceeding.
Local Average Treatment Effect (LATE). The IV estimate identifies a specific causal effect: the effect of footfall changes induced by rainfall. This is not necessarily the same as the effect of footfall changes induced by, say, a social media campaign. If the visitors who stay home when it rains are systematically different from the visitors attracted by a digital ad (and they probably are), the IV estimate applies to the rain sensitive margin of visitors, not to all visitors universally. This is an important nuance when extrapolating IV results to evaluate interventions that operate through different channels.
Conclusion
The correlation between footfall and retail sales is one of the most intuitive relationships in business analytics. But intuition and causal identification are different things, and conflating them leads to systematically inflated estimates that distort resource allocation downstream.
Instrumental variables offer a principled way to break the simultaneity between traffic and revenue. Rainfall, with its clear effect on store visits, its plausible exclusion from direct spending effects, and its unimpeachable exogeneity, is one of the cleanest instruments available in applied retail analytics. The implementation is straightforward, the diagnostics are well established, and the business implications of getting the number right are substantial.
The next time someone in your organization quotes a “revenue per visitor” figure derived from a simple regression, ask them one question: did they account for the fact that the same forces driving people into the store are also driving them to spend? If the answer is no, the number is almost certainly too high, and now you know how to fix it.
Author:
Pratik Khedekar | Data Scientist