How StakeholderWatch estimates causal effects

This page describes, for a reader who wants to check it, how StakeholderWatch's dashboards are produced, how the surveys are designed, how the data are collected, how the model is specified and estimated, what the tests on the dashboard mean and where the method stops.

The model

The model behind every dashboard is a map of how opinions form and turn into action, developed with researchers at VU Amsterdam and in daily use since 2022. People do not react to a firm directly; they react to what they believe about it. Those beliefs are formed by what reaches them: the macro issues of the day, the concerns they carry about the economy and society, the events the firm is involved in, and the channels through which they come into contact with it. Beliefs form two evaluations, a judgment of the firm's reputation and a judgment of the value of what it offers, and those evaluations drive what people will do, if they will exchange resources. The model rests on social exchange theory (Blau, 1964; Cropanzano & Mitchell, 2005), network theory (Granovetter, 1973, 1985) and stakeholder theory (Freeman, 1984; Mitchell, Agle & Wood, 1997). Reputation in the model is not one score: following Rindova et al. (2005), Lange, Lee and Dai (2011) and Mishina, Block and Mannor (2012), it separates character from capability. The full description is on the Theoretical model page.

The estimation

Every day, a fresh sample of respondents completes the client's survey. Every night, the full set of completes inside the estimation window (a trailing quarter by default; a year, 180 days, a month or the last 300 completes on request) is used to re-estimate one structural equation model (Bollen, 1989; Kline, 2016). Each path is a coefficient with a standard error, and the dashboard shows a coefficient only when it reaches the significance level the user selects (p below .05 by default; .10 and .20 are available for exploration and labelled as such).

"Causal" here means two things, and both are stated so that they can be checked. First, the model is causal in its structure: the direction of every path is fixed by theory before estimation, and the fit of the whole system is tested rather than assumed (Hu & Bentler, 1999). Second, the data are collected so that the usual threats to a causal reading are controlled: respondent-level controls for age, region, familiarity with the firm and the other standard covariates; matched comparison respondents when an event's effect is estimated; and, where a firm's own history is the comparison, difference-in-differences against peers measured on the same panel (Angrist & Pischke, 2009).

Breaks and peers

Structural break tests (Andrews, 1993; Bai & Perron, 1998, 2003) look every night for the date at which an outcome, or a coefficient, settled at a new level. A break is reported with its estimated date, the average before and after, and the date on which it was confirmed by further readings outside the old range. Peer comparison, on the brands measured alongside the client, separates the part of a move that belongs to the sector from the part that belongs to the firm.

Survey design and sample

Each questionnaire is built from a common core and a client-specific part. The core carries the model's blocks with item wording fixed across clients so that the model is comparable; the client-specific part carries the organisation's own events, channels, competitors and issues. Familiarity with the organisation is screened before any evaluation is asked and is a control in every estimation. Completes are collected at a steady daily rate; quotas on age, gender, region and education follow the national population or the client's target public, and post-stratification weights are computed daily. A segment whose completes fall below the level at which a coefficient can be estimated is shown as unavailable rather than estimated.

What the method cannot do

The coefficients describe what moved respondents' beliefs, evaluations and intentions. They do not observe purchases, applications or trades; where a client supplies behavioural data, the link from intention to behaviour can be estimated separately and reported with its own uncertainty. A coefficient is an average over the sample or segment and does not describe any individual. Like every structural model, this one cannot rule out an omitted cause that moves both an input and an outcome; the controls, the matched comparisons and the peer panel reduce this risk and are reported so that a reader can judge it. Breaks are estimated with uncertainty around their dates, and a break is reported only once it has been confirmed. We state these limits on every dashboard, because an honest account of what a number means is part of the number.

The works this page draws on are listed under Relevant literature.