The Free and Frank Series: Mapping What People Think Without Telling Them What to Think

16/05/2026

The Problem

This is the nineteenth instalment in a series exploring what free and frank advice means in Aotearoa. Last week, I explained why studying a concept that no one has defined requires an interpretive approach, and why that approach needed to be extended through Ata to accommodate the constitutional plurality that Te Tiriti o Waitangi establishes. This week, I turn to the puzzle that the interpretive framework alone could not solve: how to discover whether shared patterns exist in practitioner meaning without the researcher imposing those patterns through her own categories. By the end of this post, I hope to have shown that Q-methodology (Brown, 1980; Stephenson, 1953), combined with elite interviews and a sustained validation process involving over a hundred practitioners and scholars, provides something rare in social science: a way of mapping what people think without first telling them what to think.

Last week, I described how elite interviews can recover how individual practitioners understand free and frank advice. The richness of that recovery is considerable. Over thirty-two interviews with former prime ministers, ministers, chief executives, deputy chief executives, senior policy managers, and ministerial advisers spanning the last nine administrations, I listened to practitioners describe what free and frank advice means to them, why it matters, and what turns it on or off. The interviews were semi-structured and open-ended, shaped by Ata-whakarongo (Pohatu, 2013): I listened with reflective attention rather than filtering accounts through predetermined categories, attending to hesitations, contradictions, and the emotional weight of what was said alongside its substantive content (Rhodes, 2017; Schön, 1983).

What emerged from that first stage was a rich and detailed map of the conceptual territory. Six major thematic domains surfaced through abductive thematic analysis (Blaikie & Priest, 2019; Braun & Clarke, 2006): constitutional evolution and institutional memory; ministerial authority and democratic accountability; timing, strategy, and professional judgement; relational maintenance and trust; implementation wisdom as a distinctive value; and contested boundaries and definitional ambiguity. These domains represent the terrain practitioners navigate when they think about advisory practice. They confirmed what the literature had suggested: that the definitional absence is experienced from within daily work, not merely observed from the outside.

But the interviews left crucial questions open. How do these different themes relate to one another in any single practitioner’s thinking? Do individual practitioners draw on all six domains, or do they cluster into distinct, coherent orientations? Do the tensions and contradictions I heard reflect individual idiosyncrasy, or do they represent systematic and predictable diversity? And, most pressingly: were the patterns I thought I could see in the data genuinely there, or were they artefacts of my own twenty years in the advisory relationship?

To answer those questions, I needed a method that could reveal structure in the field of practitioner meaning without me putting it there.

What Q-Methodology Is

Q-methodology originated in the work of William Stephenson, a physicist and psychologist who proposed in the 1930s that subjectivity itself could be the phenomenon under investigation (Brown, 1980; Stephenson, 1953). Rather than treating how people see the world as noise to be controlled, Stephenson argued that systematic study could reveal shared patterns of perspective: ways of seeing that multiple people inhabit, viewpoints that organise what might otherwise appear as nothing more than idiosyncratic individual opinion (Brown, 1980, 1993; Van Exel & de Graaf, 2005).

The technical form is an inversion. Conventional factor analysis, the kind used in most survey research, correlates variables across persons. It produces dimensions along which individuals differ. Q-methodology reverses the matrix. It correlates persons across variables, producing shared perspectives: ways of thinking that groups of people enact, regardless of whether those people share a job title, an agency, or a demographic profile (Brown, 1980; Watts & Stenner, 2012). The factors that emerge are not attributes that individuals possess. They are positions that individuals occupy in a space of possible perspectives on whatever the researcher has placed before them.

This is what makes Q-methodology apt for studying a convention that has resisted definition for four decades. It does not require the researcher to define the concept in advance and test whether practice conforms. It does not impose the researcher’s categories on practitioners. Instead, it begins with what Brown (1980) called operant subjectivity: behaviour in search of theories rather than theories in search of behaviour. The practitioners provide the meaning. The mathematics reveal the structure. The researcher interprets, but the structure belongs to the participants.

There is a functional resonance here that I want to note, though it is my own observation rather than a claim in the literature. When speakers rise on the marae to address a kaupapa in whaikōrero, they do not speak as representatives of demographic categories. They speak from positions within a conversation, carrying perspectives that relate to other perspectives, building collectively toward an understanding that no single speaker possesses in advance. The perspectives are structured, but the structure emerges through the kōrero itself rather than being imposed by external classification. Q-methodology produces something analogous: a map of perspectives on free and frank advice whose structure emerges through the analysis rather than being predetermined by it. The resonance is functional rather than foundational. The genealogies are distinct. But it suggests that Q sits more comfortably alongside Ata than the disciplinary distance between quantitative technique and tikanga methodology might lead one to expect.

The Three-Stage Design

The research followed a three-stage design, and each stage was built to do what the others could not (Bevir & Rhodes, 2016; Patton, 2002).

Stage One generated meaning. Thirty-two elite interviews, conducted according to Ata principles (Hudson et al., 2010; Pohatu, 2013), produced the conceptual territory described above.

The primary output was a comprehensive concourse: a collection of statements representing the full range of views expressed by practitioners about free and frank advice, anchored in participants’ own language rather than in categories imposed by the researcher (Brown, 1980, 1993).

From that concourse, thirty-four contestable statements were carefully distilled, each drawn from what practitioners had actually said. Before the statement set was finalised, a number of Stage One participants reviewed the drafts for fidelity: whether the statements captured the ideas they had articulated, whether the language was recognisable as their own rather than my translation.

This review enacted Ata-kōrero (Pohatu, 2013): returning to participants to ensure that the raw material of Stage Two was faithful to the meanings Stage One had surfaced.

Stage Two mapped shared patterns. Fifty-eight practitioners, by design, a different group from those in Stage One, each sorted the thirty-four statements into a distribution grid ranging from +5 (most characteristic of their view) to -5 (least characteristic) (Brown, 1980; Van Exel & de Graaf, 2005).

The forced-choice ranking required participants to make genuine discriminations, to weigh competing ideas against one another, and in doing so to construct a model of their own orientation toward free and frank advice. Following the sort, each participant was asked four reflective questions, shaped by Ata-whakarongo (Pohatu, 2013).

Many reported that the sorting process itself had clarified or challenged their thinking: the Q-sort is not merely data collection but a moment of interpretive work in its own right (Brown, 1980).

The completed sorts were then subjected to by-person factor analysis, and the analytical choices involved in that process matter enough to warrant a brief detour.

Why Spearman, and What the Trialling Revealed

The first analytical step is to compute a correlation matrix: a table showing how closely every participant’s sort corresponds to every other participant’s sort. The choice of correlation method determines what the analysis can see (Brown, 1980).

The conventional choice in Q-methodological practice is Pearson’s product-moment correlation, which treats the sort data as interval-level measurement: as though the gap between ranking positions carries equal psychological weight at every point on the scale. This is a pragmatic convention rather than a strictly defensible position. When a participant moves a statement from +2 to +3, is that the same kind of shift as moving one from -4 to -3? The sorting process does not require this, and practitioners do not report experiencing it that way.

Spearman’s rank-order correlation is the method that most directly reflects what the sorting task actually asks practitioners to do (Kellstedt & Whitten, 2013). It computes the correspondence between two sorts based on their ordering rather than assuming that the gaps between positions have a precise metric meaning.

Both methods were trialled over nearly six months, during which the factor solutions produced by each were systematically compared for statistical robustness, interpretive coherence, and fidelity to practitioners’ post-sort reflections. Solutions extracted between four and eight factors were examined for each method.

Over those months, the Spearman-based solutions consistently produced factors that were cleaner in their distinguishing statements and more recognisable when tested against the qualitative data from Stage One and the fifty-eight post-sort reflections.

To take one illustration. One statement asked whether free and frank advice is the craft of senior officials, and senior officials only. Under the Spearman solution, two of the emerging postures separated sharply on this statement: one ranking it at -5, the other at +3. The post-sort reflections confirmed the divergence was real. One participant said the duty exists independently of seniority, owed by every official who touches advice. Another said it is senior business and should never go near people who do not understand what is at stake. The Spearman solution captured that divergence. The Pearson solution blurred it. The decision to use Spearman was therefore not purely statistical. It was a methodological judgement grounded in extended practical comparison and validated against qualitative data (Brown, 1980).

Stage Three: Testing the Findings

Interpretive research does not claim to discover objective facts. It constructs defensible interpretations that must then be subjected to critical scrutiny from those with standing to challenge them (Bevir & Rhodes, 2016; Lincoln & Guba, 1985; Rhodes, 2017). Stage Three tested the patterns identified in Stage Two across multiple forums, each selected because it offered a form of scrutiny the others could not provide.

Two wānanga brought together small groups of senior practitioners, regulators, and oversight officials for the kind of slow, collective dialogue that individual interviews cannot produce. The wānanga format embodies Ata-noho, extended engagement, and Ata-whakarongo, listening with patience and openness (Hudson et al., 2010; Pohatu, 2013). Unlike conventional focus groups, wānanga treat participants as knowledge-holders whose insights are essential to the construction of valid knowledge (L. T. Smith, 2012). What surfaced in the wānanga, particularly about the limits of the Westminster frame and the absence of a Te Tiriti posture in the factor solution, could not have emerged in any other forum.

Fifteen one-on-one expert interviews were conducted with individuals whose work is referenced in the thesis and with practitioners who hold or have held positions of significant responsibility for the integrity of the advisory relationship, including former officials in the central agencies and parliamentary offices, and current and former senior public service leaders with experience across multiple agencies and administrations (van Dorp & ‘t Hart, 2019). These interviews tested the findings against the individual judgement of people who had spent their careers building and maintaining the advisory system.

Two online workshops enabled practitioners to experience the Q-sort in groups and to engage with the emerging findings in real time, shaping the interpretation rather than simply receiving it.

The research was presented at three academic conferences: a Present and Connect: PhD Conference at the University of Canterbury, the Australia and Aotearoa New Zealand Public Policy Network conference in Auckland, and the International Research Society for Public Management in Perth. Each audience worked within different scholarly traditions, interpretive, policy, and Westminster, and each tested the findings from a different angle (Bevir & Rhodes, 2016). A further event for Transparency International New Zealand, attended by participants from across the transparency and accountability community, provided a cross-sector lens.

Five participants were threaded across all three stages, from initial interview in Stage One through Q-sort in Stage Two to validation in Stage Three, providing a longitudinal check on whether the transition from individual meaning to shared pattern had preserved or distorted what those practitioners understood by free and frank advice (Creswell, 2014; Lincoln & Guba, 1985).

In total, over a hundred practitioners across elected, unelected, regulatory, academic, and oversight spaces engaged with the findings before the thesis was submitted. Each forum tested something distinct. The wānanga could surface what workshops could not. Expert interviews could challenge what conferences could not. The combination creates a validation architecture, not a checklist.

What the Method Found

The mathematics, interpreted through the decentred framework (Bevir & Rhodes, 2003, 2006) and held accountable to Ata (Pohatu, 2013), revealed something that none of the individual methods could have produced alone.

Six distinct postures emerged from the data: six coherent ways of understanding and enacting free and frank advice. They are not demographic groupings. Agency, seniority, gender, ethnicity, and whether a participant holds elected or appointed office do not predict which posture a practitioner inhabits (Bevir & Rhodes, 2006; Rhodes, 2017).

The postures are traditions in Bevir’s sense: webs of belief inherited and modified through engagement with the advisory relationship, not expressions of structural position. Together, the six postures explain seventy per cent of how fifty-eight practitioners sorted the statements (Brown, 1980).

Beneath the structured disagreement, all six postures converge on a shared boundary: what I call the practical state, the integration of expertise, evidence, and practical knowledge about how policies will actually work in the operational reality of government (Head, 2008). The disagreement is real and consequential. The boundary is real and consequential. Together they constitute what the thesis terms a bounded plurality: a finite number of distinct practitioner postures, organised along four recognisable fault lines, held together by a shared commitment to implementation wisdom (Connolly, 1974; Gallie, 1956).

This combination of Q-methodology with the decentred interpretive approach, extended through Ata, within a three-stage design that generates, maps, and validates practitioner subjectivity, is, to the best of my knowledge, without precedent in the public administration literature (Brown, 1980; Bevir & Rhodes, 2003; Pohatu, 2013). The method was not borrowed from another study and applied here. It was designed for this problem: a concept that everyone invokes, that no one has defined, and whose meaning lives in the contested, contextual, daily work of practitioners who must navigate it without a map.

The method gave them a way to draw the map themselves. What that map looks like, the six postures, the four fault lines, and the shared ground beneath them, is where this series turns next.

Next in the Series

Over the coming weeks, we will enter the empirical findings. I will introduce the six postures, one by one, through the practitioners who inhabit them: the Institutional Pragmatist, the Democratic Constitutionalist, the Collaborative Sense-Maker, the Moment Chooser, the Political Servant, and the Westminster Protector. We will trace the fault lines along which they diverge and the ground on which they agree. The answer to the question we started with, what is free and frank advice, turns out to be neither a definition nor a confusion, but a structured contest operating within shared parameters. The disagreement itself may be the answer. But that is for the weeks ahead.

References

Bevir, M., & Rhodes, R. A. W. (2003). Interpreting British governance. Routledge.

Bevir, M., & Rhodes, R. A. W. (2006). Governance stories. Routledge.

Bevir, M., & Rhodes, R. A. W. (2016). Routledge handbook of interpretive political science. Routledge.

Blaikie, N., & Priest, J. (2019). Designing social research (3rd ed.). Polity Press.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.

Brown, S. R. (1980). Political subjectivity: Applications of Q methodology in political science. Yale University Press.

Brown, S. R. (1993). A primer on Q methodology. Operant Subjectivity, 16(3/4), 91–138.

Connolly, W. E. (1974). The terms of political discourse. D. C. Heath.

Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). SAGE.

Gallie, W. B. (1956). Essentially contested concepts. Proceedings of the Aristotelian Society, 56, 167–198.

Head, B. W. (2008). Three lenses of evidence-based policy. Australian Journal of Public Administration, 67(1), 1–11.

Hudson, M., Milne, M., Reynolds, P., Russell, K., & Smith, B. (2010). Te Ara Tika: Guidelines for Māori research ethics. Health Research Council of New Zealand.

Kellstedt, P. M., & Whitten, G. D. (2013). The fundamentals of political science research (2nd ed.). Cambridge University Press.

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. SAGE.

Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). SAGE.

Pohatu, T. W. (2013). Ata: Growing respectful relationships. He Pukenga Kōrero, 8(1), 1–8.

Rhodes, R. A. W. (2017). Interpretive political science: Selected essays (Vol. 2). Oxford University Press.

Schön, D. A. (1983). The reflective practitioner: How professionals think in action. Basic Books.

Smith, L. T. (2012). Decolonising methodologies: Research and Indigenous peoples (2nd ed.). Zed Books.

Stephenson, W. (1953). The study of behavior: Q-technique and its methodology. University of Chicago Press.

Van Exel, J., & de Graaf, G. (2005). Q methodology: A sneak preview. Social Sciences, 2(June), 1–30.

van Dorp, E. J., & ‘t Hart, P. (2019). Navigating advisors and advisory systems. In R. A. W. Rhodes (Ed.), Narrative policy analysis (pp. 73–96). Palgrave Macmillan.

Watts, S., & Stenner, P. (2012). Doing Q methodological research: Theory, method and interpretation. SAGE.