I Built a Fake Think Tank to Understand the Real Ones. It Works Better.
There’s a particular kind of document that lives on the websites of policy think tanks. You’ve seen it. A PDF, usually. Somewhere between 40 and 80 pages. A cover with a photo of someone looking thoughtfully into the middle distance, possibly at a whiteboard, possibly at a river. A title that takes 14 words to say something that could be said in four. And inside, buried somewhere around page 23 after the executive summary, the methodology section, the literature review, and three pages of acknowledgements — an actual idea.
One idea. Sometimes less.
I’ve spent a lot of my professional life reading these documents, working alongside the people who produce them, and occasionally having to translate what they actually mean for people who have to make decisions based on them. It is, I will tell you frankly, a deeply humbling experience. Not because the people writing them aren’t smart. Many of them are spectacularly smart. But because the gap between the intelligence in the room and the intelligence on the page is, in most cases, a chasm you could lose a city in.
So. I built my own.
Not to publish it. Not to get a grant. Not to compete. I built it as a learning device — a way of understanding what good policy thinking actually looks like when you strip away the institutional theatre, the jargon, the seventeen rounds of legal review, and the almost pathological need to hedge every single statement into meaninglessness. I wanted to know: what does a genuinely useful policy analysis actually require? What are the ingredients? What makes one piece of work move a room and another one sit unopened on a desk for six months before someone prints it out to prop up a monitor?
The answer, it turned out, was not what I expected.
Why a think tank, specifically
Let me back up a step, because “I built an AI policy think tank” sounds more dramatic than it was in the beginning.
What I actually did was start asking an AI to behave as though it were a serious policy research entity — not a chatbot answering questions, but something with a brief, a methodology, a set of analytical commitments, and a genuine point of view. I gave it a name. I gave it an area of focus. I told it what it valued and what it didn’t. I told it who it was writing for and what it was trying to change. And then I started throwing real policy questions at it.
Housing affordability. Energy transition. Aged care workforce. AI regulation. The kinds of things that have real stakes, real complexity, and no shortage of existing analysis from real institutions that you can benchmark against.
The benchmarking, it turns out, is where things got interesting.
The thing about real think tanks
Here’s what I noticed, reading maybe two hundred policy documents across probably thirty institutions over the past couple of years. The best ones — the ones that actually move the needle on thinking — do something very specific. They don’t just summarise what is known. They take a position on what it means.
Not in a partisan way. In an analytical way. They commit. They say: given what we know, and given what we value, this is what we think should happen, and here’s why. There’s intellectual skin in the game. And because there’s skin in the game, the reader can push back, probe the logic, find the assumptions, stress-test the conclusion. The document becomes a conversation, even if you’re reading it alone at 11pm.
The mediocre ones — and there are a lot of mediocre ones — do something else entirely. They balance. They present multiple perspectives. They acknowledge the complexity. They note that “further research is needed.” They arrive at the end of 60 pages having successfully avoided saying anything that anyone could disagree with, which also means successfully avoiding saying anything that anyone could act on.
It is the written equivalent of a shrug, dressed up in Calibri 11pt.
My AI think tank has no interest in shrugging.
What I built, specifically
The architecture is not complicated, and that’s part of the point.
I set up a system prompt that establishes the institution’s values, its epistemological commitments (what kinds of evidence it takes seriously, what kinds it treats with scepticism), its target audience, its stylistic preferences, and its absolute refusals. Things it will not do: bury the lede in equivocation. Pretend that all positions are equally valid when the evidence clearly favours one. Use the passive voice as a mechanism for avoiding accountability. Write a recommendation section that sounds like it was designed by committee to offend no one.
Things it will always do: name the tension explicitly before trying to resolve it. Distinguish between what the evidence shows and what values are doing the work. Be honest about the limits of its own analysis. Write as though the reader is smart and time-poor, not as though they need to be educated from first principles every single time.
I also gave it a tone. Not aggressive. Not provocative for its own sake. But confident. Committed. The kind of institutional voice that sounds like it has thought very hard about something and arrived somewhere, rather than the kind that sounds like it is trying very hard not to be blamed for arriving anywhere.
Real think tanks operate inside ecosystems. They have funders who have interests. They have relationships with ministers and shadow ministers and departmental secretaries that they need to maintain if they want their work to be read, cited, used. They have boards.
The first real test
The first serious question I threw at it was housing affordability. Partly because I know the policy space reasonably well, partly because it’s a topic where the quality of existing think tank analysis varies enormously, and partly because it’s one of those areas where the conventional wisdom is so thoroughly calcified that most institutional analysis ends up reinforcing the frame rather than challenging it.
The conventional frame, if you’ve been paying attention, goes roughly like this: housing is expensive because there isn’t enough of it, which means the solution is to build more of it. Supply, supply, supply. It’s an intellectually tidy answer. It’s also an answer that conveniently avoids the rather uncomfortable question of why housing prices going down would be politically catastrophic for a large percentage of Australian homeowners — and therefore why no government of either stripe has ever actually tried very hard to make housing cheaper, as opposed to making it slightly less expensive to get into.
I asked the AI think tank to produce an analysis of the housing affordability crisis.
It named the political economy of incumbent homeowners in paragraph three. Not hedged. Not “some critics have suggested.” Named it, explained the incentive structure, traced the electoral calculus, and then built its recommendations from there rather than pretending the elephant wasn’t in the room.
I compared it to a major recent report on the same topic from a well-known Australian institution. The real report did not mention the political economy of incumbent homeowners until page 41. In passing.
This is not a small thing. The page on which you choose to acknowledge an inconvenient truth is itself a political act.
What it turns out think tanks are actually for
I want to be careful here, because I’m not being entirely fair to the real ones, and I know it.
Real think tanks operate inside ecosystems. They have funders who have interests. They have relationships with ministers and shadow ministers and departmental secretaries that they need to maintain if they want their work to be read, cited, used. They have boards. They have reputations to protect. They are, in short, institutions — which means they are subject to all of the forces that cause institutions to sand the edges off their thinking before anyone reads it.
None of that is cynical. Most of it is rational. If you want your 15-year body of work on climate adaptation to matter, you do not start by setting fire to a relationship with a key stakeholder in year three over one report that you could have toned down slightly. That is a legitimate calculation.
My AI think tank has no stakeholders. No relationships to protect. No funders to keep happy. No grant cycles. No board retreats in the Hunter Valley. It has exactly the analytical commitments I gave it, and nothing else.
And that turns out to matter, a lot, in terms of what comes out.
What I actually learned
Several things, and I want to be precise about them because the easy version of this story — “AI is better than experts, wow” — is both wrong and dull.
The first thing I learned is that most policy analysis spends enormous effort establishing what is already known and very little effort on what should be concluded from it. This is partly an academic habit (literature reviews, methodology sections, the ritual acknowledgement of prior work), partly institutional caution, and partly, I suspect, a genuine confusion between the two jobs of analysis: describing the landscape and navigating it. My think tank skips the first job almost entirely and focuses almost exclusively on the second. That’s only possible because the first job has been done, repeatedly, by many people — and the AI can absorb all of it in a way no single researcher can.
The second thing I learned is that the quality of policy thinking scales with the quality of the values framework underneath it, not the sophistication of the technical analysis above it. Two pieces of work can look at exactly the same data and arrive at completely different conclusions because they’ve made different (often unstated) choices about what matters most. The housing example is perfect: if you believe housing is primarily an asset class, your recommendations look one way. If you believe housing is primarily a basic right, they look entirely different. Most policy documents are too frightened to say which one they believe, which means they can’t actually be useful to anyone trying to make a real choice.
The third thing — and this one surprised me most — is that the most valuable thing the AI can do is ask the question that the humans around the real problem are too close to ask. The thing everyone in the room knows but no one has said out loud. The assumption baked so deeply into the analysis that it stopped looking like an assumption years ago and started looking like gravity. Getting that named, clearly, early, without apology, changes everything about what comes after it.
The uncomfortable part
I should probably say the quiet part loud.
This exercise is, in some ways, a critique of an industry that I work within and alongside. I have good friends in think tanks. I have colleagues who have produced genuinely excellent work through those institutions. I am not suggesting the institutions are worthless, or that the people in them are lazy, or that the entire edifice should be replaced by an AI with a good system prompt.
What I am suggesting is that the constraints that produce mediocre institutional work are, in many cases, not constraints on intelligence or rigour. They’re constraints on honesty. Specifically, on the willingness to say what you actually think when what you actually think might be inconvenient to someone who matters.
That is a very human problem. And it is not obviously solvable by better research methodology, or bigger datasets, or more interdisciplinary collaboration (god help us), or any of the other things the sector tends to prescribe for itself when it senses it is losing relevance.
It might be solvable by a different relationship with the reader. By remembering that the person on the other end of a policy document is not a passive recipient of expertise, but someone trying to understand a complicated world so they can make a better decision in it. That the job is not to prove you did the research. The job is to make them see what you see.
Where it’s going
I’ve kept building it. The brief has expanded. The outputs have gotten sharper, not because the model has changed, but because I’ve gotten more precise about the values underneath it — clearer on what it’s for, more honest about what it’s not for, better at identifying the moments where it’s tempting to equivocate and explicitly refusing to.
I’ve been using it to pressure-test real policy submissions before they go in. To find the assumptions hiding inside the logic. To draft the argument you’d want to make if you weren’t worried about what anyone would think. And then, occasionally, to make a version of that argument that a human being with real relationships and real consequences might actually be able to use.
The gap between those two versions is, itself, information. It tells you exactly how much institutional friction is shaping the output. How much of what eventually goes on paper is the analysis, and how much is the management of relationships.
Sometimes that management is appropriate. Institutions are not just in the business of being right; they’re in the business of being useful over time, which requires staying alive and trusted. I understand that.
But sometimes the gap is just... cowardice. Dressed up as balance. Presented as rigour.
And you can tell the difference.
The last thing
A friend asked me recently whether I thought AI was going to replace policy analysts. I told them that was exactly the wrong question, for the same reason that asking whether word processors replaced writers was the wrong question.
The interesting question is: now that you have this tool, what are you going to say with it that you couldn’t say before?
Because if the answer is “the same thing I was saying, but faster” — well. That’s an answer. It’s just not a very exciting one.
The more interesting answer involves being willing to actually commit to something. To put a view on the page that a reader can grab hold of and either agree with or push back on. To treat policy not as a landscape to be mapped but as a problem to be navigated, with a direction, and a destination, and a reason for choosing it.
Think tanks at their best have always done this. The good ones still do.
I just built a smaller, weirder, completely unaccountable version of one, and it’s been the most clarifying professional exercise I’ve done in years.
Turns out, once you remove everything that’s stopping you from saying what you think, what you think gets a lot more interesting.
Who knew.
You can play with the AI think tank yourself here: Menzies.AI