https://acoup.blog/2022/08/29/new-acquisitions-on-the-wisdom-of-noah-smith/ Skip to content A Collection of Unmitigated Pedantry A look at history and popular culture Menu * Home * Resources for Teachers * Resources for World-Builders * Book Recommendation List * Guest Posts * Contact * About the Pedant New Acquisitions: On the Wisdom of Noah Smith Bret Devereaux New Acquisitions August 29, 2022 25 Minutes I generally try to avoid having Twitter disputes spill on to the blog. Generally what happens on Twitter is best left on Twitter and in some cases not even that. However this past week I was pulled into a Twitter debate with Noah Smith about the validity of the way that historians offer our knowledge into the public debate who then opted to continue that debate in a long-form blog post tackling my work in particular, which in turn seemed like it demanded a long-form response rather than something in 280 characters or less. Noah's initial tweet declared: My beef with academic history is not that it's woke. Nor that it's anti-woke. It's that the theories are given even more credence than macroeconomics even though they're even less empirically testable. -- Noah Smith (@Noahpinion) August 22, 2022 Carlos Morel responded with some confusion, noting that historians tend to be quite wary of what he termed 'theorizing' and perhaps foolishly I waded in because I felt Morel had made a valid point somewhat confused by the fact that Smith seemed unaware that the word 'theory' is used differently in different fields.^1 And so to compress quite a few tweets into a short statement, I offered that in fact this was a real difference: historians do not generally aim to construct laws of general applicability (quite unlike social scientists, who do), but instead to study and furnish relevant exemplars as tools (but not predictions) for thinking about current problems. To which Smith responded that he'd be putting together a post where my public writings would "form the core" showing how contra to what I said that "academic historians make strong theoretical claims that cannot be evaluated empirically." And here I want to ask you to please put a pin in the word 'empirically' now used twice because we're going to come back to it. In any event he did write that post and it is here and you can read it; unfortunately I cannot recommend it. We're going to talk about why but it is going to end up being rather involved because to explain why a shallow critique of a discipline's methods is shallow, you have to explain how that discipline functions and why it does so. But before we get to the complicated stuff we should deal with: The Bad Faith Complaints We can start with the conceit of the section titled "Historical analogies are theories" (mark that word 'theories' again because we're coming back to it) which is where my work is focused on. For the most part I just want to get most of this out of the way before I get to the real meat of Smith's complaint. There are a few problems with his reading of the essays in question which seems to speak either to bad faith or a failure in comprehension. Let's take this paragraph of his: [stupid1][stupid1] Which is building off of this paragraph of mine: [bulwark1][bulwark1] Now perhaps this is unfair of me coming as a specialist trained in the reading of texts but it sure does seem to me like Smith has stripped out quite a lot of qualifiers here to fundamentally misrepresent a paragraph that is in fact a giant caveat, instead presenting it as claim of general statistical predictability. Phrases like "stretch the scientific metaphor" before "laboratories of democracy" fairly clearly indicate that, no, I am well aware these are not actual laboratories; the double-quote marks around "data set" do the same, acknowledging that this is analogous to a data set but not an actual data set.^2 Moreover I am leading the paragraph with the idea that there is fundamental uncertainty, indeed "always risks" in "drawing comparisons across vast chasms of time and culture" before introducing the idea that there may well be other examples which might offer different lessons. Rather than "asserting that ancient Greece is an appropriate analogy for our modern politics," as Smith has it, I have explicitly opened the door to the idea that it might not be. As we'll see, this sort of stress on context and contingency over rules of general applicability is a key difference between the methods historians use and the social sciences. Smith then has a brief foray into wondering if I am constructing an argument-by-definition that tyrants are defined by their effort to repeatedly seek power (a pointless digression; I define 'tyrant' in the piece) which we may mostly skip over except to note that I think Smith's failure to familiarize himself at even a basic level with the material in question is rather exposed by his use of Richard Nixon as the example of a tyrant who would be 'No True Scotsman'd' out of this potential definition I am not actually proposing. Richard Nixon was an crook and a bad president, but I know of no definition where a politician who won election to his office and then was forced to leave it by constitutional processes and didn't use violence to attempt to retain power would qualify as a tyrant; Watergate does not have a body count. The ancient Greek definition (the one I was working with here) functionally requires the position to be extra-constitutional and violence to be used in the seizure and maintenance of power as standard definitional features.^3 Even the remarkably reductive characterization I offered in the essay, that tyranny was a neutral Greek term for one-man rule (if you think you detect an editor shaving down a more complete definition for the sake of pacing, that's because you do), still disqualifies Nixon who - quite manifestly in the instance as he was about to be removed from power by Congress - did not rule alone. The broader problem here is Smith's choice of targets, which are not a bunch of peer-reviewed journal articles but in fact a number of mostly short-form (c. 1200-1500 word) essays in traditional media publications. At first when Smith read the phrase "Would-be tyrants keep trying until they succeed" to mean "all would-be tyrants keep trying until they succeed," I assumed he was just unfamiliar with this genre of writing where editors tend to strip out caveats like 'sometimes' or 'frequently' as redundant and where the general mode is to present a single detailed example on the assumption that, as a historian we have in fact considered the broader evidence as it exists but do not have the word-space to exhaustively list all examples.^4 Indeed in most cases, Smith has removed what nuance was contained in the original essays. A conclusion of "a need to choose: Either trim down American objectives in what are, effectively, occupied countries to those which can be achieved merely by organizing the existing military and political structures or settle down to the task of building a new military organization [there] from the ground up" becomes in Smith's summation, "to predict that the U.S. military would be more effective if it reorganized itself in certain ways." The latter is frankly a misleading summary of an argument which instead offers the Romans as one tool (of potentially many, including others like the British army in India mentioned in that very essay) with which to think about the trade-offs inherent in trying to raise military force in occupied countries. Perhaps Smith has never written in this genre before and so is unfamiliar with it and its constraints . Except that Noah Smith spent some time as a columnist for Bloomberg and wrote in exactly this genre. In fact, he wrote exactly these sorts of essays, in the deep, dark past of last year. Here, for instance, is Noah Smith contending that it took Franklin Delano Roosevelt two crises to be transformative whereas at that point Joe Biden had only had one crisis and perhaps that wouldn't be enough: [stupid2][stupid2] Am I to assume that Smith has done a statistical study of every transformative president and confirmed that the mean number of crises they required to be transformative was two? Why hasn't he included these studies in his short article? Has he done empirical verification of the two-crisis theory? Of course not, he is reasoning from a single historical exemplar and drawing a conclusion from it, a fairly standard use of case-based inductive reasoning. It turns out he does know how this works. Collectively I think these problems speak to the shallow degree to which Smith is engaging with the question; critiquing the non-comprehensiveness of short, traditional media articles is a failure of seriousness. No one is going to give a complete accounting of ancient Greek tyranny in the c. 1,000 known Greek poleis in 1,500 words; demanding they do so is a literal category error, mistaking the think-piece for the footnote-laden journal article. With that out of the way, we can at last get to the meat of the disagreement. Epistemologies In his tweets (above) and in the essay itself, Smith is consistent in his call for empirical methods to be used and for historians to acknowledge that they are making "predictive theories." I think he reaches his main point most clearly here: [stupid3][stupid3] And my sense is that the problem here is that Smith has not familiarized himself with history as a discipline or historical argumentation, both how it works but also why it works that way. Consequently he's attempting to jam them into his social science (economics, particularly) framework, apparently unaware that there are different methodologies and indeed often different epistemologies than his own. That is a fairly big problem and a disappointing one, so let me explain what I mean. First, an epistemology is a theory of knowledge - that is a theory of how we can come to know things. You will note that I keep using it in the plural because there is not one epistemology in use but in fact several; actual functioning humans rely on different epistemic principles at different times and in different subjects. Empiricism is one of these epistemologies, which argues that direct, personal sense-perception is the chief or even only valid form of knowledge; in its pure form, empiricism rejects authority, testimony, and rationalism (that is the application of raw logic) as sources of knowledge. To test something empirically is thus to test it experientially, typically in the form of an experiment whose results can be observed empirically, that is with the senses. That of course includes using all sorts of tools, including statistical tools; a statistical test of data is an empirical test of that data as it outputs a result which can be observed. Empiricism, as you may gather, is the epistemology which underlies the scientific method and so is the chief epistemology in the natural sciences and the social sciences. And I am terribly fond of it; for things which can be tested empirically, it works great and is generally to be preferred over other epistemological approaches. Those who know my actual research will be well aware that I think forms of empirical testing (such as experimental archaeology) can be very valuable in helping to understand the past. The problem is that not all things can be tested empirically, either because it is impossible to do so or because it is impractical to do so. On the impossible end, we have phenomenon which are not subject to independent sense-perception, like the thoughts of a person other than yourself; we are not (yet?) able to export someone's thoughts and brain imaging provides at best a very incomplete picture of someone's mental state. The best one can do is ask the person what they are thinking but for the researcher this introduces a non-empirical break in which they are forced to rely on the subject telling the truth. Likewise, a fairly large category of things which cannot be tested empirically is everything that does not exist right now, since humans experience time in a linear, continuously forward-moving fashion, leaving all things that once existed but no longer do out of the realm of human sense-perception. That, of course, is terribly relevant to historians, because it means very little of what we study is subject to empirical tests. Most humans, indeed even famous humans, leave no empirical evidence of their existence; one could not, for instance, empirically prove the existence of Socrates. And yet we can be very confident Socrates existed!^5 On the other end, some things are impractical to test empirically; empirical tests rely on repeated experiments under changing conditions (the scientific method) to determine how something behaves. This is well enough when you are studying something relatively simple, but a difficult task when you are studying, say, a society. Social scientists look for 'natural experiments' to aid their understanding, but this is compounded by the rarity of some really important phenomena. An economist studying market interactions can statistically analyze millions of daily trades on the NYSE, but a political scientist has to wait four years to add one data-point to a data-series on presidential elections. Consequently empirical methods struggle to establish solid predictability for such rare events despite the best efforts of very talented analysts. Needless to say, asking people to do a controlled to-scale experiment in, say, warfare or pandemic control would face severe legal, ethical and practical hurdles, but at the same time these events are sufficiently rare and complex that relying only on natural experiments results imposes severe limits. Again, empiricism is great, when you can use it. Now a philosopher might then insist on pure empiricism (or, as in David Hume's formulation, pure empiricism in matters of fact and pure logic for the relations of ideas) and just declare everything outside of it unknowable, but in practice this is absurdly limiting. In our actual lives and also in the course of nearly every kind of scholarship (humanities, social sciences or STEM) we rely on a range of epistemologies. Some things are considered proved by the raw application of deductive reasoning and logic, a form of rationalism rather than empiricism (one cannot, after all, sense-perceive the square root of negative one). In some things testimony must be relied on; perhaps the most important element of history as a discipline are the systems we apply to assess the reliability of testimony for events which, having taken place in the past, cannot be viewed empirically in the present (the term for all of these methods collectively is 'the historical method.' Historians are not creative people when it comes to naming things). To insist that all knowledge must be empirical knowledge and that all theories must be empirically testable theories is to either misunderstand what the words mean or to engage in scientism - the delusion that all knowledge is empirical, scientific knowledge. The irony in this is that when Smith suggests a specific test in response to my article it is not, in fact, an empirical test. In particular he suggests that, "Before we conclude that "would-be tyrants keep trying until they succeed", we should rigorously and systematically check the historical record to see if we could identify, ex ante, a set of characteristics that allowed us to predict who would keep trying to seize power and who would give up." But without then running the experiment in the present (that is, waiting to see what kind of dictators show up and if they match the 'set of characteristics') there's no empirical test there. It is difficult to even imagine how such a test would be managed (how do you isolate the variables?), but its results would be useless to anyone today in any event because they wouldn't be known for decades. ^6 This is of course the great difficulty that political scientists wrestle with (not without significant success, mind you): the effort to apply scientific methods to the subject (large-scale human interactions which often include violence) where controlled experiments are impossible, unethical or both. Instead what he's actually suggesting isn't an empirical test at all: it is a quantitative, statistical test of data none of which can be empirically verified because it all happened in the past; only the statistical analysis is subject to empirical verification. It's a lot easier to see how this would be done: past leaders would be grouped into tyrants and non-tyrants, each assigned a series of mathematically defined qualities and then one would run a regression analysis to determine which variables are most predictive of being a tyrant. One would then 'test' the analysis against other historical tyrants not included in the original sample to see if the predictability held. That's still not an empirical test (neither the main data set, nor the test data set are empirically derived - only their comparison is empirically observed which won't correct for any problems in the non-empirical elements: the evidence and the process of reducing it to data), but this is precisely the sort of work that political scientists do and often to useful result. That said I hope in this case the difficulties implied by assigning historical figures mathematically defined qualities to enable statistical comparison, especially in the context of incomplete historical information, are not lost on the reader. There is perhaps a reason no political scientist^7 has yet tried to go and run this analysis. How History Works And now at last we get into both how and why the historical method differs from social science approaches to the past. Now the difference here shouldn't be taken as too binary; there is significant overlap in both methods and concerns. But generally the social sciences aim to establish general rules for how societies function which have strong future predictiveness; laws of the workings of society akin to the laws of the workings of physics whereby we can predict with quite a lot of precision where a ball will go when thrown; for the sake of clarity I'm going to call these 'laws of general applicability.' By contrast the focus of historians is on the past itself; while historians of past decades often toyed with the idea of 'grand narratives' akin to the social sciences' laws of general applicability, these have long since been abandoned by all but a few because the exceptions kept overwhelming the general rules. Of course historians hope that the work we do to create knowledge of the past will be useful in the present, but the discipline prioritizes the former over the latter. As a result, historians generally reject the creation of laws of general applicability, insisting that while the past is a useful teacher, efforts at strict predictability will always be overwhelmed by contingency, context and unexpected variables. That in turn leads to differences in methods. Social scientists - and here we mostly mean economists and political scientists, the two social sciences that collide with history most often - use many methods but perhaps the most important is quantitative analysis using historical data.^8 Historians by and large do not reject that method but tend to be leery of it because a key step in the process is the reduction of a whole bunch of very complex evidence to a handful of mathematical variables which can then be analyzed statistically. I use the word reduction here quite intentionally; those mathematical values can only be either simplifications or distortions of the evidence used to create them. Going back to our example above with tyrants, imagine the difficulties in reducing figures like Peisistratos or Cylon (or Napoleon, Caesar or Hitler) to mathematical expressions; there is ample room for the introduction of new bias but even if done faithfully the result will flatten out much of the nuance of these figures (or be forced to infer in places where we lack evidence). And so even when such data is generated with great care that means taking complex, difficult phenomena and flattening them. Let's take an example and we'll pick a statistical argument that I think actually has some considerable merit to it: the democratic peace theory. The theory (in its modern form), arguing that democratic countries do not (generally) go to war with each other, emerged out of statistical studies comparing historical democracies on a period-by-period basis with a list of historical conflicts and observing the lower rate of democracy-on-democracy conflict.^9 But a brief look at the original study reveals how much the complexity of the actual history was flattened to provide for statistical analysis; the United States is a democracy in 1776, but Great Britain isn't until 1832 - a look at the actual criteria (fn. a on p. 212) reveals the tortured efforts to get a binary classification which produced data that made even minimal sense. Non-European style potentially democratic states - I have in mind the Six Nations of the Iroquois Confederacy - are entirely excluded. The binary created means that the United States in 1812 is 100% a democracy despite holding a not-trivial proportion of its populace in slavery, while its enemy Great Britain is 100% not a democracy despite the main political power being vested in an elected parliament.^10 The actual complexity has to be flattened out to produce data; this isn't a critique of the theory - the flattening was unavoidable. Some more recent efforts at the problem have tried assigning democracy or liberalism 'scores' to countries but of course that itself introduces all sorts of complications. The conversion or compression of fuzzy, non-numerical evidence to data is thus not free, it is a 'lossy' process. And that leads to a fairly frequent kind of interaction between the disciplines: historians scour the evidence and produce our best assessment of them, often at fairly low confidence. Social scientists then take these assessments and turn them into data, stripping or flattening out the caveats in the process and then produce impressive looking statistics like those implied by the chart below and attempt to draw conclusions from them, while historians cry foul over how - in Smith's phasing - 'disciplining with data' cannot correct for the problems with the data. ImageImage In the case of this chart, what Max Roser has succeeded in doing is not charting global deaths in conflict, but rather in charting the rate at which evidence for battles is preserved over time and the reliability of the estimates of their casualties. One can actually see the problem with this effort at creating a map of all known battles: Every recored battle in history -also why the EU was such a a good idea pic.twitter.com/RC0BLt2mye -- Soner Cagaptay (@SonerCagaptay) May 4, 2021 The conclusion drawn in the tweet is, of course, spurious; the prominence of Europe here is an artifact of what battles are well-documented in the languages that the database was constructed with. And before one argues this is just a tweet, John Keegan tried the same thing in A History of Warfare (1993). The map and its conclusions are only so good as the evidence that informs the data it is based on. Which appears to contend that the Mongolian Steppe was a relatively peaceful place. Of course it wasn't, but it was a place that produced almost no written records and so provides almost no recorded battles to the database, though in the brief moments we can see clearly into the history of this region it was very violent indeed. But the gaps here are massive; keep in mind for instance on the first chart that prior to 1492 (so nearly the whole first sixth of the Max Roser chart) functionally none of the conflicts in North or South American can be represented because they leave no trace in the evidence. Was it peaceful? No, it doesn't seem to have been, but that's not reflected in the data. All of which is to say that while the social science approach to understanding the past has its benefits, it also has some fairly severe limitations. It is not simply a superior method. My own view is that both historians and social scientists have a fair bit to learn from each other's methods and conclusions (and caveats and criticisms) but of course this requires mutual respect and a mutual methodological understanding. Historians by contrast are constrained by two key factors: as a discipline we're taught to avoid simplifying our subjects for the sake of analysis (some historians are more careful in this than others) and rather than focusing on converting the historical research of another field into data, historians deal directly with primary sources, which in turn demand quite a lot of time and energy be invested into collecting and working through the evidence. Smith's insinuation that historians aren't doing the "hard, often unrewarding work" is incredible coming from someone who works in a discipline that has all of its historical data 'pre-chewed' for it by historians. Every dot on that chart of battle deaths above likely represents years of work by historians collecting, sorting and understanding difficult and often contradictory primary source material. Without that work, producing the chart would be impossible. Consequently that means that rather than engaging in very expansive (mile wide, inch deep) studies aimed at teasing out general laws of society, historians focus very narrowly in both chronological and topical scope. It is not rare to see entire careers dedicated to the study of a single social institution in a single country for a relatively short time because that is frequently the level of granularity demanded when you are working with the actual source evidence 'in the raw.' Nevertheless as a discipline historians have always^11 held that understanding the past is useful for understanding the present. Or as arguably the first historian, Thucydides puts it, "if it be judged useful by those inquirers who desire an exact knowledge of the past as an aid to the understanding of the future, which in the course of human affairs must resemble, if it does not reflect it, I shall be content."^12 Smith declares that this sort of use of history means that these "are social-science theories" (emphasis original), which is an absurd bit of turf-claiming from a discipline (the social sciences) which is, as a practice distinct from philosophy or history, something like 2,225 years younger than history.^13 So how do historians build arguments with present-tense implications and is this approach valid? The present-tense implications of historical research generally come in two kinds: either the history of a thing (usually an institution) that still exists is used to explain how that thing came to exist as it does or the history of something in the past is presented as analogous to something similar in the present, such that the former is a useful tool when thinking about the latter. Smith is clearly focused on the latter kind of historical argument, so we can set the history-of-a-thing-that-exists argument aside for today. The epistemic foundation of these kinds of arguments is actually fairly simple: it rests on the notion that because humans remain relatively constant situations in the past that are similar to situations today may thus produce similar outcomes. This is no new thing; the attentive will notice our good friend Thucydides laying out this very logic some c. 2,420 years ago. At the same time it comes with a caveat: historians avoid claiming strict predictability because our small-scale, granular studies direct so much of our attention to how contingent historical events are. Humans remain constant, but conditions, technology, culture, and a thousand other things do not. I think it would be fair to say that historians - and this is a serious contrast with many social scientists - generally consider strong predictions of that sort impossible when applied to human affairs. Which is why, to the frustration of some, we tend to refuse to engage counter-factuals or grand narrative predictions. ... and it's remarkable -- and honestly confusing to visitors from other fields -- the extent to which historians resist explicit reasoning about causation and counterfactual analysis even while constantly saying things that clearly implicate these ideas. -- Matthew Yglesias (@mattyglesias) August 28, 2022 We tend to refuse to engage in counterfactual analysis because we look at the evidence and conclude that it cannot support the level of confidence we'd need to have. This is not a mindless, Luddite resistance but a considered position on the epistemic limits of knowing the past or predicting the future. Instead historians are taught when making present-tense arguments to adopt a very limited kind of argument: Phenomenon A1 occurred before and it resulted in Result B, therefore as Phenomenon A2 occurs now, result B may happen. Tyrants in the past have made multiple attempts to seize power, therefore tyrants in the present may as well, therefore some concern over this possibility is warranted. The result is not a prediction but rather an acknowledgement of possibility; the historian does not offer a precise estimate of probability (in the Bayesian way) because they don't think accurately calculating even that is possible - the 'unknown unknowns' (that is to say, contingent factors) overwhelm any system of assessing probability statistically. Once again what Smith mistakes for lethargy is in fact a considered position by historians that further certainty is not possible; the critique historians make is that the methods Smith advises take that essential, unresolveable uncertainty, dress it up with numbers and pretend it has vanished (or been quantified) when it hasn't and cannot be.^14 Nevertheless this historian's approach holds significant advantages. By treating individual examples in something closer to the full complexity (in as much as the format will allow) rather than flattening them into data, they can offer context both to the past event and the current one. What elements of the past event - including elements that are difficult or even impossible to quantify - are like the current one? Which are unlike? How did it make people then feel and so how might it make me feel now? These are valid and useful questions which the historian's approach can speak to, if not answer, and serve as good examples of how the quantitative or 'empirical' approaches that Smith insists on are not, in fact, the sum of knowledge or required to make a useful and intellectually rigorous contribution to public debate. Though I've already offered several examples where Smith critiques historical methodologies without actually bothering to understand them, I want to add one more, which is Smith's apparent lack of awareness of the different uses of the word 'theory,' which came up in the debates that led to his essay. Smith uses 'theory' to mean 'hypothesis' or 'predictive theory' but within the discipline of history 'theory' as a term refers to the broad intellectual framework into which evidence is interpreted; that ought to make sense given that history is by and large a discipline of source criticism rather than one engaged in hypothesis testing (though we do a bit of that too). Historical theory is thus often concerned with questions of what sort of history is important, what questions can and ought to be asked of the sources and how their importance should be understood (e.g. mentalites within the Annales school; a study which focuses on mentalites can be described as being situated within an Annales theoretical framework), a very different beast from a hypothesis. The Wisdom of Noah Smith In conclusion, Noah Smith's analysis here cannot be recommended. While accusing historians of shrugging off the "hard, often unrewarding work" he has failed to engage meaningfully with the historical method and its epistemic systems; the easy, swiftly rewarding work of establishing a basic understanding of other disciplines. The result is an analysis which repeatedly misunderstands the claims that historians are making and deliberately ignores the way they signal uncertainty. Instead, Smith indulges in rank scientism, insisting that all claims be confirmed empirically, a calling he himself failed to answer to on the previous day in arguing for the elite overproduction hypothesis which he admits, "make[s] some questionable assumptions about how labor markets work" and which isn't empirically tested. For my own part I would note that Peter Turchin's effort to find support for this hypothesis in the ancient world is fatally flawed by the lack of evidence; Turchin has built a vast castle on sand - it can be no stronger than its meager evidentiary foundation.^15 Of course some historians absolutely do make arguments that extend out beyond what their evidence can support firmly. Sometimes they are responsible in signalling uncertainty and sometimes they are not. The problem is especially acute in compressed formats and genres. Of course social scientists do much the same; it cannot for instance both be the case that we can be certain that forgiving student loan debt both absolutely will and assuredly will not induce more inflation. On this latter point, I suspect Smith would agree and yet I don't see him suggesting that economics, as a discipline, should pack it up and go home. Great to see history majors being abandoned. Maybe now all the historians will have time to go read a book pic.twitter.com/ e5xE7Exfr7 -- Noah Smith (@Noahpinion) August 24, 2022 And it would be easy enough to dismiss one ill-advised essay on an internet full of them were it not for the fact that this kind of scientism contributes to the hardship of a discipline already under sustained attack not because of the predictions it supposedly makes but because of the true things we insist on teaching about the past, while at the same time history departments continue to shrink. Of course these trends worry me as a historian, but they ought to also worry social scientists too, who after all, as noted rely on historians to process the evidence they use to produce the data that forms the foundations of their conclusions and also benefit from historians offering a critical second look at their models and conclusions. Without historians to do that work, the ability of social scientists to reach for data earlier than the rise of the modern administrative state functionally vanishes. Yet for all of this it is the wisdom of most historians to be able to see the value in methodological and epistemological approaches other than their own. Alas, it is a wisdom Noah Smith apparently lacks. Share this: * Twitter * Facebook * 1. And before someone assumes that this is just because the humanities use a non-humanities term wrong, it seems worth noting that theory (theoria) was our word first and had our meaning first. We'll get to the difference in that meaning in a moment. 2. By the way, Noah - misquote! Punctuation has to be preserved in quotation, "data set" embedded in a quotation is "'data set.'" 3. This is, in fact, what separated in the Greek mind, kings from tyrants: a turannos ruled extra-constitutionally (that is, outside of the traditional system of government for the polis) with violence while a basileus (king) ruled in a customary or constitutional position through tradition and legitimacy. Thus Sparta and Macedon had kings, while Syracuse a series of tyrants (the Romans do not make this distinction and so often called the Syracusan tyrants kings (reges), so you will see that usage in English as well, but in the Greek sense they were tyrants). 4. In this case that's not the evidence for all tyrants everywhere, but specifically as noted with the caveats above, the evidence for ancient Greek tyranny, a much smaller set; Smith assumes this term is perfectly synonymous with 'dictator,' which it is not but again that goes to Smith's failure to grapple with terminology in disciplines other than his own. 5. Because his existence is independently attested by multiple contemporary authors, most notably Plato, Xenophon and Aristophanes. But this is testamentary evidence of the sort empiricism, as a total epistemology rejects, the absurdity of which is why attempting to treat empiricism as a total epistemology for practical purposes is derisively termed scientism, instead of science. 6. One would need, presumably, to identify the relevant characteristics, measure them quantitatively somehow, and then look at their rate of incidence in the emerging tyrant population against the rate of incidence in the general population. I shudder to imagine how long you'd have to wait until the tyrant population reached a sufficient size to achieve statistical significance in all but the most banal of observations. 7. That I know of, I cannot claim to have comprehensive knowledge of every poli-sci statistical study. 8. I am of course simplifying here to a degree to the methods that Smith is most focused on (the quantitative ones). Both economics and political science also have branches that rely heavily on game theory, sometimes as thought experiments, sometimes as empirically tested, although with theoretical approaches that rely more on rationalism than empiricism. 9. The first full version of this is M.W. Doyle, "Kant, Liberal Legacies and Foreign Affairs," Philosophy & Public Affairs 12.3/ 12.4 (1983). The models and methods for assessing them have been refined substantially since then, but the principle is generally considered well-established, though substantial argument about its causes remains, see for instance the substantial bibliography on D. Reiter, "Is Democracy a Cause of Peace" Oxford Research Encyclopedia of Politics (2017) for a sense of the debate to date. Because we're going to engage with actual published scholarship here rather than tilting at 1500 word magazine articles. 10. As is fairly typical, by the by, the statistical approach here extends only to the 18th century. Subsequent studies have pushed the method further back but not out of the early modern period. I know of no effort to try to test the democratic peace theory against ancient democracies. Attempting to define democracy in that context would doubtless introduce new problems. 11. A rare word for me to use, but here I actually mean it. 12. Trans. follows R. Crawley (1874) with minor changes. 13. Thucydides writing what I view as the first true effort at history as we understand it c. 400 BC, while the term 'social science' isn't coined until 1824 and the first clear field of it not established until the 1830s. It's fair to claim earlier figures (Machiavelli, Hobbes, even Aristotle) as predecessors of the social sciences, but they all understood themselves to be doing philosophy and also used quite different methods, whereas Thucydides understood himself to be doing history and even engages in historical source criticism (e.g. Thuc. 1.20-21). 14. Of course more responsible social scientists are well aware of limits of certainty and even quantifiable uncertainty when dealing with these sorts of complex systems. 15. Which doesn't mean he is wrong, merely that the evidence prior to very recently cannot be used to say what he says it does because of how staggeringly incomplete it is. Like this: Like Loading... * Tagged * Academia * Economics * Historical Methods * History * Oh no * Political Science * Social Science [3f4e8a2b40][3f4e8a2b40] Published by Bret Devereaux View all posts by Bret Devereaux Published August 29, 2022 Post navigation Previous Post Collections: Why No Roman Industrial Revolution? 23 thoughts on "New Acquisitions: On the Wisdom of Noah Smith" 1. [a5159][a5159] Dirichlet-to-Neumann says: August 29, 2022 at 4:04 pm An ACOUP post on Monday ? Thanks a lot Noah Smith ! Loading... Reply 2. [d3a24][d3a24] Nathan says: August 29, 2022 at 4:21 pm This makes me regret ever giving Noah money. Utterly unacceptable behavior from someone who purports to be of the side of truth. Time to put my money to a better scholar, writer, and thinker. Ugh I just feel gross now. Loading... Reply 1. [b4438][b4438] mindstalk0 says: August 29, 2022 at 5:46 pm Our host has a Patreon, if not a tip jar. Loading... Reply 3. [b8851][b8851] NAC says: August 29, 2022 at 4:32 pm Huh. Weird to see a couple different people who I sometimes read the blogs of fighting. This also gets into weird stuff like the philosophy of knowledge. I think the best point you make is about the lack of representative data in the past - you can't exactly do a research survey of dead people. This could be wrong (and, I'll admit I am biased here, I'm a chemistry student), but it seems to me that this is a consequence of emergent properties of larger systems and how complexity makes things difficult to analyze. After all, chemistry is based off of quantum mechanics, but simulating, say, one mole of gas by using that is computationally impossible, so you have to use approximations in many circumstances for things to be useful, and the same goes with biology,. You can't simulate the human brain with current technology, so we have to make approximations (this part of the brain controls this, and from that arises psychology as we can't simulate that, and in large groups you need economics and sociology, etc, etc. We can't purely reduce history to economics, sociology, for the same reason - if we had all the data, infinite computing power, and 100% accurate theories independent of culture, it might be possible. But that's never going to happen, especially as so much data is forever lost, so we have to use modern historical methods to get the best picture we can. Loading... Reply 4. [6c802][6c802] Sniffnoy says: August 29, 2022 at 4:34 pm The result is not a prediction but rather an acknowledgement of possibility; the historian does not offer a precise estimate of probability (in the Bayesian way) because they don't think accurately calculating even that is possible - the 'unknown unknowns' (that is to say, contingent factors) overwhelm any system of assessing probability statistically. An acknowledgement of possibility is a prediction, though. The space of outcomes one could write down regarding the real world is huge, so the probability that would be assigned to any one of them by a truly uninformative prediction is tiny. To say that a particular thing has a high-enough probability that it's worth noting as possible does in fact carry substantially more information than not doing so -- sometimes a lot; frequently most of the work goes into locating what possibilities are worth considering. For instance, if I were to claim that "It's possible that Donald Trump will declare himself the King of Antarctica", people would reasonably say that's a dumb statement. It's "possible" in the sense that it can't be strictly ruled out, but it's not "possible" in the sense that it's likely enough to be worth considering, which is what's meant by the acknowledgements of possibility that you talk about. Such a statement is indeed enough of a prediction that we can reasonably say that it is wrong; no, I haven't asserted that Trump will declare himself King of Antarctica, so it can't be disproved in that way, but the implicit probability I'm putting on it by calling it "possible" is far too high, and if you take action on this prediction -- by taking some cheap gamble that will only pay off if this action does occur, on the theory that hey it probably won't happen but it costs you so little and you'll win big if it does (what that would consist of in this case is unclear) -- you are making a mistake even if it only ends up costing you a hundredth of a penny. And yes obviously I am taking a Bayesian perspective here, but, like, one can consider Bayesian epistemology to be an essentially correct description of an ideal epistemology without like actually attempting to put precise probabilities on everything. Saying "it's possible that X might occur" is a statement about the probability of X even if it turning it into numbers might be a difficult exercise. (And that's still true if you want to use probability intervals rather than point probabilities, etc.) I guess the conflict here is between the weak notion of prediction and the strong level of evidence demanded? Like, to my mind such an "acknowledgement of possibility" is a prediction, but I'm obviously using a weak notion of "prediction" when I say that; I'm not sure you can simultaneously use such a weak notion of prediction while demanding a much stronger level of evidence for it, which is what Smith seems to do? Loading... Reply 1. [33fe8][33fe8] godfreyofboulogne says: August 29, 2022 at 5:03 pm I think that there's an implied shorthand here, where "possible" really means "non-trivially possible." Loading... Reply 2. [b5b08][b5b08] journcy says: August 29, 2022 at 5:04 pm Strong comment. I think the weakest part of Bret's otherwise pretty great response here is that he seems to weakly motte and bailey "historians don't like making predictions/talking about causality" (the motte) with "historians like to passive aggressively insinuate that they know X has happened before and X will happen again" (the bailey). Which (by my reading) seems to be what Matt Yglesias was talking about in the quoted tweet, which I think Bret didn't really respond to (recognize?) properly. (Also, I can't condone the oh-so-polite sniping back at NS, but I *can* say it made me laugh.) Loading... Reply 3. [6c802][6c802] Sniffnoy says: August 29, 2022 at 5:04 pm (Like to be slightly more explicit at the end there, since I left that part a bit short -- if one wants to use a Bayesian notion of "prediction", as I am, then one also has to use a Bayesian notion of "evidence", which is much weaker than the sort of evidence that Smith seems to be asking for, and it seems to me like historians' evidence is pretty adequate for the sorts of predictions they're making?) Loading... Reply 5. [3a573][3a573] Genial Precis says: August 29, 2022 at 4:37 pm I think Smith caught an IMO uncharacteristic bit of twitter poisoning. There are highly visible twitter characters cloaked in the would-be authority of history who make exactly the errors you describe. They are not representative of the careful epistemology of history as a discipline but they are much more visible. That doesn't make his arguments any better, but it makes them perfectly comprehensible at the expense their relationship to history. Or perhaps it isn't twitter's fault. But people making unreasonably confident claims on the alleged basis of historical study is very much not an empty class. You could call them pundits instead of historians but doing so risks the scotsman fallacy. Likewise, what I took from Yglesias' argument, was a claim that whoever he's calling historians are not as limited in their attribution of causation as they claim, and perhaps even think, they are. Loading... Reply 1. [18a56][18a56] NSh says: August 29, 2022 at 5:59 pm But it isn't a no true Scotsman arguement if the person clearly isn't a Scotsman and is only wearing a plaid cap. A pundit citing history is not a historian anymore than I'm a doctor when I pass on good remediation for a cold. Overall I'm confused about what's controversial here. Of course tyrants barring obstacles will seize power again generally speaking. Hasn't anyone worked on a committee of some kind or run into clique groups? So long as the power base exists so to the ambition. Loading... Reply 6. [8472a][8472a] Retief says: August 29, 2022 at 4:37 pm First of all, I think you are focusing too much on "empirical". Reading Noah's article, it's clear that he isn't using the same definition of empirical that you are. Yours might be "right" and his might be "wrong", but faulting him for an argument he isn't actually making is a bit weak. Beyond that, I think Noah's arguments that historians can cherry pick examples to support any position they wish has some validity. Like, I'm sure that if you tried, you could find an example (or several examples) of greek would-be tyrants who tried once and then willingly retired. And while an article that said "some greek would-be tyrants did just try once, so it's possible that trump would retire now" would be technically correct, that would also presumably be somewhat misleading in practice. You can add in as many qualifiers as you like, but readers will tend to read that as "trump won't try again" even if that isn't what you intended to say. Of course, this sort of criticism is hardly unique to history. "Lies, damned lies, and statistics" is a phrase for a reason, after all. A statistical approach to the question of "how often do tyrants try again" would be interesting, but the actual results would likely vary wildly depending on how you define "tyrant" and "try". So yeah, the historical approach has its share of flaws, but I'm not trying to argue that the statistical approach is necessarily better overall. Loading... Reply 1. [3f4e8][3f4e8] Bret Devereaux says: August 29, 2022 at 5:15 pm If a social scientist is using the word 'empirical' incorrectly, something has gone very wrong indeed. That's an important word to understand in any discipline that understands itself as a science! Loading... Reply 7. [33fe8][33fe8] godfreyofboulogne says: August 29, 2022 at 5:01 pm What strikes me most forcefully in this is the sheer *arrogance* of social scientists. Perhaps we would be much better off if they had never co-opted the monicker "science" for their disciplines. Loading... Reply 8. [ce699][ce699] Alexander Rose says: August 29, 2022 at 5:01 pm Kick his ass Bret Loading... Reply 9. [aeebb][aeebb] ksdale says: August 29, 2022 at 5:22 pm I'm grateful that you took the time to write this, I followed a bit on Twitter, but generally wasn't impressed by how either 'side' acquitted themselves, as is usually the case on Twitter. The discussion of different epistemologies was particularly enlightening for me, as well as the pitfalls inherent in reducing things to numerical values. The only thing I struggled with throughout the conversation was what exactly counts as a prediction. When you say "Tyrants in the past have made multiple attempts to seize power, therefore tyrants in the present may as well, therefore some concern over this possibility is warranted," I guess I don't feel like this is *just* an acknowledgement of possibility. A neutral acknowledgement of possibility implies to me that... anything else is equally likely? Just saying that "Would-be tyrants may make multiple attempts to seize power," is an acknowledgement of possibility, but it's also not particularly convincing. It makes sense that history can offer support for this possibility existing, but how does history support it if not by suggesting that it is more likely to occur? And when we offer evidence to suggest that a certain possibility is more or less likely, are we not making predictions? Loading... Reply 1. [b4438][b4438] mindstalk0 says: August 29, 2022 at 5:51 pm I mean, something can be relatively unlikely -- 10% "chance", say -- and yet still be worth worrying about, because of its high negative consequences if it happens. "A would-be tyrant might just give up on trying for power" -- yes, that's a possibility, and one we're not worried about, because he gives up. "A would-be tyrant keeps trying" and "Further would-be tyrants follow the trail of norm violations that he blazed" are things to actually worry about. Loading... Reply 2. [0f503][0f503] Koken says: August 29, 2022 at 6:14 pm This is the kind of thing I took Yglesias to be referring to. Either you are making some kind of actual predictive claim about how likely things are to happen, and need to be willing to be pinned down on that, or you're not saying much at all. If historical parallels don't help provide causal explanations or predictively useful information, what do they bring to discussion of any current event? It feels a bit like purely ornamental erudition. In practice, of course, people usually bring them up in a way that does imply some prediction. Loading... Reply 1. [d1aa8][d1aa8] ECD says: August 29, 2022 at 6:26 pm Speaking only for myself, I find the current trend towards 'you should be able to quantify the degree of chance that this will happen' tends to lend itself ironically to far more false certainty than the available evidence actually supports. To take the tyrant example, at the time it was written there was quite a lot of talk going around about how January 6th wasn't really a coup attempt because it was silly, or Trump backed off rather than push it further. Responding to that is entirely worthwhile. Claiming that you know that there's a sixty percent chance he'll try again and a thirty-five percent chance it will inspire serious follow-up attempts by other parties isn't being more helpful, even though it allegedly indicates uncertainty (this could go either way) the level of precision in the guess ironically gives a false impression of actual knowledge which we don't/ can't have. And I think that's actually the critical point of dispute. To Noah/Matt, those sorts of estimates are a sign of intellectual humility and good reasoning. They indicate what you think is actually likely in a very clear way. To Bret/most historians I saw commenting, those estimates are a sign of wild intellectual arrogance, because they seem to assume you have sufficient facts to make such an estimate. Loading... Reply 1. [0f503][0f503] Koken says: August 29, 2022 at 6:54 pm Then side with the philosophers in the article above - accept that you don't know, and say that. It's this business of saying that history suggests a possibility, but not saying whether you think it is more likely than any other possibility, that seems off. Almost anything is possible; if you're not willing to give some indication (not necessarily numerical) of what you think is likely, what are you saying? Loading... Reply 10. [b4438][b4438] mindstalk0 says: August 29, 2022 at 5:44 pm "Phenomenon A1 occurred before and it resulted in Result B, therefore as Phenomenon A2 occurs now, result B *may* happen. Tyrants in the past have made multiple attempts to seize power, therefore tyrants in the present may as well, therefore some concern over this possibility is warranted. The result is not a prediction but rather an acknowledgement of possibility;" This is *exactly* what I was thinking when I was reading Smith going on about 'predictions'. "Hey, this has happened in situations like this, so be on the lookout." And the only empirical update to that could be after much more history, likely centuries from now, or some influx of data (new way of recovering records, a time machine...) Loading... Reply 11. [24989][24989] Elise Stalker says: August 29, 2022 at 5:47 pm I noticed in the discussions around this that the Smith partisans made a lot of hay of "Marxist Historians", treating them as a clear counterexample to your claim that historians emphasize contingency and avoid endorsing grand narratives. I may be wrong here, being an amateur, but that seems like a rather stale characterization? As I understand it, whatever the initial ambitions of the Marxist historians, they have in fact mostly ended up doing a lot of work to demonstrate contingency in practice, regularly contradicting Marx along the way, and that mostly they have held onto/retreated to an emphasis on economic production and class structure and their relationship. I don't doubt there are hardliners still fighting for full dialectical determinism, but they must be a small minority, in history in particular. As an aside, the recurring phenomenon of outsiders constructing grand historical theories that are overly broad in claimed scope despite being based on a clearly rather narrow and biased pool which at best boil down to a simple and not particularly novel intution must be rather annoying to historians. To take a particular example from this post, it seems to me that when you translate "Elite Overproduction" from Turchinese you are left with the claim that political crises arise when there are more people who feel entitled to power than there is power to go around, and perhaps the corollary that this entitlement is created by the very social and political system it threatens. A fairly banal observation that is nonetheless not even going to be universally applicable. Loading... Reply 12. [8928c][8928c] Constance Sublette says: August 29, 2022 at 5:50 pm Non historians who believe they know a great deal of history get both defensive and confrontational when an historian points out flaws in what they think they know, or even what they know has been superseded by later scholarship, including archaeology. Seeing an increase of this lately online, where non-historians, who are enamored of all things, 'what if?', alternate history and counterfactuals are denouncing historians and history in general. We are in the age where everyone knows they are The Expert, but actual, practicing, credentialed and experienced expertise is belittled. Loading... Reply 13. [df307][df307] David Howard says: August 29, 2022 at 6:23 pm I really appreciate your commitment to and direct endorsement of epistemic humility here, that's one of the things I appreciate most about what you're doing with this blog. There's a stealmanning of Smith's argument as you've presented it (I've not read it, at your un-recomendation) which I've experienced regularly when reading historical arguments directed at popular audiences (i.e. thinkpieces). As a non-expert I have a tool I can use to evaluate empirical model-based arguments: I can watch what happens, and make a rough Bayesian update about the offered model. With non-empirical arguments I cannot do this, and I don't feel like I have a good tool. I can agree it's not reasonable, as Smith demands, that the discipline of history remold itself to fit the tool I have. I do think it is reasonable to ask that history seek to provide non-experts with a reasonably general purpose tool for evaluating arguments made from the results of the historical method. Referring to sources is great, but the volume of historical arguments I come across is such that I can't as a layperson buy and read an introductory review of the evidence for each - if I did, I would be a historian and not a layperson. What do you recommend I do as a non-expert to judge the strength and applicability of historical arguments, if not try to generate and test predictions from them? Loading... Reply Leave a Reply Cancel reply Search for: [ ] [Search] Updates every Friday! You can support this project via Patreon, and by spreading the word! My Tweets Recent Posts * New Acquisitions: On the Wisdom of Noah Smith * Collections: Why No Roman Industrial Revolution? * Collections: This. Isn't. Sparta. Retrospective * Collections: Logistics, How Did They Do It, Part III: On the Move * Fireside Friday, August 5, 2022 * Collections: Logistics, How Did They Do It, Part II: Foraging Questions? Requests? Have a topic you want me to post about? A question on a post? Do you want to scream incoherently into the endless void that is the internet? The best place to find me is on twitter @BretDevereaux Following me on Twitter is the best way to be informed of new posts as they appear. Tags Academia Ancient Antiquity Armor Artillery Castles Cavalry Cities Civil Military Relations Cohesion Command Content Warning Culture Customs Demography Doctrine Dune Early Medieval Early Modern Economic History Economy Empire Farming Field Fortifications Fireside Chat Food Fortifications Game of Thrones Gaming History Hollywood Tactics Horses Household Humanities Imperialism Infantry Iron Logistics Lord of the Rings Markets Medieval Middle Ages Military History Modern Morale Oh no Operations Organic Economy Paradox Political Systems Production Religion Roman Roman Empire Roman Republic Rome Rural Science Fiction Siege Siege Warfare Slavery Social History Sparta States steppe nomads Strategy Tactics Technology Textiles Trade Ukraine Video Games War and Society Warfare Weapons Archives * August 2022 (5) * July 2022 (5) * June 2022 (4) * May 2022 (4) * April 2022 (5) * March 2022 (4) * February 2022 (4) * January 2022 (4) * December 2021 (5) * November 2021 (4) * October 2021 (5) * September 2021 (4) * August 2021 (4) * July 2021 (5) * June 2021 (4) * May 2021 (4) * April 2021 (5) * March 2021 (4) * February 2021 (4) * January 2021 (5) * December 2020 (3) * November 2020 (4) * October 2020 (5) * September 2020 (4) * August 2020 (4) * July 2020 (5) * June 2020 (4) * May 2020 (5) * April 2020 (4) * March 2020 (5) * February 2020 (4) * January 2020 (5) * December 2019 (4) * November 2019 (5) * October 2019 (5) * September 2019 (5) * August 2019 (6) * July 2019 (5) * June 2019 (6) * May 2019 (12) Powered by WordPress.com. Loading Comments... Write a Comment... [ ] Email (Required) [ ] Name (Required) [ ] Website [ ] [Post Comment] %d bloggers like this: