Nuclear weapons have always prevented invasions. Then Ukrainian troops poured into Russia.

For decades, the theory of nuclear escalation assumed that countries with nuclear weapons were largely immune to attack because an attacker risked triggering apocalypse. Relatively small states such as Israel, Iran, North Korea and Libya sought nuclear weapons, among other things, to deter attacks by larger, better-armed opponents.

There are clashes between the nuclear powers: India has been involved in border skirmishes with China and Pakistan. Palestinian Hamas militias stormed Israel in October, which is widely believed to have nuclear weapons. But in general, the threat of annihilation protects the nuclear powers from large-scale attacks and ensures peace between them.

Ukraine is not a nuclear power and is inferior in weaponry to Russia, yet Kyiv has managed to control an area of ​​almost 1,200 square kilometers for more than three weeks. This is a stunning turn of events. Strategists have often imagined over the years that NATO countries would conquer Russian territory in a fight, rather than a beleaguered outsider.

Now Western politicians, military experts and nuclear theorists are puzzling over what current events mean for the prospects of Russian escalation – and for future war maneuvers. The theoretical risk is being tested in the real world, requiring a reassessment of the role of nuclear weapons in deterrence.

Russia's published nuclear doctrine states that Moscow would only resort to nuclear weapons if the country's sovereignty or territorial integrity were threatened. Although Ukraine occupies part of Russian soil, neither side appears to view the Kursk region as strategically important. So the Ukrainian attack – embarrassing as it may be for the Kremlin – does not indicate that it crosses a Russian red line.

But ambiguity and uncertainty are an essential part of nuclear tactics.

“Nobody really knows the Russian red line – they never specified it precisely,” said Nikolai Sokov, a former Soviet and Russian arms control negotiator. “Maybe we will find out later that we crossed the red line two months ago,” said Sokov, who is currently briefing Western military leaders on Russian strategic thinking.

One wildcard that Sokov notes is that the Kremlin and President Vladimir Putin seem to view threats to their regime as threats to Russia's sovereignty. Viewed from that perspective, significant Ukrainian successes or Russian losses could trigger a nuclear escalation — although it would likely begin with increased use of nonnuclear weapons rather than a sudden attack, Sokov said.

Fear of crossing Russian red lines has shaped President Biden's attitude toward the war. Not wanting it to come into direct conflict with NATO, he was reluctant to give Ukraine weapons it had asked for, including tanks, advanced missiles and fighter jets. Kyiv eventually got most of them, leading Ukrainians and their Western backers to argue that Putin's red lines were flexible.

With the attack on Kursk, Ukraine wants to show that another taboo can be broken without dire consequences. Part of the goal is to convince the White House to allow Ukraine to use more lethal and precise US weapons to attack Russia.

Many politicians in the West, especially in Washington and Berlin, remain more cautious because Putin is so unpredictable.

The uncertainty about where Russia's red lines lie is “the fundamental challenge of strategic ambiguity,” says Janice Gross Stein, a professor of conflict management at the University of Toronto. Testing boundaries and signaling through threats or promises of restraint is, as she calls it, a “contest between a strategy of manipulating uncertainty and a strategy of reducing uncertainty.”

Playing out events triggered by Ukraine's push toward Kursk is reminiscent of the Cold War, when escalation theory was a widely studied field. When the Soviets developed an atomic bomb in 1949, four years after the United States, Western strategists tried to imagine how these fearsome weapons might be used in combat.

Their enormous destructive power – and the far greater danger posed by thermonuclear weapons since 1952 – gave rise to a new field of research: nuclear deterrence thinking and analytical tools.

Game theory, developed in the 1920s by mathematician John von Neumann, became popular as a method for assessing nuclear risk appetite. To deal with these weighty unknowns, the RAND Corporation, founded as a Pentagon think tank, used theoretical constructs such as the so-called prisoner's dilemma – a situation in which two parties who cannot communicate must separately decide whether to cooperate for mutual benefit – to play out how the United States and the Soviet Union might act and react in conflict scenarios.

In 1960, theorist and economist Thomas Schelling summarized this analysis in an influential collection of academic articles entitled The Strategy of Conflict, presenting an intellectual approach to evaluating deterrence and escalation.

The atomic theory attracted brilliant thinkers. Schelling won the Nobel Prize in Economic Sciences. The escalation theory was advanced by the work of Nobel Prize-winning economist John Nash, who is featured in the film A Beautiful Mind.

Nevertheless, developing a nuclear strategy and determining an opponent's red lines remains a risky game.

“It's like we're walking toward a cliff in the dark,” said Christopher Chivvis, who has studied nuclear risk issues at RAND and as a U.S. intelligence official. “It's out there somewhere. We just don't know where it is.”

Putin has threatened to use nuclear weapons more or less directly on several occasions since launching his large-scale invasion of Ukraine in 2022. But even a decade ago, Russia's occupation of Ukraine's Crimean peninsula marked a new level of post-Cold War aggression that prompted escalation theorists to pull out their textbooks.

About six years ago, political scientist James Davis, who held a chair in international relations at the University of St. Gallen in Switzerland, recognized that nuclear decision-making had declined as an academic subject but was regaining importance. He began teaching courses on the subject, including one entitled “Fundamentals of Arms Control.”

Now he is revising some long-held assumptions.

“We have always believed that nuclear weapons can only serve as a deterrent,” he said. “We really did not believe that a non-nuclear power would attack a nuclear power.”

Translating the uncertainties of escalation theory outside the classroom, namely to the general public, can be even more puzzling. Experts point out that 75 years of successful deterrence does not mean that the approach can always prevent nuclear war.

“It's a difficult task for people who work on this issue full-time to get messages across,” says Chivvis, who is now a senior fellow at the Carnegie Endowment for International Peace. “You don't want to keep warning about it, but you also don't want to say it's not going to happen.”