The Coyote Has (Almost) Landed – ‘The Fenchurch Problem’
Here we will examine and interpret in detail the Level 1 primitive ontology and epistemological stance of the de Broglie-Bohm theory. We might disagree a bit with the conventional view…
At the end of Part III, we found a Level 1 ontological stance that seemed, for all the world, to be solid ‘ground’ upon which the Coyote could alight from his untenable position “with both feet planted firmly in mid-air” (Michael 1985) – namely, the de Broglie-Bohm (dBB) pilot-wave theory. In direct analogy with the classical Newtonian case, dBB theory also posits particles as a fundamental entity—or, what researchers in this field call the ‘primitive ontology’ (cf. Allori 2022)—albeit ‘guided’ by the ‘pilot wave’ described by the Schrödinger equation. Therein lies dBB’s attempt at a resolution of the apparent wave-particle duality of quantum physics.
In Part IV we examined the experimental results flowing from Bell’s Theorem as a way to see what types of quantum theory survive the empirical filter. Recall that experimental tests of Bell’s inequality demonstrate that Nature acts pantopically (i.e., ‘non-locally’ in the old money) at the quantum level, and reveal that quantum entities are subject to instantaneous action-at-a-distance through implexity (i.e., ‘entanglement’). I would think that this is a resolution of the EPR thought experiment (it’s not really a ‘paradox’ which is often how it is characterised) which neither Einstein nor Bohr would have liked. In terms of a primitive ontology of particles, dBB was the sole survivor, because of its inherently pantopic character (through the quantum potential $V_Q$), although Stochastic Mechanics was in the running for a good while, until other considerations ruled it out.
In Part V we examined a comprehensive nomenclature schema designed to avoid the cognitive and linguistic gymnastics necessary to avoid saying something idiotic-sounding, like ‘the wavelength of a particle’ or ‘the non-locality of a particle’. This misnaming of quantum things using words with a classical meaning was likened to the misnaming of a platypus as a ‘duckmole’, or of a cylinder as having ‘circle-rectangle duality’. As you can see, I have now begun to use this new nomenclature so as to acclimate readers to it, and to try to demonstrate how not saying stupid-sounding things can actually help support one’s thinking process. In Part VIII, I will use the key concepts from Part V to attempt to derive Quantics (i.e., ‘quantum mechanics’) from the action functional of the Lagrangian approach to Mechanics (i.e., ‘classical mechanics’). This actually falls out so quickly that I am still gobsmacked at how it happened, just by changing the language and taking that change seriously, both conceptually and especially mathematically. I am still trying to pick holes in it to ensure that it is not a case of ‘motivated reasoning’ leading one astray, so stayed tuned and we will soon find out whether or not this is a chimera …
For now, let us see just how close the Coyote came to landing, by way of the dBB theory. Very close, it turns out, but not quite all the way…
OK, But What is ‘Waving’?
There are different schools of thought even within the community of physicists who use and champion the dBB theory. As you might imagine, this depends on assumptions about the ontological status of the particles, and of the ‘pilot wave’ that ‘guides’ the particles in the dynamics of the theory.
The answers to these questions reveal a spectrum of positions. In some cases, the stance treats one as fundamental and the other as secondary. I do not pretend this sketch is anywhere near comprehensive; I just want to highlight the main positions physicists and philosophers take on this. My own view, as it has emerged during the course of this exploration, resonates with some of them, but not, I think, to the degree that it is fair to say that it is merely another version of them.
So, if we think about the particles first, the two main views you can take are that the particles are fundamental, or that the particles are derivative. The more widely-held view has it that the ‘primitive ontology’—the really fundamental thing—is particles, and that everything else is secondary, including the pilot-wave. But there is also a view that assumes the particles are ‘effective’ manifestations of an underlying field, i.e., that the ‘particles’ are ‘excitations’ of the pilot-wave field. Here the field is primary and the particles are secondary. This is actually how conventional quantum field theory treats them.
The other side of this question is the ontological status of the wavefunction, the $\psi$-ontology, and it gets a bit more complicated, with at least several positions as to what it represents. These range from the early view that $\psi$ was a real field in 3D space (which was de Broglie’s and Schrödinger’s original hope), or now more usually that it is a collection of correlated fields in 3D space; to the idea that it is a real physical field in the higher-dimensional configuration space; to the nomological view, which holds that $\psi$ is a law-like regularity in nature. To these we can further add the informational view that holds that $\psi$ encodes information about the system; to the wavefunction realism view that holds that $\psi$ is the ‘real’ reality and that 3D space is simply ‘emergent’ from that; and finally to the structural realism view that holds that $\psi$ represents structural relations between the entities, but is not an entity in itself.Each of these positions has been criticised on various grounds. I won’t go into these in detail, because there is any amount of commentary on this among physicists and philosophers of physics which can be consulted to get the full picture. For now, it is enough to have made this sketch to set the scene for the next section.
Norsen’s analogy, revisited
In Part I, we reported on Travis Norsen’s scenario of a hypothetical planet whose physicists have discovered the Hamilton-Jacobi equation but without any knowledge or inkling of its Newtonian ontological basis, i.e., they do not know about Newton’s laws of motion and how these describe particle interactions (Norsen 2022). For them, what we call Hamilton’s Principal Function $S$ is a mysterious ‘orchestrating field’ which somehow ‘guides’ Newtonian particle trajectories in 3D space even though it lives in an abstract, many-higher-dimensional configuration space, not in the 3D space that both we and the particles live in. For the case of 1 particle, however, it turns out the dynamics do—or can be modelled to—take place in normal 3D space. But for more than one particle, this is no longer true, and they must instead resort to using the abstract higher-dimensional configuration space to do their calculations.
The Hamilton-Jacobi (HJ) equation is a wave-like equation for the (Norsen’s wonderful term) “orchestrating field”, $S$:
$$
\frac{\partial S}{\partial t} = – H\left(q_i, \frac{\partial S}{\partial q_i}, t\right) .
$$
The function $S=S(q_i, t)$ depends on the ‘generalised coordinates’ $q_i, i=1\ldots 3N$, and time $t$. It is defined as a function over the whole configuration space of generally up to $3N$-dimensions (sometimes fewer if there are constraints, for example), where $N$ is the number of particles (each particle has 3 coordinates in 3D space), and $H$ is the Hamiltonian, which represents the total energy of the system (most usually ‘kinetic’ $T$ + ‘potential’ $V$). The derivatives $\frac{\partial S}{\partial q_i}$ of $S$ with respect to the generalised coordinates $q_i$ give rise to conjugate generalised momenta $p_i$, which is usually how the kinetic energy is represented. Surfaces of constant $S$ are normal (i.e., perpendicular) to the particle trajectories, and propagate through configuration space in a manner somewhat akin to shock-wave fronts. Thus, the gradient/derivative of $S$ at a point gives the direction of motion (via the momentum) of the trajectory of the $i$th particle, $p_i=\nabla_i S$. This is the origin of the kinetic energy term $\frac{(\nabla S)^2}{2m}$ in the HJ equation in Part III. The function $S$ defines what mathematicians call a congruence—a space-filling set of non-intersecting curves; a point on each one represents a possible configuration of all the particles in real space, as well as their future (and indeed past) configurations. By specifying the initial condition at some specific time, say $t_0$, the future dynamics of all the particles can be calculated.
It is a very powerful way to work out complicated many-particle dynamics (and other forms too, but let’s just stick to particles for simplicity of the discussion). Now, because on this planet we know how it was developed through multiple levels of mathematical abstraction (see Part I), the fact that the orchestrating field $S$ lives in an abstract, higher-dimensional space is not an issue that causes anyone any sleepless nights. We recognise that configuration space is merely a mathematical tool for efficiently encoding the particles’ interactions with each other in real space, derived from an understanding of how those interactions take place at the basic physical (i.e., Newtonian) level. So, configuration space is merely an encoding space, and $S$ is not a real field living in a real space, but an encoding function living in the encoding space. This is not mysterious—it is just mathematics. What it does do is allow us to describe very complicated particle motions in a very compact and considerably less complicated way than by using individual Newton’s equations for all $N$ particles (which is possible in principle but no-one would ever want to do it, especially because there is already such a better way).
However, the physicists on Norsen’s hypothetical planet are not aware of this underpinning background, and presumably spend many sleepless nights lying awake wondering, maybe even worrying, and perhaps many more hours in the bar at physics conferences arguing about the ontological status of the mysterious orchestrating field. Some say it should be treated as a real field in real space. Some say that the higher-dimensional configuration space is the true reality and that the 3D space of their direct experience is only an illusion. Others say it is just an encoding function for some as-yet-unknown law of the interactions that occur among particles. Yet others claim it merely represents our knowledge of the system of particles; and yet still others suggest other things. Each position has its proponents and detractors, and the philosophical debates continue on with no end in sight (another round, please!)
On our planet, Schrödinger used the classical HJ equation to infer the form of the quantum mechanical equation, building on the quasi-wave-like-ish character that the function $S$ is noted to possess. By postulating that the action field $S$ represented the phase of the quantum ‘wave’, $\psi\sim e^{iS/\hbar}$, he was able to derive the equation that now bears his name. Its success for the 1-particle case (in 3D) was immediate and dramatic (deriving the energy levels of the hydrogen atom from first principles). But very soon there was concern about the ontological status of this strange wave-function that lives in an abstract higher-dimensional configuration space that somehow ‘guides’ the quantum entities that live with us here in real 3D space… Wait! That sounds familiar…!
The point that Norsen made—which was such a gobsmacker for its sensibleness compared the usual sort of stuff that you’re likely to read regarding quantum physics—is that the Schrödinger equation might—just might, mind you!—be a kind of indirect and abstract representation of quantum entities, just like the HJ equation is for classical particles. And this makes total sense, given that the Schrödinger equation uses the principal object in HJ theory—Hamilton’s Principal Function, after all—as the major element of any putative wave-like representation, namely the phase of that wave. The simplest way to represent a wave is with the complex imaginary $i$ as part of the exponent of the exponential function $e$, together with a function that will represent the phase—the ‘reduced’ Planck’s constant $\hbar \equiv h/2\pi$ is present in this case to make the term $S/\hbar$ a suitably dimensionless number, as is required. Therefore, the assumption that the action $S$ is the phase of a wave almost literally requires the $e^{i S/\hbar}$ form for the wavefunction, together with some sort of function to represent its amplitude (which we have not written here, for the sake of pulling out the key point).
If we take this idea seriously as a working hypothesis, then discussions about $\psi$-ontology—the ontological status of the wavefunction—are entirely missing the mark. It is not a real wave waving around in real (3D) space, nor is it even a real wave waving around in configuration space. Rather, it is a function obeying a wave-like equation, which somehow encodes the interactions among quantons (i.e., quantum entities), represented abstractly in an encoding space, namely the higher-dimensional configuration space. It is no more nor less mysterious than Hamilton’s action function $S$. We don’t consider $S$ to be some weird and spooky orchestrating field in (classical) Mechanics; nor should we consider the function $\psi$ to be such a thing in Quantics. And the reason we don’t think so in Mechanics is because we know where it came from and what the underlying ontology is, namely Newtonian particle mechanics. This is because we can map these particles with coordinate systems in 3D that can be transformed into the ‘generalised’ coordinates in the abstract configuration space.
What differs in Quantics, though, is that we have no idea what the underlying ontology is for the quantum side of the ladder of abstraction introduced in Parts I-III. In this sense, we are in a position exactly like the physicists on Norsen’s hypothetical planet (which is of course why he tells the story)—we have discovered a strange and mysterious orchestrating field obeying a (wave-like) function that somehow guides the motions of (quantum) entities in real 3D space, which ‘lives’ in an abstract higher-dimensional configuration space, but without a clue as to how whatever is being orchestrated or ‘guided’ is being orchestrated or guided. It is a powerful mathematical system whose actual referents we are pretty-much in the dark about except for their quite unusual observed behaviour compared with macroscopic objects.
And the important thing to note is that this is not just a quantum phenomenon, either. Even in Mechanics where we know the underlying ontology (i.e., particles), given a specific configuration space representation of some 3D dynamical system, it is impossible to reconstruct the real-world 3D dynamics in the absence of knowing the exact ‘forward transformation’ of the coordinates. Mathematically, this is because the mapping of the set of all possible 3D physical systems to the set of abstract representations in configuration space is surjective but not injective. That is, it is a many-to-one map: many 3D dynamical systems can map onto the same configuration space representation. Therefore, if you do not have the context of how the generalised coordinates in configuration space were defined from those in 3D space, then the information of how to uniquely map the results of calculations from configuration space back into the original system in real 3D space is absent. But if you do have the forward map from 3D to configuration space, then the mapping is indeed invertible, and you can always easily swap back and forth between them. And this is exactly how and why Mechanics works. We always have the forward map from 3D coordinates to configuration space generalised coordinates, by construction, so it is always easy to swap back and forth.
Now, in Quantics we have the representation in configuration space, but no idea how to map that back into real 3D space because we do not have the forward map from 3D to configuration space. Admitting this as our (perhaps lamentable) position might not seem that useful, but in contrast to the Copenhagen interpretation which is—as noted by many authors, incoherent as a system of trying to understand quantum physics (e.g., ‘collapse’ of the wavefunction, shifty split / Heisenberg cut, hide-and-seek moon)—this view simply ’fesses up and admits to ignorance about what is going on (i.e., we know quantum entities do interact pantopically, we know $\psi$ somehow encodes these interactions, but we just don’t how, yet). Incoherence is a permanent structural flaw that requires tearing down and rebuilding. Ignorance, however, is a potentially temporary state of affairs that allows for the future possibility of discovery and resolution. Given the choice, I would always choose uncertain transient ignorance over certain permanent incoherence any time, because it also allows us the possibility of trialling different underlying ontologies to see whether they give rise to predictions that are consistent with experiment.
So, from this perspective, the bottom line is: that’s where we find ourselves in Quantics. We have a function $\psi$ in configuration space that somehow encodes the unknown interactions of the fundamental quantum entities (whatever they are), but no idea how. I see this position as having resonances with several of the views given above: the nomological (there are clearly some regularities going on in how the entities interact); informational (because how these regularities describe the interactions of the entities is somehow encoded in the function); and probably to some degree structural (the function seems to reflect some deeper structure, especially given the experimental fact of pantopy). But, unfortunately, we don’t have the forward map, only the configuration space representation, and with no idea what the representation actually represents. This means there is a temptation, in the absence of any actual knowledge of what is represented, to reify the representation itself. While this is completely understandable, it is also quite unhelpful. The map is not the territory; but having never been to or seen the territory, all we have is the map, which we might come to mistakenly regard as the reality.
Now, all of the above is a natural consequence of the deceptively simple idea that $\psi$ has the same status in Quantics as $S$ has in Mechanics. Therefore, any criticism that takes aim at aspects of this interpretation of $\psi$ had better be ready to apply those same criticisms to $S$. For example, the arguments against dBB that criticise it for using configuration space would also need to similarly criticise HJ theory for doing the same (which no ones does, because that would considered utterly stupid). This is unflinching intellectual honesty, which is often quite uncomfortable, because it means treating opposing ideas scrupulously fairly, applying the same logic to your own as to the opponents’. It is remarkable how this requirement short-circuits a lot of that argumentation. If you dislike something about $\psi$, then be ready to dislike the same thing about $S$. Don’t want to do that for $S$? Then you have no right to do so for $\psi$. Try it and see how many criticisms just appear dumb as a result. Julia Galef (2021) calls this the distinction between fair-minded Scout mindset (“is this true?”) and the biased Soldier mindset (“can I believe this?” for welcome ideas as opposed to “must I believe this?” for unwelcome ones). Scouts are interested in mapping the relevant territory to get as accurate a picture of the reality of the landscape as they possibly can. Soldiers are interested in assailing or defending a position. I see a lot of Soldier mindset going on in the world, and precious little Scout mindset, and not just in quantum physics, either.
In summarised point form, then, here is how I see the situation as it obtains in Quantics:
- $\psi$ indirectly represents something real happening in 3D, but is not itself in 3D (compare $S$ from which $\psi$ was constructed);
- we don’t know what the primitive dynamics are (this is the primary epistemic asymmetry between Mechanics and Quantics);
- the configuration space mapping is faithful, because it makes correct experimental predictions, but is not invertible;
- the inability to invert the mapping is an epistemic gap, tempting us to reify $\psi$ and to promulgate fruitless $\psi$-ontological arguments; and
- this is not just a minor methodological point; rather, it explains pretty much everything about the current interpretation impasse.
In this view, the debate about $\psi$-ontology is misplaced because it is premature. We are arguing about the ontological status of a representation whose referent we are entirely ignorant of. Like the prisoners in Plato’s Cave, we are arguing about whether a shadow is a real object when we have not even identified what is casting it. This is hardly a productive use of our limited time on Earth. The more productive use of time would not be refining our interpretations of $\psi$, but instead searching for suitable 3D primitives that produce predictions which are consistent with the experimental results. The more the merrier, I would think, and the more likely we may then be to potentially hit upon something that might meaningfully advance our knowledge. But, the non-invertability of the map means there could be many distinct primitive 3D dynamics that map to the same $\psi$, so that the search for a/the referent is not just difficult, it may be intrinsically impossible due—in the absence of further clarifying assumptions—to being wholly under-determined. And the trouble is that, even in this form, $\psi$ already gives us excellent practically-useful experimental predictions, so there has been very little pressure to look any further than just ‘shutting up and calculating’ (except for those bloody-minded enough to want to know “what (the bloody hell!) is going on?”).
Now, of course there is already at least one candidate for a type of 3D primitive ontology that fits the bill… Guess which one?
So, does that mean the dBB theory has landed the Coyote? Well, it is very, very close, and perhaps for all practical purposes—as a coherent replacement for the Copenhagen interpretation—maybe it has, and it will probably do, since it will produce all of the same correct numerical predictions. But, if we are looking to fully land the Coyote on solid ground, then I’m afraid I am forced to make the judgement call: ‘Not quite.’
The ‘Fenchurch’ problem
The character of Fenchurch appears in the fourth book of Douglas Adams’ Hitch Hiker’s Guide to the Galaxy trilogy (yes, you read that right), So Long and Thanks for All the Fish. Main character Arthur Dent finally manages to meet up with her for a proper date after a (typically Dentian) series of obstacles and impediments, having earlier returned to an Earth that had somehow been saved from the destruction he witnessed in the first book of the series. The rescuers were the dolphins, who have now all entirely and mysteriously vanished, much to the wonderment and confusion of humankind. The book’s actual title is in fact a translation of the final message the dolphins sent to humanity, which was executed as a complicated set of aquatic acrobatic manoeuvres. Fenchurch tells Arthur that there is something wrong with her, which she challenges him to find during the course of their date. After an afternoon and evening of much guessing and some prompting (Chapter 22), she finally tells him (spoiler alert: look away now!) that the problem is with her feet. And when he bends down to take a closer look at them at ground level he realises: “Yes, I see what’s wrong with your feet. They don’t touch the ground.”
For all of the success of the dBB theory in helping the Coyote come back down towards the ground from impossibly hanging suspended in mid-air, it nonetheless does suffer from what I am here calling the Fenchurch Problem—its feet almost but not quite touch the ground. Let’s examine why I make this claim.
One of the criticisms levelled at dBB theory is the alleged redundancy—or ‘ontological extravagance’—of having two fundamental entities: the particles which are the primitive ontology, and the pilot-wave function $\psi$ which ‘guides’ them. I find this criticism disingenuous at best and hypocritical at worst, because it is never levelled at HJ theory, where the action $S$ is the function that describes the paths of all the Newtonian particles in the system. To me, that is pretty neat. Two entities, and no criticism of that there. But I do disagree with the idea of the pilot-wave ‘guiding’ them since, under the view taken here, $\psi$ is not guiding the particles, so much as merely describing how they move. It no more guides the particles than $S$ does in HJ theory.
But as misplaced a criticism as it is, it does reveal a fundamental structural problem that even the otherwise perfectly serviceable dBB theory possesses: namely, wave-particle duality—that uneasy shotgun marriage between two incommensurable concepts from Mechanics, which Lévy-Leblond so criticised in the previous post. It is a minor, but nonetheless irreducibly present wrinkle. Like Fenchurch, who can otherwise get by in the world quite easily without the strange fact of not actually touching the ground being a glaringly obvious issue to everyone, the problem persists nonetheless. One wonders how it might be resolved. Lévy-Leblond would say to jettison the duality thinking entirely, and to embrace the inherent dual nature of quantons: “not either waves, or particles, but neither waves, nor particles” (Lévy-Leblond 1977, p.185). This means accepting both discreteness in number and continuous extension as part of the same package deal: in other words, quantons need to treated, so to speak, as cylindrical platypuses.
The following posts will seek to do just that, not just conceptually, but also—and this is something which is quite a bit more involved—mathematically.
I mentioned in Part II that I have a fondness for the Lagrangian formalism of Mechanics (which extends to Quantics, too). This is for at least four reasons. First, the Lagrangian formalism seems to be ‘closer’ to the 3D reality than the Hamiltonian or Hamilton-Jacobi formalisms which, powerful as they are, are somewhat further removed from the 3D physics we are trying to understand. This closeness is shown in the ladder of abstraction from earlier posts, where this formalism is at level 2. Second, the Lagrangian formulation is automatically amenable to relativistic extension since the action is itself a relativistic invariant. To someone brought up on relativity theory, this is a big plus, and hints at a much deeper importance in physics. Third, the Lagrangian formulation is also covariant, which means that the equations hold true in any well-defined coordinate system; again a quite desirable property, and again something that resonates from General Relativity (where the field equations are generally covariant). But also fourth: when you really think about it, it is the action that is the fundamental property of physics, both in Mechanics implicitly and in Quantics explicitly. Planck’s constant $h$ has the dimensions of energy-time, which is action. That is, one of the few fundamental constants in nature is the quantum of action. Of course, Feynman was aware of this and managed to create a new third way of doing Quantics based on the Lagrangian formalism, which uses the action as a master concept. And he was prompted along this path by a paper by Dirac (whom we quoted in Part II). Let me repeat that here for convenience (Dirac 1933):
it would seem desirable to take up the question of what corresponds in the quantum theory to the Lagrangian method of the classical theory. A little consideration shows, however, that one cannot expect to be able to take over the classical Lagrangian equations in any very direct way. … We must therefore seek our quantum Lagrangian theory in an indirect way. We must try to take over the ideas of the classical Lagrangian theory, not the equations of the classical Lagrangian theory.
In the next post, we will examine the Lagrangian formulation of Mechanics, as preparation for attempting to derive Quantics from the classical formalism, using just two assumptions, in order to try to at long last land the Coyote, in Part VIII. Furthermore, it appears to be the case that we can take over the equations of the classical theory, although with some slight modification of the technique. Whether this actually works out in the cold hard light of day as it seems to have done so far in my explorations, or not, it promises to be a pretty fun ride! Grab your hat and hang on!
Next: Part VII – The Classical Lagrangian Basis of Quantum Mechanics.