Maxwell’s Demon: A Paradox in Physics?

As a rule, the universe tends towards disorder. It can seem like a rather depressing fact to some, but no matter how concerted and deliberate you try to be, physics guarantees that your actions will always act to increase the overall amount of disorder in the world. Want to have a spoon of sugar in your tea? You’ve just ruined your sweetener’s fine crystal structure by letting it dissolve. Take it without sugar? In boiling the kettle you’ve already set the water molecules in your drink into ever faster and disordered motion just by heating them up. There’s no stopping it. This universal law is codified physically in the second law of thermodynamics, which dictates that after carrying out any irreversible process (irreversible in the sense that you cannot stir the sugar out of your tea), entropy, a measure of disorder, must necessarily have increased. 

Beyond the depression, at first this principle can seem somewhat illusive. Why does Nature decide things must be messied? The answer lies in probability. Take again the example of our cup of tea and sugar. Each sugar molecule, given the chance, can move relatively freely through the tea. They’ll bump into a water molecule here or there, another sugar molecule,  or potentially a caffeine molecule (should you not take decaf), but on average, over time, they get around the entire cup. If you consider the probability of different arrangements of the sugar molecules, you can see that an unmixing of a spoon of sugar is incredibly unlikely. For this to happen, we’d need every sugar molecule from all around the cup to conspire to all at once stick back to our spoon, meanwhile enough of the water molecules would have to decide to get out of the way to make room for our spoonful (presuming your sugar was dry to begin with). The odds of this happening are staggeringly small. They’re so astronomically small in fact that in principle we can say it’ll essentially never happen, even if we stood and stared diligently at our cup for a few billion years. The second law of thermodynamics, under this guise, and once we note that generally there’s just a higher chance of things being disordered, is simply a statement that Nature does the most probable. 

Now, in 1867, James Clerk Maxwell, feeling rather devious, proposed a simple thought experiment regarding entropy that went without a complete solution by physicists for some 115 years. What Maxwell tried to do was concoct a system whose entropy would decrease rather than increase over time, thus becoming more ordered and flying in the face of the second law of thermodynamics. To do this he imagined a container split in two. One side of the container, say the right, is completely empty to start, the other is full of gas, and the two are separated by an impenetrable wall with a small tap to let gas flow between the containers. First, we open the tap and let things progress according to the laws of physics. Predictably, the gas spreads out between the two containers, diffusing out from the left portion into the right until everything settles down into equilibrium, the gas pressure now having gone down due to its expansion. This so far is completely ordinary. If we run and check that nothing has broken yet, we’ll be assured that everything is in order. The system has undergone an irreversible process (the gas won’t magically saunter back into its original place on one side of the container), and so it should be more disordered than before, with a higher entropy. This we find to be true, since in having spread out the positions of all the molecules making up the gas are now inherently more random. The second law remains law and we have no problems so far.

Now comes the demon. Maxwell allows for a tiny creature, you can imagine them with horns if you like, to open and close the tap at will. This demon, conspiring to annoy physicists, decides it rather liked things the way they were before. It decides that should it see a particle moving to the left from the right side of the container it will open the tap and let it into the left side, whereas if it sees a particle moving to the right from the left side it’ll refuse to budge the tap. Over time, this leads to a gradual return to our original situation; all the gas is back on the left, whereas there’s nothing but vacuum on the right. Thus our demon, through a slight bit of trickery wiggling a tap open and closed, seems to have restored order and reversed entropy! Even more startling still is that if we had placed a small turbine at the tap that spins as gas flows between the containers, we could have harvested some of the energy from our experiment and, since we’ve returned everything back to its starting position, there’s no stopping us doing that again, and again, and again. We’d get more energy out of the system each time, and a free lunch.

There’s no such thing of course. The solution to the problem of Maxwell’s demon lies not in trying to manufacture a replica in a lab and generate infinite power using little devilish creatures and boxes of gas, rather it lies in trying to prove that either no such demon could exist, or in as much as they could they don’t break the laws of physics. One idea first tried out on the problem was that the demon would necessarily have to actually be able to see the gas molecules in order to know when to open the tap, and to do this it’d have to shoot light at them, using up energy and thus squashing our hopes of a sustainable future being powered by Satan. Equally, this light must reflect off the molecules, jostling them about thus increasing the randomness of their motion. This solution falls through however, since the demon, outwitting us, can always decide to use lower and lower intensity light to sense the particles, and thus use an absolute minimum of energy, eeking out an advantage and still breaking physics (this might mean they’re worse at seeing the particles, but they’ve got time and can miss a few so long as overall gas only ever travels one way through the tap).

The beginnings of a solution to the problem interestingly came from the work of pioneering computer scientist Claude Shannon. In 1948, Shannon showed that the information content of any message could be directly quantified mathematically by what he called information entropy. The more information a message holds, the higher its information entropy. On the face of it Shannon’s notion of information entropy seems quite some distance from what physicists mean when they talk about entropy. What’s the connection between the amount of information stored in a text message and how disordered my cup of tea is? The answer lies in a distinction between the measurements us physicists normally take of systems and measurements of their exact states. Normally, we don’t measure where every single gas molecule is in a system, in fact normally we can’t possibly do this. To store all the position data for just one gram of hydrogen gas at one instant in time would require something like a quadrillion gigabytes of storage. Every bit of data storage on the planet currently amounts to less than 3% of that, and that’s just for a snapshot of the particles at an instant. We’d be in an even worse state trying to measure how the particles move over time. What we normally measure in a lab are things like pressure, temperature, or volume, which all come from the behaviour of large numbers of particles rather than individual ones, and so require much less data to store. Physicists say that what we typically measure is the macrostate of systems, or how they appear on a macroscopic scale, rather than their microstate, or how they appear down to the level of atoms and molecules. In general, many different microstates can correspond to the same macrostate, since they give the same pressure, temperature, etc., just with the particles moved around a bit. Information entropy, when applied to a gas in a box, is a measurement of how much additional information we’d need to figure out the exact microstate our gas is in (where all the molecules are, how fast they’re moving, etc.) once we know what macrostate it’s in. Thus, if we know all the particles are on the left side of the box, we’d need to know less information to figure out exactly where they are, and thus the system has lower entropy than if they were spread out over the entire box.

Information entropy alone doesn’t solve the problem, however. The second step was made by physicist Rolf Landauer who, in 1961, showed that theoretically there is a lower limit to the energy efficiency of any form of computation, in what’s known as Landauer’s principle. What Landauer proved is that if a computer is to perform a calculation, whether that be adding two numbers, running a line of code, or even something as simple as storing or deleting data, there is a quantifiable minimal energy cost in doing so, and a corresponding entropy increase for having gone to the bother. This entropy increase comes from the fact that, in using up some power, whatever circuit we run our computations on must heat up, and thus either it or its surroundings must become more disordered, since this heat causes its molecules to jiggle about in more random motion.

With all of this at hand, it still took another 21 years for physicists to finally come up with a solution to Maxwell’s demon when, in 1982, Charles Bennett finally figured it out. Bennett’s stroke of genius was to reduce the demon down to its core function; an information-processing machine connected to a tap. The tap connection isn’t too important, since we can always in principle minimise its energy consumption and corresponding entropy increase, and so the solution comes directly from considering the information-processing (the problem isn’t in the demon’s body, just whatever they have between the ears). Bennett reasoned that the demon is going to need to store some data, since they need to record both where the particles near it are and how they’re moving, alongside when they have to open and close the tap to let the right ones through. He equally reasoned that the demon doesn’t have infinite storage space, and so will eventually run out of room and have to start overwriting their storage to record when next to open the tap. This overwriting, however, must take at least the theoretical minimum of energy given my Landuer’s principle, and this must correspond to a least corresponding increase in entropy. This entropy increase, when calculated, will always exceed the decrease the demon gets out of playing with the tap, and thus the demon is thwarted and the second law persists.

Maxwell’s demon is one of the many striking examples in physics of the power of a simple thought experiment and particularly how their solutions, sometimes taking years, can draw in results from seemingly unrelated fields of study entirely into a unified whole. Physicists from Galileo to Newton to Einstein have all pondered mock setups like this in their minds, daydreaming about lobbing cannonballs into orbit, jumping in elevators, or poisoning cats, and from these imaginings has sprung some of the greatest developments in the history of science and physics, alongside ever deeper insights into the minutiae of the Universe. The whole process of carrying out thought experiments, or Gedankenexperiment as Einstein liked to call them, runs down to the core of what physicists do; simplify the world to its most fundamental parts, play with them, learn something, then build it back up again.