The Dreams That Stuff is Made of

by Stephen Hawking

  But now how can we succeed in bringing the two observers to an understanding? This is a question whose answer is obviously of fundamental significance for the atomic theory. First of all, it is easy to see that the macro-observer reckons only with mean values; for what he calls density, visible velocity and temperature of the gas are, for the micro-observer, certain mean values, statistical data, which are derived from the space distribution and from the velocities of the atoms in an appropriate manner. But the micro-observer cannot operate with these mean values alone, for, if these are given at one instant of time, the progress of the process is not determined throughout; on the contrary: he can easily find with given mean values an enormously large number of individual values for the positions and the velocities of the atoms, all of which correspond with the same mean values and which, in spite of this, lead to quite different processes with regard to the mean values. It follows from this of necessity that the micro-observer must either give up the attempt to understand the unique progress, in accordance with experience, of the macroscopic changes of state—and this would be the end of the atomic theory—or that he, through the introduction of a special physical hypothesis, restrict in a suitable manner the manifold of micro-states considered by him. There is certainly nothing to prevent him from assuming that not all conceivable micro-states are realizable in nature, and that certain of them are in fact thinkable, but never actually realized. In the formularization of such a hypothesis, there is of course no point of departure to be found from the principles of dynamics alone; for pure dynamics leaves this case undetermined. But on just this account any dynamical hypothesis, which involves nothing further than a closer specification of the micro-states realized in nature, is certainly permissible. Which hypothesis is to be given the preference can only be decided through comparison of the results to which the different possible hypotheses lead in the course of experience.

  In order to limit the investigation in this way, we must obviously fix our attention only upon all imaginable configurations and velocities of the individual atoms which are compatible with determinate values of the density, the velocity and the temperature of the gas, or in other words: we must consider all the micro-states which belong to a determinate macro-state, and must investigate the various kinds of processes which follow in accordance with the fixed laws of dynamics from the different micro-states. Now, precise calculation has in every case always led to the important result that an enormously large number of these different micro-processes relate to one and the same macro-process, and that only proportionately few of the same, which are distinguished by quite special exceptional conditions concerning the positions and the velocities of neighboring atoms, furnish exceptions. Furthermore, it has also shown that one of the resulting macro-processes is that which the macroscopic observer recognizes, so that it is compatible with the second law of thermodynamics.

  Here, manifestly, the bridge of understanding is supplied. The micro-observer needs only to assimilate in his theory the physical hypothesis that all those special cases in which special exceptional conditions exist among the neighboring configurations of interacting atoms do not occur in nature, or, in other words, that the micro-states are in elementary disorder. Then the uniqueness of the macroscopic process is assured and with it, also, the fulfillment of the principle of increase of entropy in all directions.

  Therefore, it is not the atomic distribution, but rather the hypothesis of elementary disorder, which forms the real kernel of the principle of increase of entropy and, therefore, the preliminary condition for the existence of entropy. Without elementary disorder there is neither entropy nor irreversible process.ad Therefore, a single atom can never possess an entropy; for we cannot speak of disorder in connection with it. But with a fairly large number of atoms, say 100 or 1,000, the matter is quite different. Here, one can certainly speak of a disorder, in case that the values of the coordinates and the velocity components are distributed among the atoms in accordance with the laws of accident. Then it is possible to calculate the probability for a given state. But how is it with regard to the increase of entropy? May we assert that the motion of 100 atoms is irreversible? Certainly not; but this is only because the state of 100 atoms cannot be defined in a thermodynamic sense, since the process does not proceed in a unique manner from the standpoint of a macro-observer, and this requirement forms, as we have seen above, the foundation and preliminary condition for the definition of a thermodynamic state.

  If one therefore asks: How many atoms are at least necessary in order that a process may be considered irreversible?, the answer is: so many atoms that one may form from them definite mean values which define the state in a macroscopic sense. One must reflect that to secure the validity of the principle of increase of entropy there must be added to the condition of elementary disorder still another, namely, that the number of the elements under consideration be sufficiently large to render possible the formation of definite mean values. The second law has a meaning for these mean values only; but for them, it is quite exact, just as exact as the law of the calculus of probability, that the mean value, so far as it may be defined, of a sufficiently large number of throws with a six-sided die, is 3.

  These considerations are, at the same time, capable of throwing light upon questions such as the following: Does the principle of increase of entropy possess a meaning for the so-called Brownian molecular movement of a suspended particle? Does the kinetic energy of this motion represent useful work or not? The entropy principle is just as little valid for a single suspended particle as for an atom, and therefore is not valid for a few of them, but only when there is so large a number that definite mean values can be formed. That one is able to see the particles and not the atoms makes no material difference; because the progress of a process does not depend upon the power of an observing instrument. The question with regard to useful work plays no rôle in this connection; strictly speaking, this possesses, in general, no objective physical meaning. For it does not admit of an answer without reference to the scheme of the physicist or technician who proposes to make use of the work in question. The second law, therefore, has fundamentally nothing to do with the idea of useful work (cf. first lecture, p. 15).

  But, if the entropy principle is to hold, a further assumption is necessary, concerning the various disordered elements,—an assumption which tacitly is commonly made and which we have not previously definitely expressed. It is, however, not less important than those referred to above. The elements must actually be of the same kind, or they must at least form a number of groups of like kind, e.g., constitute a mixture in which each kind of element occurs in large numbers. For only through the similarity of the elements does it come about that order and law can result in the larger from the smaller. If the molecules of a gas be all different from one another, the properties of a gas can never show so simple a law-abiding behavior as that which is indicated by thermodynamics. In fact, the calculation of the probability of a state presupposes that all complexions which correspond to the state are a priori equally likely. Without this condition one is just as little able to calculate the probability of a given state as, for instance, the probability of a given throw with dice whose sides are unequal in size. In summing up we may therefore say: the second law of thermodynamics in its objective physical conception, freed from anthropomorphism, relates to certain mean values which are formed from a large number of disordered elements of the same kind.

  The validity of the principle of increase of entropy and of the irreversible progress of thermodynamic processes in nature is completely assured in this formularization. After the introduction of the hypothesis of elementary disorder, the microscopic observer can no longer confidently assert that each process considered by him in a collection of atoms is reversible; for the motion occurring in the reverse order will not always obey the requirements of that hypothesis. In fact, the motions of single atoms are always reversible, and thus far one may say, as before, that the irreversible proc
esses appear reduced to a reversible process, but the phenomenon as a whole is nevertheless irreversible, because upon reversal the disorder of the numerous individual elementary processes would be eliminated. Irreversibility is inherent, not in the individual elementary processes themselves, but solely in their irregular constitution. It is this only which guarantees the unique change of the macroscopic mean values.

  Thus, for example, the reverse progress of a frictional process is impossible, in that it would presuppose elementary arrangement of interacting neighboring molecules. For the collisions between any two molecules must thereby possess a certain distinguishing character, in that the velocities of two colliding molecules depend in a definite way upon the place at which they meet. In this way only can it happen that in collisions like directed velocities ensue and, therefore, visible motion.

  Previously we have only referred to the principle of elementary disorder in its application to the atomic theory of matter. But it may also be assumed as valid, as I wish to indicate at this point, on quite the same grounds as those holding in the case of matter, for the theory of radiant heat. Let us consider, e.g., two bodies at different temperatures between which exchange of heat occurs through radiation. We can in this case also imagine a microscopic observer, as opposed to the ordinary macroscopic observer, who possesses insight into all the particulars of electromagnetic processes which are connected with emission and absorption, and the propagation of heat rays. The microscopic observer would declare the whole process reversible because all electrodynamic processes can also take place in the reverse direction, and the contradiction may here be referred back to a difference in definition of the state of a heat ray. Thus, while the macroscopic observer completely defines a monochromatic ray through direction, state of polarization, color, and intensity, the microscopic observer, in order to possess a complete knowledge of an electromagnetic state, necessarily requires the specification of all the numerous irregular variations of amplitude and phase to which the most homogeneous heat ray is actually subject. That such irregular variations actually exist follows immediately from the well known fact that two rays of the same color never interfere, except when they originate in the same source of light. But until these fluctuations are given in all particulars, the micro-observer can say nothing with regard to the progress of the process. He is also unable to specify whether the exchange of heat radiation between the two bodies leads to a decrease or to an increase of their difference in temperature. The principle of elementary disorder first furnishes the adequate criterion of the tendency of the radiation process, i.e., the warming of the colder body at the expense of the warmer, just as the same principle conditions the irreversibility of exchange of heat through conduction. However, in the two cases compared, there is indicated an essential difference in the kind of the disorder. While in heat conduction the disordered elements may be represented as associated with the various molecules, in heat radiation there are the numerous vibration periods, connected with a heat ray, among which the energy of radiation is irregularly distributed. In other words: the disorder among the molecules is a material one, while in heat radiation it is one of energy distribution. This is the most important difference between the two kinds of disorder; a common feature exists as regards the great number of uncoordinated elements required. Just as the entropy of a body is defined as a function of the macroscopic state, only when the body contains so many atoms that from them definite mean values may be formed, so the entropy principle only possesses a meaning with regard to a heat ray when the ray comprehends so many periodic vibrations, i.e., persists for so long a time, that a definite mean value for the intensity of the ray may be obtained from the successive irregular fluctuating amplitudes.

  Now, after the principle of elementary disorder has been introduced and accepted by us as valid throughout nature, the fundamental question arises as to the calculation of the probability of a given state, and the actual derivation of the entropy therefrom. From the entropy all the laws of thermodynamic states of equilibrium, for material substances, and also for energy radiation, may be uniquely derived. With regard to the connection between entropy and probability, this is inferred very simply from the law that the probability of two independent configurations is represented by the product of the individual probabilities:W = W1 · W2,

  while the entropy S is represented by the sum of the individual entropies:S = S1 + S2.

  Accordingly, the entropy is proportional to the logarithm of the probability:

  (1)

  k is a universal constant. In particular, it is the same for atomic as for radiation configurations, for there is nothing to prevent us assuming that the configuration designated by 1 is atomic, while that designated by 2 is a radiation configuration. If k has been calculated, say with the aid of radiation measurements, then k must have the same value for atomic processes. Later we shall follow this procedure, in order to utilize the laws of heat radiation in the kinetic theory of gases. Now, there remains, as the last and most difficult part of the problem, the calculation of the probability W of a given physical configuration in a given macroscopic state. We shall treat today, by way of preparation for the quite general problem to follow, the simple problem: to specify the probability of a given state for a single moving material point, subject to given conservative forces. Since the state depends upon 6 variables: the 3 generalized coordinates , , and the three corresponding velocity components , , , and since all possible values of these 6 variables constitute a continuous manifold, the probability sought is, that these 6 quantities shall lie respectively within certain infinitely small intervals, or, if one thinks of these 6 quantities as the rectilinear orthogonal coordinates of a point in an ideal six-dimensional space, that this ideal “state point” shall fall within a given, infinitely small “state domain.” Since the domain is infinitely small, the probability will be proportional to the magnitude of the domain and therefore proportional to

  But this expression cannot serve as an absolute measure of the probability, because in general it changes in magnitude with the time, if each state point moves in accordance with the laws of motion of material points, while the probability of a state which follows of necessity from another must be the same for the one as the other. Now, as is well known, another integral quite similarly formed, may be specified in place of the one above, which possesses the special property of not changing in value with the time. It is only necessary to employ, in addition to the general coordinates , , the three so-called momenta ψ1, ψ2, ψ3, in place of the three velocities , , , as the determining coordinates of the state. These are defined in the following way:

  wherein H denotes the kinetic potential (Helmholz). Then, in Hamiltonian form, the equations of motion are:

  (E is the energy), and from these equations follows the “condition of incompressibility”:

  Referring to the six-dimensional space represented by the coordinates ϕ1, ϕ2, ϕ3, ψ 1, ψ 2, ψ 3, this equation states that the magnitude of an arbitrarily chosen state domain, viz.:

  does not change with the time, when each point of the domain changes its position in accordance with the laws of motion of material points. Accordingly, it is made possible to take the magnitude of this domain as a direct measure for the probability that the state point falls within the domain.

  From the last expression, which can be easily generalized for the case of an arbitrary number of variables, we shall calculate later the probability of a thermodynamic state, for the case of radiant energy as well as that for material substances.

  Chapter Two

  The question of whether matter is infinitely divisible has troubled philosophers for millennia. In approximately 450 BCE, the Greek philosopher Democritus speculated that there must be some smallest unit of matter from which every material thing is built. He named this the atom, which means “indivisible” in Greek. However, by the end of the nineteenth century, it was known that what we now call atoms are, in fact, divisible. This is not to say that Dem
ocritus was wrong, for we have good reasons to believe that elementary particles like electrons are fundamental and indivisible. It simply means what we call an atom was misnamed.

  In the latter part of the nineteenth century, atoms were known to be composed of positively charged protons and negatively charged electrons, but it was not known how the protons and electrons were structured in the atom. From 1909 through 1911, Ernest Rutherford and his assistant Hans Geiger conducted experiments to explore this question. The results of their study, are presented in the groundbreaking paper, “The Scattering of α and β Particles by Matter and the Structure of the Atom.” They bombarded gold foil with α particles in the hope that by watching how the α particles interacted with the atoms in the foil they could determine the structure of the gold atoms. α Particles have a strong positive electrical charge, so they are ideally suited to studying where the positive and negative charges reside inside an atom. The experiment seems straightforward, but what Rutherford and Geiger discovered was totally unexpected. They found that all the protons were very tightly clumped together at the center of the atom, in what we now call the nucleus. The electrons were found surrounding the nucleus. Rutherford speculated that the electrons orbit the nucleus in much the same way that planets orbit the sun. Consequently, Rutherford’s model of the atom was called the planetary model. The planetary model was completely unexpected because it seems to violate well-known physical laws. For example, we know that electrical charges of the same sign repel each other. All protons are positively charged, so the protons in the nucleus should strongly repel each other and blow the nucleus apart—but they don’t! What could be holding it together? Rutherford did not know. He simply postulated that there must be some force holding it together. This force would not be well understood until the 1970s with the advent of quantum chromodynamics.