We have now seen how the mind is forced by the very nature of inference itself to make use of induction and hypothesis.

But the question arises how these conclusions come to receive their justification by the event.  Why are most inductions and hypotheses true?  I reply that they are not true.  On the contrary, experience shows that of the most rigid and careful inductions and hypotheses only an infinitesimal proportion are never found to be in any respect false.

And yet it is a fact that all careful inductions are nearly true and all well‑grounded hypotheses resemble the truth;  why is that?  If we put our hand in a bag of beans the sample we take out has perhaps not quite but about the same proportion of the different colours as the whole bag.  Why is that?

The answer is that which I gave a week ago.  Namely, that there is a certain vague tendency for the whole to be like any of its parts taken at random because it is composed of its parts.  And, therefore, there must be some slight preponderance of true over false scientific inferences.  Now the falsity in conclusions is eliminated and neutralized by opposing falsity while the slight tendency to the truth is always one way and is accumulated by experience.  The same principle of balancing of errors holds alike in observation and in reasoning.

(Peirce 1866, pp. 470–471)

A similar line of thought may be gone through in reference to hypothesis.  In this case we must start with the consideration of the term:

spherical, bright, fragrant, juicy, tropical fruit.

Such a term, formed by the sum of the comprehensions of several terms, is called a conjunctive term.  A conjunctive term has no extension adequate to its comprehension.  Thus the only spherical bright fragrant juicy tropical fruit we know is the orange and that has many other characters besides these.  Hence, such a term is of no use whatever.  If it occurs in the predicate and something is said to be a spherical bright fragrant juicy tropical fruit, since there is nothing which is all this which is not an orange, we may say that this is an orange at once.  On the other hand, if the conjunctive term is subject and we know that every spherical bright fragrant juicy tropical fruit necessarily has certain properties, it must be that we know more than that and can simplify the subject.  Thus a conjunctive term may always be replaced by a simple one.

So if we find that light is capable of producing certain phenomena which could only be enumerated by a long conjunction of terms, we may be sure that this compound predicate may be replaced by a simple one.  And if only one simple one is known in which the conjunctive term is contained, this must be provisionally adopted.

(Peirce 1866, p. 470)

Accordingly, if we are engaged in symbolizing and we come to such a proposition as “Neat, swine, sheep, and deer are herbivorous”, we know firstly that the disjunctive term may be replaced by a true symbol.  But suppose we know of no symbol for neat, swine, sheep, and deer except cloven‑hoofed animals.  There is but one objection to substituting this for the disjunctive term;  it is that we should, then, say more than we have observed.  In short, it has a superfluous information.

But we have already seen that this is an objection which must always stand in the way of taking symbols.  If therefore we are to use symbols at all we must use them notwithstanding that.  Now all thinking is a process of symbolization, for the conceptions of the understanding are symbols in the strict sense.  Unless, therefore, we are to give up thinking altogether we must admit the validity of induction.  But even to doubt is to think.  So we cannot give up thinking and the validity of induction must be admitted.

(Peirce 1866, p. 469)

Yet there are combinations of words and combinations of conceptions which are not strictly speaking symbols.  These are of two kinds of which I will give you instances.  We have first cases like:

man and horse and kangaroo and whale,

and secondly, cases like:

spherical bright fragrant juicy tropical fruit.

The first of these terms has no comprehension which is adequate to the limitation of the extension.  In fact, men, horses, kangaroos, and whales have no attributes in common which are not possessed by the entire class of mammals.  For this reason, this disjunctive term, man and horse and kangaroo and whale, is of no use whatever.  For suppose it is the subject of a sentence;  suppose we know that men and horses and kangaroos and whales have some common character.  Since they have no common character which does not belong to the whole class of mammals, it is plain that mammals may be substituted for this term.  Suppose it is the predicate of a sentence, and that we know that something is either a man or a horse or a kangaroo or a whale;  then, the person who has found out this, knows more about this thing than that it is a mammal;  he therefore knows which of these four it is for these four have nothing in common except what belongs to all other mammals.  Hence in this case the particular one may be substituted for the disjunctive term.  A disjunctive term, then, — one which aggregates the extension of several symbols, — may always be replaced by a simple term.

Hence if we find out that neat are herbivorous, swine are herbivorous, sheep are herbivorous, and deer are herbivorous;  we may be sure that there is some class of animals which covers all these, all the members of which are herbivorous.  Now a disjunctive term — such as neat swine sheep and deer, or man, horse, kangaroo, and whale — is not a true symbol.  It does not denote what it does in consequence of its connotation, as a symbol does;  on the contrary, no part of its connotation goes at all to determine what it denotes — it is in that respect a mere accident if it denote anything.  Its sphere is determined by the concurrence of the four members, man, horse, kangaroo, and whale, or neat swine sheep and deer as the case may be.

Now those who are not accustomed to the homologies of the conceptions of men and words, will think it very fanciful if I say that this concurrence of four terms to determine the sphere of a disjunctive term resembles the arbitrary convention by which men agree that a certain sign shall stand for a certain thing.  And yet how is such a convention made?  The men all look upon or think of the thing and each gets a certain conception and then they agree that whatever calls up or becomes an object of that conception in either of them shall be denoted by the sign.

In the one case, then, we have several different words and the disjunctive term denotes whatever is the object of either of them.  In the other case, we have several different conceptions — the conceptions of different men — and the conventional sign stands for whatever is an object of either of them.  It is plain the two cases are essentially the same, and that a disjunctive term is to be regarded as a conventional sign or index.  And we find both agree in having a determinate extension but an inadequate comprehension.

(Peirce 1866, pp. 468–469)

For this purpose, I must call your attention to the differences there are in the manner in which different representations stand for their objects.

In the first place there are likenesses or copies — such as statues, pictures, emblems, hieroglyphics, and the like.  Such representations stand for their objects only so far as they have an actual resemblance to them — that is agree with them in some characters.  The peculiarity of such representations is that they do not determine their objects — they stand for anything more or less;  for they stand for whatever they resemble and they resemble everything more or less.

The second kind of representations are such as are set up by a convention of men or a decree of God.  Such are tallies, proper names, &c.  The peculiarity of these conventional signs is that they represent no character of their objects.

Likenesses denote nothing in particular;  conventional signs connote nothing in particular.

The third and last kind of representations are symbols or general representations.  They connote attributes and so connote them as to determine what they denote.  To this class belong all words and all conceptions.  Most combinations of words are also symbols.  A proposition, an argument, even a whole book may be, and should be, a single symbol.

(Peirce 1866, pp. 467–468)

Let us now return to the information.  The information of a term is the measure of its superfluous comprehension.  That is to say that the proper office of the comprehension is to determine the extension of the term.  For instance, you and I are men because we possess those attributes — having two legs, being rational, &c. — which make up the comprehension of man.  Every addition to the comprehension of a term lessens its extension up to a certain point, after that further additions increase the information instead.

Thus, let us commence with the term colour;  add to the comprehension of this term, that of red.  Red colour has considerably less extension than colour;  add to this the comprehension of dark;  dark red colour has still less [extension].  Add to this the comprehension of non‑blue — non‑blue dark red colour has the same extension as dark red colour, so that the non‑blue here performs a work of supererogation;  it tells us that no dark red colour is blue, but does none of the proper business of connotation, that of diminishing the extension at all.  Thus information measures the superfluous comprehension.  And, hence, whenever we make a symbol to express any thing or any attribute we cannot make it so empty that it shall have no superfluous comprehension.

I am going, next, to show that inference is symbolization and that the puzzle of the validity of scientific inference lies merely in this superfluous comprehension and is therefore entirely removed by a consideration of the laws of information.

(Peirce 1866, p. 467)