Four populations with nothing to separate

Two methods now exist here for answering the same question about a crease pattern: does it have a mountain-valley lettering whose arcs close no circle?

The old one samples. Draw letterings that pass every condition at every vertex, test each one for a circle, report the share. The new one searches: test the arcs while choosing the letters instead of after choosing them all, and stop at the first lettering that survives.

They disagree spectacularly on one pattern, where two thousand draws find nothing and the search finds one in five hundred and sixty-one steps. The obvious next question is how often they disagree in general, and this collection keeps four standing populations of crease patterns precisely so that questions of that shape have somewhere to be asked.

The answer is that they never disagree on any of them.

What the four populations are #

They were assembled for an earlier question about what a typical instance looks like, and the assembly is deliberate rather than convenient.

The printed shelf — eight patterns a reader can fold. It is somebody’s list, in the sense that these are patterns chosen because they are worth folding, and that is its whole value: it is the population closest to a reader’s own paper.

Twist tessellations — one construction run over four tilings at three turn angles, assembled unit by unit on the sheet rather than clipped. Twelve patterns from one machine, which is what a family generated by a rule looks like.

Quadrilateral meshes — six developable meshes with no two vertices alike, each satisfying every condition at every interior vertex. The population with the least regularity in it.

Fold-and-cut outlines — seven shapes, each with the crease pattern that folds them onto a single line. The population with essentially no circuits: a straight skeleton is a tree, so these patterns cannot argue with themselves at all.

Together they are meant to span what this subject produces: the deliberate, the generated, the irregular, and the degenerate. They are the same four the collection uses whenever a claim has to be shown to hold somewhere other than the pattern it was noticed on, and the five ways of refusing a crease pattern were ranked over exactly these.

Every member, twice #

Each of the twenty-eight was put through both methods. Forty letterings drawn at random from the admissible set, tested for a circle; then a search with the circle test inside the choice.

Every member is given a consistent lettering by both. Not one of the twenty-eight fails to produce one in forty draws, so there is no member on which the sampler returns a nought and the search could correct it. And the search finds one on every member in at most sixty nodes, which is fewer than most of them have panels — meaning it walked straight to an answer without ever taking a letter back.

So the populations are silent on the question. Anyone choosing between the two methods on this evidence would conclude that it does not matter which is used, and would then meet the rhombille patch.

Why the populations cannot show it #

The reason is structural rather than a matter of the populations being too small, and it is worth stating carefully because it applies to every test set this collection has.

Three of the four are made of things a construction produces. A twist tessellation, a quadrilateral mesh and a fold-and-cut pattern are all outputs of a procedure that ends in a propagation, and a propagation returns whichever consistent lettering it reached first. So every member arrives already carrying a lettering that works. A pattern that is hard to letter would have been a pattern the construction failed on, and a construction that failed would not have contributed a member.

The fourth is a list of patterns worth folding, which is a stronger filter still. A pattern whose letters are hard to find is a pattern nobody prints.

So all four populations are, in the precise sense, conditioned on the question being easy. That is not a defect in how they were assembled; it is what assembling a population means. A set of instances gathered from the things a field actually produces is a set of instances the field’s procedures can handle.

The one place they nearly separate #

There is a single member of the four populations where the two methods could plausibly have parted company, and looking at why they do not is the most informative row in the table.

The quadrilateral meshes are the least regular patterns here: six meshes with no two vertices alike, generated by solving a closure condition rather than by laying down a repeating unit. Four of the six have no arrangement of their nine panels at all — every ordering breaks one of the non-crossing rules — which makes them the population’s hardest instances by any measure this collection has.

They are also the cheapest to search. Six nodes at worst, on a pattern with nine panels and twelve creases, which is a search that decided every crease the propagation left open and never revised one.

The reason is that a mesh’s difficulty is not in its letters. Its letterings are consistent — the arcs close no circle on any of them — and the refusal comes from the two non-crossing rules, which the search never asks about and the sampler never asks about either. Both methods answer the same easy question and agree, and the hard question is somewhere neither is looking.

That is worth carrying, because it is the general shape of an agreement between two methods. They agree when they are answering the same question, and whether that question is the interesting one is a separate matter that no amount of agreement addresses.

Where the separating instances came from #

The five clipped tessellation patches are not in the twist population, and the difference is one option.

The population’s members are assembled unit by unit on the sheet: whole twist polygons with their pleats, laid down until the sheet runs out, with anything that would not fit left off. The clipped patches are generated over the plane and then cut to the square, so the rim carries whatever the cut produced — half-polygons, truncated pleats, and in one case twelve fragments a micrometre long.

Clipping is the more honest depiction of a tessellation — a real sheet has an edge and the pattern does not stop politely at it — and it was introduced for exactly that reason. The side effect is that the clipped patches are the only patterns here that were not produced by a procedure guaranteeing its own success.

That is the whole of it. The instances that separate two methods are the ones nothing certified.

How a population would have to be built to show it #

It is worth asking what would have had to be different, because the answer is not more members.

Adding forty more twist tessellations to the twist population would add forty more patterns generated by the same construction, every one of which arrives with a working lettering. The population would triple in size and the table would not move by a row. Size is not the axis.

Nor is variety, in the sense the populations already have it. The four span deliberate, generated, irregular and degenerate patterns, and the two methods agree on all four kinds. What is missing is a different axis entirely: instances that were generated and then damaged — clipped at an awkward place, or drawn at a parameter the construction was not designed for.

The five clipped patches occupy that axis by accident, and there are five of them, which is not a population. What they show is that the axis exists and that it is where the difference lives.

What forty draws is, as a test #

One number in the table deserves defending, since the whole comparison rests on it: forty draws per member rather than four hundred or four thousand.

Forty is enough to distinguish every draw works from most draws work from hardly any draw works, which is the resolution the comparison needs. A pattern whose consistent share is above about a twentieth will produce at least one clean lettering in forty draws nine times in ten; a pattern below a hundredth will usually produce none. So a nought at forty draws is a signal that a pattern’s share is low, and there are no noughts.

It is not enough to distinguish a share of one in four hundred from a share of nothing, which is exactly the distinction the triangular patch turns on. That is the limitation, and it does not bite here because no member of any population is anywhere near that regime. Every one of the twenty-eight is consistent in a substantial fraction of its draws.

So the forty is doing what a screening test should: it is cheap, it would catch a member in the interesting regime, and it caught none because none is there.

The detection curve, exactly #

The defence of forty draws can be made precise, and the precision is worth having because the same number is quoted throughout this collection.

A pattern whose consistent share is $p$ produces no clean lettering in forty draws with probability $(1-p)^{40}$. Setting that to a half gives $p = 1.7$ per cent; setting it to a tenth gives $p = 5.6$ per cent.

So forty draws is a test whose half-detection point is a share of one in sixty and whose ninety-per-cent point is one in eighteen. Above one in eighteen a nought is a surprise; below one in sixty a nought is the expected result and carries almost no information.

Every member of the four populations sits far above the upper figure, which is why no nought appears and why the screening test is doing its job rather than being lucky.

Which says the famous nought was not bad luck #

The same curve reads the case the comparison turns on, and it reads it differently from the essay above.

The triangular patch’s share is five in two thousand, which is one in four hundred. At that share, forty draws return nothing 90 per cent of the time, and two hundred draws return nothing 61 per cent of the time.

So the nought at two hundred draws was not a sampling accident. It was the more likely of the two outcomes, and a sampler asked that question would report nothing on that patch more often than it reported anything. Calling it an accident makes the sampler sound unlucky when it was behaving exactly as a sampler at that sample size must.

The rest of the arithmetic agrees. To find at least one clean lettering nine times in ten at a share of one in four hundred takes about 920 draws, and the run that found five took two thousand — where the expected count at that share is five exactly.

The sampler was never wrong; it was under-powered, and the power was calculable in advance. That is a different and more actionable failure than an accident, because a required sample size can be worked out from the smallest share worth detecting, and an accident cannot be planned around.

It also sharpens what the search bought. The search settled the same patch in 561 steps — fewer than the 920 draws the sampler would have needed for a fair chance, and each step is cheaper than a draw. The two methods do not merely differ in kind here; on this instance the categorical one is also the cheaper one.

What a population can and cannot license #

Three kinds of negative result are available about a pattern, and they are not interchangeable.

None found in forty draws. This licenses very little, and the collection now has the case that shows how little: the triangular patch gives none in two hundred draws and five in two thousand, so a nought at the smaller sample was pure sampling accident. A share of one in four hundred is missed by two hundred draws more often than not.

None among all enumerated. This is a proof, and it is available only where the lettering space can be listed — twelve creases or so, which covers three of the printed patterns and none of the patches.

None found by an exhausted search. This is also a proof and it scales, because a search that runs out of options has established that the options are gone rather than that it did not look hard enough. It is what makes the new method different in kind rather than merely faster.

None of the four populations exercises the third. Every member is settled by a witness, so the exhaustion path is never taken, and a check that never takes a path is not checking it.

The population this suggests building #

What is missing is a population of instances that were not produced by a procedure that guarantees an answer, and the clipped patches are an accident rather than a design.

Building one deliberately is not difficult to describe. Take a tiling, take a turn angle, take a clip position, and generate the patch — including the positions where the clip lands awkwardly, which are exactly the ones the unit-by-unit assembly avoids. Sweep the three parameters and keep everything the construction places, rather than everything it places nicely.

That population would contain instances where the sampler fails and the search succeeds, instances where both succeed at very different costs, and — if any exist — instances where the search exhausts and proves absence. It would be the first test set here whose members were not selected for tractability.

It is not built here, and the reason is worth recording rather than glossing: generating it is cheap and characterising it is not. Every measurement in this collection about a population reports what fraction of it does something, and a population whose members take between eighty and twenty thousand nodes each is one where that fraction costs hours rather than seconds to establish.

The cost of the missing population, priced #

It is worth pricing the thing that was not done, because not built is a claim that should come with a number.

A parameter sweep over four tilings, six turn angles and four clip offsets is ninety-six patches. Generating them is a second each. Sampling two hundred letterings from each is about half a second on the small ones and five on the large. Searching each once is milliseconds.

Characterising them is where the cost sits. To say anything about the distribution of search cost — which is the quantity that would make the population worth having, since the cost on a hard patch is a distribution rather than a number — each member needs a hundred or more runs with a budget large enough to be informative. On the one hard patch here that is about forty seconds; on a patch twice the size it could be many minutes, and nothing predicts which members will be which until they have been run.

Ninety-six members at an unknown cost each, with the unknown being exactly the quantity of interest, is a measurement that has to be designed rather than launched. That is the honest reason it is deferred, and it is a better reason than not having got round to it.

What this changes about the collection’s habits #

One habit, and it is a small correction rather than a reversal.

Every claim here of the form this holds across all four populations remains exactly as strong as it was. The populations are real, the measurements over them are real, and the claims are about what those twenty-eight patterns do.

What such a claim cannot do is establish that a method is adequate, because the populations are made of instances the methods already handle. The same thing was found earlier from the other direction: the layer refusal fired on none of the thirty-three patterns in the populations, and the zero was not a test with nothing to do but a test whose subjects had all been built by procedures that cannot produce the failure.

Twice now, the populations have failed to exhibit something real, and both times for the same reason. That is enough to make it a rule rather than an anecdote: a population assembled from what a field produces cannot show what a field’s procedures miss. For that, the instances have to be built to be awkward, and nothing here has been.