Why getting voting right is hard, Part II: Hand-Counted Paper Ballots

Posted by ekr on 14 Dec 2020

This post originally appeared on the Mozilla Blog

In Part I we looked at desirable properties for voting system. In this post, I want to look at the details of a specific system, hand-counted paper ballots.

Hand-counted paper ballots are probably the simplest voting system in common use (though mostly outside the US). In practice, the process usually looks something like the following:

1. Election officials pre-print paper ballots and distribute them to polling places. Each paper ballot has a list of contests and the choices for each contest, and a box or some other location where the voter can indicate their choice, as shown above.

2. Voters arrive at the polling place, identify themselves to election workers, and are issued a ballot. They mark the section of the ballot corresponding to their choice. They cast their ballots by putting them into a ballot box, which can be as simple as a cardboard box with a hole in the top for the ballots.

3. Once the polls close, the election workers collect all the ballots. If they are to be locally counted, then the process is as below; if they are to be centrally counted, they are transported back to election headquarters for counting.

The counting process varies between jurisdictions, but at a high level the process is simple. The vote counters go through each ballot one at a time and determine which choice it is for. Joseph Lorenzo Hall provides a good description of the procedure for California's statutory 1% tally here:

In practice, the hand-counting method used by counties in California seems very similar. The typical tally team uses four people consisting of two talliers, one caller and one witness:

• The caller speaks aloud the choice on the ballot for the race being tallied (e.g., "Yes...Yes...Yes..." or ``Lincoln...Lincoln...Lincoln...").
• The witness observes each ballot to ensure that the spoken vote corresponded to what was on the ballot and also collates ballots in cross-stacks of ten ballots.
• Each tallier records the tally by crossing out numbers on a tally sheet to keep track of the vote tally.

Talliers announce the tally at each multiple of ten ("10", "20", etc.) so that they can roll-back the tally if the two talliers get out of sync.

Obviously other techniques are possible, but as long as people are able to observe, differences in technique are mostly about efficiency rather than accuracy or transparency. The key requirement here is that any observer can look at the ballots and see that they are being recorded as they are cast. Jurisdictions will usually have some mechanism for challenging the tally of a specific ballot.

Security and Verifiability #

The major virtue of hand-counted paper ballots is that they are simple, with security and privacy properties that are easy for voters to understand and reason about, and for observers to verify for themselves

It's easiest to break the election in two phases:

• Voting and collecting the ballots
• Counting the collected ballots

If each of these is done correctly, then we can have high confidence that the election was correctly decided.

Voting #

The security properties of the voting process mostly come down to ballot handling, namely that:

• Only authorized voters get ballots and only one ballot. Note that it's necessary to ensure this because otherwise it's very hard to prevent multiple voting, where an authorized voter puts in two ballots.
• Only the ballots of authorized voters make it into the ballot box.
• All the ballots in the ballot box and only the ballots from the ballot box make it to election headquarters.

The first two of these properties are readily observed by observers -- whether independent or partisan. The last property typically relies on technical controls. For instance, in Santa Clara county ballots are taken from the ballot box and put into clear tamper-evident bags for transport to election central, which limits the ability for poll workers to replace the ballots. When put together all three properties provide a high degree of confidence that the right ballots are available to be counted. This isn't to say that there's no opportunity for fraud via sleight-of-hand or voter impersonation (more on this later) but it's largely one-at-a-time fraud, affecting a few ballots at a time, and is hard to perpetrate at scale.

Counting #

The counting process is even easier to verify: it's conducted in the open and so observers have their own chance to see each ballot and be confident that it has been counted correctly. Obviously, you need a lot of observers because you need at least one for each counting team, but given that the number of voters far exceeds the number of counting teams, it's not that impractical for a campaign to come up with enough observers.

Probably the biggest source of problems with hand-counted paper ballots is disputes about the meaning of ambiguous ballots. Ideally voters would mark their ballots according to the instructions, but it's quite common for voters to make stray marks, mark more than one box, fill in the boxes with dots instead of Xs, or even some more exotic variations, as shown in the examples below. In each case, it needs to be determined how to handle the ballot. It's common to apply an "Intent of the voter" standard, but this still requires judgement. One extra difficulty here is that at the point where you are interpreting each ballot, you already know what it looks like, so naturally this can lead to a fair amount of partisan bickering about whether to accept each individual ballot, as each side tries to accept ballots that seem like they are for their preferred candidate and disqualify ballots that seem like they are for their opponent.

A related issue is whether a given ballot is valid. This isn't so much an issue with ballots cast at a polling place, but for vote-by-mail ballots there can be questions about signatures on the envelopes, the number of envelopes, etc. I'll get to this later when I cover vote by mail in a later post.

Privacy/Secrecy of the Ballot #

The level of privacy provided by paper ballots depends a fair bit on the precise details of how they are used and handled. In typical elections, voters will be given some level of privacy to fill out their ballot, so they don't have to worry too much about that stage (though presumably in theory someone could set up cameras in the polling place). Aside from that, we primarily need to worry about two classes of attack:

1. Tracking a given voter's ballot from checkin to counting.
2. Determining how a voter voted from the ballot itself.

Ideally -- at least from the perspective of privacy -- the ballots are all identical and the ballot box is big enough that you get some level of shuffling (how much is an open question), then it's quite hard to correlate the ballot a voter was given to when it's counted, though you might be able to narrow it down some by looking at which polling place/box the ballot came in and where it was in the box. In some jurisdictions, ballots have serial numbers, which might make this kind of tracking easier, though only if records of which voter gets which ballot are kept and available. Apparently the UK has this kind of system but tightly controls the records.

It's generally not possible to tell from a ballot itself which voter it belongs to unless the voter cooperates by making the ballot distinctive in some way. This might happen because the voter is being paid (or threatened) to cast their vote a certain way. While some election jurisdictions prohibit distinguishing marks, as a practical matter it's not really possible to prevent voters from making such marks if they really want to. This is especially true when the ballots need not be machine readable and so the voter has the ability to fill in the box somewhat distinctively (there are a lot of ways to write an X!). In elections with a lot of contests, as with many places on the US, it is also possible to use what's called a "pattern voting" attack in which you vote one contest the way you are told and then vote the downballot contests in a way that uniquely identifies you. This sort of attack is very hard to prevent, but actually checking that people voted they way they were told is of course a lot of work. There are also more exotic attacks such as fingerprinting paper stock, but none of these are easy to mount in bulk.

Accessibility #

One big drawback of hand-marked ballots is that they are not very accessible, both to people with disabilities and to non-native speakers. For obvious reasons, if you're blind or have limited dexterity it can be hard to fill in the boxes (this is even harder with optical scan type ballots). Many jurisdictions that use paper ballots will also have some accommodation for people with disabilities. Paper ballots work fine in most languages, but each language must be separately translated and then printed, and then you need to have extras of each ballot type in case more people come than you expect, so at the end of the day the logistics can get quite complicated. By contrast, electronic voting machines (which I'll get to later) scale much better to multiple languages.

Scalability #

Although hand-counting does a good job of producing accurate and verifiable counts, it does not scale very well^[1]. Estimates of how expensive it is to count ballots vary quite a bit, but a 2010 Pew study of hand recounts in Washington and Minnesota (the 2004 Washington gubernatorial and 2008 Minnesota US Senate races) put the cost of recounting a single contest at between $0.15 and $0.60 per ballot. Of course, as noted above some of the cost here is that of disputing ambiguous ballots. If the races is not particularly competitive then these ballots can be set aside and only need to be carefully adjudicated if they have a chance of changing the result.

Importantly, the cost of hand-counting goes up with the number of ballots times the number of contests on the ballot. In the United States it's not uncommon to have 20 or more contests per election. For example, here is a sample ballot from the 2020 general election in Santa Clara County, CA. This ballot has the following contests

In an election like this, the cost to count could be several dollars per ballot. Of course, California has an exceptionally large number of contests, but in general hand-counting represents a significant cost.

Aside from the financial impact of hand counting ballots, it just takes a long time. Pew notes that both the Washington and Minnesota recounts took around seven months to resolve, though again this is partly due to the small margin of victory. As another example, California law requires a "1% post-election manual tally" in which 1% of precincts are randomly selected for hand-counting. Even with such a restricted count, the tally can take weeks in a large county such as Los Angeles, suggesting that hand counting all the ballots would be prohibitive in this setting. This isn't to say that hand counting can never work, obviously, merely that it's not a good match for the US electoral system, which tends to have a lot more contests than in other countries.

Up Next: Optical Scanning #

The bottom line here is that while hand counting works well in many jurisdictions it's not a great fit for a lot of elections in the United States. So if we can't count ballots by hand, then what can we do? The good news is that there are ballot counting mechanisms which can provide similar assurance and privacy properties to hand counting but do so much more efficiently, namely optical scan ballots. I'll be covering that in my next post.