Automated Detection and Fingerprinting of Censorship Block Pages

Page length comparison at a 30.19% size-difference threshold achieves a 95.03% true positive rate and 1.371% false positive rate for block page detection, outperforming DOM similarity (95.35% TP, 3.732% FP) on false positive rate and cosine similarity (97.94% TP, 1.938% FP, 74.23% precision) on precision. These metrics were evaluated via ten-fold cross-validation on the ONI dataset of ~500,000 entries from 49 countries spanning 2007–2012.

§4.2, Table 1 evaluation dpimeasurement-platformpacket-injection generic

Five commercial filtering products (FortiGuard, Squid, Netsweeper, Websense, WireFilter) were identified in 7 of 36 block-page clusters via copyright notices in HTML comments, HTTP header strings, or URL path patterns; the remaining 29 clusters contained no identifying markup. WireFilter was first detected in the wild in Saudi Arabia (AS 25019) in 2011, representing a newly deployed filtering product not previously observed in measurements.

§5.2, Table 2 detection dpimeasurement-platform saaegeneric

Within a single country mandate, different ISPs implement censorship with different filtering tools and mechanisms: Thailand's AS 9737 and AS 17552 use structurally distinct block-page templates (vector 17 is ~1,000 bytes using div layout; vector 8 is ~6,000 bytes using table layout). Both ISPs actively obfuscate their filtering product by reporting generic 'Server: Apache/2.2.9 (Debian)' or 'Server: Apache' HTTP headers instead of the actual product identifier.

§6 detection dpimiddlebox-interferencemeasurement-platform generic

Applying automated block-page detection to the ONI dataset (49 countries, 2007–2012) reveals that Burma's (AS 18399) censorship mechanism shifted from DNS redirection to a transparent proxy returning a custom block page in mid-2009, then block pages largely disappeared after Burma's late-2011 political liberalization. Saudi Arabia (AS 25019) shows a similar transition with WireFilter replacing an unidentified prior tool in 2011, with two concurrent block-page templates suggesting multiple simultaneous filtering devices.

§6 evaluation dns-poisoningmeasurement-platformpacket-injection mmsa

Term frequency clustering of block pages achieves an F-1 measure of 0.98, correctly recovering manually identified block-page templates; page-length clustering performs far worse at F-1 of 0.64. Across the full ONI dataset, only 37 distinct term frequency vectors were found from five years of measurements, indicating that filtering vendors rarely change block-page HTML structure.

§5.1, §5.2 evaluation measurement-platformml-classifier generic