An open API service indexing awesome lists of open source software.

https://github.com/travisbrown/publicsquare

Extract business data from PublicSquare, a right-wing advertising service
https://github.com/travisbrown/publicsquare

Last synced: 6 months ago
JSON representation

Extract business data from PublicSquare, a right-wing advertising service

Awesome Lists containing this project

README

          

# PublicSquare data extraction

[![Rust build status](https://img.shields.io/github/actions/workflow/status/travisbrown/publicsquare/ci.yaml?branch=main)](https://github.com/travisbrown/publicsquare/actions)
[![Coverage status](https://img.shields.io/codecov/c/github/travisbrown/publicsquare/main.svg)](https://codecov.io/github/travisbrown/publicsquare)

This is a small Rust project for extracting business data from [Wayback Machine][wbm] snapshots of "marketplace"
pages from [PublicSquare][publicsquare], a Trump-aligned company offering advertising and payment processing
services for right-wing American businesses.

PublicSquare seems to have removed all of their marketplace pages, possibly because they were being used to boycott
businesses supporting Trump. Many of these pages were archived by the Wayback Machine when they were live, and
this repository provides code for processing those archive pages.

## Overview

If you collect the Wayback Machine snapshots listed in the `metadata/featured-marketplace.json.zst` file and run this
code on them, you will get a line-delimited JSON file with 56,688 business objects. Some are duplicates, but there are
34,568 distinct email addresses and 31,513 distinct phone numbers, so this seems to find a reasonable subset of the
75,000 businesses that [PublicSquare reported][report] "on the platform" (not necessarily marketplace advertisers) in
December 2023.

## Motivation

People should be able to avoid businesses that supported Trump's campaign and administration and other right-wing
causes. It's difficult to navigate the Wayback Machine snapshots for PublicSquare directly, because they are only
indexed by URL, and the marketplace URLs don't include the business name, state, etc. (only opaque identifiers of the
form `e30348e0-1f76-11ef-aaaf-51da97b10f4a`).

There are also many archived pages that don't render in the browser. If you view the source for [this page][example],
for example, you'll see that it's a listing for a BarleyBean coffee shop in Austin, Texas, but the rendered view is blank.

This code is provided for the sake of transparency, so that anyone can verify that the list of 56,688 business
data objects (distributed separately) is extracted from archive snapshots provided by the Wayback Machine.

## Limitations

I've only written parsing code for snapshots archived under the `https://www.publicsquare.com/featured/marketplace/`
URL prefix. There seems to be relevant data saved under other prefixes, but I haven't taken the time to see how
consistent it is, or if it's usefully different from what we get here.

The list of businesses generated by this project is not necessarily complete, anyway, since it only includes businesses
listed on marketplace pages that were archived by the Internet Archive while they were live on PublicSquare's site.

## Other notes

This repository includes metadata about snapshots from the Wayback Archive's CDX index, but does not contain any other
data, either from the Wayback Machine or accessed from PublicSquare directly.

## License

This software is licensed under the [GNU General Public License v3.0][gpl-v3] (GPL-3.0).

[example]: https://web.archive.org/web/20240614165155/https://www.publicsquare.com/featured/marketplace/e30348e0-1f76-11ef-aaaf-51da97b10f4a
[gpl-v3]: https://www.gnu.org/licenses/gpl-3.0.en.html
[publicsquare]: https://www.huffpost.com/entry/publicsquare-trump-critics-boycott-businesses_n_680900d2e4b00850c6839b0b
[report]: https://web.archive.org/web/20251002065418/https://investors.publicsquare.com/news/news-details/2024/PublicSquare-Reports-Year-End-2023-Financial-Results/default.aspx
[wbm]: https://web.archive.org/