https://github.com/pauldotknopf/twitter-dump

Dump data from Twitter's public API.
https://github.com/pauldotknopf/twitter-dump

Last synced: 17 days ago
JSON representation

Dump data from Twitter's public API.

• Host: GitHub
• URL: https://github.com/pauldotknopf/twitter-dump
• Owner: pauldotknopf
• Created: 2020-01-01T19:48:54.000Z (almost 5 years ago)
• Default Branch: develop
• Last Pushed: 2023-02-13T01:49:42.000Z (almost 2 years ago)
• Last Synced: 2024-10-02T16:14:03.516Z (about 1 month ago)
• Language: C#
• Size: 106 KB
• Stars: 118
• Watchers: 6
• Forks: 15
• Open Issues: 15
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# About

Extract all tweets for a given query. This uses Twitters private API (via twitter.com) to bypass quotas/limits.

# Installation

```

dotnet tool install --global TwitterDump

```

# General

First, you must setup the authentication.

```

twitter-dump auth

```

After you have followed the instructions and succesfully authenticated, you can now dump a search to a JSON file.

```

twitter-dump search -q "(from:realDonaldTrump)" -o trump.json

```

# Installation in Ubuntu 20.04

To install on Unbuntu 20.04 do:

```

$ wget https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb

$ sudo dpkg -i packages-microsoft-prod.deb

$ sudo apt-get update

$ sudo apt-get install -y apt-transport-https && sudo apt-get update

$ sudo apt-get install dotnet-sdk-3.1

$ export PATH="$PATH:$HOME/.dotnet/tools"

$ dotnet tool install --global TwitterDump

```

The software is now installed. You now need to get a token from a logged in session:

* Using the Chromium browser log in to Twitter and then navigate to: https://twitter.com/search

* Open Chromium developer tools (Ctrl-Shift I)

* Open the Network tab on the developer tools

* Filter requests for "adaptive.json"

* Search for anything in the web page (doesn't matter what)

* Scroll down until a network request for "adapative.json" is made

* Right click the request and click "Copy -> Copy as cURL"

* In a terminal run: perl -pe 's/\\\n//g' | twitter-dump auth

* Paste the contents of your clipboard

* Press Ctrl-D to close the input to 'perl'

Now you can run:

```

$ twitter-dump search -q "(from:GnuParallel)" -o gnuparallel.json

```

# License

MIT