https://github.com/sentdm/twilio-message-export
Export the last 30 days of Twilio SMS/MMS message records to JSONL using the Bulk Export API
https://github.com/sentdm/twilio-message-export
bulk-export cli data-export messaging migration mms python sms twilio
Last synced: 8 days ago
JSON representation
Export the last 30 days of Twilio SMS/MMS message records to JSONL using the Bulk Export API
- Host: GitHub
- URL: https://github.com/sentdm/twilio-message-export
- Owner: sentdm
- License: mit
- Created: 2025-12-30T14:10:29.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2025-12-30T14:10:31.000Z (5 months ago)
- Last Synced: 2026-05-25T07:08:03.290Z (18 days ago)
- Topics: bulk-export, cli, data-export, messaging, migration, mms, python, sms, twilio
- Language: Python
- Homepage: https://www.sent.dm
- Size: 5.86 KB
- Stars: 4
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Twilio Message Export
Export the last 30 days of SMS/MMS message records from your Twilio account using the [Bulk Export API](https://www.twilio.com/docs/usage/bulkexport).
## Quick Start
```bash
# Clone the repo
git clone https://github.com/sent-dm/twilio-message-export.git
cd twilio-message-export
# Install dependencies
pip install -r requirements.txt
# Set your Twilio credentials
export TWILIO_ACCOUNT_SID="ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
export TWILIO_AUTH_TOKEN="your_auth_token"
# Run the export
python twilio_message_export.py
```
## Requirements
- Python 3.7+
- Twilio Account SID and Auth Token ([find them here](https://console.twilio.com/))
## What It Does
1. Creates a Bulk Export job for the last 30 days of messages
2. Polls Twilio until the export completes (typically 5-30 minutes)
3. Downloads each day's data and combines into a single file
4. Outputs a `.jsonl` file with one message per line
## Output Format
The script produces a [JSON Lines](https://jsonlines.org/) file:
```
messages_export_2024-11-28_to_2024-12-28.jsonl
```
Each line is a JSON object containing all fields from the [Twilio Bulk Export API](https://www.twilio.com/docs/usage/bulkexport):
```json
{
"sid": "SMxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"account_sid": "ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"messaging_service_sid": "MGxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"date_created": "2024-12-15T14:32:01Z",
"date_sent": "2024-12-15T14:32:02Z",
"date_updated": "2024-12-15T14:32:05Z",
"from": "+15551234567",
"to": "+15559876543",
"body": "Your message content here",
"status": "delivered",
"direction": "outbound-api",
"num_segments": 1,
"num_media": 0,
"price": "-0.00750",
"error_code": null
}
```
### Available Fields
| Field | Description |
|-------|-------------|
| `sid` | Unique message identifier (SM...) |
| `account_sid` | Your Twilio Account SID (AC...) |
| `messaging_service_sid` | Messaging Service SID if used (MG...) |
| `date_created` | When the message was created (UTC, ISO 8601) |
| `date_sent` | When the message was sent (UTC, ISO 8601) |
| `date_updated` | When the message was last updated (UTC, ISO 8601) |
| `from` | Sender phone number |
| `to` | Recipient phone number |
| `body` | Message content |
| `status` | Delivery status: `queued`, `sending`, `sent`, `delivered`, `undelivered`, `failed` |
| `direction` | `inbound`, `outbound-api`, `outbound-call`, `outbound-reply` |
| `num_segments` | Number of SMS segments |
| `num_media` | Number of MMS media attachments |
| `price` | Cost of the message (may not be present on all records) |
| `error_code` | Error code if failed (null if successful) |
> **Note**: Bulk Export excludes `price_units`, `api_version`, `error_message`, `uri`, and `subresource_uris` fields. The `price` field may not be present on all records.
## Configuration
| Environment Variable | Required | Description |
|---------------------|----------|-------------|
| `TWILIO_ACCOUNT_SID` | Yes | Your Twilio Account SID (starts with `AC`) |
| `TWILIO_AUTH_TOKEN` | Yes | Your Twilio Auth Token |
## Notes
- **Date Range**: Exports cover 30 days ending 2 days ago (Twilio API requirement)
- **Processing Time**: Large accounts may take 10-30 minutes
- **Rate Limits**: Maximum 366 days of exports per UTC day
- **File Retention**: Twilio deletes export files after 7 days
## Troubleshooting
**"Missing Twilio credentials"**
- Ensure both `TWILIO_ACCOUNT_SID` and `TWILIO_AUTH_TOKEN` are set
- Check for typos or extra whitespace
**"Authentication failed"**
- Verify credentials at [console.twilio.com](https://console.twilio.com/)
- Ensure you're using the main Account SID, not a subaccount
**"Rate limited"**
- You've exceeded 366 days of exports in a UTC day
- Wait until midnight UTC and try again
**Export job stuck**
- The script will timeout after 2 hours
- Check job status at [console.twilio.com](https://console.twilio.com/)
## License
MIT