https://github.com/gwen001/dataextractor

A Burp Suite extension to extract datas from source code while browsing.
https://github.com/gwen001/dataextractor

bugbounty burpsuite pentesting private python secrets security-tools

Last synced: about 2 months ago
JSON representation

A Burp Suite extension to extract datas from source code while browsing.

• Host: GitHub
• URL: https://github.com/gwen001/dataextractor
• Owner: gwen001
• License: mit
• Created: 2021-10-21T09:27:00.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2024-03-20T03:30:13.000Z (11 months ago)
• Last Synced: 2024-12-11T20:10:55.208Z (about 2 months ago)
• Topics: bugbounty, burpsuite, pentesting, private, python, secrets, security-tools
• Language: Python
• Homepage:
• Size: 571 KB
• Stars: 153
• Watchers: 6
• Forks: 33
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • Funding: .github/FUNDING.yml
  • License: LICENSE.md

Awesome Lists containing this project

README

        
DataExtractor


A Burp Suite extension to extract data from source code while browsing.




    

    

    



---

## Features

- in scope parsing  

- file extensions to ignore  

- files exclusion based on regexp  

- multi tabs for multiple purpose  

- data extraction based on regexp  

- results exclusion based on regexp  

- data export  

## Install

First of all, ensure you have [Jython](https://www.jython.org/) installed and loaded and setup before the next step.

- clone this repository  

- in the Extender tab, click the button `Add`  

- set `Extension type` to `Python`  

- browse to the cloned folder and select `DataExtractor.py` as the `Extension file`  

The extension should load and is now ready to perform a passive scan.

## Help

- A single click on any `Apply changes` button will save all your settings

- Settings / Follow scope rules:

do not parse out of scope urls as defined in the target scope tab

- Settings / Remove duplicates:

remove duplicates from data tabs

- Settings / Ignore extensions:

do not parse urls with those extensions

- Settings / Ignore files:

do not parse those files (regexps allowed), JSON format: `["jquery.min.js",".png",...]`  

important: regexps here are case insentitive by design

- Custom tab / Config:

list of regexps to search, JSON format: `{"key1":"regexp1","?key2":"regexp2",...}`  

if the first character of the key is a `?` or a `*`, the key will not be printed in the data tab  

important: regexps here are NOT case sentitive, use `(?i)` as a prefix of the whole regexp to make it insentitive  

important: you should have at least 1 group configured using parenthesis `()` to be able to catch something,

`group(1)` is used as a result so to ignore a group, please use `?:` as a prefix of the group itself

- Custom tab / Remove from results:

remove those results from data tab (regexps allowed), JSON format: `["http://$","application/javacript",...]`  

important: regexps here are case insentitive by design

## Examples of regexp config

All config textareas should be valid JSON format associative arrays (key/value), so take care of every single comma and quote.

As soon as you save your settings, a check is performed so you can ensure that everything is fine in the output/errors tab of the exender tab.  

Note that all keys should be different.

Subdomains (case insensitive):

```

{

  "*dummykey1":"(?i)(([0-9a-z_\\-\\.]+)\\.github\\.com)"

}

```

AWS keys ans Slack tokens (case sensitive):

```

{

  "slack token": "(xox[pboa]-[0-9]{10,12}-[0-9]{10,12}(-[0-9]{10,12})?-[a-zA-Z0-9]{24,32})",

  "aws key": "((AKIA|A3T|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{12,})"

}

```

See the file [myregexp](https://github.com/gwen001/DataExtractor/blob/main/myregexp) to get all my regexps.

## Example of ignore/remove

All ignore/remove textareas should be valid JSON format arrays, so take care of every single comma and quote.

As soon as you save your settings, a check is performed so you can ensure that everything is fine in the output/errors tab of the exender tab.  

```

[

  ".png$",

  "application/javascript",

  "googleapis.com",

  "sha256.*$"

]

```

---









---

Feel free to [open an issue](/../../issues/) if you have any problem with the script.