Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/defasium/vkcaptchabreaker

VKontakte captcha bypass with pseudoCRNN model running as a chrome extension, Python, JS, 2020
https://github.com/defasium/vkcaptchabreaker

browser-extension captcha captcha-breaking captcha-recognition captcha-solving chrome-extension firefox-addon firefox-webextension javascript keras onnx python vkontakte

Last synced: 4 months ago
JSON representation

VKontakte captcha bypass with pseudoCRNN model running as a chrome extension, Python, JS, 2020

Host: GitHub
URL: https://github.com/defasium/vkcaptchabreaker
Owner: Defasium
Created: 2020-11-01T02:50:25.000Z (about 4 years ago)
Default Branch: main
Last Pushed: 2021-11-12T11:29:08.000Z (about 3 years ago)
Last Synced: 2024-10-09T18:21:34.798Z (4 months ago)
Topics: browser-extension, captcha, captcha-breaking, captcha-recognition, captcha-solving, chrome-extension, firefox-addon, firefox-webextension, javascript, keras, onnx, python, vkontakte
Language: Jupyter Notebook
Homepage:
Size: 14.7 MB
Stars: 71
Watchers: 1
Forks: 7
Open Issues: 7
Metadata Files:
- Readme: README.md

Awesome Lists containing this project

README

        # vkCaptchaBreaker

Use this VKontakte captcha breaker with 91% accuracy right in browser with chrome extension, Python, JS, 2020



      



### HTML Live Demo from above picture: [__🔥DEMO🔥__](https://csb-so9x3.netlify.app/)

### Code  Sandbox for HTML with ONNX.js+ONNXruntimeWeb: [__vk-captcha-breaker__](https://codesandbox.io/s/vk-captcha-breaker-inxnl?file=/index.html)

_Images in Demo or Sandbox are broken? [__Solution__](https://github.com/Defasium/vkCaptchaBreaker/issues/3)_

### Extension can be downloaded here:



  

    

  

    

  

    



### Recognition speed comparison on different devices with different CPU specs (in milliseconds):



  



____

## Table of Contents

  * [Features](#features)

  * [Installation](#installation)

    * [Windows/Linux/MacOS](#windowslinuxmacos)

    * [Android](#android)

  * [Description](#description)

  * [DataMining](#datamining)  

  * [ONNXTroubles](#onnxtroubles)

  * [Results](#results)

  * [Sources](#sources)

____

## Features

Currently supported actions (on mobile domain m.vk.com only):

* messages ✉️ 

* likes ❤️

* comments 💬

* adding to friends 🧑‍🤝‍🧑

Please, note that captcha on login page and promocode page is not supported!

## Installation

# Update v1.6: 30% faster, CORS is no longer a problem 

Update v1.5: recognition in background, Firefox addon 

Update v1.4: added support for other captcha scenarios

Update v1.3: now it works for mobile domain only (m.vk.com)

Google Chrome Extension and Firefox Addon for VKontakte which utilizes trained AI models to predict captcha from image and automatically enters it.

~~Currently only captcha from messages are supported (no likes or comments for you, my bad)~~

You can download latest version from: [__VKCaptchaBreaker.crx__](https://github.com/Defasium/vkCaptchaBreaker/releases)

### Windows/Linux/MacOS

To install the extension, simply follow this tutorial:

* For Chrome: [__www.cnet.com/how-to/how-to-install-chrome-extensions-manually__](https://www.cnet.com/how-to/how-to-install-chrome-extensions-manually/)

* For Firefox: [__www.muo.com/tag/how-to-install-firefox-add-ons-manually-even-from-github__](https://www.muo.com/tag/how-to-install-firefox-add-ons-manually-even-from-github/)

Finally, activate it:



  



### Android

To install the extension, simply follow this tutorial:

[__qwikfone.medium.com/how-to-install-chrome-extension-on-android-3cd89f3a771c__](https://qwikfone.medium.com/how-to-install-chrome-extension-on-android-3cd89f3a771c)

Installation and activation example:



  



If something won't work, just reactivate extension.

____

## Description



  



Keras implementation of Convolutional Recurrent Neural Network with [`ONNX.js`](https://github.com/microsoft/onnxjs) support.

To achieve better accuracy, [`CRNN`](python/crnn.py) utilizes CTC loss and Knowledge distillation with Label smoothing.

CRNN comes in two versions: the one BIG with recurrent layers (RNNs)

and the other small one without them.

The reason for doing so is that if we want to use trained models on end devices, such as

Mobile devices/Client-side browsers, we want to get a model with a minimum number of parameters.

For the BIG version, Google's Inception-like CNN encoder is used with Bidirectional GRUs as RNN decoder.

For the small version, the ShuffleNetV2 encoder is used with 1d convs instead of RNN cells.

____

## DataMining

To parse captcha you can use any URL that Vkontakte provided, e.g. https://vk.com/captcha.php?sid=625944628258&s=1

Here `625944628258` is a random seed.

It's important to mention that some seeds will generate old captchas with Cyrillic symbols.

Using URLs with different seeds over 1.5 million images was parsed.

At first 5000 captcha's was manually labelled (that process take approximately 1 year).

Thus, an initial dataset was created, let's call it Dataset5k.

Next, on this dataset, BIG CRNN was trained and achieved 80% accuracy on unseen data.

After that using Vkontakte account without phone verification more images were automatically labelled online.

Thus, a new dataset was created, and the process continues.

After achieving 99% accuracy with BIG CRNN model, it was used for labelling the rest of 1.5 million images offline.

Datasets with 850k images and 1500k images were constructed this way.

This huge "filthy" datasets (implying that there are mistakes in predictions) were used for training small pseudoCRNN model.

____

## ONNXTroubles



  



Unfortunately, in the case of using models on Client-side, e.g. with JavaScript, existing frameworks (Open Neural Network Exchange (ONNX.js)) does not support RNNs.

The only one which supports them is Tensorflow.js, but it is slower.

It's important to note that RNNs also are quite slow due to its nature.

To mitigate this problem this project introduces a small pseudoCRNN architecture without RNN blocks. Instead of bidirectional Gated Recurrent Units (GRU) as in the BIG version, it uses one-dimensional convolutions. 

Other troubles:

 * Can't use dynamic reshaping, e.g Reshape(None, -1, 32) will crash

 * Can't use 1d Convolutions, instead, the following trick was used:

 

 (None, 32, 128) --Reshape--> (None, 32, 1, 128) --Conv2d--> (None, 32, 1, filters) --Reshape--> (None, 32, filters)

 * Can't use Droupout, it's important to reconstruct model without this layer

 * Can't use Casting operation in ONNX.js, e.g. INT32 -> FLOAT32 conversion

 * Can't use Ragged Tensors in ONNX.js, e.g. for CTC algorithm

 * Can't use tf.where, because Nonzero is not implemented in ONNX.js

 

You can check the list of supported operators in ONNX.js [`HERE`](https://github.com/microsoft/onnxjs/blob/master/docs/operators.md)

ONNX is a great thing, but its JavaScript Framework needs more attention from Microsoft developers.

As a result, in the next project, Tensorflow.js should be used instead of ONNX.js

____

## Results

After [`training`](python/FilthyCaptchaLearning.ipynb) on 1.5 Million filthy labeled captcha for 300k steps, the [__student model__](https://github.com/Defasium/models/blob/main/captcha_model.onnx) achieved ~90% accuracy (while teacher accuracy was 99%)



  



Comparision:

|Model|Dataset Size|Steps|Knowledge Distillation, %|Accuracy, %|Size, MB|Images/sec|

|:--------------|:--:|:--:|:--:|---:|---:|---:|

|BIG CRNN|5k|70k|-|80|66|~284|

|BIG CRNN|16k|70k|-|93|66|~284|

|BIG CRNN|40k|70k|-|__98.9__|66|~284|

|small pseudoCRNN|40k|70k|-|64|4|~2155|

|small pseudoCRNN|850k*|70k|-|82|4|~2155|

|small pseudoCRNN|850k*|70k|50|83|4|~2155|

|small pseudoCRNN|1.5M*|140k|50|86.7|4|~2155|

|small pseudoCRNN|1.5M*|280k|30|91.2|__4**__|__~2155__|

\* - means that dataset was made with filthy labels from the teacher

\** - after conversion the final size of the model is 1 MB

Thus, you can achieve up to 10 times faster inference time with the small version.

Optimized pseudoCRNN student architecture after [`convertion to ONNX format`](python/onnx_crnn_conversion.py):



  



____

## Sources

This project was inspired by number-plate recognition work and uses its modified version of CRNN implementation, [@RIA.com](https://github.com/ria-com/nomeroff-net).

ONNX.js reference example, created by [@elliotwaite](https://github.com/elliotwaite/pytorch-to-javascript).

For small pseudoCRNN CNN encoder, this system uses a modified version of ShuffleNetV2, implemented by [@opconty](https://github.com/opconty/keras-shufflenetV2).

Tensorflow Dataset class with caching implementation is the modified version of [@krasserm](https://github.com/krasserm/super-resolution).

For the model's graph visualization was used the `Netron` application, by [@lutzroeder](https://github.com/lutzroeder/netron)