Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/osmcha/osmcha

Python package to detect suspicious OSM changesets
https://github.com/osmcha/osmcha

gis openstreetmap osm qa

Last synced: 4 months ago
JSON representation

Python package to detect suspicious OSM changesets

Awesome Lists containing this project

README 

        
osmcha

=======



OSM Changeset Analyser, ``osmcha``, is a Python package to detect suspicious OSM changesets.

It was designed to be used with `osmcha-django `_,

but also can be used standalone or in other projects.



You can report issues or request new features in the the

`osmcha-frontend repository `_.



.. image:: https://badge.fury.io/py/osmcha.svg

    :target: http://badge.fury.io/py/osmcha



.. image:: https://coveralls.io/repos/OSMCha/osmcha/badge.svg

    :target: https://coveralls.io/r/willemarcel/osmcha



.. image:: https://img.shields.io/badge/Contributor%20Covenant-v2.0%20adopted-ff69b4.svg

    :target: CODE_OF_CONDUCT.md



Installation

============



.. code-block:: console



    pip install osmcha



Usage

=====



Python Library

--------------



You can read a `replication changeset file `_

directly from the web:



.. code-block:: python



  from osmcha.changeset import ChangesetList

  c = ChangesetList('https://planet.openstreetmap.org/replication/changesets/002/236/374.osm.gz')



or from your local filesystem.



.. code-block:: python



  c = ChangesetList('tests/245.osm.gz')



``c.changesets`` will return a list containing data of all the changesets listed in the file.



You can filter the changesets passing a `GeoJSON` file with a polygon with your

interest area to `ChangesetList` as the second argument.



Finally, to analyse an especific changeset, do:



.. code-block:: python



  from osmcha.changeset import Analyse

  ch = Analyse(changeset_id)

  ch.full_analysis()



Customizing Detection Rules

~~~~~~~~~~~~~~~~~~~~~~~~~~~



You can customize the detection rules by defining your prefered values when

initializing the ``Analyze`` class. See below the default values.



.. code-block:: python



  ch = Analyse(changeset_id, create_threshold=200, modify_threshold=200,

    delete_threshold=30, percentage=0.7, top_threshold=1000,

    suspect_words=[...], illegal_sources=[...], excluded_words=[...])



Command Line Interface

----------------------



The command line interface can be used to verify an especific changeset directly

from the terminal.



Usage: ``osmcha ``



Detection Rules

===============



``osmcha`` works by analysing how many map features the changeset created, modified

or deleted, and by verifying the presence of some suspect words in the ``comment``,

``source`` and ``imagery_used`` fields of the changeset. Furthermore, we also

consider if the software editor used allows to import data or to do mass edits.

We consider ``powerfull editors``: JOSM, Merkaartor, level0, QGIS and ArcGis.



In the ``Usage`` section, you can see how to customize some of these detection rules.



Possible Import

---------------



We tag a changeset as a ``possible import`` if the number of created elements is

greater than 70% of the sum of elements created, modified and deleted and if it

creates more than 1000 elements or 200 elements case it used one of the ``powerfull editors``.



Mass Modification

-----------------



We consider a changeset as a ``mass modification`` if the number of modified elements

is greater than 70% of the sum of elements created, modified and deleted and if it

modifies more than 200 elements.



Mass Deletion

-------------



All changesets that delete more than 1000 elements are considered a ``mass deletion``.

If the changeset deletes between 200 and 1000 elements and the number of deleted

elements is greater than 70% of the sum of elements created, modified and deleted

it's also tagged as a ``mass deletion``.



Suspect words

-------------



The suspect words are loaded from a `yaml file `_.

You can customize the words by setting another default file with a environment

variable:



.. code-block:: console



  export SUSPECT_WORDS=



or pass a list of words to the ``Analyse`` class, more information on the section

``Customizing Detection Rules``. We use a list of illegal sources to analyse the

``source`` and ``imagery_used`` fields and another more general list to examine

the comment field. We have also a list of excluded words to avoid false positives.



New mapper

-----------



Verify if the user has less than 5 edits or less than 5 mapping days.



User has multiple blocks

------------------------



Changesets created by users that has received more than one block will be

flagged.



OSM Server Configuration

------------------------



If you need to use OSMCha with another OSM server instance, you need to configure the OSM_SERVER_URL environment variable, without trailing slash. Example:



.. code-block:: console



  export OSM_SERVER_URL='https://www.openhistoricalmap.org'



Tests

======



To run the tests on `osmcha`:



.. code-block:: console



  git clone https://github.com/osmcha/osmcha.git

  cd osmcha

  pip install -e .[test]

  py.test -v



Publishing a new version

=========================



Update the version number in ``osmcha/__init__.py`` and execute

the following commands:



.. code-block:: console



  python setup.py bdist_wheel

  twine upload dist/osmcha-{version}...



Changelog

=========



Check `CHANGELOG `_ for the version history.



Related projects

================



* `osmcha-django `_ - backend and API

* `osmcha-frontend `_ - frontend of the `OSMCha `_ application

* `osm-compare `_ - library that analyse OSM features to input it to OSMCha



License

=======



GPLv3