{"id":15657813,"url":"https://github.com/pythonicninja/agregacje3","last_synced_at":"2025-03-30T02:21:50.819Z","repository":{"id":23378835,"uuid":"26740328","full_name":"PythonicNinja/agregacje3","owner":"PythonicNinja","description":null,"archived":false,"fork":false,"pushed_at":"2014-11-19T22:18:01.000Z","size":1484,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-05T04:46:13.295Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PythonicNinja.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-11-17T04:41:16.000Z","updated_at":"2014-11-21T10:10:16.000Z","dependencies_parsed_at":"2022-07-13T23:50:31.182Z","dependency_job_id":null,"html_url":"https://github.com/PythonicNinja/agregacje3","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PythonicNinja%2Fagregacje3","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PythonicNinja%2Fagregacje3/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PythonicNinja%2Fagregacje3/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PythonicNinja%2Fagregacje3/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PythonicNinja","download_url":"https://codeload.github.com/PythonicNinja/agregacje3/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246266570,"owners_count":20749815,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-03T13:09:50.679Z","updated_at":"2025-03-30T02:21:50.802Z","avatar_url":"https://github.com/PythonicNinja.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"**Spis treści**\n\n\n- [Wojciech Nowak](#wojciech-nowak)\n    - [Sprzęt](#sprzęt)\n    - [Wersje oprogramowania](#wersje-oprogramowania)\n- [Zadania 1](#zadania-1)\n    - [Zadanie 1a](#zadanie-1a)\n        - [MongoDB](#mongodb)\n            - [2.6](#import-26)\n            - [2.8.0-rc0](#import-280-rc0)\n            - [2.8.0-rc0 z WiredTiger](#import-280-rc0-wiredtiger)\n            - [summary](#mongodb-imports)\n        - [PostgreSQL](#postgresql)\n    - [Zadanie 1b](#zadanie-1b)\n    - [Zadanie 1c](#zadanie-1c)\n    - [Zadanie 1d](#zadanie-1d)\n- [Zadanie 2](#zadanie-2)\n    - [Agregacja 1](#agregacja-1)\n        - [JS](#js)\n        - [Python](#python)\n    - [Agregacja 2](#agregacja-2)\n        - [JS](#js-1)\n        - [Python](#python-1)\n    - [Agregacja 3](#agregacja-3)\n        - [JS](#js-2)\n        - [Python](#python-2)\n    - [Agregacja 4](#agregacja-4)\n        - [JS](#js-3)\n        - [Python](#python-3)\n\n# Wojciech Nowak \n* nr albumu 206354 Informatyka I rok Magisterskie\n\n####Sprzęt\n* procesor: \n\n\t\tModel:\t\t\t\t\t\t\t\t  Intel Core i7-4650U\n\t\tSzybkość procesora:\t\t\t\t\t1,7 GHz  \n\t\tŁączna liczba rdzeni:\t\t\t\t\t2    \n\t\tPamięć podręczna L2 (na rdzeń): \t  256KB    \n\t\tPamięć podręczna L3: \t\t\t\t 4MB\n\t\t\n* pamięć ram:\n\n\t\tRozmiar: \t\t8GB \n\t\tTyp:\t\t\tDDR3\n\t\tPrędkość:\t   1600MHz\n\n* dysk: \n\n\t\tmodel: \t\t\tSSD Samsung sm0256f \n\t\tmagistrala:\t\tPCIe SSD\n\t\t\n####Wersje oprogramowania\n* MongoDB shell version: \t\n        2.8.0-rc0\n        2.6\n* WiredTiger: \t            1.4.2\n* Postgresql version: \t\t9.3.5\n* OS: Mac OS X \t\t\t\t10.10 (14A389)\n* Python:\n       3.4.2\n       2.7.8\n\n\n#Zadania 1\n\n###Zadanie 1a \nPolega na zaimportowaniu, do systemów baz danych uruchomionych na swoim komputerze, danych z pliku Train.csv bazy:\n\n####MongoDB\n\nkomenda:\n\n\t\u003e\u003e time mongoimport --type csv -c Train --file Desktop/Train.csv --headerline\n\njednak:\n\n\t\u003e\u003edb.Train.findOne()\n\n\t{\n\t\t\"_id\" : ObjectId(\"5463ddb4ab50c94d02a24f07\"),\n\t\t\"Id\" : \"\u003cp\u003eIs there a way to check if the uploaded file is an image apart from checking the file extension using PHP?\u003c/p\u003e\"\n\t}\n\t\npo krótkiej analizie co jest problemem, dochodzimy do wniosku, że należy usunąć \"\\n\"\n\n\t\u003e\u003e time cat Train.csv | tr \"\\n\" \" \" | tr \"\\r\" \"\\n\" \u003e Train_prepared_for_mongoimport.csv\n\treal\t19m42.573s\n\tuser\t37m9.401s\n\tsys\t\t0m38.146s\n\t\n######import 2.6\n\t\n\t\u003e\u003etime mongoimport --type csv -c Train --file Train_prepared_for_mongoimport.csv --headerline\n\tconnected to: 127.0.0.1\n\t2014-11-13T01:27:39.002+0100 \t\tProgress: 44715137/7253917400     0%\n\t...\n\t2014-11-13T01:35:26.016+0100 \t\tProgress: 7212784202/7253917400\t99%\n\t2014-11-13T01:35:26.016+0100 \t\t\t6000000  12765/second\n\t2014-11-13T01:35:28.844+0100 check 9 6034197\n\t2014-11-13T01:35:28.898+0100 imported 6034196 objects\n\n\treal\t7m52.488s\n\tuser\t2m25.171s\n\tsys\t0m26.896s\n\t\n######import 2.8.0-rc0\n\t\n\t\u003e\u003etime mongoimport --type csv -c Train --file Train_prepared_for_mongoimport.csv --headerline\n\t2014-11-17T01:45:04.590+0100\timported 6034196 documents\n\n\treal\t6m49.952s\n\tuser\t6m16.081s\n\tsys\t0m53.392s\n\t\n\n######2.8.0-rc0 + WiredTiger\n    \ninstalacja WiredTiger [install.html](http://source.wiredtiger.com/1.4.2/install.html)\t\n    \nuruchomienie mongo z storageEngine jako WiredTiger.\n\n    \u003e\u003emongod --storageEngine wiredtiger\n    \n    \u003e\u003etime mongoimport --type csv -c Train --file Train_prepared_for_mongoimport.csv --headerline\n\t2014-11-19T18:30:10.829+0100\timported 6034196 documents\n    real\t4m19.000s\n    user\t5m57.282s\n    sys\t1m2.356s\n    \nJak widać projekt jest bardzo ciekawy i przyszłościowy, swój udział w tym storageEngine mają najwięksi gracze:\n\n    - Google\n    \n        api/leveldb/leveldb/*\tGoogle, Inc.\n        \n        src/support/hash_city.c\tGoogle, Inc.\n        \n    - Facebook\n    \n        api/leveldb/rocksdb/*\tFacebook, Inc.\n        \n    -University of Berkley\n    \n        src/utilities/util_getopt.c\tUniversity of California, Berkeley\n        \nWizja lepszego wykorzystania zasobów procesora jest na tyle kusząca, że Dr Michael Cahill (główny twórca)\n\n    \"raised over US$30 million from investors including Intel Capital and JP Morgan during his three year tenure\"\n\n\n![Alt text](import/images/2.8.0-rc0WiredTiger.png)\n\n\n######Mongo.db imports\n\n![Alt text](import/images/cpuPeek.png)\n\n\n| 2.6       | 2.8.0-rc0 | 2.8.0-rc0 with WiredTiger |\n|-----------|:---------:|--------------------------:|\n| 7m52.488s | 6m49.952s |                 4m19.000s |\n    \n\n####PostgreSQL\n\nnajpierw użyjemy exportu z mongo do json'a\n\n\t\u003e\u003e time mongoexport --db test --collection Train --out train.json --journal\n\t2014-11-17T00:14:50.951+0100\texported 6034196 records\n\n\treal\t5m46.904s\n\tuser\t2m55.293s\n\tsys\t\t0m36.245s\n\t\ntworzymy bazę\n\n\t\u003e\u003e createdb Train\n\nwłączamy hstora\n\n\t\u003e\u003e psql Train\t\n\t\n\t\u003e\u003e CREATE EXTENSION hstore;\n\ntworzymy tabele\n\n\tCREATE TABLE train (\n    \tId int,\n\t    Title text,\n    \tBody text,\n\t    Tags text[],\n\t);\n\nimport w pythonie z mongodb -\u003e postgresa:\n\n[python script](import/mongo2postgres.py)\t\n\t\n\t\u003e\u003e time python mongo2postgres.py\n\treal\t68m41.412s\n\tuser\t10m26.866s\n\tsys\t6m58.740s\t\n\n\n###Zadanie 1b\nZliczyć liczbę zaimportowanych rekordów (Odpowiedź: powinno ich być 6_034_195).\n\t\n\t\u003e\u003e mongo\n\t\u003e db.Train.count()\n\t6034196\n\t\n###Zadanie 1c\nZamiana formatu danych.) Zamienić string zawierający tagi na tablicę napisów z tagami następnie zliczyć wszystkie tagi i wszystkie różne tagi.\n\t\n\t\u003e\u003etime python3 tags_to_list_v0.py \n\tAll tags 17407667\n\tAll distinct 42048\n\n\treal    71m5.795s\n\tuser    47m8.053s\n\tsys     3m53.134s\n\n[slow version Link](import/tags_to_list_v0.py)\n\n![Alt text](import/images/lowCPU.png)\n\t\nZbyt wolno :(, dodatkowo robiłem budowanie słownika ale głównym spowolnieniem było nie zastosowanie zrównolegniania.\n\t\nUznałem, że da się dużo lepiej używając greenletów:\n\n\t\u003e\u003e time python tags_to_list_v1.py \n\treal    32m16.310s\n\tuser    9m25.584s\n\tsys     0m57.192s\n\n[greenlet Link](import/tags_to_list_v1.py)\n\n![Alt text](import/images/highCPU.png)\n\n| tags_to_list_v0       | tags_to_list_v1.py |\n|-----------------------|:------------------:|\n| 71m5.795s             | 32m16.310s         |\n\n###Zadanie 1d\nZadanie 1d. Wyszukać w sieci dane zawierające obiekty GeoJSON. Następnie dane zapisać w bazie MongoDB.\n\nSzukając danych geojson, natrafiłem na bazę punktów poi. Postanowiłem zanalizować rejestratory w polsce.\n\n[link do pliku](http://www.poiplaza.com/download/zipfiles/1424/PL-FixSpeedcam_mio_moov.zip)\n\n\t\nprzykładowy rekord:\n\n\t52.64373, 19.19789, 50@ PL-Modzerowo Fix Speedcam\n\t\nmusimy przygotowac do importu do mongodb\n\n1. usunąć polskie znaki\n\n\t\t\u003e\u003e iconv -f utf8 -t utf8 -c PL-speedcam.csv \u003e PL-speedcam_ASCII.csv\n\t\n2. usunąć \" \", \"@\", \"PL-\", \"Fix Speedcam\", nazwę miasta oraz podzielić przecinkami\n\n\t\t\u003e\u003e cat PL-speedcam_ASCII.csv | cut -d \"@\" -f1 \u003e PL-speedcam_toImport.csv\n\n3. import do mongodb\n\n\t\t\u003e\u003etime mongoimport --type csv -c speedcam --file PL-speedcam_toImport.csv --fields lon,lat,speed\n\t\t2014-11-17T04:29:26.515+0100\tconnected to: localhost\n\t\t2014-11-17T04:29:26.589+0100\timported 2538 documents\n\n\t\treal\t0m0.106s\n\t\tuser\t0m0.034s\n\t\tsys\t0m0.018s\n\t\t\n4. przykładoy wpis:\n\n\t\t\u003e\u003edb.speedcam.findOne()\n\t\t{\n\t\t\t\"_id\" : ObjectId(\"54696b964fa64c9a2a9113fe\"),\n\t\t\t\"lon\" : 52.88185,\n\t\t\t\"lat\" : 21.32492,\n\t\t\t\"speed\" : 50\n\t\t}\n\t\t\n5. musimy przemienić go na geojson'a\n\t\n\n```python\n# -*- coding: utf-8 -*-\n# CREATED ON DATE: 17.11.14\n__author__ = 'vojtek.nowak@gmail.com'\n\n\nimport csv,json, decimal, codecs\n\nfile = open(\"PL-speedcam_toImport.csv\")\nreader = csv.reader(file, delimiter=\",\", quotechar='\"')\n\nkeys = ['lon', 'lat', 'speed']\n\nwith codecs.open(\"speedcamGeo.json\", \"w\", encoding=\"utf-8\") as out:\n    for r in reader:\n        line = dict(zip(keys, r))\n\n        out.write(json.dumps({\n            'speed': int(line.get('speed')),\n            'loc': [float(line.get('lon')), float(line.get('lat'))]\n        })+\"\\n\")\n\nfile.close()\n```\n\t\n\t\u003e\u003e python csv2geojson.py\n\n\n6. import do mongodb\n\t\n\t\t\u003e\u003etime mongoimport --type json -c speedcam --file speedcamGeo.json\n\t\t2014-11-17T04:53:24.583+0100\tconnected to: localhost\n\t\t2014-11-17T04:53:24.615+0100\timported 2538 documents\n\n\t\treal\t0m0.042s\n\t\tuser\t0m0.033s\n\t\tsys\t0m0.008s\n\t\t\n\nAGREGACJE GEOJSON:\n\nNapisałem skrypt [Link](speedcam/csv2geojson.py) w pythonie generujący geojsony dla zapytania do bazy mongodb:\n\n\tdef to_geo_json_points(cursor, json_name, type=\"Point\"):\n    \tgeo_json = {\n        \t\"type\": \"FeatureCollection\",\n\t        \"features\": [\n    \t        {\n        \t        \"type\": \"Feature\",\n            \t    \"geometry\":\n                \t    {\n                   \t     \"_id\": str(point.get('speed')) + \" \" + str(point.get('loc')),\n                   \t     \"type\": type,\n                   \t     \"coordinates\": [point.get('loc')[1], point.get('loc')[0]]\n                   \t }\n\t            } for point in cursor\n    \t    ]\n\t    }\n\t\n    with open(json_name, 'w') as out_json:\n        json.dump(geo_json, out_json)\n\n1. fotoradary w całej Polsce.\n\n\t[Link](speedcam/speed_cams_poland.geojson)\n\t\n\t![Alt text](speedcam/images/polsza.png)\n\t\n\t\t\u003e\u003e to_geo_json_points(\n\t    cursor=db.speedcam.find(),\n    \tjson_name='speed_cams_poland.geojson')\t\n    \t    \t\n2. fotoradary w okolicy Gdańska.\n\n\t[Link](speedcam/speed_cams_near_gdansk.geojson)\n\t\n\t![Alt text](speedcam/images/near_gdansk.png)\n\t\n\t\t\u003e\u003e to_geo_json_points(\n\t    cursor=db.speedcam.find({\n    \t    'loc': {\n        \t    '$near': {'$geometry': {'type': \"Point\", 'coordinates': [54.349683, 18.643335]}, '$maxDistance': 20000}}}),\n\t    json_name='speed_cams_near_gdansk.geojson')\n\n    \t\n3. fotoradary w Pomorskim.\n\n\t[Link](speedcam/speed_cams_pomorskie.geojson)\n\t\n\t![Alt text](speedcam/images/pomorskie.png)\n\t\n\t\t\u003e\u003e to_geo_json_points(\n    \tcursor=db.speedcam.find({\n        \t'loc': {\n            \t'$geoIntersects': {\n                \t'$geometry': {\n                   \t 'type': \"Polygon\",\n                   \t 'coordinates': [\n                   \t     [\n                   \t         [54.570489, 16.702425],\n                   \t         [54.950828, 19.201752],\n                   \t         [53.611260, 19.310739],\n                   \t         [53.611260, 16.893747],\n           \t                 [54.570489, 16.702425]\n                        ]\n                    ]\n                }\n            }\n        }\n\t    }),\n    \tjson_name='speed_cams_pomorskie.geojson')\t\n4. fotoradary w okolicy Karkowa \u003c 100km\n\n\t[Link](speedcam/speed_cams_around_krakow.geojson)\n\t\n\t![Alt text](speedcam/images/100_around_krakow.png)\n\t\n\t\t\u003e\u003eto_geo_json_points(\n\t    cursor=db.speedcam.find({\n    \t    'loc': {\n        \t    '$near': {'$geometry': {'type': \"Point\", 'coordinates': [50.059441, \t19.940328]}}}}).limit(100),\n    \tjson_name='speed_cams_around_krakow.geojson')    \t\n    \t\n5. fotoradary w na drodze.\n\n\t[Link](speedcam/speed_cams_route.geojson)  \n\n\t\t\u003e\u003eto_geo_json_points(\n    \t\t\tcursor=db.speedcam.find({\n        \t\t'loc': {'$geoIntersects':\n            \t\t        {'$geometry':\n                \t\t         {\n                   \t\t          'type': \"LineString\",\n                       \t      'coordinates': [\n                   \t    \t          [52.64373, 19.19789],\n                   \t        \t      [51.17473, 19.46548]\n\t                   \t          ]\n    \t               \t      }\n        \t           \t }\n\t        \t}\n\t    \t}),\n\t\t    json_name='speed_cams_route.geojson')\n\n\n6. fotoradary near Warsaw but not in warsaw center 4km \u003c center \u003c 20km\n\t[Link](speedcam/speed_cams_around_center_of_warsaw.geojson)\n\t\n\t![Alt text](speedcam/images/around_warsaw.png)\n\n\t\t\u003e\u003eto_geo_json_points(\n    \tcursor=db.speedcam.find({\n        \t'loc': {\n            '$near': {'$geometry': {'type': \"Point\", 'coordinates': [52.232728, 21.010382]}, \t'$maxDistance': 20000, '$minDistance': 4000}}}),\n    \tjson_name='speed_cams_around_center_of_warsaw.geojson')\n\n\n7. fotoradary 15 around polish seaside\n\t[Link](speedcam/speed_cams_near_seaside.geojson)\n\t\n\t![Alt text](speedcam/images/near_seaside.png)\n\t\n\t\t\u003e\u003eto_geo_json_points(\n    \tcursor=db.speedcam.find({\n        \t'loc': {\n            '$near': {\n                '$geometry': {\n                    'type': \"Point\",\n                    'coordinates': [55.634283, 15.403922]\n                }\n            }\n        }\n\t    }).limit(15),\n    \tjson_name='speed_cams_near_seaside.geojson')\n\t\n#Zadanie 2\n\ndane pochodzą z [http://data.police.uk/data/ \n](http://data.police.uk/data/) z ostatniego roku tzn. 09.2013 - 09.2014\n\npołączenie csv\n\n\t\u003e\u003e cat 201*/*.csv \u003emerged.csv\n\nzakładana liczba rekordów\n\n\t\u003e\u003ecat merged.csv | wc -l\n\t10 277 390\n\t\nimport do bazy mongodb\n\n\t\u003e\u003e time mongoimport --type csv -c crime --file merged.csv --headerline\n\t2014-11-19T03:29:58.883+0100\timported 10277389 documents\n\n\treal\t4m47.820s\n\tuser\t8m29.818s\n\tsys\t\t1m2.077s\n\t\ntest ilości\n\n\t\u003e\u003e db.crime.count()\n\t10277389\n\t\nprzykładowy wpis:\n\n\t\u003e\u003e \u003e db.crime.findOne()\n\t{\n\t\"_id\" : ObjectId(\"546bff87fe4798e24132403f\"),\n\t\"Crime ID\" : \"15939afdf6964e9edea28bb0ba7aa3cb684fe879726910bf03945b3f8ceb85a3\",\n\t\"Month\" : \"2013-09\",\n\t\"Reported by\" : \"Avon and Somerset Constabulary\",\n\t\"Falls within\" : \"Avon and Somerset Constabulary\",\n\t\"Longitude\" : \"\",\n\t\"Latitude\" : \"\",\n\t\"Location\" : \"No location\",\n\t\"LSOA code\" : \"\",\n\t\"LSOA name\" : \"\",\n\t\"Outcome type\" : \"Suspect charged as part of another case\"\n\t}\n\n###Agregacja 1\nilość zakończeń przestępstwa w danym typie.\n\t\n\t\n####JS\t\n\t\t\u003e\u003edb.crime.aggregate(\n        \t[\n    \t        {\n\t                '$group': {\n                \t    '_id':  \"$Outcome type\",\n            \t        'count': {'$sum': 1}\n        \t        }\n    \t        },\n\t            {\n                \t'$sort': {'count': -1}\n            \t}\n        \t])\n        time: 64.59 sec.\n        \n####Python \nskrypt z generowaniem słownika do highcharts [link](crime/aggregation1.py)\n\n\n        \u003e\u003etime python aggregation1.py \n        real    1m1.465s\n        user    0m0.070s\n        sys     0m0.067s\n\n\n[Interactive Chart](http://jsfiddle.net/bmrhL57u/2/)\t\n![Alt text](crime/images/agg1.png)\n\n        \t\n###Agregacja 2\nilość przestępstw w danym miesiącu.\n\n####JS\t\n\t\t\u003e\u003e\u003e\u003edb.crime.aggregate([\n        \t{\n                '$group': {\n                    '_id':  \"$Month\",\n                    'count': {'$sum': 1}\n                }\n        \t},\n            {\n                '$sort': {'count': -1}\n            }\n\t\t])\n        time: 56.883 sec.\n        \n####Python \nskrypt z generowaniem słownika do highcharts [link](crime/aggregation2.py)\n\n\n        \u003e\u003etime python aggregation2.py \n        real    0m57.950s\n        user    0m0.076s\n        sys     0m0.067s\n\n\n[Interactive Chart](http://jsfiddle.net/bmrhL57u/3/)\t\n![Alt text](crime/images/agg2.png)\n\t\t\n\t\t\n[\n    {\n        \"_id\" : \"2014-09\",\n        \"count\" : 812034\n    },\n    {\n        \"_id\" : \"2014-06\",\n        \"count\" : 817447\n    },\n    {\n        \"_id\" : \"2014-07\",\n        \"count\" : 879294\n    },\n    {\n        \"_id\" : \"2014-05\",\n        \"count\" : 815277\n    },\n    {\n        \"_id\" : \"2013-12\",\n        \"count\" : 726761\n    },\n    {\n        \"_id\" : \"2014-03\",\n        \"count\" : 782209\n    },\n    {\n        \"_id\" : \"2014-04\",\n        \"count\" : 761371\n    },\n    {\n        \"_id\" : \"2014-01\",\n        \"count\" : 753897\n    },\n    {\n        \"_id\" : \"2014-08\",\n        \"count\" : 815111\n    },\n    {\n        \"_id\" : \"2013-10\",\n        \"count\" : 854526\n    },\n    {\n        \"_id\" : \"2014-02\",\n        \"count\" : 691036\n    },\n    {\n        \"_id\" : \"2013-09\",\n        \"count\" : 784351\n    },\n    {\n        \"_id\" : \"2013-11\",\n        \"count\" : 782932\n    }\n]\n\n\n###Agregacja 3\nkiedy dawano uniewinnienie?\n\n####JS\t\n\t\t\u003e\u003edb.crime.aggregate([\n        { '$match' : { 'Outcome type' : \"Offender given absolute discharge\" }},\n        {\n                '$group': {\n                    '_id':  \"$Month\",\n                    'count': {'$sum': 1}\n                }\n        },\n        {\n            '$sort': {'count': -1}\n        }\n        ])\n        time: 48.487 sec.\n        \n####Python \nskrypt z generowaniem słownika do highcharts [link](crime/aggregation3.py)\n\n\n        \u003e\u003etime python aggregation3.py \n        real    0m51.672s\n        user    0m0.078s\n        sys     0m0.069s\n\n[Interactive Chart](http://jsfiddle.net/bmrhL57u/4/)\t\n![Alt text](crime/images/agg3.png)\n\n[\n    {\n        \"_id\" : \"2014-01\",\n        \"count\" : 145\n    },\n    {\n        \"_id\" : \"2014-07\",\n        \"count\" : 136\n    },\n    {\n        \"_id\" : \"2014-04\",\n        \"count\" : 132\n    },\n    {\n        \"_id\" : \"2013-10\",\n        \"count\" : 130\n    },\n    {\n        \"_id\" : \"2013-09\",\n        \"count\" : 127\n    },\n    {\n        \"_id\" : \"2013-11\",\n        \"count\" : 127\n    },\n    {\n        \"_id\" : \"2014-03\",\n        \"count\" : 123\n    },\n    {\n        \"_id\" : \"2014-08\",\n        \"count\" : 115\n    },\n    {\n        \"_id\" : \"2014-06\",\n        \"count\" : 111\n    },\n    {\n        \"_id\" : \"2014-05\",\n        \"count\" : 103\n    },\n    {\n        \"_id\" : \"2014-02\",\n        \"count\" : 101\n    },\n    {\n        \"_id\" : \"2013-12\",\n        \"count\" : 93\n    },\n    {\n        \"_id\" : \"2014-09\",\n        \"count\" : 80\n    }\n]\n\n###Agregacja 4\nUniewinnienia i ich lokalizacje\n\n####JS\t\n\t\t\u003e\u003edb.crime.aggregate([\n        { '$match' : { 'Outcome type' : \"Offender given absolute discharge\" }},\n        {\n                '$group': {\n                    '_id':  \"$Location\",\n                    'count': {'$sum': 1}\n                }\n        },\n        {\n            '$sort': {'count': -1}\n        },\n        {\n            '$limit': 10\n        }\n\t    ])\n        time: 49.382 sec.\n        \n####Python \nskrypt z generowaniem słownika do highcharts [link](crime/aggregation4.py)\n\n\n        \u003e\u003etime python aggregation4.py \n        real    0m49.514s\n        user    0m0.078s\n        sys     0m0.082s\n\n\t\n[Interactive Chart](http://jsfiddle.net/bmrhL57u/5/)\t\n![Alt text](crime/images/agg4.png)\n\t\n\t\t[\n    {\n        \"_id\" : \"On or near Supermarket\",\n        \"count\" : 143\n    },\n    {\n        \"_id\" : \"On or near Shopping Area\",\n        \"count\" : 111\n    },\n    {\n        \"_id\" : \"On or near Parking Area\",\n        \"count\" : 58\n    },\n    {\n        \"_id\" : \"On or near Petrol Station\",\n        \"count\" : 49\n    },\n    {\n        \"_id\" : \"No location\",\n        \"count\" : 20\n    },\n    {\n        \"_id\" : \"On or near Sports/Recreation Area\",\n        \"count\" : 18\n    },\n    {\n        \"_id\" : \"On or near Nightclub\",\n        \"count\" : 17\n    },\n    {\n        \"_id\" : \"On or near Pedestrian Subway\",\n        \"count\" : 15\n    },\n    {\n        \"_id\" : \"On or near Police Station\",\n        \"count\" : 11\n    },\n    {\n        \"_id\" : \"On or near NEWGATE STREET\",\n        \"count\" : 10\n    }\n]\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpythonicninja%2Fagregacje3","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpythonicninja%2Fagregacje3","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpythonicninja%2Fagregacje3/lists"}