{"id":24429209,"url":"https://github.com/peterk/soulab","last_synced_at":"2026-04-28T13:03:11.188Z","repository":{"id":33177584,"uuid":"36818770","full_name":"peterk/SOUlab","owner":"peterk","description":"Snabblabb med SOU-data","archived":false,"fork":false,"pushed_at":"2015-06-18T07:17:27.000Z","size":228,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-20T13:35:33.503Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/peterk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-06-03T17:18:12.000Z","updated_at":"2021-06-27T11:37:32.000Z","dependencies_parsed_at":"2022-08-17T21:45:09.871Z","dependency_job_id":null,"html_url":"https://github.com/peterk/SOUlab","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peterk%2FSOUlab","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peterk%2FSOUlab/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peterk%2FSOUlab/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peterk%2FSOUlab/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/peterk","download_url":"https://codeload.github.com/peterk/SOUlab/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243456567,"owners_count":20293905,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-20T13:33:50.154Z","updated_at":"2025-12-29T13:59:42.846Z","avatar_url":"https://github.com/peterk.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SOUlab\n\nSnabblabb med SOU-data efter blogginlägg. Läs mer på http://www.christopherkullenberg.se/relationer-i-souniversum/\n\nHar du labbat själv? Bidra gärna med kod/analyser här.\n\n## Installera dependencies\n\n```\npip install -r requirements.txt\n```\n\n## Experiment\n\nDessa utgår från att textfiler för SOU:er finns tillgängliga i katalogen\n\"data\". Ladda hem [SOU-zipfil (ca 800 Mb)](http://scientometrics.flov.gu.se/files/SOUtxtDecadeFiles.zip) och packa upp i mappen \"data\".\n\n### 1. Skapa grafdata för användning i Gephi\n\nMycket simpel strängmatchning av \"SOU \\d\\d\\d\\d:...\" i SOU-texterna för\natt möjliggöra referensanalys. Bygg SOU-databas med build_db.py först (en SQLite-databas kommer skapas):\n\n```\npython build_db.py ./data\n```\n\nLäs ut relationerna mellan SOU:er:\n\n```\npython parsesou.py ./data\n```\n\nBygg grafdata i gexf-format med gen_graph.py:\n\n```\npython gen_graph.py soudata.gexf\n```\n\nDen sista filen kan sedan användas i verktyget Gephi för visualisering.\n\n\n### 2. Frekvensanalys\n\nGivet att grunddata har byggts upp enligt ovan är det enkelt att\ngenerera antalet SOU:er per år med:\n\n```\npython gen_count.py \u003e frekvens.csv\n```\n\n### 3. Hitta platsnamn\n\nFilen features.txt innehåller svenska ortnamn hämtade från geonames.org.\nVerktyget grep har stöd för [strängmatchning med Aho-Corasick](http://en.wikipedia.org/wiki/Aho–Corasick_string_matching_algorithm). För att mappa SOU:er mot ortnamn som förekommer i dem kör:\n\n```\ngrep -owf features.txt data/30tal/*.txt\n```\n\n### 4. Deep learning med word2vec\n\nWord2vec tar en corpus av text och omvandlar till ordvektorer. Med dessa\ngår det att göra andra typer av analyser, t.ex. avständ mellan ord mm.\nDet borde gå att arbeta med SOU-korpusen i\n[word2vec](https://docs.google.com/a/peterkrantz.se/file/d/0B7XkCwpI5KDYRWRnd1RzWXQ2TWc/edit)\nför att kunna göra\nandra typer av analyser även om OCR-underlaget troligtvis kommer skapa\nen del problem. gen_word2vec.py tränar en modell med SOU-datat:\n\n```\npython gen_word2vec.py ./data/\n```\n\nKörtiden kan vara någon timme för att skapa informationen. När det är\nklart utforskar man enklast datamängden från python på kommandoraden.\nStarta en ny python-tolk och testa enligt nedan:\n\n```python\nimport gensim\n\n# ladda den genererade modellen\nmodel = gensim.models.Word2Vec.load(\"gensim2.model\")\n\n\nmodel.most_similar(u\"amatör\")\n# [(u'amatörteater', musikföreningar', 0.7612131834030151), (u'folkdans', 0.7437361478805542), (u'symfonisk', 0.7418662309646606), (u'allsång', 0.7417885661125183)] ...\n\n\nmodel.doesnt_match(\"stat kommun landsting ambassad\".split())\n# ambassad\n\n\nmodel.most_similar(positive=[u'integritet', u'lag'], topn=1)\n# [(u'datalagen', 0.7018075585365295)]\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeterk%2Fsoulab","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpeterk%2Fsoulab","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeterk%2Fsoulab/lists"}