{"id":19922557,"url":"https://github.com/sa7mon/csv-sql-tools","last_synced_at":"2026-03-05T12:02:28.559Z","repository":{"id":84343577,"uuid":"134494243","full_name":"sa7mon/csv-sql-tools","owner":"sa7mon","description":null,"archived":false,"fork":false,"pushed_at":"2018-05-23T14:41:59.000Z","size":66212,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-01-29T21:26:39.764Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sa7mon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2018-05-23T01:14:38.000Z","updated_at":"2020-12-04T18:53:09.000Z","dependencies_parsed_at":"2023-07-06T22:31:48.754Z","dependency_job_id":null,"html_url":"https://github.com/sa7mon/csv-sql-tools","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sa7mon/csv-sql-tools","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sa7mon%2Fcsv-sql-tools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sa7mon%2Fcsv-sql-tools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sa7mon%2Fcsv-sql-tools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sa7mon%2Fcsv-sql-tools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sa7mon","download_url":"https://codeload.github.com/sa7mon/csv-sql-tools/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sa7mon%2Fcsv-sql-tools/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30123731,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-05T11:11:57.947Z","status":"ssl_error","status_checked_at":"2026-03-05T11:11:29.001Z","response_time":93,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T22:11:27.616Z","updated_at":"2026-03-05T12:02:28.554Z","avatar_url":"https://github.com/sa7mon.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# CSV SQL Tools\n\n## Dataset \nSource: [Los Angeles Crime Data: 2010 - Present](https://data.lacity.org/A-Safe-City/Crime-Data-from-2010-to-Present/y8tr-7khq)\n\nDownloaded la\\_crime.csv. Grabbed mo_codes.pdf, then `pdftotext -layout input.pdf output.txt` and cleaned to make mo_codes.csv.\n\n\n## Testing Notes\n\n[Idea](https://news.ycombinator.com/item?id=16781294)\n\n### Tools\n\n* [https://harelba.github.io/q](https://harelba.github.io/q)\n* [https://github.com/BurntSushi/xsv](https://github.com/BurntSushi/xsv)\n* [http://bigbash.it/ - Possibly use to compare with coreutils](http://bigbash.it/)\n* [https://github.com/dinedal/textql](https://github.com/dinedal/textql)\n* [http://johnkerl.org/miller/doc/](http://johnkerl.org/miller/doc/)\n* [https://github.com/dbohdan/sqawk](https://github.com/dbohdan/sqawk)\n* [https://csvkit.readthedocs.io/en/1.0.3/](https://csvkit.readthedocs.io/en/1.0.3/)\n* [https://pandas.pydata.org/pandas-docs/stable/comparison_with_sql.html](https://pandas.pydata.org/pandas-docs/stable/comparison_with_sql.html)\n* [https://github.com/BatchLabs/charlatan#charlatan](https://github.com/BatchLabs/charlatan#charlatan )\n* [https://drill.apache.org/docs/querying-plain-text-files/](https://drill.apache.org/docs/querying-plain-text-files/)\n* [https://jsvine.github.io/intro-to-visidata/](https://jsvine.github.io/intro-to-visidata/)\n* [https://metacpan.org/pod/distribution/App-fsql/bin/fsql](https://metacpan.org/pod/distribution/App-fsql/bin/fsql)\n* [https://github.com/noborus/trdsql](https://github.com/noborus/trdsql)\n* [https://github.com/tobimensch/termsql](https://github.com/tobimensch/termsql)\n* [https://github.com/tjunier/sqawk](https://github.com/tjunier/sqawk)\n* [https://github.com/turicas/rows](https://github.com/turicas/rows)\n* [https://github.com/wireservice/csvkit](https://github.com/wireservice/csvkit)\n* [https://github.com/agershun/alasql](https://github.com/agershun/alasql)\n* [https://github.com/dbohdan/structured-text-tools/blob/master/sql-based.md](https://github.com/dbohdan/structured-text-tools/blob/master/sql-based.md)\n\n### Test Cases\n* ``` SELECT * FROM la_crime WHERE victim_age \u003c 20 ORDER BY dr_number```\n   * q: ```q -H -d '@' \"SELECT * FROM la_crime.asv WHERE victim_age \u003c 20 ORDER BY dr_number\"```\n* ```SELECT weapon_description FROM la_crime WHERE weapon_description LIKE '%knife%'```\n* ``` SELECT crime.date_occurred, crime.crime_code_description, mo.mo_description FROM la_crime AS crime JOIN la_crime_mo_codes AS mo ON (crime.mo_codes = mo.mo_code)``` \u003c- Little janky because mo_codes column can have multiple codes\n   * q: ```SELECT crime.date_occurred, crime.crime_code_description, mo.mo_description FROM la_crime crime JOIN la_crime_mo_codes.asv mo ON (crime.mo_codes = mo.mo_code)```\n* ``` SELECT COUNT(*) FROM la_crime.asv```\n* ``` SELECT DISTINCT(area_name) FROM la_crime.asv ORDER BY area_name```\n* select multiple where multiple fields (strings = target, number \u003e target \u003e number)\n\n### To Measure mem/time\n* [https://superuser.com/a/767491](https://superuser.com/a/767491)\n\n### Template to generate bash baseline with BigBash\n\u003cpre\u003e\nCREATE TABLE crimes (dr_number,date_reported,date_occurred,time_occurred,area_id,area_name,reporting_district,crime_code,crime_code_description,mo_codes,victim_age,victim_sex,victim_descent,premise_code,premise_description,weapon_used_code,weapon_description,status_code,status_description,crime_code_1,crime_code_2,crime_code_3,crime_code_4,address,cross_street,location);\nMAP crimes to 'la_crime-stats_2010-present.csv' DELIMITER ',';\nSELECT Crime_Code_Description FROM crimes;\n\u003c/pre\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsa7mon%2Fcsv-sql-tools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsa7mon%2Fcsv-sql-tools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsa7mon%2Fcsv-sql-tools/lists"}