{"id":20102517,"url":"https://github.com/clickhouse/houseclick","last_synced_at":"2025-05-06T08:30:46.200Z","repository":{"id":185417094,"uuid":"629470604","full_name":"ClickHouse/HouseClick","owner":"ClickHouse","description":"House prices app","archived":false,"fork":false,"pushed_at":"2023-11-28T00:32:50.000Z","size":3314,"stargazers_count":24,"open_issues_count":5,"forks_count":2,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-09T10:05:18.631Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ClickHouse.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-18T11:35:17.000Z","updated_at":"2024-09-23T05:10:22.000Z","dependencies_parsed_at":"2024-11-13T17:35:31.843Z","dependency_job_id":"c1f6c220-03de-456c-b905-a33771b6565d","html_url":"https://github.com/ClickHouse/HouseClick","commit_stats":null,"previous_names":["clickhouse/houseclick"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClickHouse%2FHouseClick","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClickHouse%2FHouseClick/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClickHouse%2FHouseClick/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClickHouse%2FHouseClick/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ClickHouse","download_url":"https://codeload.github.com/ClickHouse/HouseClick/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252648447,"owners_count":21782391,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T17:31:31.963Z","updated_at":"2025-05-06T08:30:46.184Z","avatar_url":"https://github.com/ClickHouse.png","language":"JavaScript","readme":"# HouseClick - ClickHouse + Supabase demo\n\nHouseClick is a fake estate agency business, powered by Supabase and ClickHouse via the ClickHouse Foreign Data Wrapper. Supabase for transactional data and application state, ClickHouse for analytics - all the through the Supabase API.\n\nThis demo complements the blog post [Adding Real Time Analytics to a Supabase Application](https://clickhouse.com/blog/adding-real-time-analytics-to-a-supabase-application). This requires you to have the Foreign Data Wrapper for ClickHouse available in your Supabase project.\n\n**Important**\n\n- This is a demo only. \n- The code has no tests and comments are currently minimal - although the code is very simple. \n- Rough and ready. Use for inspiration only. \n- PRs welcome to improve the code.\n- Due to concerns regards image writes, we do not provide a generated dataset. Steps to generate are included.\n- This requires the [Foreign Data Wrapper](https://supabase.github.io/wrappers/clickhouse/) (FDW) for ClickHouse to be available in Supabase.\n\n![House Click](./house_click.png)\n\n![House Click Search](./house_click_search.png)\n\n## Setup\n\n### 1. Requirements\n\n- Python 3.10+ - for data generation scripts\n- Node v16.15.1+\n- Yarn 1.22.19\n- Supabase (free tier is sufficient) account or local instance. You will need [Foreign Data Wrapper](https://supabase.github.io/wrappers/clickhouse/)  enabled on this instance.\n- OpenAPI key. $10 free credit is sufficient to generate 1000 properties. This is used to generated property descriptions.\n\n### 2. Installation\n\n```bash\ngit clone git@github.com:ClickHouse/HouseClick.git\n# setup script dependencies\ncd scripts\n# optionally create a virutal env e.g. virtualenv -p python3 .venv \u0026\u0026 source .venv\npip install -r requirements.txt\n# Install frontend\ncd ../HouseClick\nyarn install\n```\n\n### 3. Loading data into ClickHouse\n\nFrom the ClickHouse client. This data is around 28m rows and takes several minutes to load on most internet connections. Further details [here](https://clickhouse.com/docs/en/getting-started/example-datasets/uk-price-paid).\n\n```sql\nCREATE TABLE uk_price_paid\n(\n    price UInt32,\n    date Date,\n    postcode1 LowCardinality(String),\n    postcode2 LowCardinality(String),\n    type Enum8('terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4, 'other' = 0),\n    is_new UInt8,\n    duration Enum8('freehold' = 1, 'leasehold' = 2, 'unknown' = 0),\n    addr1 String,\n    addr2 String,\n    street LowCardinality(String),\n    locality LowCardinality(String),\n    town LowCardinality(String),\n    district LowCardinality(String),\n    county LowCardinality(String)\n)\nENGINE = MergeTree\nORDER BY (postcode1, postcode2, addr1, addr2);\n\nINSERT INTO uk_price_paid\nWITH\n   splitByChar(' ', postcode) AS p\nSELECT\n    toUInt32(price_string) AS price,\n    parseDateTimeBestEffortUS(time) AS date,\n    p[1] AS postcode1,\n    p[2] AS postcode2,\n    transform(a, ['T', 'S', 'D', 'F', 'O'], ['terraced', 'semi-detached', 'detached', 'flat', 'other']) AS type,\n    b = 'Y' AS is_new,\n    transform(c, ['F', 'L', 'U'], ['freehold', 'leasehold', 'unknown']) AS duration,\n    addr1,\n    addr2,\n    street,\n    locality,\n    town,\n    district,\n    county\nFROM url(\n    'http://prod.publicdata.landregistry.gov.uk.s3-website-eu-west-1.amazonaws.com/pp-complete.csv',\n    'CSV',\n    'uuid_string String,\n    price_string String,\n    time String,\n    postcode String,\n    a String,\n    b String,\n    c String,\n    addr1 String,\n    addr2 String,\n    street String,\n    locality String,\n    town String,\n    district String,\n    county String,\n    d String,\n    e String'\n) SETTINGS max_http_get_redirects=10;\n\n\n-- Create parameterized view for an example FDW visualization\nCREATE VIEW default.sold_by_duration AS\nSELECT\n    duration AS name,\n    count() AS value\nFROM default.uk_price_paid\nWHERE (duration != 'unknown') AND ((postcode1 = {_postcode:String}) OR (district = {_district:String}) OR (town = {_town:String}))\nGROUP BY duration\n\n```\n\n### 4. Generating Property listings data\n\nThis step requires the `uk_price_paid` data to be loaded into ClickHouse (see previous step).\n\n\nFrom the base directory. Adjust settings as required. Below assumes local instance.\n\n```\nexport CLICKHOUSE_HOST=localhost\nexport CLICKHOUSE_PORT=8123\nexport CLICKHOUSE_USER=default\nexport CLICKHOUSE_PASSWORD=password\nexport CLICKHOUSE_PROTO=http\nexport OPENAI_API_KEY=blah\n\n```\n\nIf you need to adjust the number of properties see [here](https://github.com/ClickHouse/house-prices/blob/main/scripts/generate_data.py#L14-L16). \nThis script could be improved - specifically, the [price to bedrooms extrapolation](https://github.com/ClickHouse/house-prices/blob/0681ed1c6862397a7b6d3e696cbfd7686ffe6c4b/scripts/generate_data.py#L29). Improvements welcome.\n\nThe following will generate a file `house_prices.csv` in the base directory. This process can be slow depending on your connection and speed of the OpenAI API. This step generates image links using the Bing API.\n\n```bash\npython scripts/generate_data.py\n```\n\n### 5. Setup Supabase tables\n\nExecute the following from your Supabase project SQL console. This will create the main `uk_house_listings` table and add a search index.\nIt also adds the Foreign Data Wrapper for ClickHouse and connects to the earlier created view.\n\n```sql\n\n-- create the table\nCREATE TABLE uk_house_listings\n(\n   id integer primary key,\n   date Date,\n   addr1 varchar(100),\n   addr2 varchar(100),\n   street varchar(60),\n   locality varchar(35),\n   town varchar(35),\n   district varchar(40),\n   county varchar(35),\n   postcode1 varchar(8),\n   postcode2 varchar(3),\n   type varchar(13),\n   duration varchar(9),\n   is_new SMALLINT,\n   price INTEGER,\n   rooms SMALLINT,\n   title text,\n   description text,\n   urls Text[],\n   sold boolean,\n   sold_date Date,\n   features text\n)\n\n-- add a search index\n\nalter table\n  uk_house_listings\nadd column\n  fts tsvector generated always as (to_tsvector('english', description || ' ' || title || ' ' || postcode1 || ' '|| postcode2)) stored;\n\ncreate index listings_fts on uk_house_listings using gin (fts); -- generate the index\n\n-- install clickhouse wrapper\ncreate extension if not exists wrappers;\n\ncreate foreign data wrapper clickhouse_wrapper\nhandler click_house_fdw_handler\nvalidator click_house_fdw_validator;\n\n-- modify the following to point to your ClickHouse instance\ncreate server clickhouse_server\nforeign data wrapper clickhouse_wrapper\noptions (\nconn_string 'tcp://default:\u003cpassword\u003e@\u003chost\u003e:9440/default?connection_timeout=30s\u0026ping_before_query=false\u0026secure=true'\n);\n\ncreate foreign table sold_by_duration (\n  name text,\n  value bigint,\n  postcode1 text, -- parameter column, used for input parameter,\n  district text,\n  town text\n)\nserver clickhouse_server\n  options (\n    table '(select * from sold_by_duration(_postcode=${postcode1},_district=${district}, _town=${town}))',\n    rowid_column 'duration'\n);\n\n```\n\n### 6. Loading data into Supabase\n\nWe will need a Supabase account (free tier is sufficient) or instance. Ensure you use a [Project API keys](https://supabase.com/docs/guides/api/api-keys) for this step which have write access. \nThe data should of been generated in the previous step and be present in the base directory as `house_prices.csv`.\n\nFrom the base directory:\n\n```bash\nexport SUPABASE_URL=url\nexport SUPABASE_PRIVATE_KEY=write_key\n\npython scripts/import_data.py\n```\n\n### 7. Running in Development mode\n\nFrom the `HouseClick` directory.\n\n**Important:** The Supabase key here should be the [anon key](https://supabase.com/docs/guides/api/api-keys. You should ensure row access security is configured and this key has only read access to the `uk_house_listings` table.\n\n```bash\nexport CLICKHOUSE_HOST=http://localhost:8123\nexport CLICKHOUSE_USER=default\nexport CLICKHOUSE_PASSWORD=password\n\nexport NEXT_PUBLIC_SUPABASE_URL=url\nexport NEXT_PUBLIC_SUPABASE_ANON_KEY=anon_key\n\nyarn dev\n```\n\nVisit [localhost:3000](http://localhost:3000)!","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclickhouse%2Fhouseclick","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fclickhouse%2Fhouseclick","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclickhouse%2Fhouseclick/lists"}