{"id":20532586,"url":"https://github.com/cimpress-mcp/j2v","last_synced_at":"2025-04-14T06:31:44.732Z","repository":{"id":50940965,"uuid":"179290197","full_name":"Cimpress-MCP/j2v","owner":"Cimpress-MCP","description":"Creates Looker Views and Explore based on provided JSON(s).","archived":false,"fork":false,"pushed_at":"2021-05-27T09:19:36.000Z","size":258,"stargazers_count":12,"open_issues_count":1,"forks_count":5,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-04-12T01:40:02.428Z","etag":null,"topics":["big-data","database","json","json-parser","looker","lookml","python3","snowflake","sql","sql-query"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Cimpress-MCP.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-04-03T12:56:35.000Z","updated_at":"2022-03-11T03:26:14.000Z","dependencies_parsed_at":"2022-08-25T13:51:37.963Z","dependency_job_id":null,"html_url":"https://github.com/Cimpress-MCP/j2v","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cimpress-MCP%2Fj2v","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cimpress-MCP%2Fj2v/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cimpress-MCP%2Fj2v/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cimpress-MCP%2Fj2v/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Cimpress-MCP","download_url":"https://codeload.github.com/Cimpress-MCP/j2v/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248833506,"owners_count":21168876,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["big-data","database","json","json-parser","looker","lookml","python3","snowflake","sql","sql-query"],"created_at":"2024-11-16T00:15:46.941Z","updated_at":"2025-04-14T06:31:44.710Z","avatar_url":"https://github.com/Cimpress-MCP.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![PyPI version](https://badge.fury.io/py/j2v.svg)](https://badge.fury.io/py/j2v) \n\n[![CI/CD](https://github.com/Cimpress-MCP/j2v/workflows/Test/badge.svg)](https://github.com/Cimpress-MCP/j2v/actions?query=workflow%3ATest)\n\n# JSONs to Looker views (J2V)\n\nJ2V is a simple command-line tool to convert JSON to [Looker](https://looker.com/) readable files in forms of [Looker Views](https://docs.looker.com/reference/view-params/view) and [Looker Explores](https://docs.looker.com/reference/explore-params/explore).\n\nAlso it outputs an SQL with proper paths and explosion expressions.\n\nThis is useful to be used in combination with databases that are focusing on schema-on-read, and data is stored in raw JSON instead of exploded into columns of a table or view.\n\n## Example use case\n\nYou have a table in your database. This table contains a column containing JSONs (one JSON per row). You are very curious how these data look like exploded, but you do not want to spend 2h going through the JSON structure and specifying all the fields just to surface them in Looker.\n\nWith J2V all the structures are discovered automatically and two files are generated - a Looker View and Looker Explore. All you need to do is copy/paste the output of this command line tool into your Looker project and you can start exploring.\n\n# Usage\n\n## Requirements\n\n[Python 3](https://www.python.org/downloads/) must be installed.\n\n## How to run\n* use code from github or\n* `pip install j2v`\n\n## Parameters\n\n* `json_files`: Files in JSON format, representing the data stored in a table\n* `output_view`: Name of Looker View output file to be created\n* `output_explore`: Name of Looker model output file to be created\n* `sql_table_name`: Name of the DB table to be used (this is only used in the LookML files; no actual connection to a database will be done as part of this tool)\n* `table_alias`: Name of the table alias \n* `column_name`: Name of the column in the DB table as specified in `sql_table_name`. (this is only used in the LookML files; no actual connection to a database will be done as part of this tool)\n* `primary_key`: Name of the primary key from JSON field\n* `sql_dialect`: Specifies the sql dialect of the output. [snowflake | bigquery]\n\n## Output\n\n* `output_view`: File containing definitions of Looker views (see [examples](./examples/) directory in this repository)\n* `output_explore`: File containing definition of looker explore exploding the structures (see [examples](./examples/) directory in this repository)\n\n## Example usage\n\n### Using all parameters\n\n`python main.py --json_files data1.json data2.json --output_view RESTAURANT_CHAIN --output_explore RESTAURANT_CHAIN --column_name DATA --sql_table_name RESTAURANT_DETAILS --table_alias chains_table --handle_null_values_in_sql true --primary_key apiVersion`\n### Using only mandatory parameters\n\n`python3 main.py --json_files order_example.json order_example2.json order_example3.json`\u003cbr /\u003e\n\n# Contribution\n\n## Project structure:\n\n* `j2v` - source code of a package\n* `examples` - working examples\n* `tests` - tests\n\n## Contribute\n\n1. If unsure, open an issue for a discussion\n1. Create a fork\n1. Make your change\n1. Make a pull request\n1. Happy contribution!\n\n## EXAMPLE\n\n### Input: \n```json\n{\n  \"apiVersion\": \"v3.4\",\n  \"data Provider\": \"Eat me\",\n  \"restaurants\": [\n    {\n      \"name\": \"Super Burger\",\n      \"city\": \"Sydney\",\n      \"country\": \"Australia\",\n      \"address\": \"Big Street 3\",\n      \"currency\": \"AUD\",\n      \"openTime\": 1571143824,\n      \"menu\": [\n        {\n          \"dishName\": \"BurgerPlus\",\n          \"price\": 10,\n          \"ingredients\": [\"Meat\", \"Cheese\", \"Bun\"]\n        }\n      ]\n    }\n  ],\n  \"headquarter\": {\n    \"employees\": 36,\n    \"city\": \"Olsztyn\",\n    \"country\": \"Poland\",\n    \"building\": {\n      \"address\": \"3 Maja 10\",\n      \"floors\": [1, 2, 7]\n    }\n  },\n  \"dataGenerationTimestamp\": \"2019-03-30T11:30:00.812Z\",\n  \"payloadPrimaryKeyValue\": \"3ab21b54-22d6-473c-b055-4430f8927d4c\",\n  \"version\": null\n}\n```\n\n### Ouput:\n\n#### SQL output:\n\n- Snowflake [Default]\n- BigQuery\n\n```SNOWFLAKE SQL\n\n ---VIEW WITH NUll VALUE HANDLING---\n\n\nSELECT\n---chains_table Information\nIFNULL(chains_table.\"DATA\":\"apiVersion\"::string,'N/A') AS API_VERSION,\nIFNULL(chains_table.\"DATA\":\"data Provider\"::string,'N/A') AS DATA_PROVIDER,\nIFNULL(chains_table.\"DATA\":\"headquarter\":\"building\":\"address\"::string,'N/A') AS HEADQUARTER_BUILDING_ADDRESS,\nIFNULL(chains_table.\"DATA\":\"headquarter\":\"city\"::string,'N/A') AS HEADQUARTER_CITY,\nIFNULL(chains_table.\"DATA\":\"headquarter\":\"country\"::string,'N/A') AS HEADQUARTER_COUNTRY,\nIFNULL(chains_table.\"DATA\":\"headquarter\":\"employees\"::number,0) AS HEADQUARTER_EMPLOYEES,\nIFNULL(chains_table.\"DATA\":\"payloadPrimaryKeyValue\"::string,'N/A') AS PAYLOAD_PRIMARY_KEY_VALUE,\nIFNULL(chains_table.\"DATA\":\"version\"::string,'N/A') AS VERSION,\nchains_table.\"DATA\":\"dataGenerationTimestamp\"::timestamp AS DATA_GENERATION_TIMESTAMP,\n---restaurants Information\nIFNULL(restaurants.VALUE:\"address\"::string,'N/A') AS RESTAURANTS_ADDRESS,\nIFNULL(restaurants.VALUE:\"city\"::string,'N/A') AS RESTAURANTS_CITY,\nIFNULL(restaurants.VALUE:\"country\"::string,'N/A') AS RESTAURANTS_COUNTRY,\nIFNULL(restaurants.VALUE:\"currency\"::string,'N/A') AS RESTAURANTS_CURRENCY,\nIFNULL(restaurants.VALUE:\"name\"::string,'N/A') AS RESTAURANTS_NAME,\nIFNULL(restaurants.VALUE:\"openTime\"::number,0) AS RESTAURANTS_OPEN_TIME,\n---restaurants_menu Information\nIFNULL(restaurants_menu.VALUE:\"dishName\"::string,'N/A') AS RESTAURANTS_MENU_DISH_NAME,\nIFNULL(restaurants_menu.VALUE:\"price\"::number,0) AS RESTAURANTS_MENU_PRICE,\n---restaurants_menu_ingredients Information\nIFNULL(restaurants_menu_ingredients.VALUE::string,'N/A') AS RESTAURANTS_MENU_INGREDIENTS_VALUE,\n---headquarter_building_floors Information\nIFNULL(headquarter_building_floors.VALUE::number,0) AS HEADQUARTER_BUILDING_FLOORS_VALUE\nFROM RESTAURANT_DETAILS AS chains_table,\nLATERAL FLATTEN(OUTER =\u003e TRUE, INPUT =\u003e chains_table.\"DATA\":\"restaurants\") restaurants,\nLATERAL FLATTEN(OUTER =\u003e TRUE, INPUT =\u003e restaurants.VALUE:\"menu\") restaurants_menu,\nLATERAL FLATTEN(OUTER =\u003e TRUE, INPUT =\u003e restaurants_menu.VALUE:\"ingredients\") restaurants_menu_ingredients,\nLATERAL FLATTEN(OUTER =\u003e TRUE, INPUT =\u003e chains_table.\"DATA\":\"headquarter\":\"building\":\"floors\") headquarter_building_floors\n```\n\n``` BIGQUERY SQL\n\n ---VIEW WITH NUll VALUE HANDLING---\nSELECT\n---chains_table Information\nIFNULL(chains_table.DATA.apiVersion,'N/A') AS API_VERSION,\nIFNULL(chains_table.DATA.data Provider,'N/A') AS DATA_PROVIDER,\nIFNULL(chains_table.DATA.headquarter.building.address,'N/A') AS HEADQUARTER_BUILDING_ADDRESS,\nIFNULL(chains_table.DATA.headquarter.city,'N/A') AS HEADQUARTER_CITY,\nIFNULL(chains_table.DATA.headquarter.country,'N/A') AS HEADQUARTER_COUNTRY,\nIFNULL(chains_table.DATA.headquarter.employees,0) AS HEADQUARTER_EMPLOYEES,\nIFNULL(chains_table.DATA.payloadPrimaryKeyValue,'N/A') AS PAYLOAD_PRIMARY_KEY_VALUE,\nIFNULL(chains_table.DATA.version,'N/A') AS VERSION,\nchains_table.DATA.dataGenerationTimestamp AS DATA_GENERATION_TIMESTAMP,\n---headquarter_building_floors Information\nIFNULL(headquarter_building_floors.,0) AS HEADQUARTER_BUILDING_FLOORS,\n---restaurants Information\nIFNULL(restaurants.address,'N/A') AS RESTAURANTS_ADDRESS,\nIFNULL(restaurants.city,'N/A') AS RESTAURANTS_CITY,\nIFNULL(restaurants.country,'N/A') AS RESTAURANTS_COUNTRY,\nIFNULL(restaurants.currency,'N/A') AS RESTAURANTS_CURRENCY,\nIFNULL(restaurants.name,'N/A') AS RESTAURANTS_NAME,\nIFNULL(restaurants.openTime,0) AS RESTAURANTS_OPEN_TIME,\n---restaurants_menu Information\nIFNULL(restaurants_menu.dishName,'N/A') AS RESTAURANTS_MENU_DISH_NAME,\nIFNULL(restaurants_menu.price,0) AS RESTAURANTS_MENU_PRICE,\n---restaurants_menu_ingredients Information\nIFNULL(restaurants_menu_ingredients.,'N/A') AS RESTAURANTS_MENU_INGREDIENTS\nFROM RESTAURANT_DETAILS AS chains_table\nLEFT JOIN UNNEST(chains_table.DATA.headquarter.building.floors) AS headquarter_building_floors\nLEFT JOIN UNNEST(chains_table.DATA.restaurants) AS restaurants\nLEFT JOIN UNNEST(restaurants.menu) AS restaurants_menu\nLEFT JOIN UNNEST(restaurants_menu.ingredients) AS restaurants_menu_ingredients\n```\n\n#### Ouput files:\n\n##### View file:\n\n```LookML\n\n\nview: chains_table { \n  sql_table_name: RESTAURANT_DETAILS ;;\n\n  dimension: address {\n    description: \"Address\"\n    type: string\n    sql: ${TABLE}.\"DATA\":\"headquarter\":\"building\":\"address\"::string ;;\n    group_label: \"Building\"\n  }\n    \n  dimension: api_version {\n    description: \"Api version\"\n    primary_key: yes\n    type: string\n    sql: ${TABLE}.\"DATA\":\"apiVersion\"::string ;;\n  }\n    \n  dimension: city {\n    description: \"City\"\n    type: string\n    sql: ${TABLE}.\"DATA\":\"headquarter\":\"city\"::string ;;\n    group_label: \"Headquarter\"\n  }\n    \n  dimension: country {\n    description: \"Country\"\n    type: string\n    sql: ${TABLE}.\"DATA\":\"headquarter\":\"country\"::string ;;\n    group_label: \"Headquarter\"\n  }\n    \n  dimension: data_provider {\n    description: \"Data provider\"\n    type: string\n    sql: ${TABLE}.\"DATA\":\"data Provider\"::string ;;\n  }\n    \n  dimension: employees {\n    description: \"Employees\"\n    type: number\n    sql: ${TABLE}.\"DATA\":\"headquarter\":\"employees\"::number ;;\n    group_label: \"Headquarter\"\n  }\n    \n  dimension: payload_primary_key_value {\n    description: \"Payload primary key value\"\n    type: string\n    sql: ${TABLE}.\"DATA\":\"payloadPrimaryKeyValue\"::string ;;\n  }\n    \n  dimension: version {\n    description: \"Version\"\n    type: string\n    sql: ${TABLE}.\"DATA\":\"version\"::string ;;\n  }\n    \n  dimension_group: data_generation_timestamp {\n    description: \"Data generation timestamp\"\n    type: time\n    timeframes: [\n        raw,\n        time,\n        date,\n        week,\n        month,\n        quarter,\n        year\n    ]\n    sql: ${TABLE}.\"DATA\":\"dataGenerationTimestamp\"::timestamp ;;\n  }\n    \n}\n\nview: restaurants { \n\n  dimension: address {\n    description: \"Address\"\n    type: string\n    sql: ${TABLE}.VALUE:\"address\"::string ;;\n  }\n    \n  dimension: city {\n    description: \"City\"\n    type: string\n    sql: ${TABLE}.VALUE:\"city\"::string ;;\n  }\n    \n  dimension: country {\n    description: \"Country\"\n    type: string\n    sql: ${TABLE}.VALUE:\"country\"::string ;;\n  }\n    \n  dimension: currency {\n    description: \"Currency\"\n    type: string\n    sql: ${TABLE}.VALUE:\"currency\"::string ;;\n  }\n    \n  dimension: name {\n    description: \"Name\"\n    type: string\n    sql: ${TABLE}.VALUE:\"name\"::string ;;\n  }\n    \n  dimension_group: open_time {\n    description: \"Open time\"\n    datatype: epoch\n    type: time\n    timeframes: [\n        raw,\n        time,\n        date,\n        week,\n        month,\n        quarter,\n        year\n    ]\n    sql: ${TABLE}.VALUE:\"openTime\"::number ;;\n  }\n    \n}\n\nview: restaurants_menu { \n\n  dimension: dish_name {\n    description: \"Dish name\"\n    type: string\n    sql: ${TABLE}.VALUE:\"dishName\"::string ;;\n  }\n    \n  dimension: price {\n    description: \"Price\"\n    type: number\n    sql: ${TABLE}.VALUE:\"price\"::number ;;\n  }\n    \n}\n\nview: restaurants_menu_ingredients { \n\n  dimension: value {\n    description: \"Value\"\n    type: string\n    sql: ${TABLE}.VALUE::string ;;\n  }\n    \n}\n\nview: headquarter_building_floors { \n\n  dimension: value {\n    description: \"Value\"\n    type: number\n    sql: ${TABLE}.VALUE::number ;;\n  }\n    \n}\n\n\n```\n\n##### Explore file:\n\n```LookML\n\ninclude: \"restaurant_chain.view.lkml\"\n   \nexplore: chains_table {\n  view_name: chains_table\n  from: chains_table\n  label: \"chains_table explore\"\n  description: \"chains_table explore\"\n\n  join: restaurants {\n     from: restaurants\n     sql:,LATERAL FLATTEN(OUTER =\u003e TRUE, INPUT =\u003e chains_table.\"DATA\":\"restaurants\") restaurants;;\n     relationship: one_to_many \n  }\n  \n  join: restaurants_menu {\n     from: restaurants_menu\n     sql:,LATERAL FLATTEN(OUTER =\u003e TRUE, INPUT =\u003e restaurants.VALUE:\"menu\") restaurants_menu;;\n     relationship: one_to_many \n     required_joins: [restaurants]\n  }\n  \n  join: restaurants_menu_ingredients {\n     from: restaurants_menu_ingredients\n     sql:,LATERAL FLATTEN(OUTER =\u003e TRUE, INPUT =\u003e restaurants_menu.VALUE:\"ingredients\") restaurants_menu_ingredients;;\n     relationship: one_to_many \n     required_joins: [restaurants_menu]\n  }\n  \n  join: headquarter_building_floors {\n     from: headquarter_building_floors\n     sql:,LATERAL FLATTEN(OUTER =\u003e TRUE, INPUT =\u003e chains_table.\"DATA\":\"headquarter\":\"building\":\"floors\") headquarter_building_floors;;\n     relationship: one_to_many \n  }\n  \n}\n\n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcimpress-mcp%2Fj2v","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcimpress-mcp%2Fj2v","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcimpress-mcp%2Fj2v/lists"}