{"id":18984434,"url":"https://github.com/timvw/adobe-analytics-datafeed-datasource","last_synced_at":"2025-04-19T20:23:29.405Z","repository":{"id":206369257,"uuid":"715101401","full_name":"timvw/adobe-analytics-datafeed-datasource","owner":"timvw","description":"Apache Spark data source for Adobe Analytics Data Feed","archived":false,"fork":false,"pushed_at":"2025-03-17T07:10:47.000Z","size":3216,"stargazers_count":3,"open_issues_count":9,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-29T12:51:20.634Z","etag":null,"topics":["adobe-analytics","clickstream","python","scala","spark"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/timvw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-06T13:35:25.000Z","updated_at":"2024-12-09T19:54:33.000Z","dependencies_parsed_at":"2023-11-15T10:28:33.566Z","dependency_job_id":"e1eb75f7-4d61-447a-8222-fa6072374ea3","html_url":"https://github.com/timvw/adobe-analytics-datafeed-datasource","commit_stats":null,"previous_names":["timvw/adobe-analytics-datafeed-datasource"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timvw%2Fadobe-analytics-datafeed-datasource","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timvw%2Fadobe-analytics-datafeed-datasource/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timvw%2Fadobe-analytics-datafeed-datasource/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timvw%2Fadobe-analytics-datafeed-datasource/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/timvw","download_url":"https://codeload.github.com/timvw/adobe-analytics-datafeed-datasource/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249250812,"owners_count":21237961,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adobe-analytics","clickstream","python","scala","spark"],"created_at":"2024-11-08T16:21:27.596Z","updated_at":"2025-04-19T20:23:29.358Z","avatar_url":"https://github.com/timvw.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Maven Central](https://maven-badges.herokuapp.com/maven-central/be.icteam/adobe-analytics-datafeed-datasource_2.12/badge.svg)](https://central.sonatype.com/artifact/be.icteam/adobe-analytics-datafeed-datasource_2.12)\n\n# Datasource for Adobe Analytics Data Feed\n\nAdobe Analytics [Data feeds](https://experienceleague.adobe.com/docs/analytics/export/analytics-data-feed/data-feed-contents/datafeeds-contents.html?lang=en) are a means to get raw data out of Adobe Analytics.\n\nThis project implements an [Apache Spark data source](https://spark.apache.org/docs/latest/sql-data-sources.html) leveraging [uniVocity TSV Parser](https://github.com/uniVocity/univocity-parsers/tree/master) and does not suffer from the flaws found in many online examples which treat the [(hit)data files](https://experienceleague.adobe.com/docs/analytics/export/analytics-data-feed/data-feed-contents/datafeeds-contents.html?lang=en#hit-data-files) as CSV.\nConcretly, escaped values are not handled correctly by a CSV parser due to inherent [differences between CSV and TSV](https://github.com/eBay/tsv-utils/blob/master/docs/comparing-tsv-and-csv.md).\n\n## Features\n\n* Correct handling of records which contain [special characters](https://experienceleague.adobe.com/docs/analytics/export/analytics-data-feed/data-feed-contents/datafeeds-spec-chars.html?lang=en)\n* Lookup values are replaced with their actual value in the [Lookup files](https://experienceleague.adobe.com/docs/analytics/export/analytics-data-feed/data-feed-contents/datafeeds-contents.html?lang=en#lookup-files)\n  * [Dynamic lookups](https://experienceleague.adobe.com/docs/analytics/export/analytics-data-feed/data-feed-contents/dynamic-lookups.html?lang=en) are supported as well\n* [Events](https://experienceleague.adobe.com/docs/analytics/implementation/vars/page-vars/events/events-overview.html?lang=en) are parsed as array of (key, value)\n* [Products](https://experienceleague.adobe.com/docs/analytics/implementation/vars/page-vars/products.html?lang=en) are parsed as product with name, category, quantity, price, events and evars.\n* Capability to filter found manifest files through:\n  * [Path Glob Filter](https://spark.apache.org/docs/latest/sql-data-sources-generic-options.html#path-glob-filter)\n  * [Modification Time Path Filters](https://spark.apache.org/docs/latest/sql-data-sources-generic-options.html#modification-time-path-filters)\n\nAll available options are here:\n[DatafeedOptions.scala](./src/main/scala/be/icteam/adobe/analytics/datafeed/DatafeedOptions.scala)\n\n## Usage\n\nMake sure the package is in the classpath, eg: by using the --packages option:\n\n```bash\nspark-shell --packages \"be.icteam:adobe-analytics-datafeed-datasource_2.12:$version\"\n```\n\nAnd you can read the feed as following:\n\n```scala\nval df = spark.read\n  .format(\"be.icteam.adobe.analytics.datafeed\")\n  .load(\"./src/test/resources/randyzwitch\")\n```\n\nHere is what it looks like:\n\n```scala\ndf.show(3, false)\n\n+------------------------------------------------------+----------------------------------+------------------+------------+---------------+------------------------+----------+------------------------+-----------+-----------+----------------+--------------------+-------------------+---------------+----------+-----+-----+-----+-----+-----+-----+-----+-----+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+-----------+------------------+-------------------------------------------------------------------+--------------------------+------------------+---------+-----------+-------+----------+--------+--------------+-----------------+-----------------+----------------------+---------+-------------------+------------------+-------------+------------+------------+-------------+----------------------+----------+----------+----------+----------+----------+----------+----------+----------+----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+----------+----------+----------+----------+----------+-------------+---------------+--------------------+--------------------+--------------------+---------------------------------------------------------------------+--------------------+--------------+---------------------------------------------------------------------+----------------------+--------------------------------------------------------------------+----------+----------+-------------+-----------------+-----------------------------------------------------------+------------+----------+----------+-----------+-----------+------------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+---------------+----------------------------------------------------------------------+------------------+----------+-------------------+-------------------+---------+----------+-------------+-------+-------------------------------------------------------------------------------------------------------------------------+--------------+---------+--------------+-------------------------------------+-------------------+--------------------+-------------------------------------------------------------------+--------------------+\n|post_event_list                                       |post_product_list                 |browser           |browser_type|connection_type|country                 |javascript|language                |os         |resolution |ref_type        |accept_language     |date_time          |domain         |evar1     |evar2|evar3|evar4|evar5|evar6|evar7|evar8|evar9|evar10|evar11|evar12|evar13|evar14|evar15|evar16|evar17|evar18|evar19|evar20|evar21|evar22|evar23|evar24|evar25|evar26|evar27|evar28|evar29|evar30|evar31|evar32|evar33|evar34|evar35|evar36|evar37|evar38|evar39|evar40|evar41|evar42|evar43|evar44|evar45|evar46|evar47|evar48|evar49|evar50|evar51|evar52|evar53|evar54|evar55|evar56|evar57|evar58|evar59|evar60|evar61|evar62|evar63|evar64|evar65|evar66|evar67|evar68|evar69|evar70|evar71|evar72|evar73|evar74|evar75|exclude_hit|first_hit_pagename|first_hit_page_url                                                 |first_hit_referrer        |first_hit_time_gmt|geo_city |geo_country|geo_dma|geo_region|geo_zip |ip            |last_hit_time_gmt|last_purchase_num|last_purchase_time_gmt|new_visit|post_browser_height|post_browser_width|post_campaign|post_channel|post_cookies|post_currency|post_cust_hit_time_gmt|post_evar1|post_evar2|post_evar3|post_evar4|post_evar5|post_evar6|post_evar7|post_evar8|post_evar9|post_evar10|post_evar11|post_evar12|post_evar13|post_evar14|post_evar15|post_evar16|post_evar17|post_evar18|post_evar19|post_evar20|post_evar21|post_evar22|post_evar23|post_evar24|post_evar25|post_evar26|post_evar27|post_evar28|post_evar29|post_evar30|post_evar31|post_evar32|post_evar33|post_evar34|post_evar35|post_evar36|post_evar37|post_evar38|post_evar39|post_evar40|post_evar41|post_evar42|post_evar43|post_evar44|post_evar45|post_evar46|post_evar47|post_evar48|post_evar49|post_evar50|post_evar51|post_evar52|post_evar53|post_evar54|post_evar55|post_evar56|post_evar57|post_evar58|post_evar59|post_evar60|post_evar61|post_evar62|post_evar63|post_evar64|post_evar65|post_evar66|post_evar67|post_evar68|post_evar69|post_evar70|post_evar71|post_evar72|post_evar73|post_evar74|post_evar75|post_hier1|post_hier2|post_hier3|post_hier4|post_hier5|post_keywords|post_page_event|post_page_event_var1|post_page_event_var2|post_page_event_var3|post_pagename                                                        |post_pagename_no_url|post_page_type|post_page_url                                                        |post_persistent_cookie|post_prop1                                                          |post_prop2|post_prop3|post_prop4   |post_prop5       |post_prop6                                                 |post_prop7  |post_prop8|post_prop9|post_prop10|post_prop11|post_prop12       |post_prop13|post_prop14|post_prop15|post_prop16|post_prop17|post_prop18|post_prop19|post_prop20|post_prop21|post_prop22|post_prop23|post_prop24|post_prop25|post_prop26|post_prop27|post_prop28|post_prop29|post_prop30|post_prop31|post_prop32|post_prop33|post_prop34|post_prop35|post_prop36|post_prop37|post_prop38|post_prop39|post_prop40|post_prop41|post_prop42|post_prop43|post_prop44|post_prop45|post_prop46|post_prop47|post_prop48|post_prop49|post_prop50|post_prop51|post_prop52|post_prop53|post_prop54|post_prop55|post_prop56|post_prop57|post_prop58|post_prop59|post_prop60|post_prop61|post_prop62|post_prop63|post_prop64|post_prop65|post_prop66|post_prop67|post_prop68|post_prop69|post_prop70|post_prop71|post_prop72|post_prop73|post_prop74|post_prop75|post_purchaseid|post_referrer                                                         |post_search_engine|post_state|post_visid_high    |post_visid_low     |post_zip |prev_page |ref_domain   |service|user_agent                                                                                                               |visit_keywords|visit_num|visit_page_num|visit_referrer                       |visit_search_engine|visit_start_pagename|visit_start_page_url                                               |visit_start_time_gmt|\n+------------------------------------------------------+----------------------------------+------------------+------------+---------------+------------------------+----------+------------------------+-----------+-----------+----------------+--------------------+-------------------+---------------+----------+-----+-----+-----+-----+-----+-----+-----+-----+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+-----------+------------------+-------------------------------------------------------------------+--------------------------+------------------+---------+-----------+-------+----------+--------+--------------+-----------------+-----------------+----------------------+---------+-------------------+------------------+-------------+------------+------------+-------------+----------------------+----------+----------+----------+----------+----------+----------+----------+----------+----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+----------+----------+----------+----------+----------+-------------+---------------+--------------------+--------------------+--------------------+---------------------------------------------------------------------+--------------------+--------------+---------------------------------------------------------------------+----------------------+--------------------------------------------------------------------+----------+----------+-------------+-----------------+-----------------------------------------------------------+------------+----------+----------+-----------+-----------+------------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+---------------+----------------------------------------------------------------------+------------------+----------+-------------------+-------------------+---------+----------+-------------+-------+-------------------------------------------------------------------------------------------------------------------------+--------------+---------+--------------+-------------------------------------+-------------------+--------------------+-------------------------------------------------------------------+--------------------+\n|[{Instance of eVar1, null}, {Instance of eVar2, null}]|[{null, , null, null, null, null}]|Safari 7.1        |Apple       |LAN/Wifi       |Commercial (mostly U.S.)|1.6       |English (United States) |OS X 10.9.5|1400 x 864 |Search Engines  |en-us               |2015-07-13 00:26:18|netvigator.com |logged-out|guest|null |null |null |null |null |null |null |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |0          |null              |http://randyzwitch.com/broken-macbook-pro-hinge-fixed-free/        |https://www.google.com.hk/|1436761578        |hong kong|hkg        |0      |no region |0       |219.77.75.182 |0                |0                |0                     |1        |687                |1347              |null         |null        |Y           |USD          |1436761578            |logged-out|guest     |null      |null      |null      |null      |null      |null      |null      |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null      |null      |null      |null      |null      |::empty::    |0              |null                |null                |null                |http://randyzwitch.com/broken-macbook-pro-hinge-fixed-free           |null                |null          |http://randyzwitch.com/broken-macbook-pro-hinge-fixed-free           |Y                     |Broken MacBook Pro Hinge? Apple will fix for free! | randyzwitch.com|1173      |post      |single-post  |technology       |apple,customer-service,genius-bar,macbook-pro              |Randy Zwitch|1         |2012      |06         |25         |June 25, 2012     |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null           |https://www.google.com.hk/                                            |557               |null      |2791471528899189638|791228704714081521 |::hash::0|0         |google.com.hk|ss     |Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/600.1.17 (KHTML, like Gecko) Version/7.1 Safari/537.85.10     |::empty::     |1        |1             |https://www.google.com.hk/           |557                |null                |http://randyzwitch.com/broken-macbook-pro-hinge-fixed-free/        |1436761578          |\n|[{Instance of eVar1, null}, {Instance of eVar2, null}]|[{null, , null, null, null, null}]|Google Chrome 43.0|Google      |LAN/Wifi       |Japan                   |1.6       |English (United States) |Windows 8.1|1280 x 800 |Search Engines  |en-US,en;q=0.8      |2015-07-13 00:56:09|aist.go.jp     |logged-out|guest|null |null |null |null |null |null |null |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |0          |null              |http://randyzwitch.com/rsitecatalyst-website-pathing-sankey-charts/|https://www.google.com/   |1436426719        |tsukuba  |jpn        |0      |08        |305-0005|150.29.149.177|1436754129       |0                |0                     |1        |777                |1293              |null         |null        |Y           |USD          |1436763369            |logged-out|guest     |null      |null      |null      |null      |null      |null      |null      |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null      |null      |null      |null      |null      |::empty::    |0              |null                |null                |null                |http://randyzwitch.com/rsitecatalyst-website-pathing-sankey-charts   |null                |null          |http://randyzwitch.com/rsitecatalyst-website-pathing-sankey-charts   |Y                     |Visualizing Website Pathing With Sankey Charts                      |3047      |post      |single-post  |digital-analytics|adobe-analytics,data-visualization,omniture,r,rsitecatalyst|Randy Zwitch|1         |2014      |09         |10         |September 10, 2014|7          |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null           |https://www.google.com/                                               |57                |null      |3037297388874966800|6917530475045353754|::hash::0|0         |google.com   |ss     |Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.132 Safari/537.36            |::empty::     |4        |1             |https://www.google.com/              |57                 |null                |http://randyzwitch.com/rsitecatalyst-website-pathing-sankey-charts/|1436763369          |\n|[{Instance of eVar1, null}, {Instance of eVar2, null}]|[{null, , null, null, null, null}]|Google Chrome 43.0|Google      |LAN/Wifi       |Network (mostly U.S.)   |1.6       |English (United States) |OS X 10.10 |1280 x 800 |Search Engines  |en-US,en;q=0.8      |2015-07-13 00:48:36|comcast.net    |logged-out|guest|null |null |null |null |null |null |null |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |null  |0          |null              |http://randyzwitch.com/hive-five-hard-won-lessons/                 |https://www.google.com/   |1435962984        |san jose |usa        |807    |ca        |95126   |50.136.222.167|1436200856       |0                |0                     |1        |777                |1197              |null         |null        |Y           |USD          |1436762916            |logged-out|guest     |null      |null      |null      |null      |null      |null      |null      |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null      |null      |null      |null      |null      |::empty::    |0              |null                |null                |null                |http://randyzwitch.com/hive-five-hard-won-lessons                    |null                |null          |http://randyzwitch.com/hive-five-hard-won-lessons                    |Y                     |Five Hard-Won Lessons Using Hive | randyzwitch.com                  |2680      |post      |single-post  |data-science     |big-data,hadoop,hive,python,r                              |Randy Zwitch|1         |2014      |06         |12         |June 12, 2014     |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null       |null           |https://www.google.com/                                               |57                |null      |3083707027358817578|6917535643501355093|::hash::0|0         |google.com   |ss     |Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.132 Safari/537.36|::empty::     |4        |1             |https://www.google.com/              |57                 |null                |http://randyzwitch.com/hive-five-hard-won-lessons/                 |1436762916          |\n```\n\n## Store in delta/iceberg format and be done with the madness:\n   \n```scala\nval df = spark.read\n  .format(\"be.icteam.adobe.analytics.datafeed\")\n  .option(ClickstreamOptions.MODIFIED_AFTER, checkpoint)\n  .load(\"s3://bucket/landing/feed\")\n\ndf.write.format(\"delta\").save(\"s3://bucket/conformed/feed\")\n```\n\n## Development\n\nPublish your own version in your local m2 repository:\n\n```bash\nsbt publishM2\n```\n\n## Releases\n\nThis project leverages [sbt-ci-release](https://github.com/sbt/sbt-ci-release) to create and publish to Sonatype and Maven Central from GitHub Actions.\n\nCreate and push the appropiate tag (vX.Y.Z) and [ci.yml](./github/workflows/ci.yml) will make sure a release is built\n```bash\ngit tag v0.1.0\ngit push --tags\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimvw%2Fadobe-analytics-datafeed-datasource","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftimvw%2Fadobe-analytics-datafeed-datasource","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimvw%2Fadobe-analytics-datafeed-datasource/lists"}