https://github.com/lamastex/spark-gdelt-examples

Example applications of GDELT mass media intelligence data
https://github.com/lamastex/spark-gdelt-examples

apache-spark delta-lake events-of-interest finance gas gdelt gdelt-data gdelt-events mass-media oil persons-of-interest time-series trends

Last synced: 6 months ago
JSON representation

Example applications of GDELT mass media intelligence data

• Host: GitHub
• URL: https://github.com/lamastex/spark-gdelt-examples
• Owner: lamastex
• License: apache-2.0
• Created: 2020-08-21T09:10:53.000Z (about 5 years ago)
• Default Branch: master
• Last Pushed: 2022-01-25T16:29:16.000Z (over 3 years ago)
• Last Synced: 2025-03-27T21:52:10.232Z (6 months ago)
• Topics: apache-spark, delta-lake, events-of-interest, finance, gas, gdelt, gdelt-data, gdelt-events, mass-media, oil, persons-of-interest, time-series, trends
• Language: HTML
• Homepage: https://lamastex.github.io/spark-gdelt-examples/
• Size: 27.7 MB
• Stars: 2
• Watchers: 1
• Forks: 3
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Spark GDELT Examples

## Detecting Events and Entities of Interest about Trend in OIL Price

![oiltrendsevents](notebooks/db/images/oiltrendsevents.png)

Johannes Graner, Albert Nilsson and Raazesh Sainudiin

2020, Uppsala, Sweden

This project was supported by Combient Mix AB through summer internships at:

Combient Competence Centre for Data Engineering Sciences, 

Department of Mathematics, 

Uppsala University, Uppsala, Sweden

See Example notebooks to detect events and persons or entities of interest

- [notebooks/ScaDaMaLe/000b-gdelt-utils](notebooks/ScaDaMaLe/000b_gdelt_utils.md)

- [notebooks/db/gdelt-EOI-detection](notebooks/ScaDaMaLe/030a_gdelt_EOI_detection.md)

- [notebooks/db/gdelt-POI-detection](notebooks/ScaDaMaLe/030b_gdelt_POI_detection.md)

## Investigating events in Swedish media

![sweden_goldstein](notebooks/db/images/sweden_goldstein.png)

![cameo_sweden](notebooks/db/images/cameo_sweden.png)

Yaser Kaddoura, Mohammad Mustakim Ur Rahman, and Raazesh Sainudiin

2021, Uppsala, Sweden

This project was supported by Combient Mix AB through Data Science Project Fellowships between 2021-10 and 2021-12 to Yaser Kaddoura and Mohammad Mustakim Ur Rahman at Combient Competence Centre for Data Engineering Sciences, Department of Mathematics, Uppsala University, Uppsala, Sweden, and databricks University Alliance with infrastructure credits from AWS.

Combient Competence Centre for Data Engineering Sciences, Department of Mathematics, Uppsala University, Uppsala, Sweden

See Example notebooks:

- [notebooks/db/2022_00_GDELT_base](notebooks/db/2022_00_GDELT_base.md)

- [notebooks/db/2022_00a_GDELT_download_CAMEO](notebooks/db/2022_00a_GDELT_download_CAMEO.md) 

- [notebooks/db/2022_00b_GDELT_download_csv](notebooks/db/2022_00b_GDELT_download_csv.md) 

- [notebooks/db/2022_01_GDELT_main](notebooks/db/2022_01_GDELT_main.md)

- [notebooks/db/2022_02a_GDELT_extract_shooting_articles](notebooks/db/2022_02a_GDELT_extract_shooting_articles.md)

- [notebooks/db/2022_02b_GDELT_scraping_Swedish_articles](notebooks/db/2022_02b_GDELT_scraping_Swedish_articles.md)

# Resources

This builds on the following libraries and its antecedents therein:

- [https://github.com/aamend/spark-gdelt](https://github.com/aamend/spark-gdelt)

- [https://github.com/lamastex/spark-trend-calculus](https://github.com/lamastex/spark-trend-calculus)

## This work was inspired by:

- Antoine Aamennd's [texata-2017](https://github.com/aamend/texata-r2-2017)

- Andrew Morgan's [Trend Calculus Library](https://github.com/ByteSumoLtd/TrendCalculus-lua)

---

---

[edit on GitHub](https://github.com/lamastex/spark-gdelt-examples/edit/master/README.md)