https://github.com/pchampio/egc-2020
:mag: :chart_with_upwards_trend: Using techniques of knowledge discovery and text mining the goal is to explain the structure and the evolution of the EGC community
https://github.com/pchampio/egc-2020
data-mining knowledge-discovery univ-lemans
Last synced: 3 months ago
JSON representation
:mag: :chart_with_upwards_trend: Using techniques of knowledge discovery and text mining the goal is to explain the structure and the evolution of the EGC community
- Host: GitHub
- URL: https://github.com/pchampio/egc-2020
- Owner: pchampio
- Created: 2018-10-19T12:33:31.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2018-11-22T08:52:27.000Z (over 6 years ago)
- Last Synced: 2025-02-02T17:18:28.386Z (5 months ago)
- Topics: data-mining, knowledge-discovery, univ-lemans
- Language: Jupyter Notebook
- Homepage: http://www.egc.asso.fr/manifestations/defi-egc/defi-egc-2020-20-ans-dhistoire-pour-quel-avenir.html
- Size: 18.1 MB
- Stars: 2
- Watchers: 4
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# EGC_2020
EGC 2020 Challenge: 20 years of history for which future?
The goal of this challenge is to take stock at the evolution of the EGC community over the past 20
years and try to predict the future. The principle is to apply techniques of knowledge discovery and
data mining to explain the structure and evolution.
## Dataset
The data set consists of 1200 titles and abstracts from the articles published at the EGC conference between 2004 and 2018.
Fields:
- years
- title
- abstract
- authors
## Pipeline
- filter_extreme
- tf-idf
- LDA (Coherence Score)
- K-Means (Silhouette scores)
## Cluster (Topics) evolution / Time
Our system deducted a sharp increase in articles related to the social network analysis over the past years 20 (Label 1).
On the other hand, rule-based algorithms seem to have declined drastically (Label 6).
## Evaluation (Hyper-parameters defined in the [Jupiter-notebook](./EGC.ipynb))
The pipeline used in this project doesn't seem to find a lot of structure for one cluster (Label 9), sadly this cluster represents ~30% of our training data (Silhouette plot below).
Silhouette plot for 10 clusters
#### There is still room for improvement.