Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/lux-org/lux

Automatically visualize your pandas dataframe via a single print! 📊 💡
https://github.com/lux-org/lux

data-science exploratory-data-analysis jupyter pandas python visualization visualization-tools

Last synced: 28 days ago
JSON representation

Automatically visualize your pandas dataframe via a single print! 📊 💡

Awesome Lists containing this project

README 

        


A Python API for Intelligent Visual Discovery





    

        Build Status

    

    

        PyPI version

    

    

        Documentation Status

    

    

        Slack

    

    

        Mailing List

    

    

        Binder

    

    

        CodeCov

    

    

        Twitter Follow

    




Lux is a Python library that facilitate fast and easy data exploration by automating the visualization and data analysis process. By simply printing out a dataframe in a Jupyter notebook, Lux recommends a set of visualizations highlighting interesting trends and patterns in the dataset. Visualizations are displayed via [an interactive widget](https://github.com/lux-org/lux-widget) that enables users to quickly browse through large collections of visualizations and make sense of their data.



Highlighted Visual Dataframe Workflow with Lux

     

Here is a [1-min video](https://www.youtube.com/watch?v=qmnYP-LmbNU) introducing Lux, and [slides](http://dorisjunglinlee.com/files/Zillow_07_2020_Slide.pdf) from a more extended talk.



Check out our [notebook gallery](https://lux-api.readthedocs.io/en/latest/source/reference/gallery.html) with examples of how Lux can be used with different datasets and analyses. 


Or try out Lux on your own in a [live Jupyter Notebook](https://mybinder.org/v2/gh/lux-org/lux-binder/master?urlpath=lab/tree/demo/hpi_covid_demo.ipynb)! 



# Getting Started



To start using Lux, simply add an extra import statement along with your Pandas import.



```python    

import lux

import pandas as pd

```



Lux can be used without modifying any existing Pandas code. Here, we use Pandas's [read_csv](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html) command to load in a [dataset of colleges](https://github.com/lux-org/lux-datasets/blob/master/data/college.csv) and their properties.



```python    

df = pd.read_csv("https://raw.githubusercontent.com/lux-org/lux-datasets/master/data/college.csv")

df

```

When the dataframe is printed out, Lux automatically recommends a set of visualizations highlighting interesting trends and patterns in the dataset.



Basic recommendations in Lux



Voila! Here's a set of visualizations that you can now use to explore your dataset further!



### Next-step recommendations based on user intent: 



In addition to dataframe visualizations at every step in the exploration, you can specify to Lux the attributes and values you're interested in. Based on this intent, Lux guides users towards potential next-steps in their exploration.



For example, we might be interested in the attributes `AverageCost` and `SATAverage`.



```python

df.intent = ["AverageCost","SATAverage"]

df

```

Next-step Recommendations Based on User Context

 

 The left-hand side of the widget shows the current visualization, i.e., the current visualization generated based on what the user is interested in. On the right, Lux generates three sets of recommendations, organized as separate tabs on the widget:



 - `Enhance` adds an additional attribute to the current selection, essentially highlighting how additional variables affect the relationship of `AverageCost` and `SATAverage`. We see that if we breakdown the relationship by `FundingModel`, there is a clear separation between public colleges (shown in red) and private colleges (in blue), with public colleges being cheaper to attend and with SAT average of lower than 1400.

    Enhance Recommendations

 - `Filter` adds a filter to the current selection, while keeping attributes (on the X and Y axes) fixed. These visualizations show how the relationship of  `AverageCost` and `SATAverage` changes for different subsets of data. For instance, we see that colleges that offer Bachelor's degree as its highest degree show a roughly linear trend between the two variables.

    Filter Recommendations

 - `Generalize` removes an attribute to display a more general trend, showing the distributions of `AverageCost` and `SATAverage` on its own. From the `AverageCost` histogram, we see that many colleges with average cost of around $20000 per year, corresponding to the bulge we see in the scatterplot view.

    Generalize Recommendations



 See [this page](https://lux-api.readthedocs.io/en/latest/source/guide/intent.html) for more information on additional ways for specifying the intent.



### Easy programmatic access and export of visualizations: 



Now that we have found some interesting visualizations through Lux, we might be interested in digging into these visualizations a bit more or sharing it with others. We can save the visualizations generated in Lux as a [static, shareable HTML](https://lux-api.readthedocs.io/en/latest/source/guide/export.html#exporting-widget-visualizations-as-static-html) or programmatically access these visualizations further in Jupyter. Selected `Vis` objects can be translated into [Altair](http://altair-viz.github.io/), [Matplotlib](https://matplotlib.org/), or [Vega-Lite](https://vega.github.io/vega-lite/) code, so that they can be further edited.



Easily exportable visualization object

     

Learn more about how to save and export visualizations [here](https://lux-api.readthedocs.io/en/latest/source/guide/export.html#exporting-widget-visualizations-as-static-html).



### Quick, on-demand visualizations with the help of automatic encoding: 

We've seen how `Vis`s are automatically generated as part of the recommendations. Users can also create their own Vis via the same syntax as specifying the intent. Lux is built on the philosophy that users should always be able to visualize anything they want, without having to think about *how* the visualization should look like. Lux automatically determines the mark and channel mappings based on a set of [best practices](http://hosteddocs.ittoolbox.com/fourshowmeautomaticpresentations.pdf). The visualizations are rendered via [Altair](https://github.com/altair-viz/altair/tree/master/altair) into [Vega-Lite](https://github.com/vega/vega-lite) specifications.



```python    

from lux.vis.Vis import Vis

Vis(["Region=New England","MedianEarnings"],df)

```    



Specified Visualization



### Powerful language for working with collections of visualizations:



Lux provides a powerful abstraction for working with collections of visualizations based on a partially specified queries. Users can provide a list or a wildcard to iterate over combinations of filter or attribute values and quickly browse through large numbers of visualizations. The partial specification is inspired by existing work on visualization query languages, including [ZQL](http://www.vldb.org/pvldb/vol10/p457-siddiqui.pdf) and [CompassQL](https://github.com/vega/compassql).



For example, we are interested in how the `AverageCost` distribution differs across different `Region`s.



```python    

from lux.vis.VisList import VisList

VisList(["Region=?","AverageCost"],df)

```    



Example Vis List



To find out more about other features in Lux, see the complete documentation on [ReadTheDocs](https://lux-api.readthedocs.io/).



# Installation & Setup



> **Note**: Lux's official package name is `lux-api` (not `lux`). After installing the package, remember to run the setup instructions for your notebook IDE, e.g., [jupyter notebook](#setup-in-jupyter-notebook-vscode-jupyterhub) and [jupyter lab](#setup-in-jupyter-lab).



To get started, please follow both the installation and setup instructions in your command line.

`lux-api` can be installed through [PyPI](https://pypi.org/project/lux-api/) or [conda-forge](https://github.com/conda-forge/lux-api-feedstock). 



```bash

pip install lux-api

```



If you use [conda](https://docs.conda.io/en/latest/), you can install `lux-api` via:



```bash

conda install -c conda-forge lux-api

```



Both the PyPI and conda installation include includes the Lux Jupyter widget frontend, [lux-widget](https://pypi.org/project/lux-widget/). 



## Setup in Jupyter Notebook, VSCode, JupyterHub



To use Lux with any Jupyter notebook-based frontends (e.g., [Jupyter notebook](https://github.com/jupyter/notebook), [JupyterHub](https://github.com/jupyterhub/jupyterhub), or [VSCode](https://code.visualstudio.com/docs/python/jupyter-support)), activate the notebook extension:



```bash

jupyter nbextension install --py luxwidget

jupyter nbextension enable --py luxwidget

```



If the installation happens correctly, you should see two `- Validating: OK` after executing the two lines above.

Note that you may have to restart the Jupyter Notebook server to ensure that the widget is displaying correctly.



## Setup in Jupyter Lab



Lux is compatible with both Jupyter Lab version 2 and 3. To use Lux in [Jupyter Lab](https://github.com/jupyterlab/jupyterlab), activate the lab extension:



```bash

jupyter labextension install @jupyter-widgets/jupyterlab-manager

jupyter labextension install luxwidget

```

Note that JupyterLab and VSCode is supported only for lux-widget version >=0.1.2, if you have an earlier version, please upgrade to the latest version of [lux-widget](https://pypi.org/project/lux-widget/). Lux has only been tested with the Chrome browser. 



If you encounter issues with the installation, please refer to [this page](https://lux-api.readthedocs.io/en/latest/source/guide/FAQ.html#troubleshooting-tips) to troubleshoot the installation. Follow [these instructions](https://lux-api.readthedocs.io/en/latest/source/getting_started/installation.html#manual-installation-dev-setup) to set up Lux for development purposes.



# Support and Resources



Lux is undergoing active development. If you are using Lux, we would love to hear from you! 



Drop us a note [here](https://docs.google.com/forms/d/e/1FAIpQLSe4YL2WsblqjB13FpkNsjbKGfHTM10KCgtjz9Yui2ZkiqErcQ/viewform?usp=sf_link) to share your experiences — any feedback, suggestions, and contributions are welcome!



**Links to additional resources:**



- Follow us on [Twitter](https://twitter.com/intent/follow?original_referer=https%3A%2F%2Fpublish.twitter.com%2F&ref_src=twsrc%5Etfw&screen_name=lux_api&tw_p=followbutton) or sign up to our [mailing list](https://forms.gle/XKv3ejrshkCi3FJE6) to stay tuned for upcoming releases and updates. 

- Visit [ReadTheDoc](https://lux-api.readthedocs.io/en/latest/) for more detailed documentation.

- Try out these hands-on [exercises](https://mybinder.org/v2/gh/lux-org/lux-binder/master?urlpath=lab/tree/exercise) or [tutorials](https://mybinder.org/v2/gh/lux-org/lux-binder/master?urlpath=lab/tree/tutorial) on [Binder](https://mybinder.org/v2/gh/lux-org/lux-binder/master?urlpath=lab/). Or clone and run [lux-binder](https://github.com/lux-org/lux-binder) locally.

- Join our community [Slack](https://communityinviter.com/apps/lux-project/lux) to discuss and ask questions.

- Check out [our paper](https://arxiv.org/pdf/2105.00121.pdf) for more details on how Lux works under the hoods.

- Report any bugs, issues, or requests through [Github Issues](https://github.com/lux-org/lux/issues).