Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/pengfeng/ask.py

A simple Python program to implement the search-extract-summarize flow.
https://github.com/pengfeng/ask.py

agent ai-search llm perplexity python rag

Last synced: 4 months ago
JSON representation

A simple Python program to implement the search-extract-summarize flow.

Awesome Lists containing this project

README 

          
---

title: ask.py

app_file: ask.py

sdk: gradio

sdk_version: 5.3.0

---



[![License](https://img.shields.io/github/license/pengfeng/ask.py)](LICENSE)



- [🚀 **Updates!** 🚀](#-updates-)

- [Introduction](#introduction)

  - [Demo use cases](#demo-use-cases)

  - [The search-extract-summarize flow](#the-search-extract-summarize-flow)

- [Quick start](#quick-start)

- [Use Different LLM Endpoints](#use-different-llm-endpoints)

  - [Use local Ollama inference and embedding models](#use-local-ollama-inference-and-embedding-models)

  - [Use DeepSeek API inference with OpenAI embedding models](#use-deepseek-api-inference-with-openai-embedding-models)

- [GradIO Deployment](#gradio-deployment)

- [Community](#community)



# 🚀 **Updates!** 🚀



A full version with db support and configurable components is open sourced here:

[LeetTools](https://github.com/leettools-dev/leettools). Please check it out!



We also added support for local Ollama inference and embedding models, as well as for other API

providers such as DeepSeek. Please see the [`Use Different LLM Endpoints`](#use-different-llm-endpoints) secton for more details.



> [UPDATE]

> - 2025-01-20: add support for separate API endpoints for inference and embedding

> - 2025-01-20: add support for .env file switch and Ollama example

> - 2025-01-20: add support for default search proxy

> - 2024-12-20: add the full function version link

> - 2024-11-20: add Docling converter and local mode to query against local files

> - 2024-11-10: add Chonkie as the default chunker

> - 2024-10-28: add extract function as a new output mode

> - 2024-10-25: add hybrid search demo using DuckDB full-text search

> - 2024-10-22: add GradIO integation

> - 2024-10-21: use DuckDB for the vector search and use API for embedding

> - 2024-10-20: allow to specify a list of input urls

> - 2024-10-18: output-language and output-length parameters for LLM

> - 2024-10-18: date-restrict and target-site parameters for seach



# Introduction



A single Python program to implement the search-extract-summarize flow, similar to AI search

engines such as Perplexity.



- You can run it with local Ollama inference and embedding models.

- You can run it on command line or with a GradIO UI.

- You can control the output behavior, e.g., extract structured data or change output language,

- You can control the search behavior, e.g., restrict to a specific site or date, or just scrape

  a specified list of URLs.

- You can run it in a cron job or bash script to automate complex search/data extraction tasks.

- You can ask questions against local files.



We have a running UI example [in HuggingFace Spaces](https://huggingface.co/spaces/LeetTools/ask.py).



![image](https://github.com/user-attachments/assets/0483e6a2-75d7-4fbd-813f-bfa13839c836)



## Demo use cases



- [Search like Perplexity](demos/search_and_answer.md)

- [Only use the latest information from a specific site](demos/search_on_site_and_date.md)

- [Extract information from web search results](demos/search_and_extract.md)

- [Ask questions against local files](demos/local_files.md)

- [Use Ollama local LLM and Embedding models](demos/run_with_ollama.md)



> [!NOTE]

>

> - Our main goal is to illustrate the basic concepts of AI search engines with the raw constructs.

>   Performance or scalability is not in the scope of this program.

> - We are planning to open source a real search-enabled AI toolset with real DB setup, real document

>   pipeline, and real query engine soon. Star and watch this repo for updates!



## The search-extract-summarize flow



Given a query, the program will



- in search mode: search Google for the top 10 web pages

- in local mode: use the local files under the 'data' directory

- crawl and scape the result documents for their text content

- chunk the text content into chunks and save them into a vectordb

- perform a hybrid search (vector and BM25 FTS) with the query and find the top 10 matched chunks

- [Optional] use a reranker to re-rank the top chunks

- use the top chunks as the context to ask an LLM to generate the answer

- output the answer with the references



Of course this flow is a very simplified version of the real AI search engines, but it is a good starting point to understand the basic concepts.



One benefit is that we can manipulate the search function and output format.



For example, we can:



- search with date-restrict to only retrieve the latest information.

- search within a target-site to only create the answer from the contents from it.

- ask LLM to use a specific language to answer the question.

- ask LLM to answer with a specific length.

- crawl a specific list of urls and answer based on those contents only.



This program can serve as a playground to understand and experiment with different components in

the pipeline.



# Quick start



```bash

# We recommend using uv as the virtual environment manager

# First install uv if you haven't:

% curl -LsSf https://astral.sh/uv/install.sh | sh



# Create a new virtual environment and install dependencies

% uv venv

% source .venv/bin/activate  # On Windows use: .venv\Scripts\activate

% uv pip install -e .



# Alternatively, if you prefer not to install in editable mode, you can use:

% uv pip install .



# modify .env file to set the API keys or export them as environment variables as below



# you need to set the Google search API

% export SEARCH_API_KEY="your-google-search-api-key"

% export SEARCH_PROJECT_KEY="your-google-cx-key"



# right now we use OpenAI API, default using OpenAI

# % export LLM_BASE_URL=https://api.openai.com/v1

% export LLM_API_KEY=



# By default, the program will start a web UI. See GradIO Deployment section for more info.

# Run the program on command line with -c option

% python ask.py -c -q "What is an LLM agent?"



# You can also query your local files under the 'data' directory using the local mode

% python ask.py -i local -c -q "How does Ask.py work?"



# we can specify more parameters to control the behavior such as date_restrict and target_site

% python ask.py --help

Usage: ask.py [OPTIONS]



  Search web for the query and summarize the results.



Options:

  -q, --query TEXT                Query to search

  -i, --input-mode [search|local]

                                  Input mode for the query, default is search.

                                  When using local, files under 'data' folder

                                  will be used as input.

  -o, --output-mode [answer|extract]

                                  Output mode for the answer, default is a

                                  simple answer

  -d, --date-restrict INTEGER     Restrict search results to a specific date

                                  range, default is no restriction

  -s, --target-site TEXT          Restrict search results to a specific site,

                                  default is no restriction

  --output-language TEXT          Output language for the answer

  --output-length INTEGER         Output length for the answer

  --url-list-file TEXT            Instead of doing web search, scrape the

                                  target URL list and answer the query based

                                  on the content

  --extract-schema-file TEXT      Pydantic schema for the extract mode

  --inference-model-name TEXT     Model name to use for inference

  --vector-search-only            Do not use hybrid search mode, use vector

                                  search only.

  -c, --run-cli                   Run as a command line tool instead of

                                  launching the Gradio UI

  -e, --env TEXT                  The environment file to use, absolute path

                                  or related to package root.

  -l, --log-level [DEBUG|INFO|WARNING|ERROR]

                                  Set the logging level  [default: INFO]

  --help                          Show this message and exit.

```



# Use Different LLM Endpoints



## Use local Ollama inference and embedding models

We can run Ask.py with different env files to use different LLM endpoints and other

related settings. For example, if you have a local Ollama serving instance, you can set

to use it as follows:



```bash

# you may need to pull the models first

% ollama pull llama3.2

% ollama pull nomic-embed-text

% ollama serve



% cat > .env.ollama < .env.deepseek <

DEFAULT_INFERENCE_MODEL=deepseek-chat



EMBED_BASE_URL=https://api.openai.com/v1

EMBED_API_KEY=

EMBEDDING_MODEL=text-embedding-3-small

EMBEDDING_DIMENSIONS=1536

EOF



% python ask.py -e .env.deepseek -c -q "How does DeepSeek work?"

```



# GradIO Deployment



> [!NOTE]

> Original GradIO app-sharing document [here](https://www.gradio.app/guides/sharing-your-app).



**Quick test and sharing**



By default, the program will start a web UI and share through GradIO.



```bash

% python ask.py

* Running on local URL:  http://127.0.0.1:7860

* Running on public URL: https://77c277af0330326587.gradio.live



# you can also specify SHARE_GRADIO_UI to only run locally

% export SHARE_GRADIO_UI=False

% python ask.py

* Running on local URL:  http://127.0.0.1:7860

```



**To share a more permanent link using HuggingFace Spaces**



- First, you need to [create a free HuggingFace account](https://huggingface.co/welcome).

- Then in your [settings/token page](https://huggingface.co/settings/tokens), create a new token with Write permissions.

- In your terminal, run the following commands in you app directory to deploy your program to

  HuggingFace Spaces:



```bash

% pip install gradio

% gradio deploy

Creating new Spaces Repo in '/home/you/ask.py'. Collecting metadata, press Enter to accept default value.

Enter Spaces app title [ask.py]: ask.py

Enter Gradio app file [ask.py]:

Enter Spaces hardware (cpu-basic, cpu-upgrade, t4-small, t4-medium, l4x1, l4x4, zero-a10g, a10g-small, a10g-large, a10g-largex2, a10g-largex4, a100-large, v5e-1x1, v5e-2x2, v5e-2x4) [cpu-basic]:

Any Spaces secrets (y/n) [n]: y

Enter secret name (leave blank to end): SEARCH_API_KEY

Enter secret value for SEARCH_API_KEY: YOUR_SEARCH_API_KEY

Enter secret name (leave blank to end): SEARCH_PROJECT_KEY

Enter secret value for SEARCH_API_KEY: YOUR_SEARCH_PROJECT_KEY

Enter secret name (leave blank to end): LLM_API_KEY

Enter secret value for LLM_API_KEY: YOUR_LLM_API_KEY

Enter secret name (leave blank to end):

Create Github Action to automatically update Space on 'git push'? [n]: n

Space available at https://huggingface.co/spaces/your_user_name/ask.py

```



Now you can use the HuggingFace space app to run your queries.



# Community



**License and Acknowledgment**



The source code is licensed under MIT license. Thanks for these amazing open-source projects and API

providers:



- [Google Search API](https://developers.google.com/custom-search/v1/overview)

- [OpenAI API](https://beta.openai.com/docs/api-reference/completions/create)

- [Jinja2](https://jinja.palletsprojects.com/en/3.0.x/)

- [bs4](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)

- [DuckDB](https://github.com/duckdb/duckdb)

- [Docling](https://github.com/DS4SD/docling)

- [GradIO](https://github.com/gradio-app/gradio)

- [Chonkie](https://github.com/bhavnicksm/chonkie)