Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/dave-nachman/awesome-ai-engineering

A list of resources for AI engineers
https://github.com/dave-nachman/awesome-ai-engineering

List: awesome-ai-engineering

ai-engineer ai-engineering awesome awesome-list

Last synced: about 1 month ago
JSON representation

A list of resources for AI engineers

Awesome Lists containing this project

README 

        
# Awesome AI Engineering



> A curated list of awesome resources for [AI engineers](https://www.latent.space/p/ai-engineer)



## Contents



- [Learning](#learning)

  - [Guides](#guides)

  - [Podcasts](#podcasts)

  - [Conference videos](#conference-videos)

  - [Talks](#talks)

  - [Articles](#articles)

  - [Survey papers](#survey-papers)

  - [Books](#books)

  - [Blogs](#blogs)

  - [Not quite "AI engineer" but relevant](#not-quite-ai-engineer-but-relevant)

  - [Related awesome lists](#related-awesome-lists)

- [Tools](#tools)

  - [Frameworks](#frameworks)

  - [Libraries](#libraries)

  - [LLM APIs](#llm-apis)

  - [Open models](#open-models)

  - [Observability / evals](#observability--evals)

  - [App building](#app-building)

  - [Inference](#inference)

  - [Fine-tuning](#fine-tuning)



## Learning



### Guides



- [Anthropic courses](https://github.com/anthropics/courses/tree/master)

- [LLM Bootcamp (Spring 2023)](https://fullstackdeeplearning.com/llm-bootcamp/) (from The Full Stack)

- [OpenAI Cookbook](https://cookbook.openai.com/)

- [Patterns for Building LLM-based Systems & Products](https://eugeneyan.com/writing/llm-patterns/) (from Eugene Yan, 2023)

- [Pinecone learning center](https://www.pinecone.io/learn/)

- [Prompt Engineering Guide](https://www.promptingguide.ai/)

- [RAG Techniques](https://github.com/NirDiamant/RAG_Techniques) (from Nir Diamant)

- [Vector databases](http://thedataquarry.com/posts/vector-db-1/) (four blog posts series from Prashanth Rao)

- [What We've Learned From A Year of Building with LLMs](https://applied-llms.org/) (from Applied LLMs)



### Podcasts



- [Gradient Dissent](https://wandb.ai/site/resources/podcast) (from Weights & Biases)

- [High Agency](https://humanloop.com/podcast) (from Humanloop)

- [Latent Space](https://www.latent.space/podcast)

- [No Priors](https://www.youtube.com/playlist?list=PLMKa0PxGwad7jf8hwwX8w5FHitXZ1L_h1)

- [Vanishing Gradients](https://vanishinggradients.fireside.fm/)



### Conference videos



- [AI Engineer Summit (2023)](https://www.ai.engineer/summit/2023)

- [AI Engineer World Fair (2024)](https://www.ai.engineer/worldsfair)

- [Mastering LLMs conference](https://parlance-labs.com/education/)



### Talks



- [A Hacker's Guide to Language Models](https://www.youtube.com/watch?v=jkrNMKz9pWU) (talk by Jeremy Howard)

- [The Brief History of AI Agents (2023-2024) ](https://www.youtube.com/watch?v=f9YleTc8AwE) (talk by swyz)

- [How to Construct Domain Specific LLM Evaluation Systems](https://www.youtube.com/watch?v=eLXF0VojuSs) (talk by Hamel Husain and Emil Sedgh)



### Articles



- [All the Hard Stuff Nobody Talks About when Building Products with LLMs](https://www.honeycomb.io/blog/hard-stuff-nobody-talks-about-llm) (from Honeycomb / Phillip Carter)

- [Evaluating the Effectiveness of LLM-Evaluators (aka LLM-as-Judge)](https://eugeneyan.com/writing/llm-evaluators/?utm_source=pocket_saves) (from Eugene Yan)

- [How to Interview and Hire ML/AI Engineers](https://eugeneyan.com/writing/how-to-interview/?utm_source=pocket_saves) (from Eugene Yan)

- [LLM Evaluation doesn't need to be complicated](https://www.philschmid.de/llm-evaluation) (from Phil Schmid)

- [LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/) (from Lilian Weng)

- [Observability for Large Language Models](https://www.oreilly.com/library/view/observability-for-large/9781098159757/) (from Phillip Carter; paywall)

- [Prompt Engineering](https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/) (from Lilian Weng)

- [Successful language model evals](https://www.jasonwei.net/blog/evals) (from Jason Wei)

- [The Rise of the AI Engineer](https://www.latent.space/p/ai-engineer) (from Swyx & Alessio Fanelli)

- [Your AI Product Needs Evals](https://hamel.dev/blog/posts/evals/) (from Hamel Husain)

- [What AI Engineers Should Know About Search ](https://softwaredoug.com/blog/2024/06/25/what-ai-engineers-need-to-know-search?utm_source=pocket_shared) (from Doug Turnbell)



### Survey papers



- [The Prompt Report: A Systematic Survey of Prompting Techniques](https://trigaten.github.io/Prompt_Survey_Site/)



### Books



- [AI Engineering](https://learning.oreilly.com/library/view/-/9781098166298/) (by Chip Huyen, Early Release)

- [Building LLMs for Production: Enhancing LLM Abilities and Reliability with Prompting, Fine-Tuning, and RAG](https://www.amazon.com/Building-LLMs-Production-Reliability-Fine-Tuning-ebook/dp/B0D3G58GDD) (by Louis-François Bouchard and Louie Peters)

- [Prompt Engineering for LLMs](https://learning.oreilly.com/library/view/prompt-engineering-for/9781098156145/) (by John Berryman & Albert Ziegler, Early Release)



### Blogs



- [Answer.AI](https://answer.ai)

- [Hamel Husain](https://hamel.dev/)

- [Lilian Weng](https://lilianweng.github.io/)

- [Simon Willison](https://simonwillison.net/)



### Not quite "AI engineer" but relevant



- [Build a Large Language Model (From Scratch)](https://www.manning.com/books/build-a-large-language-model-from-scratch) - book by Sebastian Raschka.

- [Building Recommendation Systems in Python and JAX](https://learning.oreilly.com/library/view/building-recommendation-systems/9781492097983/) (book by Bryan Bischof & Hector Yee)

- [GPU Mode Discord](https://discord.gg/gpumode) (formerly CUDA Mode)

- [GPU Mode YouTube videos](https://www.youtube.com/@CUDAMODE/featured) (formerly CUDA Mode)

- [Designing Machine Learning Systems](https://www.amazon.com/Designing-Machine-Learning-Systems-Production-Ready/dp/1098107969) (book by Chip Huyen)

- [fast.ai courses](http://fast.ai)

- [Neural Networks - Zero to Hero](https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ) (videos from Andrej Karpathy)



### Related awesome lists



- [Awesome AI engineering](https://github.com/boxabirds/awesome-ai-engineering)

- [Awesome AI engineering reads](https://github.com/aipengineer/awesome-ai-engineering-reads)

- [Awesome LLM planning and reasoning](https://github.com/samkhur006/awesome-llm-planning-reasoning?utm_source=pocket_shared)

- [Awesome LLM resources](https://github.com/marco-jeffrey/awesome-llm-resources)

- [Awesome RAG](https://github.com/frutik/Awesome-RAG)



## Tools



### Frameworks



- [LangChain](https://www.langchain.com/) - "LangChain is a framework for developing applications powered by large language models (LLMs)".

- [LlamaIndex](https://www.llamaindex.ai/) - "LlamaIndex is the leading data framework for building LLM applications".



### Libraries



- [Guardrails](https://github.com/guardrails-ai/guardrails) - "Adding guardrails to large language models".

- [LiteLLM](https://www.litellm.ai/) - "Call all LLM APIs using the OpenAI format".

- [Instructor](https://python.useinstructor.com/) - "Structured LLM Outputs".

- [Outlines](https://github.com/outlines-dev/outlines) - "Outlines provides ways to control the generation of language models to make their output more predictable".



### LLM APIs



- [Anthropic](https://docs.anthropic.com/en/docs/welcome)

- [Gemini (from Google)](https://ai.google.dev/gemini-api)

- [OpenAI](https://platform.openai.com/docs/concepts)



### Open models



- [Gemma](https://ai.google.dev/gemma)

- [Llama 3](https://llama.meta.com/docs/overview)



### Observability / evals



- [Evaluate](https://huggingface.co/docs/evaluate/en/index) (from HuggingFace) - "A library for easily evaluating machine learning models and datasets".

- [Langfuse](https://langfuse.com/) — "Traces, evals, prompt management and metrics to debug and improve your LLM application".

- [LangSmith](https://www.langchain.com/langsmith) - "LangSmith is an all-in-one developer platform for every step of the LLM-powered application lifecycle, whether you’re building with LangChain or not".

- [Inspect](https://inspect.ai-safety-institute.org.uk/) - "An open-source framework for large language model evaluations".

- [Weights & Biases Weave](https://wandb.ai/site/weave/) — "W&B Weave is here to help developers build and iterate on their AI applications with confidence."



### App building



- [fasthtml](https://github.com/AnswerDotAI/fasthtml) - "The fastest way to create an HTML app".

- [Gradio](https://www.gradio.app/) - "Build & Share Delightful Machine Learning Apps".

- [Streamlit](https://streamlit.io/) - "A faster way to build and share data apps".



### Inference



- [text-generation-inference from HuggingFace](https://github.com/huggingface/text-generation-inference) - "A Rust, Python and gRPC server for text generation inference. Used in production at Hugging Face to power Hugging Chat, the Inference API and Inference Endpoint".

- [vLLM](https://github.com/vllm-project/vllm) - "vLLM is a fast and easy-to-use library for LLM inference and serving".



### Fine-tuning



- [Axolotl](https://github.com/axolotl-ai-cloud/axolotl) - "Axolotl is a tool designed to streamline the fine-tuning of various AI models, offering support for multiple configurations and architectures".