Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/msalemor/sk-dev-training

Semantic Kernel Demos
https://github.com/msalemor/sk-dev-training

Last synced: 7 days ago
JSON representation

Semantic Kernel Demos

Host: GitHub
URL: https://github.com/msalemor/sk-dev-training
Owner: msalemor
Created: 2023-07-18T23:33:22.000Z (over 1 year ago)
Default Branch: main
Last Pushed: 2024-09-17T04:49:05.000Z (3 months ago)
Last Synced: 2024-10-14T19:02:40.466Z (about 2 months ago)
Language: Jupyter Notebook
Size: 257 KB
Stars: 6
Watchers: 2
Forks: 5
Open Issues: 0
Metadata Files:
- Readme: README.md

Awesome Lists containing this project

awesome-semantickernel - openai-sk-demos

README

# Introduction to Semantic Kernel Development

## About this repo

This repo includes C# interactive notebooks at `notebooks/` that showcase how to run scenarios with and without Semantic Kernal. The idea is to showcase the differences between writing LLM/Chatbot apps with and without Semantic Kernel. The samples include:

- `notebooks/cars.ipynb`: A sales description generator for used cars implemented without Semantic Kernel.
- `notebooks/sk-cars.ipynb`: A sales description generator for used cars implemented with Semantic Kernel.
- `notebooks/chat.ipynb`: A GPT chatbot implementation without Semantic Kernel.
- `notebooks/sk-chat.ipynb`: A GPT chatbot implementation with Semantic Kernel.
- `notebooks/sk-pipes.ipynb`: Showcases how to use Semantic Kernel pipes.
- **Note:** This code comes from the original dotnet/SK solution. I just have adapted it to run in C# interactive.
- `notebooks/sk-planner.ipynb`: Showcases how to use the Semantic Kernel sequential planner.
- **Note:** This code comes from the original dotnet/SK solution. I just have adapted it to run in C# interactive.
- `notebooks/sk-memory.ipynb`: Demonstrates how to use the Semantic Kernel memories functionality.
- **Note:** This code comes from the original dotnet/SK solution. I just have adapted it to run in C# interactive.
- `notebooks/sk-memory-rag.ipynb`: Demonstrate a RAG pattern implementation using Semantic Kernel volatile memory. Useful to understand the concepts of this pattern.
- `notebooks/sk-memory-raasg.ipynb`: Demonstrate a RAASG pattern implementation using Semantic Kernel. Useful to understand the concepts of this pattern.

**Note:** I am using C# interactive notebooks because I find them helpful for documenting, explaining, and debugging the different concepts. In productions applications, I have deployed c# projects with WebAPIs and React frontends.

## Running the demos

Requirements:

- .NET 6 or 7 SDK
- VS Code
- VS Code Polyglot extension

Running:

- In Azure create OpenAI endpoints for:
- GPT 3.5 or 4
- ADA Embeddings
- Copy the deployment name, endpoint, and API key information
- Clone this repo
- Navigate to the notebooks folder within this repo
- Make sure the VS Code is installed as well as the VS Code Polyglot extension and either .NET 6 or 7
- Create a file called: `notebooks/.env` and add the following values from

```bash
GPT_OPENAI_KEY=
GPT_OPENAI_DEPLOYMENT_NAME=
GPT_OPENAI_ENDPOINT=https://.openai.azure.com/
GPT_OPENAI_FULL_ENDPOINT=https://.openai.azure.com/openai/deployments//chat/completions?api-version=2023-03-15-preview

DAVINCI_OPENAI_KEY=
DAVINCI_OPENAI_DEPLOYMENT_NAME=
DAVINCI_OPENAI_ENDPOINT=https://.openai.azure.com/
DAVINCI_OPENAI_FULL_ENDPOINT=https://.openai.azure.com/openai/deployments//completions?api-version=2022-12-01
```

- Type: 'code .'
- Open any one of the notebooks
- Click the play button at each cell and review the headings and comments

## What are some important concepts in LLM/Chatbot development

* Understanding that generative AI is a Foundational model
* A foundational model can be used to solve many problems
* Understanding the difference between LLM/Chatbot model
* An LLM (Davinci) model is for answer/response
* A chat (GPT) model can keep a conversation, but it can also be used in answer/response
* Understanding tokens, token limits, and working around these limits
* Understanding the prompt and completion models
* Understanding and applying prompt engineering

### This is how simple a GPT call looks like

```text
Summarize the following text:

Text: """
{text input here}
"""
```

You don't even have to write a program, you can just `curl`:

```text
curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT_NAME/completions?api-version=2023-05-15\
-H "Content-Type: application/json" \
-H "api-key: YOUR_API_KEY" \
-d "{
\"prompt\": \"What is the speed of light?\",
\"max_tokens\": 5
}"
```

### So where is the complexity then? It is everything else.

- For example:
- As a foundational model that can solve many problems, applying this concept is complex in itself.
- Making resilient applications that can handle disconnections and throttling
- Developing complex orchestrations
- Preparing the prompts and applying prompt engineering (Grounding)
- Saving and recalling text from embeddings, for example in the RAG pattern
- Processing the completions

**Semantic Kernel can help with everything else.**

#### Resiliency considerations and recommendations

- Azure GPT models have two kinds of throttling:
- By token per minute (TPM)
- By Number of requests per minute
- When throttling occurs, the API will return a RetryAfter (in seconds) in the header
- Set the number of tokens as the number of tokens that you are expecting (not how many you will send)
- Consider using (currently as of 11/2023) GPT 3.5 Turbo over GPT 4. GPT 3.5 is faster and offers a higher TPM
- Consider load balancing between two accounts
- Handle retries at the client application and consider the RetryAfter leveraging the timeout in the RetryAfter header

## What is Semantic Kernel?

- Is an open-source SDK that provides a set of connectors and plugins that allow you to orchestrate AI models and skills with natural language semantic functions, traditional native code functions, and embeddings-based memory.
- It supports prompt templating, function chaining, vectorized memory, and intelligent planning capabilities.
- It enables you to create AI apps that combine the best of both worlds: natural language understanding and conventional programming logic

## Where does SK land in the development of LLM/Chatbot applications?

```mermaid
graph LR;
A(Basic
Development) --> B(Development
with SDKs
_____________

SK
lands here)
B-->C(Application Patterns
and Architectures)
C-->D(Applications &
Solutions)
D-->E(WAF
Deployment)
classDef someclass fill:blue,color:white
class B someclass

```

## Advantages of using SK vs making REST calls or using other SDKs

- Kernel can be configured via plugins that are implemented by interfaces.
- Resilient HttpClient client that can handle timeouts and throttling.
- Note: HttpClient in the main network object to perform Http operations in many languages.
- Semantic functions.
- Generally, an SK function is defined as a templated prompt.
- ie: `var skfJokeDefinition = "Write a joke about {{$$input}}.";`
- ie: `var skSpanishTranslator = "Translate the following text from English to Spanish:\n\n\nText:"""\n{{$$input}}\n"""";`
- SK functions can be piped together:
- ie:

```c#
var result = await kernel.RunAsync("a chicken crossing the road",
skfJokeGenerator,
skfSpanishTranslator,
text["uppercase"]);
```

- Output: `¿POR QUÉ CRUZÓ EL POLLO LA CARRETERA? PORQUE QUERÍA VER SI LA HIERBA ERA MÁS VERDE AL OTRO LADO. PERO RESULTA QUE ERA ARTIFICIAL Y SE QUEDÓ ATASCADO EN EL CÉSPED. ¡QUÉ POLLO TAN TONTO!`
- SK functions can be in-line, coded, or loaded from files.
- SK planner, dynamically can create complex orchestrations.
- Memories can be used to save contents with embeddings and retrieve content using the same embeddings, for example for use in the RAG pattern.

## When and when not to choose Semantic Kernel

**Note:** Semantic Kernel is still in preview. It is not yet recommended for production.

### When

- There's language support, particularly for C# and Python (more language support coming).
- The app can take advantage of the functionality provided by Semantic Kernel such as pipes, orchestration, memories, etc.
- If you need complex SK function orchestrations.
- If you need to work with embeddings.

### When not to

- SK may be too complex for simple applications.
- Obviously, if a language is not supported.

## Semantic Kernel vs Langchain

| Semantic Kernel | LangChain |
| ------------------------ | ------------------------------- |
| Connectors | Language Models (LLM/Chat) |
| SK functions | Prompt Templates |
| Variables in functions | Variables in prompt templates |
| Pipes | Chains |
| Planner | Chains |
| Memories | Memory |
| C#, Python, Java | Python |

**Memories:** Usually are helpers to generate, store, retrieve, and compare embeddings from vector databases.