Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/rozek/blockly-ai-playground
experiment with AI and build your own AI agents using Blockly - even if you are a complete beginner! (also suitable for school lessons)
https://github.com/rozek/blockly-ai-playground
agent ai blockly
Last synced: 21 days ago
JSON representation
experiment with AI and build your own AI agents using Blockly - even if you are a complete beginner! (also suitable for school lessons)
- Host: GitHub
- URL: https://github.com/rozek/blockly-ai-playground
- Owner: rozek
- License: mit
- Created: 2024-06-01T05:45:31.000Z (7 months ago)
- Default Branch: main
- Last Pushed: 2024-08-17T05:44:15.000Z (4 months ago)
- Last Synced: 2024-08-17T06:42:42.726Z (4 months ago)
- Topics: agent, ai, blockly
- Language: HTML
- Homepage:
- Size: 13.6 MB
- Stars: 4
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
Prerequisites |
Browser Troubleshooting |
Future plans |
Playground Context |
Playground UI |
AI Basics |
AI Mezzanines |
AI Support |
Miscellany |
Object Support |
Simple Examples |
AI Examples |
DIY AI Applications
# blockly-ai-playground #
experiment with AI and build your own AI agents using Blockly - even if you are a complete beginner! (also suitable for school lessons)
**Build your own Copilot - with Blockly - the video will show you how well it can perform (even in comparison with the original)!** (sorry for the bad quality: the video had to be created in a hurry)
[](https://www.youtube.com/watch?v=_JJzPLHObiY)
> (Work in progress - it was my contribution to the [Backdrop Build v5](https://backdropbuild.com/) contest and had to be launched as an MVP before their deadline. I'm now working on polishing it up...)
[Live Demo](https://rozek.github.io/blockly-ai-playground/LiveDemo) **Warning: please read [Overview](#overview) and [Prerequisites](#prerequisites) first. And [Browser Troubleshooting](#browser-troubleshooting) may help you in case of problems**
> **Recommendation: start with the [basic Settings example](#basic-settings) and enter your `APIServer`, `APIKey` and `default_model` settings, at least. Then use the [PoC example](#text-completion-proof-of-concept) and run your first inferences. Later, you may also try the [AI-assisted Web Search](#ai-web-search) and the [AI Agent](#ai-agent) to learn about the benefits of "AI agents". Finally, when you are ready, build your own AI-assisted applications - perhaps, based on the [DIY AI Applications](#diy-ai-applications) examples**
> **Please note: the "blockly-ai-playground" has only been tested with Perplexity AI (and Ollama with Llama 3.1) on a Mac so far. Additionally, the performance of generative AI is hard to predict - as a consequence, your mileage may vary when compared to what you saw in the video**
> You have comments, wishes or ideas? You found a bug? Then [open an issue](https://github.com/rozek/blockly-ai-playground/issues). You like this tool and plan to use it? Then leave a star!
## Overview ##
AI agents are "cool" these days and can be found at many places - but for most of us, they are basically just "black boxes" which do not reveal how they work internally.
[Blockly](https://developers.google.com/blockly) is a visual programming environment developed by Google that allows users to create code using drag-and-drop "Lego"-like blocks. It is designed to simplify coding by abstracting away the syntax, making it accessible for beginners and educational purposes.
By combining these two technologies, the "blockly-ai-playground" offers beginners and casual programmers an easy way to play and experiment with AI and build their own AI agents!
## Prerequisites ##
The "blockly-ai-playground" can be used with any AI provider offering an [OpenAI compatible API](https://platform.openai.com/docs/api-reference) - this may be [OpenAI](https://openai.com/) itself, [Perplexity](https://www.perplexity.ai/) (which was used for development), [Ollama](https://ollama.com/) (which runs LLMs locally on one's computer) and, presumably, many others.
As a consequence, you will need
* the **URL of an API entry point** (either locally on your machine or on the internet) and
* an **API Access Key** if you are using an official AI provider
## Browser Troubleshooting ##
Modern browser security may cause a lot of trouble when a web application wants to communicate with servers that have not been configured properly. If you know what you are doing, you may lower certain security levels in order to get the experience you expect.
### CORS ###
Cross-Origin Resource Sharing (CORS) is a security feature implemented in web browsers that allows or restricts web pages from making requests to a domain different from the one that served the web page. It is needed to prevent malicious websites from accessing sensitive data on other domains without permission, thereby protecting user data and maintaining web security.
Incorrect CORS settings of a server(!) can block legitimate requests from different domains, leading to functionality issues in web applications. This often results in CORS errors, where the browser blocks the request due to a lack of proper authorization from the server.
To get rid of CORS errors as a browser user, you may install a browser extension that adds the necessary CORS headers to server requests, allowing cross-origin requests to succeed. However, this method should be used only for development purposes, as it can introduce security risks if used indiscriminately.
Here is an example for a free extension for Chrome browsers (and compatible ones) - [this](https://chromewebstore.google.com/detail/cross-domain-cors/mjhpgnbimicffchbodmgfnemoghjakai) one works fine and is free.
### Communicating with your own PC ###
Modern browsers require HTTPS instead of HTTP for several security and trust reasons.
HTTPS requires servers to provide valid certificates to establish a secure, encrypted connection between the client and server, ensuring data integrity and authenticity.
However, this requirement makes it difficult for a web application to communicate with "localhost" because certificate authorities (CAs) do not issue certificates for "localhost" since it is not a unique, publicly-owned domain. Sometimes, self-signed certificates are used instead, but these are not trusted by browsers by default.
A better alternative is to configure a browser to trust "localhost" independend of a certificate:
1. Open Google Chrome (or a similar browser) and type chrome://flags in the address bar
2. Now activate the settings shown below (you may have to search for them in the list the browser will present to you)
The last setting is optional and only required if you want to access a different computer in your LAN at home
3. You may have to restart your browser afterwards
## Current Status and Future Plans ##
As you can see in the video, the "blockly-ai-playground" is principally working. However, due to the haste with which development had to take place, the result should only be considered as a "minimum viable product" (MVP) as it still requires a lot of "polishing" before it will be as easy to use as it should be for beginners.
These are my plans for the near future:
* make sure that everything is working properly in the most important browsers and on the most important platforms (including tablets, but not smartphones - their displays are just too small)
* quick notes on how to get and install Ollama (perhaps together with Open-WebUI)
* create a tutorial for real beginners (perhaps with tutorial videos?)
* use a bundler (vite) to make a real web app out of the playground
* use some of the available Blockly plugins to enhance the user interface
* create a desktop application (NW.js or Electron.js)
* create installers for Windows/Linux/macOS, provide update paths (e.g., similar to the GitHub Desktop)
* detect browser security violations (TLS, CORS etc.) and give user-friendly error messages with suggestions how to fix them
The idea is to have a beginner-friendly version ready for the next lecture season (starting in October)
## Custom Blocks ###
Besides common blocks which can be found in many Blockly environments, the "blockly-ai-playground" also provides a set of custom blocks to manage an internal context, a simple (but dynamic) user interface, basic and enhanced functions for AI, auxiliary tools for AI agents and other useful functions.
### Playground Context ###
The playground manages a dynamic "context" which is a collection of "items", consisting of a "value" (a string) and its unique "name". Context items can store settings (e.g., the URL of an API server, its access key or the AI model to use) or intermediate results generated by an agent or they provide a link between an agent and its user interface.
> Please note, that context item names are case-sensitive.
Some context items are already used internally, these are
* `APIServer` - the URL (and base path) of the OpenAI-compatible API you plan to use
* `APIKey` - your API Access Key (if required)
* `SearXNGServer` - the URL of your own SearXNG server (just leave it empty if you don't have one - in that case, a random public server will be looked-up and used instead)
the inferencing parameters
* `default_model` - the model to use for completions (no preset, as it highly depends on the configured API Server)
* `default_max_tokens` - the maximum number of completion tokens to be returned (preset to 2048)
* `default_num_ctx` - the maximum context size (no preset, if set it must be greater than "max_tokens")
* `default_temperature` - the "response randomness" setting (0...2, preset to 0)
* `default_top_p` - the "nucleus sampling threshold" setting (0...1, no preset)
* `default_top_k` - the "top-k filtering setting" setting (0...2048, preset to 1)
* `default_presence_penalty` - the "token presence penalty" setting (-2...2, preset to 1)
* `default_frequency_penalty` - the "token frequency penalty" setting (0..., no preset)
* `default_SystemTemplate` - the default template for the "system" message in a chat (preset to `Be precise and concise.`)
* `default_UserTemplate` - the default template for the "user" message in a chat (preset to `{{Prompt}}`)
and
* `Console` - the current contents of the built-in "Console" (initially empty)
In addition to the items listed above, "AI Mezzanine" blocks provide their own context items in order to allow customization of their internal API calls. To find these items, simply replace the `default` prefix of the inferencing parameter items by the prefix for the given mezzanine (e.g., 'summarizer_max_tokens' specifies the token generation limit for text summarization). If any mezzanine-specific context item is removed (or empty) the related default setting is used instead.
#### Preserve and Restore ####
During development it is sometimes useful to keep settings and values beyond the life-time of a single program. Typical use case are:
* you use one Blockly program to define APIServer, APIKey or other settings and use these settings in other Blockly programs
* you save intermediate results of API calls in order to avoid having to rerun them after restarting an AI-assisted application
These two blocks provide such a functionality: use the first block to preserve a given context item and the second to restore it later
#### Context-related Blocks ####
> returns a list with the names of all currently defined context items (see [related example](#context))
> clears the context (i.e., deletes all currently defined context items) (see [related example](#context))
> returns the current value of a given context item. If the requested item does not exist, `undefined` is returned (see [related example](#context))
> sets the given context item to the given value. If the addressed item does not exist, it will be created (see [related example](#context))
> removes the given context item. It is safe to remove an item which does not exist (see [related example](#context))
> preserves the given context item (as a string) in the Browser Storage. If the addressed item does not exist, an empty string will be preserved (see [related example](#preserve-restore))
> restores the given context item (as a string) from the Browser Storage. If the addressed item was not preserved before, the context item will be set to an empty string (see [related example](#preserve-and-restore))
### Playground UI ###
The playground also manages a programmable reactive "user interface" (aka "UI"). The playground UI is basically a vertical list of user interface "elements", often preceeded by a "label".
Every UI element has a unique "name" which is bound to a context item with the same name. As a consequence,
* changing the value of a context item automatically updates the contents of a bound UI element and
* entering data into a UI element automatically updates the assciated context item
This concept makes UI programming super easy.
#### Event Loop ####
Often, a program has to react on user inputs. In the simplest way, this just means to present a few input fields to the user (e.g., to enter `APIServer`, `APIKey` and a `Prompt`) and to start inferencing as soon as the user has pressed a button. In the meantime, the UI may also be used to inform the user about the validity of any inputs while (s)he is entering them.
This is achieved by an "event loop".
The core of any event loop is one of the blocks shown above. Each of them waits for incoming UI "events" (which indicate that the user has entered some data or clicked on a button) and then executes a sequence of inner blocks (the so called "loop body").
Within the loop body you may now check what the user actually did and react accordingly - this is what the two blocks shown above are good for:
* the first one returns `true` if a button with the given name was pressed and
* the second one returns `true` if the user entered new data into an input element with the given name
Both blocks are typically placed into the condition field of an "if-then" block to run the "then" clause only if the given button was pressed or the given input element received new data.
The loop body is executed over and over again - until the loop is exited using the block shown above.
A Blockly program may contain several event loops - but only one of them may be running at the same tine (i.e., **event loops must not be nested**).
#### Enabling ####
Often, a programmer wants to prevent buttons to be clicked or new data entered into input fields. Typical situations are
* users should not be able to start inferencing by pressing a "Submit" button before `APIServer`, `APIKey` and `Prompt` were entered (and valid)
* users should not be able to interact with the UI while an API request is running
This behaviour is achieved by "enabling" or "disabling" either the UI as a whole or specific UI elements only.
The two blocks shown above enable or disable the playground UI in its entirety. Disabling the whole UI disables all UI elements, while "enabling" the whole UI means that the enabling of every UI element depends on its own setting (i.e., elements may still be individually disabled)
These two blocks enable or disable specific UI elements.
#### UI-related Blocks ####
> switches to the playground workspace pane
> switches to the playground user interface pane (see [related example](#hello-world))
> (re-)enables the playground UI in its entirety. Please note that "enabling the whole UI" only means that the UI is no longer disabled in its entirety and every UI element is now enabled or disabled depending on its own setting (see [related example](#enable-and-disable))
> disables the playground UI in its entirety (see [related example](#enable-and-disable))
> returns a list with the names of all currently defined UI elements
> returns `true` if the UI currently contains an element with the given name - or `false` otherwise
> clears the user interface (i.e., deletes all currently defined UI elements)
> removes the element with the given name from the UI. It is safe to remove an element which does not exist
> appends a textline input element (based on an [HTML input element of type "text"](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/text)) with the given name and label to the UI. If an element with the same name already exists, it is removed before the new one is appended. This element supports the options `Placeholder`, `readonly`, `minLength`, `maxLength`, `Pattern`, `SpellChecking` and `Enabling` (see [related example](#input-elements))
> appends a password input element (based on an [HTML input element of type "password"](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/password)) with the given name and label to the UI. If an element with the same name already exists, it is removed before the new one is appended. This element supports the options `Placeholder`, `readonly`, `minLength`, `maxLength`, `Pattern` and `Enabling` (see [related example](#input-elements))
> appends a number input element (based on an [HTML input element of type "number"](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/number)) with the given name and label to the UI. If an element with the same name already exists, it is removed before the new one is appended. This element supports the options `Placeholder`, `readonly`, `Minimum`, `Maximum`, `Stepping` and `Enabling` (see [related example](#input-elements))
> appends a URL input element (based on an [HTML input element of type "url"](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/url)) with the given name and label to the UI. If an element with the same name already exists, it is removed before the new one is appended. This element supports the options `Placeholder`, `readonly`, `minLength`, `maxLength`, `Pattern` and `Enabling` (see [related example](#input-elements))
> appends a (multiline) text input element (based on an [HTML textarea](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/textarea)) with the given name and label to the UI. If an element with the same name already exists, it is removed before the new one is appended. This element supports the options `Placeholder`, `readonly`, `minLength`, `maxLength`, `LineWrapping`, `SpellChecking` and `Enabling` (see [related example](#input-elements))
> appends an input element (with the given name and label) to the UI that can be used to listen to and recognize what a user says into a microphone. If an element with the same name already exists, it is removed before the new one is appended. This element supports the options `Placeholder`, `readonly`, `minLength`, `maxLength`, `LineWrapping`, `SpellChecking` and `Enabling` (see [related example](#speechinput))
> Nota bene: speech recognition requires microphone access - for that reason the browser may ask for permission to use your PC's microphone. Recognition will only work if you grant that permission - or else fail.
> **Important: not all browsers support speech recognition (Chrome seems to, but other browsers may not - even if they are based on webkit). Additionally, the playground must be served over valid (and secure!) HTTPS**
> appends a checkbox element (based on an [HTML input element of type "checkbox"](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/checkbox)) with the given name and label to the UI. If an element with the same name already exists, it is removed before the new one is appended. This element only supports the option `Enabling` (see [related example](#checkbox))
> appends a radio button group element (based on an [HTML input elements of type "radio"](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/radio)) with the given name and label to the UI. If an element with the same name already exists, it is removed before the new one is appended.
>
> Radiobutton groups are a bit special as they contain a whole group of radio buttons, each with its own label. As a consequence, after appending a new radio button group element, you will have to define a list of radio buttons using the "append Choice" block - and the value of the whole group (i.e., the associated context item) is the (0-based) index of the currently checked radio button (or `-1` if no button is checked) (see [related example](#radiobutton-group))
> appends a drop-down element with the given name and label to the UI. If an element with the same name already exists, it is removed before the new one is appended
>
> After adding a new DropDown to the UI, you still have to define the entries from which the user may select. This can be done using the "append Choice" block - one per selectable entry. The context item associated with the DropDown will then contain the (0-based) index of the currently selected entry (see [related example](#dropdown))
> appends a button element with the given name and label to the UI. If an element with the same name already exists, it is removed before the new one is appended (see [related example](#input-elements))
> appends a textline output element with the given name and label to the UI. If an element with the same name already exists, it is removed before the new one is appended (see [related example](#output-elements))
> appends a number output element with the given name and label to the UI. If an element with the same name already exists, it is removed before the new one is appended (see [related example](#output-elements))
> appends a (multiline) text output element with the given name and label to the UI. If an element with the same name already exists, it is removed before the new one is appended (see [related example](#output-elements))
> appends an unlabelled text view element with the given name to the UI. If an element with the same name already exists, it is removed before the new one is appended (see [related example](#output-elements))
> appends an unlabelled "fine print" view element with the given name to the UI. If an element with the same name already exists, it is removed before the new one is appended (see [related example](#output-elements))
> appends a URL input element customized for the selection or input of the `APIServer` context item with the given label to the UI. If an element with the name `APIServer` already exists, it is removed before the new one is appended. This element supports the options `Placeholder`, `readonly`, `minLength`, `maxLength`, `Pattern` and `Enabling` (see [related example](#apiserver-input))
> sets the current value of a given option for an UI element with the given key to a given value. If no element with the given key can be found, an exception is thrown and the program aborted. Which options and option values are meaningful (and, thus, used by the UI), depends on the type of the given UI element (see [related example](#textline-input))
> returns the current value of a given option for an UI element with the given key. If no element with the given key can be found, an exception is thrown and the program aborted. The requested option does not have to exist
> returns `true` if an UI element with the given key is enabled or `false` otherwise. If no element with the given key can be found, an exception is thrown and the program aborted.
> enables the UI element with the given key. If no element with the given key can be found, an exception is thrown and the program aborted (see [related example](#enable-and-disable))
> disables the UI element with the given key. If no element with the given key can be found, an exception is thrown and the program aborted (see [related example](#enable-and-disable))
> returns `true` if an UI element with the given key (exists and) is enabled. If no element with the given key can be found, an exception is thrown and the program aborted.
> clears all currently defined choices of a radiobutton group or a drop-down element
> appends the given text to the list of choices for a radiobutton group or a drop-down element (see [related example](#dropdown))
> clears the built-in "Console" (see [related example](#console)).
> appends the given string to the built-in "Console" (see [related example](#console))
> appends the given string to the built-in "Console" and starts a new line (see [related example](#console))
> removes all pending UI events from the internal queue. If there are no pending events, this statement is simply ignored
> starts an event loop and runs the given statements whenever a new UI event was received. If the given timeout is `0` the loop is exited as soon as no more UI events are pending. If the given timeout is greater than `0` the loop is exited if no event was received within the given number of seconds. With a negative timeout, the loop waits indefinitely for the next UI event
> starts an event loop and runs the given statements whenever a new UI event was received, waiting indefinitely for new events (see [related example](#event-loop))
> returns `true` if the currently processed UI event is a click on a button with the given name - or `false` otherwise (see [related example](#button-clicks))
> returns `true` if the currently processed UI event indicates that new input for the UI element with the given name was received - or `false` otherwise (see [related example](#webspeech))
> leaves the current event loop
### AI Basics ###
The following blocks perform basic AI functions
> performs a chat completion for the given list of messages and returns the string created by the LLM. The message list must consist of an initial "system" message, a "user" message and (optionally) a sequence of "assistant" and "user" messages. If provided, any required options (such as `model`, `temperature`, `top_k` etc.) are taken from context items whose names start with the given option prefix. If such context items do not exist, are empty or the prefix is missing, any options are taken from context items whose names start with `default_` (see [related example](#text-completion-proof-of-concept))
> performs a text completion for the given prompt. The prompt is used to generate a "system" and a "user" message which are then used to generate a "chat completion". The "system" message is generated from the `SystemTemplate` context item, the "user" message from `UserTemplate`. If provided, any required options (such as `model`, `temperature`, `top_k` etc.) are taken from context items whose names start with the given option prefix. If such context items do not exist, are empty or the prefix is missing, any options are taken from context items whose names start with `default_` (see [related example](#chat-completion))
### AI Mezzanines ###
The following blocks internally use AI requests themselves to perform some tasks which are often found in AI agents:
> returns a summary of the given text. The block uses its own set of context items (prefixed with `Summarizer_`) to configure the inferencing parameters for text summarization - missing (or empty) items are taken from the default set (see [related example](#text-summary))
> generates a sequence of steps that have to be worked through in order to achieve a given goal. A single "step" is an object with the following properties:
>
> * `Tool`: name of a selected tool (either `ResearchTool`, `ComputationTool` or `ResponseTool`)
> * `Objective`: a short statement of what the chosen tool should achieve
> * `Justification`: a brief explanation of why this tool and objective were chosen
>
> The result of this block is a list of such steps (see [related example](#steps-derived-from-a-given-objective))
> uses the given "objective" to generate an adequate search phrase, search the internet, fetch all documents found by the search engine, extract any relevant information from these documents and produce a common result from all individual findings. Internally, this block uses the search phrase generator, information extractor and validator, solution merger and solution encoder described below (see [related example](#ai-research-tool))
> generates a "phrase" which can be used to perform an internet search using a common search engine (or SearXNG) (see [AI Web Search](#ai-search-phrase-generation) example)
> a web search usually returns a list of URLs - one per document that seems relevant. This block now fetches one of theses documents from the given URL, "reads" it and extracts any information that seems relevant for the given objective (see [AI Web Search](#ai-information-extraction) example)
> extracting information from a document may return nonsense. In order not to dilute the final result, this block checks if the given extract is actually relevant for the given objective or not. It returns `true` if it is or `false` otherwise (see [AI Web Search](#ai-information-validation) example)
> extracting information from several documents yields a list of separate extracts which may overlap in their contents. This block takes two such extracts and combines them into a single response relevant for a given objective (see [AI Web Search](#ai-web-search) example)
> AI responses are often "noisy", i.e. they contain additional "decorative" words besides the actually requested information. This block therefore extracts the plain response and returns it as a list of facts (see example [AI Web Search](#ai-web-search))
> generates a JavaScript function to achieve the given "objective", executes it and returns the result. This block internally uses the parameter list, function and argument generators described below (see [related example](#ai-computation-tool))
> uses the current "knowledge" (taken from the context item `Knowledge`) and generates a (possibly empty) list of parameter specifications for a Javascript function to achieve the given "objective") (see [related example](#ai-parameter-list-generation))
> uses the given parameter specification list and generates a JavaScript function to achieve the given "objective" (see [related example](#ai-code-generation-and-execution))
> uses the current "knowledge" (taken from the context item `Knowledge`) and generates an argument value suitable for the given parameter specification.
> uses the current "knowledge" (taken from the context item `Knowledge`) and generates a human-readable response to the given "task".
> takes a given "fact", generates a concise description based on the given "objective" that was used to find or compute this data and appends it to the current "knowledge" of the running AI agent. The new "knowledge" will be writen to the context item `Knowledge` but not automatically preserved.
### AI Support ###
The following blocks have been made to support the creation of AI agents.
> returns `true` if the given value represents valid agent "knowledge" - or `false` otherwise. Agent "knowledge" is a (possibly empty) list of JavaScript objects containing the following properties:
>
> * `Description`: a concise description of what the actual `fact` represents
> * `Fact`: the actual knowledge as a JSON-encoded JavaScript value
>
> The `Description` will help the AI to find relevant facts for its task, while the `Fact` contains the actual data.
> returns `true` if the given value represents a single working step for an AI agent - or `false` otherwise. A "working step" is a Javascript object containing the following properties:
>
> * `Tool`: name of a selected tool (either `ResearchTool`, `ComputationTool` or `ResponseTool`)
> * `Objective`: a short statement of what the chosen tool should achieve
> * `Justification`: a brief explanation of why this tool and objective were chosen
> returns `true` if the given value represents a sequence of working steps for an AI agent - or `false` otherwise.
> returns `true` if the given value a single function parameter specification - or `false` otherwise. A "parameter specification" is a Javascript object containing the following properties:
>
> `Name`: the parameter name (a valid JavaScript identifier)
> `Description`: a short description of the parameter's purpose
>
> both settings can be used to create a JavaScript function and derive its argument values from the already given knowledge
> returns `true` if the given value represents a list of function parameter specifications - or `false` otherwise.
> returns `true` if the given value is a string containing a Wikipedia URL - or `false` otherwise
> returns `true` if the given value is a string containing a Wikipedia page name - or `false` otherwise
> converts the given HTML into formatted text
> returns the first JSON object found in a given text - or `undefined` if no such JSON specification could be found (or the specification was incorrect)
> fetches the (english) Wikipedia article with the given page name and returns its contents as plain text
> searches the web for documents related to a given search phrase and returns a list with the URLs of the first found documents. If context item `SearXNGServer` contains a valid URL, the search request is sent to that server and only documents with a score greater than or equal to the threshold specified in context item `SearXNGScoreThreshold` (a number in the range 0...10 which defaults to 2) will be listed. Otherwise, a random public server will be looked up and used to perform the search. In both cases, up to 10 URLs will be returned, at most.
> searches the (english) Wikipedia for articles related to a given search phrase and returns a list with the Wikipedia page names of the first found articles
### Miscellany ###
The following blocks do not fit into the categories shown before - may may still be helpful when creating agents:
> this block has no specific functionality but serves as a visual separator between adjacent blocks only - this may help identifying related blocks
> throws the given exception (which is then shown in an alert dialog) and aborts the running program (see related [example](#throw))
> waits for a given number of seconds and continues (see [Console example](#console))
> opens a browser "alert" pop-up dialog, displays the given message and continues as soon as the user has closed the dialog (see related [example](#alert-confirm-prompt))
> opens a browser "confirmation" pop-up dialog, displays the given message and continues as soon as the user has responded with "ok" or "Cancel". If cancelled, the block returns `false`, otherwise it returns `true` (see related [example](#alert-confirm-prompt))
> opens a browser "prompt" pop-up dialog, displays the given message and continues as soon as the user has entered a response. If cancelled, the block returns an empty string, otherwise it returns the entered string (see related [example](#alert-confirm-prompt))
> uses text-to-speech to speak the given (english!) message (see related [example](#text-to-speech))
> returns `true` if the given value contains a string that is neither empty nor contains whitespace only - or `false` otherwise
> returns `true` if the given value contains a number - or `false` otherwise
> returns `true` if the given value contains a number with a value ranging from the given minimum to the given maximum - or `false` otherwise. The additional arguments "withMinimum" and "withMaximum" specify wether the given "minimum" and "maximum" values are themselves part of that range or not
> returns `true` if the given value contains an integer number - or `false` otherwise
> returns `true` if the given value contains an integer number with a value ranging from the given minimum to the given maximum - or `false` otherwise
> returns `true` if the given value contains an "ordinal" number (i.e., an integer greater than or equal to 0) - or `false` otherwise
> returns `true` if the given value contains a "cardinal" number (i.e., an integer greater than or equal to 1) - or `false` otherwise
> returns `true` if the given value contains a string that matches a given JavaScript regular expression - or `false` otherwise
> returns `true` if the given value contains a string that looks like a URL - or `false` otherwise
> returns `true` if the given value contains a string that looks like a Wikipedia document URL - or `false` otherwise
### Object Support ###
The following blocks help to create and manipulate objects
> returns a newly created empty object (see related [example](#objects))
> returns a (possibly empty) list with the keys of all own properties of a given object (see related [example](#objects))
> returns `true` if the given object contains a proeprty with the given key - or `false` otherwise (see related [example](#objects))
> returns the current value of the given object's property with the given key (see related [example](#objects))
> sets the given object's property with the given key to a given value (see related [example](#objects))
> deletes the given object's property with the given key (see related [example](#objects))
## Examples ##
Here are a few examples which you can upload into the Blockly workspace to get familiar with Blockly itself and the custom blocks specifically made for the "blockly-ai-playground". The linked Blockly workspace files should first be downloaded onto your computer and then uploaded into your Blockly workspace
### Hello, World ###
> this is the typical "Hello, World" example (see [Blockly workspace file](./Examples/Hello_World.json) - download it onto your computer and then upload it into the Blockly workspace)
### Alert, Confirm, Prompt ###
> if you do not want to implement a "real" user interface (with an event loop), it may already be sufficient to use the browser's built-in pop-up dialogs `alert`, `confirm` and `prompt` - for simple use cases, at least. This example demonstrates how to use them (see [Blockly workspace file](./Examples/Alert_Confirm_Prompt.json) - download it onto your computer and then upload it into the Blockly workspace)
### Console ###
> this example demonstrates all Console functions (see [Blockly workspace file](./Examples/Console.json) - download it onto your computer and then upload it into the Blockly workspace)
### Suspend, Resume, Abort ###
> use this little Blockly program to get familiar with "Suspend", "Resume" and "Abort" (see [Blockly workspace file](./Examples/Suspend_Resume_Abort.json) - download it onto your computer and then upload it into the Blockly workspace)
### throw ###
> this example illustrates how to use the "throw" block (see [Blockly workspace file](./Examples/throwError.json) - download it onto your computer and then upload it into the Blockly workspace) - please note, that all blocks following a "throw" will not be evaluated as throwing an exception will terminate a Blockly program
### Context ###
> this example demonstrates all local context functions (see [Blockly workspace file](./Examples/Context.json) - download it onto your computer and then upload it into the Blockly workspace)
### preserve and restore ###
> this example demonstrates how to preserve and restore context items (see [Blockly workspace file](./Examples/preserve_restore.json) - download it onto your computer and then upload it into the Blockly workspace)
### Input Elements ###
> this example shows all available input elements (see [Blockly workspace file](./Examples/InputElements.json) - download it onto your computer and then upload it into the Blockly workspace)
### Textline Input ###
> this example shows textline input elements in several variations (see [Blockly workspace file](./Examples/TextlineInput.json) - download it onto your computer and then upload it into the Blockly workspace). the other input elements work quite similarly
### SpeechInput ###
> this example demonstrates speech recognition using the Browser's built-in WebSpeech API (see [Blockly workspace file](./Examples/SpeechInput.json) - download it onto your computer and then upload it into the Blockly workspace)
### Checkbox ###
> Checkboxes support context values `'true'` and `'false'` (actually, any value different from `'true'` is considered `'false'`) (see [Blockly workspace file](./Examples/Checkbox.json) - download it onto your computer and then upload it into the Blockly workspace)
> if you want to react on changes of a checkbox's state, just check for new user input.
### Radiobutton Group ###
> Radiobutton groups may manage an arbitrary number of individual radio buttons, from which at most one may be checked at any time. These buttons have to be added using "append Choice" blocks (see [Blockly workspace file](./Examples/RadiobuttonGroup.json) - download it onto your computer and then upload it into the Blockly workspace). The value of a radiobutton group is then the (0-based) index of the checked radiobutton - any invalid context value will uncheck all buttons.
> if you want to react on changes of a radiobutton group's state, just check for new user input.
### DropDown ###
> Drop-down menus may manage an arbitrary number of separate choices, from which at most one may be selected at any time. These choices have to be added using "append Choice" blocks (see [Blockly workspace file](./Examples/DropDown.json) - download it onto your computer and then upload it into the Blockly workspace). The value of a drop-down menu is then the (0-based) index of the selected choice - any invalid context value will deselect all choices.
> if you want to react on changes of a drop-down menu's state, just check for new user input.
### APIServer Input ###
> the "APIServerInput" element is just a special URLInput element: in contrast to the normal one it adds a drop-down menu with predefined URls for common API servers (see [Blockly workspace file](./Examples/APIServerInput.json) - download it onto your computer and then upload it into the Blockly workspace).
> here you see what an APIServerInput element looks like
### Output Elements ###
> this example shows all output elements of the playground UI (see [Blockly workspace file](./Examples/OutputElements.json) - download it onto your computer and then upload it into the Blockly workspace)
> here you see what the output elements look like
### enable and disable ###
> this example demonstrates the effects of enabling or diabling either the whole UI or individual UI elements (see [Blockly workspace file](./Examples/enable_disable.json) - download it onto your computer and then upload it into the Blockly workspace)
### Event Loop ###
> user input and button clicks are handled within "event loops". This example illustrates how you can implement a simple program which waits for the user to enter some text, enables a "Start" button whenever such text exists and - after clicking the button - disables the whole user interface in order to avoid unwanted changes while the computer does some lengthy processing, displays any results and then re-enables the user interface again (see [Blockly workspace file](./Examples/EventLoop.json) - download it onto your computer and then upload it into the Blockly workspace)
> here is what the user interface looks like - of course, the example does not implement any real "processing" but simply uses a short delay to create the illusion of some lengthy work.
### Button Clicks ###
> this example demonstrates how to react on a button click (see [Blockly workspace file](./Examples/ButtonWasClicked.json) - download it onto your computer and then upload it into the Blockly workspace)
### Objects ###
> this example demonstrates how to work with objects (see [Blockly workspace file](./Examples/Objects.json) - download it onto your computer and then upload it into the Blockly workspace)
### Text-to-Speech ###
> this example demonstrates text-to-speech synthesis using the Browser's built-in WebSpeech API (see [Blockly workspace file](./Examples/speak.json) - download it onto your computer and then upload it into the Blockly workspace)
### WebSpeech ###
> this example illustrates how to use the Browser's built-in WebSpeech API (see [Blockly workspace file](./Examples/WebSpeech.json) - download it onto your computer and then upload it into the Blockly workspace)
### Web Search ###
> the "blockly-ai-playground" can search the web for information which may be relevant to perform a given task. This example illustrates how such a search works
> here is what you may get when searching for "Joseph Weizenbaum"
### Web Document Retrieval ###
> after having identified which web documents could be relevant for a certain task, this document has to be loaded and converted to plain text so that the AI will be able to process it. Binary files and other kinds of documents will have to be ignored (i.e., they will just result in an empty text string `''`) - right now, the text loader only supports documents with the MIME type `text/*`
> here is the output after reading a document decribing the "Weizenbaum Institute" in Germany.
### Wikipedia Search ###
> Wikipedia is often a good source for reliable information. This example demonstrates how to search Wikipedia based on a given search phrase
> here is the list of Wikipedia articles you may get when searching fo "Joseph Weizenbaum"
### Wikipedia Article Retrieval ###
> after having identified a relevant Wikipedia article, that document has to be loaded and prepared for the AI to be processed. This example demonstrates how this can be achieved (loading Wikipedia articles is a bit more complicated than loading a plain text or HTML file because the actual content is surrounded by a lot of "boilerplate" such as Wikipedia-related links)
> here you see the beginnings of a Wikipedia article about Joseph Weizenbaum
### Conversion of HTML to Text ###
> HTML is not what an AI should process directly - it's actual contents should first be converted into plain text, perhaps with some proper formatting for titles, paragraphs and links to help the AI identifying them more reliably. This example demonstrates how to perform this conversion
> here you see the conversion result
### Extraction of JSON from a Text ###
> even when instructed to output plain JSON only, an AI may still "decorate" JSON output with explanations and comments. In this example you will see how to extract a JSON specification from a given text
> and here you see the actually extracted JSON specification
## AI Examples ##
The following examples actually include AI-related blocks. As before, the Blockly workspace files should first be downloaded onto your computer and then uploaded into your Blockly workspace
### Basic Settings ###
> use this example to enter your basic AI-related settings - they will be stored in the browser's storage and may then be used by other programs without having to re-enter them over and over again (see [Blockly workspace file](./Examples/basicSettings.json) - download it onto your computer and then upload it into the Blockly workspace)
> as a hint, here are the settings used by the author himself
### Text Completion (Proof-of-Concept) ###
> this is the "proof-of-concept" mentioned above - play around as you like! (see [Blockly workspace file](./Examples/TextCompletion.json) - download it onto your computer and then upload it into the Blockly workspace)
> this is what you may get as the result
### Chat Completion ###
> here is an example for a simple ChatCompletion (see [Blockly workspace file](./Examples/ChatCompletion.json) - download it onto your computer and then upload it into the Blockly workspace)
> this is what you the result may look like
### Text Summary ###
> this example summarizes a given text (see [Blockly workspace file](./Examples/TextSummarizer.json) - download it onto your computer and then upload it into the Blockly workspace)
> this is the author's run to let the AI summarize the beginnings of "Alice in Wonderland" - your mileage may vary, of course
### AI Research Tool ###
> this is a demonstraton of the built-in "ResearchTool" for AI Agents: enter what you are looking for and let SearXNG and the AI look for relevant documents, read them for you and present a summary of what the system learned (see [Blockly workspace file](./Examples/AIResearchTool.json) - download it onto your computer and then upload it into the Blockly workspace)
> it's definitely not easy to find a good answer for the question "to which programming languages has Joseph Weizenbaum's famous Eliza program been ported?" (even if you try yourself) - here is what the author's attempt produced as output: definitely incomplete, but not bad for just a few seconds of work...
### AI Web Search ###
> you don't have to use the built-in "ResearchTool", you may also implement a similar functionality yourself: this example illustrates how to use several AI Mezzanines to implement an AI-assisted web search (see [Blockly workspace file](./Examples/AIWebSearch.json) - download it onto your computer and then upload it into the Blockly workspace)
>
> By the way: the objective was "to which programming languages has Joseph Weizenbaum's famous "Eliza" program be ported?"
> this is what the example may produce as output
### AI Search Phrase Generation ###
> if you want to conduct a web research for a given topic, you may have to generate a search phrase first - this example does that for you (see [Blockly workspace file](./Examples/SearchPhraseGenerator.json) - download it onto your computer and then upload it into the Blockly workspace)
> and here you see what a search phrase may look like
### AI Information Extraction ###
> if you have a text that contains information you need for a given task, you want the AI to read that text and extract what is relevant - here is an example of how to do that (see [Blockly workspace file](./Examples/InformationExtractor.json) - download it onto your computer and then upload it into the Blockly workspace)
> here you have an example for a useful information extraction
### AI Information Validation ###
> letting the AI extract relevant information from a given text may sound nice, but in fact, the AI often produces complete nonsens - here is an example how you can let the AI validate the actual relevance of its output itself (see [Blockly workspace file](./Examples/InformationValidator.json) - download it onto your computer and then upload it into the Blockly workspace)
> useful excerpts should display "true", nonsense "false"
### AI Computation Tool ###
> the "Computation Tool" takes an "Objective" and uses the already existing "Knowledge" to create and execute a JavaScript function that tries to compute the requested result
> this is what you may get after running the above example
### AI Parameter List Generation ###
> letting the AI generate and execute a Javascript function to achieve a given objective first requires a well-defined list of parameters so that the function knows what data will be passed as its arguments and the AI knows what kind of argument list to create. Here is an example how you can generate such a parameter list (see [Blockly workspace file](./Examples/ParameterListGenerator.json) - download it onto your computer and then upload it into the Blockly workspace)
> a parameter's "Index" property refers to the knowledge list entry from which the parameter was derived.
### AI Code Generation and Execution ###
> this examples shows how to let the AI generate a JavaScript function to compute the given objective based on a list of parameters (see [Blockly workspace file](./Examples/FunctionGenerator.json) - download it onto your computer and then upload it into the Blockly workspace)
> sometimes, the generated code is really impressive...
### AI Task Decomposer (without existing knowledge) ###
> at the core of an AI agent there is a component which breaks down a given task into smaller steps which may then be worked through. This example demonstrates how the AI does so from scratch (see [Blockly workspace file](./Examples/TaskDecomposer_I.json) - download it onto your computer and then upload it into the Blockly workspace)
> this is what you may get - not too bad!
### AI Task Decomposer (using already existing knowledge) ###
> often, an AI agent does not have to start its work from scratch but may use already existing "knowledge" - here is an example for generating the remaining steps to achieve a given objective in such a situation (see [Blockly workspace file](./Examples/TaskDecomposer_II.json) - download it onto your computer and then upload it into the Blockly workspace)
> if you already liked the previous example, this one will impress you: yes, the AI is really able to take existing knowledge into account!
### AI Knowledge Generator ###
> both the "ResearchTool" and the "ComputationTool" produce new "facts" which have to be added to existing knowledge for an AI agent to continue its work and (in the end) produce a response. This example shows how that can be done (see [Blockly workspace file](./Examples/KnowledgeGenerator.json) - download it onto your computer and then upload it into the Blockly workspace)
> this is what "knowledge" generated from a "fact" may look like. In this case, the "fact" is already part of the description - which wasn't planned, but does not hurt.
### AI Answer Generator ###
> every run of an AI agent should finally produce a response - this example demonstrates how to do so (see [Blockly workspace file](./Examples/AnswerGenerator.json) - download it onto your computer and then upload it into the Blockly workspace)
> formulating sentences is something that AI is good at.
### AI Agent ###
> this example contains a full-blown AI agent (it's a bit like your own, personal "Copilot"): give it a "task" and watch how it decomposes it into smaller steps it is then working through. The agent has two "tools" at hand: a "ResearchTool" that can conduct web researches (like the [AI Web Search](#ai-web-search) from above) and a "ComputationTool" which creates and runs a JavaScript function that is used whenever some information has to be computed rather than looked up in the Internet. The outcome of each step is added to the agent's "knowledge" and stored in the browser - this gives the agent the possibility to skip over already completed steps when it was aborted and restarted later (provided that the "task" was not changed in between)
> this is what you may get as a result. **Feel free to compare this output with what MS Copilot or Perplexity AI will tell you - and be impressed!** Nota bene: your mileage will definitely vary from run to run - use that experience to treat every response from an AI (or an AI Agent) with care, even if you asked a "professional" one!
## DIY AI Applications ##
Now that you saw numerous examples for the custom blocks provided by the "blockly-ai-playground", here are some simple but useful examples of actual AI-assisted applications you (or your students) could build yourself/themselves.
### Translator ###
> this is a simple application which translates a given text into english. If you replace text input and output by the blocks for speech recognition and synthesis, you may even talk to that application - isn't that cool?
> and here is how a translation run could look like
## License ##
[MIT License](LICENSE.md)