https://github.com/fabriziosalmi/ethicalai

Automate ethical AI assessments via GitHub Actions
https://github.com/fabriziosalmi/ethicalai

assessment assessments automated automated-testing automation automation-framework automation-testing ethical ethical-ai ethical-artificial-intelligence ethical-design ethical-digital-standards ethical-hacking github-actions

Last synced: 16 days ago
JSON representation

Automate ethical AI assessments via GitHub Actions

• Host: GitHub
• URL: https://github.com/fabriziosalmi/ethicalai
• Owner: fabriziosalmi
• License: gpl-3.0
• Created: 2023-11-12T18:19:08.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2024-09-20T09:18:17.000Z (about 2 months ago)
• Last Synced: 2024-10-11T02:34:03.460Z (about 1 month ago)
• Topics: assessment, assessments, automated, automated-testing, automation, automation-framework, automation-testing, ethical, ethical-ai, ethical-artificial-intelligence, ethical-design, ethical-digital-standards, ethical-hacking, github-actions
• Language: Python
• Homepage: https://github.com/fabriziosalmi/You-Know-What-AI-Mean?tab=readme-ov-file#you-know-what-ai-mean
• Size: 116 KB
• Stars: 5
• Watchers: 3
• Forks: 1
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# EthicalAI automated assessment framework

The EthicalAI automated assessment framework project aims to provide insightful metrics in the realm of AI ethics, contributing to the broader AI ecosystem.

> 📘 _Read my draft about ethical AI [here](https://github.com/fabriziosalmi/You-Know-What-AI-Mean?tab=readme-ov-file#you-know-what-ai-mean)._

### Project Overview

EthicalAI automated assessment frameowrk is developed in Python and is specifically crafted to analyze and understand the ethical dimensions of AI chatbots that are compatible with OpenAI's API. It operates by posing a series of questions to the AI, gathering responses, and evaluating them against established ethical benchmarks. This tool is structured to function as a weekly GitHub Action.

#### Features

- Automated Assessments: Conducts weekly automated evaluations of AI's ethical understanding.

- Results Documentation: Generates a detailed results summary in a Markdown-formatted table (model.md).

- Future Enhancements: I am exploring several avenues for improvement, including diverse prompt designs, understanding AI's knowledge limits, multi-domain responses analysis, and integrating game-based assessments.

### Improvements

I am continuously refining our assessment test suites and expanding our coverage to include more AI API endpoints. Your contributions in these areas are highly valued and welcomed.

### Reports

- [davinci003](https://github.com/fabriziosalmi/AI-automated-assessment/blob/main/davinci003.md)