{"id":20046693,"url":"https://github.com/apteco/aptecopsframework","last_synced_at":"2025-03-02T08:14:35.807Z","repository":{"id":205063247,"uuid":"713327199","full_name":"Apteco/AptecoPSFramework","owner":"Apteco","description":"Apteco Framework for handling up- and downstream data with PowerShell and .NET","archived":false,"fork":false,"pushed_at":"2024-04-18T15:47:30.000Z","size":445,"stargazers_count":0,"open_issues_count":7,"forks_count":0,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-04-18T17:19:53.880Z","etag":null,"topics":["apteco","cleverreach","dynamics-365","hubspot","inxmail","microsoft","powershell","powershell-module","salesforce"],"latest_commit_sha":null,"homepage":"https://www.powershellgallery.com/packages/AptecoPSFramework","language":"PowerShell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Apteco.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-11-02T09:49:28.000Z","updated_at":"2024-04-23T12:15:17.737Z","dependencies_parsed_at":"2023-11-02T11:45:19.294Z","dependency_job_id":"8f8609d3-7a00-42e1-bea0-1c822a6557e8","html_url":"https://github.com/Apteco/AptecoPSFramework","commit_stats":null,"previous_names":["apteco/aptecopsframework"],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Apteco%2FAptecoPSFramework","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Apteco%2FAptecoPSFramework/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Apteco%2FAptecoPSFramework/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Apteco%2FAptecoPSFramework/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Apteco","download_url":"https://codeload.github.com/Apteco/AptecoPSFramework/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241476458,"owners_count":19968916,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apteco","cleverreach","dynamics-365","hubspot","inxmail","microsoft","powershell","powershell-module","salesforce"],"created_at":"2024-11-13T11:28:45.049Z","updated_at":"2025-03-02T08:14:35.790Z","avatar_url":"https://github.com/Apteco.png","language":"PowerShell","readme":"[![License](https://img.shields.io/badge/License-MIT-blue.svg)](https://github.com/PlagueHO/PSAuth/blob/dev/LICENSE)\n\n\n# Apteco Customs - AptecoPSFramework\n\nThis framework is mainly created for installing and using custom channels in Apteco Orbit and PeopleStage. The channel implementations are written in PowerShell and already implemented as \"Plugins\" in this module. But there is a function implemented so you can refer to your own channels that are not getting overwritten if you update this module. The development of this module is done here, but it is published through [PowerShell Gallery](https://www.powershellgallery.com/packages?q=%23apteco).\n\n# Current integrations/plugins\n\nType|Vendor|API name|Technology|Features\n-|-|-|-|-\nEmail|CleverReach|v3|REST|Tagging, Upload, Broadcast-Preparation, Broadcast, Receiver- and Response-Download\nCRM|SalesForce SalesCloud|REST/Bulk API|REST|Load CRM Data, Upload to CampaignMembers\nCRM|Hubspot|CRM API v3|REST|Download all CRM object data full/delta, Upload to Marketing Lists\nCRM|Microsoft Dynamics 365 CRM|DataVerse oData WebAPI|REST|Download all CRM object data and picklists full/delta\nEmail|emarsys (An SAP Company)|Emarsys Core API|REST|Adding Lists and Contacts, get lists, campaigns, fields, contact data\nEmail|ActiveCampaign||REST|Still under development\nEmail|Apteco email||REST|Get lists and contacts on that list\nCRM|Fundraising Box||REST|Get contacts, donations, projects and much more to build a system\n\n# Installation / Update / Uninstall\n\nThis has been put into the wiki: https://github.com/Apteco/AptecoPSFramework/wiki/Installation-and-Update\n\nBut this is the Quickstart\n\n```PowerShell\nInstall-Module WriteLog\nInstall-Script Install-Dependencies, Import-Dependencies\nInstall-Module AptecoPSFramework\nImport-Module AptecoPSFramework\nInstall-AptecoPSFramework\n```\n\n# Getting started with the Framework\n\nAfter the installation you are reade to use this module and create a settings json file and connect that with your Apteco system. So please import your module like \n\n```PowerShell\nImport-Module AptecoPSFramework -Verbose\n```\n\nIf you get error messages during the import, that is normal, because there are modules missing yet. They need to be installed with `Install-AptecoPSFramework`\n\nPlease go ahead to a directory where the script files should be placed like `D:\\Scripts\\AptecoPSFramework`. Then you can now start the installation\n\n```PowerShell\nInstall-AptecoPSFramework -Verbose\n```\n\nThis command installs dependencies like scripts, modules and nuget packages. After installing the dependencies it copies a \"Boilerplate\" into your current directory and gives you more hints about how to connect this \"Boilerplate\" with a PeopleStage PowerShell channel.\n\n\n\n\n\nNow you should find a `create_settings.ps1` file in your boilerplate folder. Please execute this one to create a new `settings.json` file with\n\n```PowerShell\n. .\\create_settings.ps1\n```\n\n\n\n\n# Getting started with Plugins\n\nYou are able to use already integrated plugins, but can also develop your own plugins and load it via this module and use the already existing framework with logging, error handling, encryption and much more comfort. The idea is to have a `settings.json` which can also have a different name. That file contains all information to drive the module and integration with Apteco. You can have as many settings files for the same or different plugins as you want.\n\nPlease consider to create a channel multiple times, e.g. for every table level.\n\n## Create your settings json/yaml file\n\nThe first target is to create your settings json file, because that contains all the information that the module needs for being triggered by PeopleStage. In the boiletplate you can find a rough example of how to create that file. But specific for the chosen plugins you can set more or less parameters. They can be changed through that `create_settings.ps1` script or later in your json file. It depends on how often you change the settings json file. This path to the settings file json needs to be exchanged in the channel editor settings in the integration parameter. This needs to be an absolute path.\n\nTool tip: If you want to visualise (and edit) the JSON, this tool can help: https://jsoncrack.com/editor\n\n```PowerShell\n\n#-----------------------------------------------\n# IMPORT THE FRAMEWORK MODULE\n#-----------------------------------------------\n\nImport-Module \"AptecoPSFramework\"\n\n\n#-----------------------------------------------\n# CHOOSE A PLUGIN\n#-----------------------------------------------\n\n$plugin = @(, (get-plugins | Select guid, name, version, update, path | Out-GridView -PassThru ))\n\nIf ($plugin.count -gt 1) {\n    Write-Host \"Sorry, you have chosen more than 1 plugin\"\n    exit 1\n} elseif ( $plugin.count -eq 0 ) {\n    Write-Host \"Sorry, you have chosen less than 1 plugin\"\n    exit 1\n}\n\n\n#-----------------------------------------------\n# LOAD THE PLUGIN\n#-----------------------------------------------\n\nImport-Plugin -guid $plugin.guid\n\n\n#-----------------------------------------------\n# LOAD THE SETTINGS (GLOBAL + PLUGIN)\n#-----------------------------------------------\n\n$settings = Get-settings\n#$settings.pluginGuid = $plugin.guid\n\n\n#-----------------------------------------------\n# CHANGE PARAMETERS\n#-----------------------------------------------\n\n$settings.logfile = \".\\file.log\"\n\n# Specific for CleverReach just as an example, it is commented out currently\n\u003c#\n$settings.token.tokenUsage = \"consume\"\n$settings.token.tokenFilePath = \"D:\\Scripts\\CleverReach\\check-token\\cr.token\"\n#\u003e\n\n\n#-----------------------------------------------\n# SET AND EXPORT SETTINGS\n#-----------------------------------------------\n\nSet-Settings -PSCustom $settings\nExport-Settings -Path \".\\settings.yaml\"\n```\n\n\n## Create your own plugin or change an existing one\n\nThese steps are needed\n\n1. Get a copy of the demo plugin as a template or get another plugin from this modules folder\n1. Open the `Plugin.ps1` and exchange the guid with a new random one like `[guid]::NewGuid().ToString()`\n1. Go through the different files and change the code. Please be aware to not touch the functions names in the `peoplestage` folder\n\n\nPlease have a look at the [Demo channel](plugins/Demo) as a kind of template/boilerplate to start with. The guid needs to be changed in the `Plugin.ps1` to allow loading the plugin, but that is explained later.\n\nPlease note that the plugin that gets dynamically created when calling `import-plugin`, a copy of the internal variables is used. So after `import-plugin` all changes in the imported module don't have an effect to the dynamic plugin.\n\nPlease think about a [pull request](https://github.com/Apteco/AptecoPSModules/pulls) if you want to add more integrations into this repository\n\nThe structure for the plugin folder should look like this\n\nMandatory|Folder|Filename|Description\n-|-|-|-\nyes|.|ReadMe.md|Description about this plugin\nyes|.|Plugin.ps1|Metadata about the plugin. The filename should always be `Plugin.ps1`. If you create a new plugin, the guid needs to be recreated. Output a new guid with `[guid]::newGuid().toString()`\nyes|./settings|defaultsettings.ps1|These settings are plugin dependent and will be merged with the overall settings\nno|./public/setup|install-plugin.ps1|This script does a plugin dependent installation\nno|./public/peoplestage|test-login.ps1|Script to test the login via channel editor\nyes|./public/peoplestage|get-messages.ps1|Script to load messages\nno|./public/peoplestage|get-groups.ps1|Script to load groups or lists\nyes|./public/peoplestage|invoke-upload.ps1|Script to upload records into a group or list\nno|./public/peoplestage|invoke-broadcast.ps1|Script to trigger/broadcast a message (not needed for \"upload only\")\nno|./public/peoplestage|show-preview.ps1|Script to render a message as html\n\nGenerally the `private` folder is for internal functions that are used by the plugin, but are not usable in your PowerShell session. The `public` folders functions are exported to your sessions, which means you can use them as CmdLets automatically after the plugin has been loaded.\n\n\n### Loading your plugin\n\nYou do not need to put the plugin back to the modules folders, otherwise it can be deleted or overwritten with an update. Use this command to define your own plugins root directory after `Import-Module AptecoPSFramework`\n\n```PowerShell\nAdd-PluginFolder -Folder \"C:\\temp\\plugins\"\nRegister-Plugins\n```\n\nAfter you have added the directory and re-registered all plugins you should see a combination of the default plugins and your new developed one with\n\n```PowerShell\nGet-Plugins\n```\n\nAfter you have seen the list of plugins there are multiple ways to choose one manually with an `Get-Plugins | Out-GridView -PassThrough` or just copy the guid. Choose the plugin for the module with something similar like\n\n```PowerShell\nImport-Plugin -guid \"07f5de5b-1c83-4300-8f17-063a5fdec901\"\n```\n\n### Use plugin specific functionality\n\nThere are default functionalities per plugin like `get-messages`, `invoke-upload` etc. But there are maybe more functionalities that could be useful. To use this you can find in the `Plugin.ps1` a class, that is automatically loaded into the module, when importing the plugin. So you could do something like\n\n```PowerShell\nImport-Module AptecoPSFramework -Verbose\nSet-DebugMode -DebugMode $true          # Not necessarily needed, only for testing purposes\nImport-Settings -Path \".\\settings.json\" # The plugin gets automatically loaded from the json file\n\n# Now you can execute plugin specific commands you can see with\nGet-Module | Format-List\n\n# And get a result like\n\u003c#\n\nName              : AptecoPSFramework\nPath              : D:\\Scripts\\PSModules\\AptecoPSFramework\\AptecoPSFramework.psm1\nDescription       : Apteco PS Modules - Framework\nModuleType        : Script\nVersion           : 0.0.1\nNestedModules     : {WriteLog, MeasureRows, EncryptCredential, ExtendFunction...}\nExportedFunctions : {Add-PluginFolder, Export-Settings, Get-Debug, Get-Plugin...}\nExportedCmdlets   :\nExportedVariables :\nExportedAliases   :\n\nName              : Invoke CleverReach\nPath              : D:\\scripts\\CleverReach\\PSCleverReachModule\\f5c74d5e-a3db-46d9-a0f4-51edf2a79e6b\nDescription       :\nModuleType        : Script\nVersion           : 0.0\nNestedModules     : {WriteLog, MeasureRows, EncryptCredential, ExtendFunction...}\nExportedFunctions : {Get-Groups, Get-Messages, Invoke-Broadcast, Invoke-Upload...}\nExportedCmdlets   :\nExportedVariables :\nExportedAliases   :\n\n#\u003e\n\n# Here is another example to list all functions\nget-module | where { $_.Name -like \"Invoke*\" } | Select Name -ExpandProperty ExportedFunctions\n\n# Which generates this output\n\u003c#\n\nKey              Value\n---              -----\nGet-Groups       Get-Groups\nGet-Messages     Get-Messages\nInvoke-Broadcast Invoke-Broadcast\nInvoke-Upload    Invoke-Upload\nShow-Preview     Show-Preview\nTest-Login       Test-Login\nTest-Send        Test-Send\n\n#\u003e\n```\n\n# DuckDB support\n\nSince version `0.3.0` this module integrates DuckDB out-of-the-box, if you run `Install-AptecoPSFramework`. This downloads a few modules into a local \"lib\" folder.\n\nThis folder should be the folder, where you put your settings json/yaml files.\n\n## Using DuckDB from CLI\n\nThis is a quick start example to use DuckDB\n\n```PowerShell\n# this should be the place where you have your json/yaml settings files and your lib folder\nSet-Location \"c:\\Users\\MMustermann\\Scripts\\AptecoPSFramework\" \n\n# Import the Framework\nImport-Module AptecoPSFramework\n\n# Load Dependencies without loading a plugin\n# Loading a plugin will automatically load the packages in that folder\nImport-Lib\n\n\u003c#\n\nYou should see something in the console like\n\nWARNING: There is no variable '$processId' present on 'Script' scope. Created one with\n'bd8a69ce-0795-4f0a-b5ea-395c54473f60'\nVERBOSE: ----------------------------------------------------\nVERBOSE: Using PowerShell version 5.1.22621.3672 and Desktop edition\nVERBOSE: Using OS: Windows\nVERBOSE: User: 9709FBAA-7534-4\\WDAGUtilityAccount\nVERBOSE: Elevated: True\nVERBOSE: Loaded 0 modules\nVERBOSE: Loading libs...\nVERBOSE: There are 3 packages to load\nVERBOSE: Load status:\nVERBOSE:   Modules loaded: 0\nVERBOSE:   Lib/ref loaded: 3\nVERBOSE:   Lib/ref failed: 0\nVERBOSE:   Runtime loaded: 1\nVERBOSE:   Runtime failed: 0\n\n#\u003e\n\n# Open the connection to the default DuckDB database\n# You can also use Add-DuckDBConnection for more connections, but Open-DuckDBConnection\n# will add the default connection from settings automatically\nOpen-DuckDBConnection\n\n# Perform a simple scalar query, which returns just a number\nRead-DuckDBQueryAsScalar -Query \"Select 10+15\"\n\n# Should return 25\n\n# Perform a more advanced example to load a remote parquet file and query it in one go\n$q = \"CREATE TABLE train_services AS FROM 's3://duckdb-blobs/train_services.parquet';SELECT * FROM train_services LIMIT 10;\"\nRead-DuckDBQueryAsReader $q -ReturnAsPSCustom | ft\n\n# Instead of the combined command you could do it also separately\nInvoke-DuckDBQueryAsNonExecute -Query \"CREATE TABLE train_services AS FROM 's3://duckdb-blobs/train_services.parquet'\"\nRead-DuckDBQueryAsReader -Query \"SELECT * FROM train_services LIMIT 10\" -ReturnAsPSCustom | ft\n\n# To stream data of a bigger resultset, you should add the -AsStream switch like\nRead-DuckDBQueryAsReader -Query \"SELECT * FROM train_services LIMIT 10\" -ReturnAsPSCustom -AsStream\n\n# Close default connection\nClose-DuckDBConnection \n\n```\n\nTo do something directly with a sqlite database (instead of attaching it in the query), you can do something like this\n\n```PowerShell\n# Make sure you have the sqlite extension installed\nInvoke-DuckDBQueryAsNonExecute -Query \"INSTALL sqlite\"\n\n# Add a sqlite database directly in the connection string\nAdd-DuckDBConnection -Name \"Aachen\" -ConnectionString \"DataSource=C:\\Users\\WDAGUtilityAccount\\Downloads\\ac.sqlite;ACCESS_MODE=READ_ONLY\"\n\n# Open the database connection\nOpen-DuckDBConnection -Name \"Aachen\"\n\n# Show first 10 rows of the table and show as Out-GridView\nRead-DuckDBQueryAsReader -Query \"SELECT * FROM ac LIMIT 10\" -ReturnAsPSCustom -ConnectionName \"Aachen\" | Out-GridView\n\n# Close connection\nClose-DuckDBConnection -Name \"Aachen\"\n\n```\n\nYou can influence the database (default in-memory, but could also be a persistent one) by changing your settings at `defaultDuckDBConnection`\n\n## Using DuckDB from Orbit/PeopleStage\n\nBe aware that when Orbit/PeopleStage are calling PowerShell, this is currently 32bit and DuckDB is 64bit only. For this case you can put\nanother parameter in your `IntegrationParameters` in your channel to enforce 64bit. To do this, this parameter needs to be defined like\n`settingsFile=C:\\Apteco\\Scripts\\AptecoPSFramework\\settings.yaml;Force64bit=true` or similar.\n\n## Using DuckDB to work with csv files\n\nTo find out more about the csv file and their columns, you could do\n\n```PowerShell\nImport-Module AptecoPSFramework\nImport-Settings \".\\inx.yaml\"\nOpen-DuckDBConnection\n\n# Count the rows in a file\n$c = Read-DuckDBQueryAsScalar -Query \"Select count(*) from read_csv('.\\test.txt', sample_size=1000, delim='\\t')\"\n\n\n# This query then trys to find out more about the csv file like the delimiter, headers, date formats, column data types and much more\n# The delimiter does not need to be defined explicitely, it expects UTF-8\n$c = Read-DuckDBQueryAsReader -Query \"Select * from sniff_csv('.\\test.txt', sample_size=1000, delim='\\t') limit 10\" -ReturnAsPSCustom\n```\n\n# Errors\n\n## Die Eingabezeichenfolge hat das falsche Format\n\nWhen you get this error\n\n```PowerShell\nPS C:\\Users\\Administrator\u003e find-module aptecopsframework -IncludeDependencies -AllVersions\nDer Wert \"0.0.17-alpha\" kann nicht in den Typ \"System.Version\" konvertiert werden. Fehler: \"Die Eingabezeichenfolge\nhat das falsche Format.\"\n```\n\nPlease install a newer version of `PowerShellGet`\n\n```PowerShell\nInstall-Module PowerShellGet -Force -AllowClobber\n```\n\n## Channel not working after update\n\nPlease restart your FastStats service if you have problems after an update.\n\n## Unable to get messages with Exception / Unable to get lists with Exception\n\nThis should not happen, but shows that your PSModulePath cannot be loaded properly from C# runspaces.\n\nTo manually fix this, just add `C:\\Program Files\\WindowsPowerShell\\Modules` to the system environment variable `PSModulePath`\n\n## Missing dependency, execute: 'Install-Script Install-Dependencies'\n\nThere is a script missing to install other dependencies. To solve this, simply run\n\n```PowerShell\nInstall-Script Install-Dependencies\n```\n\nopen a new PowerShell window and after that you can proceed with your installation like\n\n```PowerShell\nImport-Module AptecoPSFramework\nInstall-AptecoPSFramework\n```\n\n## DuckDB cannot be opened\n\nSince 0.3.0 there is DuckDB integrated in this framework. If you have problems to get it work like\n\n```PowerShell\nImport-Module AptecoPSFramework\nOpen-DuckDBConnection\n```\n\nand it fails with something like\n\n```PowerShell\nPS C:\\Users\\WDAGUtilityAccount\u003e Open-DuckDBConnection\nException calling \"Open\" with \"0\" argument(s): \"The type initializer for\n'DuckDB.NET.Data.DuckDBConnectionStringBuilder' threw an exception.\"\nAt C:\\Users\\WDAGUtilityAccount\\Downloads\\AptecoPSFramework\\public\\duckdb\\Open-DuckDBConnection.ps1:18 char:13\n+             $Script:duckDb.Open()\n+             ~~~~~~~~~~~~~~~~~~~~~\n    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException\n    + FullyQualifiedErrorId : TypeInitializationException\n```\n\nthen you need to install the newest version of `vcredist` from: https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170\n\nThe direct permalink is: https://aka.ms/vs/17/release/vc_redist.x64.exe\n\nIf you already have installed the AptecoPSFramework, make sure to re-install the dependencies in your settings file directory with `Install-AptecoPSFramework`\n\n## There was no parameter found named \"-Destination\"\n\nThis message could be slightly different. But this results from an older 32 bit package of `Packagemanagement` that comes with Windows. And that one is outdated and not supported by this Framework. Make sure you have a newer version available. Best is to open a PowerShell as Administrator and install it via `Install-Module \"PackageManagement\" -Scope AllUsers`. Or update an existing one with `Update-Module PackageManagement`. Because when PowerShell is called via C#, it can load the older PackageManagement. You can safely remove it from `C:\\Program Files (x86)\\WindowsPowerShell\\Modules`. The new installed one should be in `C:\\Program Files\\WindowsPowerShell\\Modules`.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapteco%2Faptecopsframework","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapteco%2Faptecopsframework","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapteco%2Faptecopsframework/lists"}