Self-Hosting-Guide

Self-Hosting Guide. Learn all about locally hosting (on premises & private web servers) and managing software applications by yourself or your organization. Including Cloud, LLMs, WireGuard, Automation, Home Assistant, and Networking.
https://github.com/mikeroyal/Self-Hosting-Guide

Last synced: 4 days ago
JSON representation

Kubernetes Tools, Frameworks, and Projects
- Interfaces
  - VMware Tanzu
  - KubeInit
  - Rancher
  - K3s - constrained, remote locations or inside IoT appliances.
  - Mac Development Ansible Playbook by Jeff Geerling
  - Grafana Kubernetes App
  - Charmed Kubernetes - key, conformant Kubernetes platform, optimized for your multi-cloud environments developed by Canonical.
  - Container Engine for Kubernetes (OKE) - managed container orchestration service that can reduce the time and cost to build modern cloud native applications. Unlike most other vendors, Oracle Cloud Infrastructure provides Container Engine for Kubernetes as a free service that runs on higher-performance, lower-cost compute.
  - Red Hat Openshift - premises, hybrid, and multicloud deployments.
ML frameworks & applications
- Interfaces
  - Amazon SageMaker
  - Open Neural Network Exchange(ONNX) - in operators and standard data types.
  - Anaconda
  - NVIDIA cuDNN - accelerated library of primitives for [deep neural networks](https://developer.nvidia.com/deep-learning). cuDNN provides highly tuned implementations for standard routines such as forward and backward convolution, pooling, normalization, and activation layers. cuDNN accelerates widely used deep learning frameworks, including [Caffe2](https://caffe2.ai/), [Chainer](https://chainer.org/), [Keras](https://keras.io/), [MATLAB](https://www.mathworks.com/solutions/deep-learning.html), [MxNet](https://mxnet.incubator.apache.org/), [PyTorch](https://pytorch.org/), and [TensorFlow](https://www.tensorflow.org/).
  - Jupyter Notebook - source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Jupyter is used widely in industries that do data cleaning and transformation, numerical simulation, statistical modeling, data visualization, data science, and machine learning.
  - Apache PredictionIO
  - BigDL
  - Apache Spark™ MLflow
  - MLflow Tracking
  - MLflow Projects
  - MLflow Models
  - Model Registry
  - Eclipse Deeplearning4J (DL4J) - based(Scala, Kotlin, Clojure, and Groovy) deep learning application. This means starting with the raw data, loading and preprocessing it from wherever and whatever format it is in to building and tuning a wide variety of simple and complex deep learning networks.
  - Chainer - based deep learning framework aiming at flexibility. It provides automatic differentiation APIs based on the define-by-run approach (dynamic computational graphs) as well as object-oriented high-level APIs to build and train neural networks. It also supports CUDA/cuDNN using [CuPy](https://github.com/cupy/cupy) for high performance training and inference.
  - Microsoft Cognitive Toolkit (CNTK) - source toolkit for commercial-grade distributed deep learning. It describes neural networks as a series of computational steps via a directed graph. CNTK allows the user to easily realize and combine popular model types such as feed-forward DNNs, convolutional neural networks (CNNs) and recurrent neural networks (RNNs/LSTMs). CNTK implements stochastic gradient descent (SGD, error backpropagation) learning with automatic differentiation and parallelization across multiple GPUs and servers.
  - Tensorman
  - Azure Databricks - based big data analytics service designed for data science and data engineering. Azure Databricks, sets up your Apache Spark environment in minutes, autoscale, and collaborate on shared projects in an interactive workspace. Azure Databricks supports Python, Scala, R, Java, and SQL, as well as data science frameworks and libraries including TensorFlow, PyTorch, and scikit-learn.
  - Apache MXNet
  - PlaidML
  - OpenCV - time computer vision applications. The C++, Python, and Java interfaces support Linux, MacOS, Windows, iOS, and Android.
  - Caffe
  - Theano - dimensional arrays efficiently including tight integration with NumPy.
  - nGraph - of-use to AI developers.
  - Apache Spark Connector for SQL Server and Azure SQL - performance connector that enables you to use transactional data in big data analytics and persists results for ad-hoc queries or reporting. The connector allows you to use any SQL database, on-premises or in the cloud, as an input data source or output data sink for Spark jobs.
  - Cluster Manager for Apache Kafka(CMAK)
  - Numba - aware optimizing compiler for Python sponsored by Anaconda, Inc. It uses the LLVM compiler project to generate machine code from Python syntax. Numba can compile a large subset of numerically-focused Python, including many NumPy functions. Additionally, Numba has support for automatic parallelization of loops, generation of GPU-accelerated code, and creation of ufuncs and C callbacks.
  - cuML - learn.
  - AutoGluon - accuracy deep learning models on tabular, image, and text data.
  - TensorFlow - to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML powered applications.
Models of Raspberry Pi boards
- Setting up WireGuard on Home Assistant
Networking Tools & Concepts
- Configure WDT service
  - cURL - line tool for transferring data using various network protocols(HTTP, HTTPS, FTP, FTPS, SCP, SFTP, TFTP, DICT, TELNET, LDAP LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP or SMTPS). cURL is also used in cars, television sets, routers, printers, audio equipment, mobile phones, tablets, settop boxes, media players and is the Internet transfer engine for thousands of software applications in over ten billion installations.
  - Authelia - source highly-available authentication server providing single sign-on capability and two-factor authentication to applications running behind [NGINX](https://nginx.org/en/).
  - nginx(engine x)
  - cURL Fuzzer
  - DoH - alone application for DoH (DNS-over-HTTPS) name resolves and lookups.
  - HTTPie - line HTTP client. Its goal is to make CLI interaction with web services as human-friendly as possible. HTTPie is designed for testing, debugging, and generally interacting with APIs & HTTP servers.
  - HTTPStat
  - Wuzz
  - Websocat - line client for WebSockets, like netcat (or curl) for ws:// with advanced socat-like functions.
Network Protocols
- Interfaces
  - OAuth 2.0 - party applications to access the user account.
  - Containerd.io
Node.js Learning Resources
- In-memory data grids
Node.js Tools
- In-memory data grids
  - NPM
  - nvm - user, and invoked per-shell. nvm works on any POSIX-compliant shell (sh, dash, ksh, zsh, bash), in particular on these platforms: unix, macOS, and windows WSL.
  - Express
  - Meteor - simple environment for building modern web applications with JavavScript.
  - NW.js
  - PM2 - in load balancer. It allows you to keep applications alive forever, to reload them without downtime and to facilitate common system admin tasks.
  - NestJS
  - Strapi
  - Standard
  - Hexo
  - node-gyp - platform command-line tool written in Node.js for compiling native addon modules for Node.js. It contains a vendored copy of the gyp-next project that was previously used by the Chromium team, extended to support the development of Node.js native addons.
  - Mocha
  - AVA
  - egg
  - mysqljs
  - axios
Online ML Learning Resources
- In-memory data grids
  - openHAB - platform software with the aim to integrate all kinds of Smart Home technologies, devices, etc.
  - Eclipse SmartHome - to-use solution. It offers a large set of features to choose from and leaves enough possibilities to design a Smart Home solution specific to your expectations. Its modular design brings millions of combinations and proves to be easily extensible by custom parts.
  - The Thing System
  - Hazelcast - memory data grid based on Java.
  - Home Assistant
  - Ehcache - based cache that boosts performance, offloads your database, and simplifies scalability. It's the most widely-used Java-based cache because it's robust, proven, full-featured, and integrates with other popular libraries and frameworks.
- Interfaces
  - RetroPie
  - Ubuntu Server for ARM
  - Manjaro Linux ARM
  - Arch Linux ARM
  - Ubuntu MATE for Raspberry Pi
  - Ubuntu Core on a Raspberry Pi
  - Kali Linux for the Raspberry Pi
  - RetroArch
  - LibreELEC
  - Machine Learning by Stanford University from Coursera
  - Machine Learning Courses Online from Udemy
  - Learn Machine Learning with Online Courses and Classes from edX
  - DBus - process communication, and remote procedure call mechanism that allows communication between multiple processes running concurrently on the same machine.
  - Constrained Application Protocol (CoAP) - to-machine (M2M) applications such as smart energy and building automation."
  - openSUSE
  - SUSE
  - Simba
  - HeliOS
  - VxWorks - leading real-time operating systems (RTOS) for building embedded devices and systems for more than 30 years.
  - Zephyr OS - oriented RTOS with a small-footprint kernel designed for use on resource-constrained and embedded systems. Zephyr has a small-foorprint Kernel focusing on embedded devices compatible with x86, ARM, RISC-V, Xtensa and [others](https://docs.zephyrproject.org/latest/boards/index.html).
  - FreeRTOS - time operating system for microcontrollers that makes small, low-power edge devices easy to program, deploy, secure, connect, and manage.
  - Contiki-os - constrained systems with a focus on low-power wireless Internet of Things devices.
  - IoTSyS
  - Raspberry Pi OS
  - LWM2M
  - DuinOS
  - OpenIoT - connected objects, while at the same time generating and processing important business/applications events.
  - OpenRemote
  - Debian
  - Advanced Message Queuing Protocol (AMQP)
  - Trampoline
  - SOAP
  - gRPC
  - LWM2M
  - Constrained Application Protocol (CoAP) - to-machine (M2M) applications such as smart energy and building automation."
  - Extensible Messaging and Presence Protocol (XMPP) - oriented middleware based on XML (Extensible Markup Language). It enables the near real-time exchange of structured yet extensible data between any two or more network entities.
  - OASIS Message Queuing Telemetry Transport (MQTT) - subscribe network protocol that transports messages between devices. The protocol usually runs over TCP/IP; however, any network protocol that provides ordered, lossless, bi-directional connections can support MQTT.
  - Very Simple Control Protocol (VSCP) - or home-automation is in the main focus. Its main advantage is that each VSCP-node can work completely autonomous, being part of distributed network of other nodes.
  - Arm Mbed TLS
  - Kaa - duty, enterprise-grade IoT solutions in mind. It banishes a monolithic approach to architecture in favour of highly portable microservices, which allow for flexible rearrangement and customization even in the middle of the solution's lifecycle.
- Toolkits
  - Thingful
  - openPicus FlyportPro - on-module is the best solution for those customers looking for flexibility and for development time and risk reduction.
  - Pinoccio - Internet access to all yout IoT devices, and it is Arduino compatible. Each board can assume the role of Scout in a Troop and one of the Scouts is the Lead to connect internet: Field Scouts talk to each other using a mesh network (called a Troop), using an extremely low-power radio.
  - KinomaJS
  - IoT Toolkit - performance collection of libraries optimized for minimum memory consumption in RAM, ROM, high speed, and versatility working on any device.
  - Freeboard - time dashboard builder for IOT and other web mashups. A free open-source alternative to Geckoboard.
  - BeagleBone - power open-source single-board computer produced by Texas Instruments. It runs Android, Ubuntu and other Linux flavors.
  - ThingSpeak
  - Arduino Ethernet Shield 2
  - Raspberry Pi - board computers developed in the United Kingdom by the Raspberry Pi Foundation to promote teaching of basic computer science in schools and in developing countries. Price range from $10-45 depending on model.
Open Source Security Learning Resources
- Interfaces
Operating System
- In-memory data grids
  - Rust on Chrome OS
  - Bottlerocket OS - source Linux-based operating system meant for hosting containers. Bottlerocket focuses on security and maintainability, providing a reliable, consistent, and safe platform for container-based workloads.
Python Frameworks and Tools
- In-memory data grids
  - Scikit-Learn
  - Python Package Index (PyPI)
  - PyCharm
  - Django - level Python Web framework that encourages rapid development and clean, pragmatic design.
  - Web2py - source web application framework written in Python allowing allows web developers to program dynamic web content. One web2py instance can run multiple web sites using different databases.
  - Falcon - performance Python web framework for building large-scale app backends and microservices with support for MongoDB, Pluggable Applications and autogenerated Admin.
  - Pillow
  - IPython
  - Pandas
  - Matplotlib - quality figures in a variety of hardcopy formats and interactive environments across platforms.
  - Python Tools for Visual Studio(PTVS)
  - Python Tools for Visual Studio(PTVS)
  - Pylance
  - Pyright
  - AWS Chalice
  - Pipenv
  - Python Fire
  - Bottle - framework for Python. It is distributed as a single file module and has no dependencies other than the [Python Standard Library](https://docs.python.org/library/).
  - Neural Network Intelligence(NNI)
  - Luigi - in.
  - Locust
  - spaCy
  - PuLP
  - Scrapy - level web crawling and web scraping framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.
  - Sanic
  - GraphLab Create - scale, high-performance machine learning models.
  - Sentry
  - Web2py - source web application framework written in Python allowing allows web developers to program dynamic web content. One web2py instance can run multiple web sites using different databases.
  - CherryPy - oriented HTTP web framework.
Python Learning Resources
- In-memory data grids

Programming Languages

Python 102 Go 74 TypeScript 66 JavaScript 62 Rust 54 C++ 43 Shell 27 C 26 Jinja 17 Java 16

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

Self-Hosting-Guide

Kubernetes Tools, Frameworks, and Projects

Interfaces

ML frameworks & applications

Interfaces

Models of Raspberry Pi boards

Setting up WireGuard on Home Assistant

Networking Tools & Concepts

Configure WDT service

Network Protocols

Interfaces

Node.js Learning Resources

In-memory data grids

Node.js Tools

In-memory data grids

Online ML Learning Resources

In-memory data grids

Interfaces

Toolkits

Open Source Security Learning Resources

Interfaces

Operating System

In-memory data grids

Python Frameworks and Tools

In-memory data grids

Python Learning Resources

In-memory data grids