Distributed-Systems-Guide

Distributed Systems Guide
https://github.com/mikeroyal/Distributed-Systems-Guide

Last synced: 11 days ago
JSON representation

DevOps
- Amazon web service(AWS) - to-use and cost-effective cloud computing solutions. The AWS platform is developed with a combination of infrastructure as a service (IaaS), platform as a service (PaaS) and packaged software as a service (SaaS) offerings.
- Azure Draft - native applications on Kubernetes.
File systems & Storage
- GlusterFS - the-shelf hardware, you can create large, distributed storage solutions for media streaming, data analysis, and other data- and bandwidth-intensive tasks.
- Ceph - defined storage solution designed to address the object, block, and file storage needs of data centers adopting open source as the new norm for high-growth block storage, object stores and data lakes. Ceph provides enterprise scalable storage while keeping [CAPEX](https://corporatefinanceinstitute.com/resources/knowledge/modeling/how-to-calculate-capex-formula/) and [OPEX](https://www.investopedia.com/terms/o/operating_expense.asp) costs in line with underlying bulk commodity disk prices.
- ZFS - ready open source file system and volume manager with unprecedented flexibility and an uncompromising commitment to data integrity.
- OpenZFS - source storage platform. It includes the functionality of both traditional file systems and volume manager. It has many advanced features including:
- OpenZFS - source storage platform. It includes the functionality of both traditional file systems and volume manager. It has many advanced features including:
- VMware
- NAS (Network Attached Storage)
Java Learning Resources
Java Tools, Libraries, and Frameworks
- DBeaver - platform database tool for developers, SQL programmers, database administrators and analysts. Supports any database which has JDBC driver (which basically means - ANY database). EE version also supports non-JDBC datasources (MongoDB, Cassandra, Redis, DynamoDB, etc).
- Gradle - language software development. From mobile apps to microservices, from small startups to big enterprises, Gradle helps teams build, automate and deliver better software, faster. Write in Java, C++, Python or your language of choice.
- Java SE
- JDK Development Tools
- IntelliJ IDEA
- RxJava - based programs by using observable sequences. It extends the [observer pattern](http://en.wikipedia.org/wiki/Observer_pattern) to support sequences of data/events and adds operators that allow you to compose sequences together declaratively while abstracting away concerns about things like low-level threading, synchronization, thread-safety and concurrent data structures.
- Guava
- Retrofit - safe HTTP client for Android and Java develped by Square.
- Apache Flink - and batch-processing capabilities with elegant and fluent APIs in Java and Scala.
- Fastjson
- libGDX - platform Java game development framework based on OpenGL (ES) that works on Windows, Linux, Mac OS X, Android, your WebGL enabled browser and iOS.
- Jenkins - source automation server. Built with Java, it provides over 1700 [plugins](https://plugins.jenkins.io/) to support automating virtually anything, so that humans can actually spend their time doing things machines cannot.
- Redisson - Memory Data Grid. Over 50 Redis based Java objects and services: Set, Multimap, SortedSet, Map, List, Queue, Deque, Semaphore, Lock, AtomicLong, Map Reduce, Publish / Subscribe, Bloom filter, Spring Cache, Tomcat, Scheduler, JCache API, Hibernate, MyBatis, RPC, and local cache.
- GraalVM - based languages like Java, Scala, Clojure, Kotlin, and LLVM-based languages such as C and C++.
- Apache Groovy - typing and static compilation capabilities, for the Java platform aimed at improving developer productivity thanks to a concise, familiar and easy to learn syntax. It integrates smoothly with any Java program, and immediately delivers to your application powerful features, including scripting capabilities, Domain-Specific Language authoring, runtime and compile-time meta-programming and functional programming.
- JaCoCo
- Junit
- Mockito
- SpotBugs
- YourKit
- Java Design Patterns
- Elasticsearch
- okhttp
- LeakCanary
- Apache Groovy - typing and static compilation capabilities, for the Java platform aimed at improving developer productivity thanks to a concise, familiar and easy to learn syntax. It integrates smoothly with any Java program, and immediately delivers to your application powerful features, including scripting capabilities, Domain-Specific Language authoring, runtime and compile-time meta-programming and functional programming.
- NetBeans
Julia Learning Resources
Julia Tools, Libraries and Frameworks
- JuliaPro
- Profile (Stdlib)
- JuliaGPU - level syntax and flexible compiler, Julia is well positioned to productively program hardware accelerators like GPUs without sacrificing performance.
- CUDA.jl - friendly array abstraction, a compiler for writing CUDA kernels in Julia, and wrappers for various CUDA libraries.
- JuMP.jl - specific modeling language for [mathematical optimization](https://en.wikipedia.org/wiki/Mathematical_optimization) embedded in Julia.
- Flux.jl - Julia stack, and provides lightweight abstractions on top of Julia's native GPU and AD support.
- Debugger.jl
- Revise.jl - compile.
- IJulia.jl
- AWS.jl
- Nanosoldier.jl
- Optim.jl
- RCall.jl
- PyCall.jl
- MXNet.jl - of-art deep learning to Julia.
- Distributions.jl
- IRTools.jl
- Juno
- XLA.jl
- Julia for VSCode
- JavaCall.jl
- Knet
- DataFrames.jl
- Cassette.jl - in-time (JIT) compilation cycle, enabling post hoc analysis and modification of "Cassette-unaware" Julia programs without requiring manual source annotation or refactoring of the target code.
Learning Resources for ML
MATLAB Learning Resources
MATLAB Tools, Libraries, Frameworks
- Deep Learning Toolbox™ - term memory (LSTM) networks to perform classification and regression on image, time-series, and text data. You can build network architectures such as generative adversarial networks (GANs) and Siamese networks using automatic differentiation, custom training loops, and shared weights. With the Deep Network Designer app, you can design, analyze, and train networks graphically. It can exchange models with TensorFlow™ and PyTorch through the ONNX format and import models from TensorFlow-Keras and Caffe. The toolbox supports transfer learning with DarkNet-53, ResNet-50, NASNet, SqueezeNet and many other pretrained models.
- Reinforcement Learning Toolbox™ - making algorithms for complex applications such as resource allocation, robotics, and autonomous systems.
- Deep Learning HDL Toolbox™ - built bitstreams for running a variety of deep learning networks on supported Xilinx® and Intel® FPGA and SoC devices. Profiling and estimation tools let you customize a deep learning network by exploring design, performance, and resource utilization tradeoffs.
- Parallel Computing Toolbox™ - intensive problems using multicore processors, GPUs, and computer clusters. High-level constructs such as parallel for-loops, special array types, and parallelized numerical algorithms enable you to parallelize MATLAB® applications without CUDA or MPI programming. The toolbox lets you use parallel-enabled functions in MATLAB and other toolboxes. You can use the toolbox with Simulink® to run multiple simulations of a model in parallel. Programs and models can run in both interactive and batch modes.
- ROS Toolbox
- Robotics Toolbox™ - holonomic vehicle. The Toolbox also including a detailed Simulink model for a quadrotor flying robot.
- Image Processing Toolbox™ - standard algorithms and workflow apps for image processing, analysis, visualization, and algorithm development. You can perform image segmentation, image enhancement, noise reduction, geometric transformations, image registration, and 3D image processing.
- Computer Vision Toolbox™
- Model Predictive Control Toolbox™ - loop simulations, you can evaluate controller performance.
- Vision HDL Toolbox™ - streaming algorithms for the design and implementation of vision systems on FPGAs and ASICs. It provides a design framework that supports a diverse set of interface types, frame sizes, and frame rates. The image processing, video, and computer vision algorithms in the toolbox use an architecture appropriate for HDL implementations.
- UAV Toolbox
- Lidar Toolbox™ - camera cross calibration for workflows that combine computer vision and lidar processing.
- Mapping Toolbox™
- Statistics and Machine Learning Toolbox™
- Partial Differential Equation Toolbox™
- MATLAB and Simulink Services & Applications List
- MATLAB in the Cloud - cloud) including [AWS](https://aws.amazon.com/) and [Azure](https://azure.microsoft.com/).
- Simulink - Based Design. It supports simulation, automatic code generation, and continuous testing of embedded systems.
- Simulink Online™
- MATLAB Drive™
- SoC Blockset™
- Wireless HDL Toolbox™ - verified, hardware-ready Simulink® blocks and subsystems for developing 5G, LTE, and custom OFDM-based wireless communication applications. It includes reference applications, IP blocks, and gateways between frame and sample-based processing.
- ThingSpeak™ - of-concept IoT systems that require analytics.
- hctsa - series analysis using Matlab.
- YALMIP
- hctsa - series analysis using Matlab.
- MATLAB Schemer
- SEA-MAT
- Gramm - level interface to produce publication-quality plots of complex data with varied statistical visualizations. Gramm is inspired by R's ggplot2 library.
- GNU Octave - level interpreted language, primarily intended for numerical computations. It provides capabilities for the numerical solution of linear and nonlinear problems, and for performing other numerical experiments. It also provides extensive graphics capabilities for data visualization and manipulation.
Microservices
- AWS ECS - performance container orchestration service that supports Docker containers and allows you to easily run and scale containerized applications on AWS. Amazon ECS eliminates the need for you to install and operate your own container orchestration software, manage and scale a cluster of virtual machines, or schedule containers on those virtual machines.
- AWS CodeBuild
- CFEngine - source configuration management system, written by Mark Burgess.Its primary function is to provide automated configuration and maintenance of large-scale computer systems, including the unified management of servers, desktops, consumer and industrial devices, embedded networked devices, mobile smartphones, and tablet computers.
- Octpus Deploy - premises or in the cloud.
- AWS CodeDeploy - premises servers. AWS CodeDeploy makes it easier for you to rapidly release new features, helps you avoid downtime during application deployment, and handles the complexity of updating your applications.
- AWS Lambda - driven, serverless computing platform provided by Amazon as a part of the Amazon Web Services. It is a computing service that runs code in response to events and automatically manages the computing resources required by that code.
- Traefik - source Edge Router that makes publishing your services a fun and easy experience. It receives requests on behalf of your system and finds out which components are responsible for handling them. What sets Traefik apart, besides its many features, is that it automatically discovers the right configuration for your services.
ML Frameworks, Libraries, and Tools
- Amazon SageMaker
- Apple CoreML - tune models, all on the user's device. A model is the result of applying a machine learning algorithm to a set of training data. You use a model to make predictions based on new input data.
- Fuzzy logic - tree processing and better integration with rules-based programming.
- ResearchGate
- OpenClipArt
- Convolutional Neural Networks (R-CNN)
- CS231n
- Slideteam
- DeepAI
- wikimedia
- Decision trees - structured models for classification and regression.
- CMU
- Naive Bayes - theorem.html) with strong independence assumptions between the features.
- mathisfun
- wikimedia
- wikimedia
- wikimedia
- wikimedia
- wikimedia
- wikimedia
- wikimedia
- wikimedia
- wikimedia
- wikimedia
- wikimedia
- wikimedia
- wikimedia
- Tensorflow_macOS - optimized version of TensorFlow and TensorFlow Addons for macOS 11.0+ accelerated using Apple's ML Compute framework.
- nGraph - of-use to AI developers.
- Tensorman
- Numba - aware optimizing compiler for Python sponsored by Anaconda, Inc. It uses the LLVM compiler project to generate machine code from Python syntax. Numba can compile a large subset of numerically-focused Python, including many NumPy functions. Additionally, Numba has support for automatic parallelization of loops, generation of GPU-accelerated code, and creation of ufuncs and C callbacks.
- cuML - learn.
- Support Vector Machine (SVM) - group classification problems.
- TensorFlow - to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML powered applications.
Networking Tools & Concepts
- cURL - line tool for transferring data using various network protocols(HTTP, HTTPS, FTP, FTPS, SCP, SFTP, TFTP, DICT, TELNET, LDAP LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP or SMTPS). cURL is also used in cars, television sets, routers, printers, audio equipment, mobile phones, tablets, settop boxes, media players and is the Internet transfer engine for thousands of software applications in over ten billion installations.
- cURL Fuzzer
- DoH - alone application for DoH (DNS-over-HTTPS) name resolves and lookups.
- HTTPie - line HTTP client. Its goal is to make CLI interaction with web services as human-friendly as possible. HTTPie is designed for testing, debugging, and generally interacting with APIs & HTTP servers.
- HTTPStat
- Wuzz
- Websocat - line client for WebSockets, like netcat (or curl) for ws:// with advanced socat-like functions.
Network Learning Resources
Network Protocols
- OAuth 2.0 - party applications to access the user account.
NLP Learning Resources

Programming Languages

Python 21 C++ 12 Julia 10 C 6 Java 5 Scala 5 HTML 3 C# 3 Rust 3 Kotlin 3

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

Distributed-Systems-Guide

DevOps

File systems & Storage

Java Learning Resources

Java Tools, Libraries, and Frameworks

Julia Learning Resources

Julia Tools, Libraries and Frameworks

Learning Resources for ML

MATLAB Learning Resources

MATLAB Tools, Libraries, Frameworks

Microservices

ML Frameworks, Libraries, and Tools

Networking Tools & Concepts

Network Learning Resources

Network Protocols

NLP Learning Resources