https://github.com/cheatsnake/backend-cheats
π White paper for Backend developers
https://github.com/cheatsnake/backend-cheats
List: backend-cheats
architectural-patterns architecture awesome awesome-list backend backend-development cheat-sheets cheatsheet databases development helpful internet knowledge-sharing language-learning operating-system-learning patterns roadmap security teaching-materials
Last synced: 8 months ago
JSON representation
π White paper for Backend developers
- Host: GitHub
- URL: https://github.com/cheatsnake/backend-cheats
- Owner: cheatsnake
- License: mit
- Created: 2022-02-27T15:27:14.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2025-02-02T14:26:58.000Z (over 1 year ago)
- Last Synced: 2025-09-11T02:49:41.945Z (9 months ago)
- Topics: architectural-patterns, architecture, awesome, awesome-list, backend, backend-development, cheat-sheets, cheatsheet, databases, development, helpful, internet, knowledge-sharing, language-learning, operating-system-learning, patterns, roadmap, security, teaching-materials
- Homepage:
- Size: 10 MB
- Stars: 3,810
- Watchers: 46
- Forks: 381
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- ultimate-awesome - backend-cheats - π White paper for Backend developers. (Other Lists / TeX Lists)
README
This repository is a visual cheatsheet on the main topics in Backend-development. All material is divided into chapters that include different topics. There are three main parts to each topic:
- **Visual part** - various images/tables/cheatsheets for better understanding (may not be available). All pictures and tables are made from scratch, specifically for this repository.
- **Summary** - A very brief summary with a list of key terms and concepts. The terms are hyperlinked to the appropriate section on Wikipedia or a similar reference resource.
- **References to sources** - resources where you may find complete information on a particular issue (they are hidden under a spoiler, which opens when clicked). If possible, the most authoritative sources are indicated, or those that provide information in as simple and comprehensible language as possible.
> π Available translations: [`English`](https://github.com/cheatsnake/backend-cheats/blob/master/README.md) [`Π ΡΡΡΠΊΠΈΠΉ`](https://github.com/cheatsnake/backend-cheats/blob/master/README_RUS.md)
> π€ If you want to help the project, feel free to send your [issues](https://github.com/cheatsnake/backend-cheats/issues) or [pull requests](https://github.com/cheatsnake/backend-cheats/pulls).
> π For better experience enable [dark theme](https://github.com/settings/appearance).
## Contents
- [Network & Internet](#network--internet)
* [How the Internet works](#how-the-internet-works)
* [What is a domain name](#what-is-a-domain-name)
* [IP address](#ip-address)
* [What is DNS](#what-is-dns)
* [Web application design](#web-application-design)
* [Browsers and how they work](#browsers-and-how-they-work)
* [VPN and Proxy](#vpn-and-proxy)
* [Hosting](#hosting)
* [OSI network model](#osi-network-model)
* [HTTP Protocol](#http-protocol)
* [TCP/IP stack](#tcpip-stack)
* [Network problems](#network-problems)
* [Network diagnostics](#network-diagnostics)
- [PC device](#pc-device)
* [Main components (hardware)](#main-components-hardware)
* [Operating system design](#operating-system-design)
* [Processes and threads](#processes-and-threads)
* [Concurrency and parallelism](#concurrency-and-parallelism)
* [Inter-process communication](#inter-process-communication)
- [Linux basics](#linux-basics)
* [Working with shell](#working-with-shell)
* [Package manager](#package-manager)
* [Bash scripts](#bash-scripts)
* [Users, groups, and permissions](#users-groups-and-permissions)
* [Working with processes](#working-with-processes)
* [Working with SSH](#working-with-ssh)
* [Network utils](#network-utils)
* [Task scheduler](#task-scheduler)
* [System logs](#system-logs)
* [Main issues with Linux](#main-issues-with-linux)
- [General knowledge](#general-knowledge)
* [Numeral systems](#numeral-systems)
* [Logical connective](#logical-connective)
* [Data structures](#data-structures)
* [Basic algorithms](#basic-algorithms)
* [Algorithm complexity](#algorithm-complexity)
* [Data storage formats](#data-storage-formats)
* [Text encodings](#text-encodings)
- [Programming Language](#programming-language)
* [Classification of programming languages](#classification-of-programming-languages)
* [Language Basics](#language-basics)
* [Object-oriented programming](#object-oriented-programming)
* [Server development](#server-development)
* [Asynchronous programming](#asynchronous-programming)
* [Multitasking](#multitasking)
* [Advanced Topics](#advanced-topics)
* [Code quality](#code-quality)
- [Databases](#databases)
* [Database classification](#database-classification)
* [Relational database](#relational-database)
* [MongoDB](#mongodb)
* [Redis](#redis)
* [ACID Requirements](#acid-requirements)
* [Designing databases](#designing-databases)
- [API development](#api-development)
* [REST API](#rest-api)
* [GraphQL](#graphql)
* [WebSockets](#websockets)
* [RPC (Remote Procedure Call)](#rpc-remote-procedure-call)
* [WebRTC](#webrtc)
- [Software](#security)
* [Git version control system](#git-version-control-system)
* [Docker](#docker)
* [Postman/Insomnia](#postmaninsomnia)
* [Web servers](#web-servers)
* [Message brokers](#message-brokers)
* [Ngrok](#ngrok)
* [AI tools](#ai-tools)
- [Security](#security)
* [Web application vulnerabilities](#web-application-vulnerabilities)
* [Environment variables](#environment-variables)
* [Hashing](#hashing)
* [Authentication and authorization](#authentication-and-authorization)
* [SSL/TLS](#ssltls)
- [Testing](#testing)
* [Unit Tests](#unit-tests)
* [Integration tests](#integration-tests)
* [E2E tests](#e2e-tests)
* [Load testing](#load-testing)
* [Regression testing](#regression-testing)
- [Deployment (CI/CD)](#deployment-cicd)
* [Cloud services](#cloud-services)
* [Container orchestration](#container-orchestration)
* [Automation tools](#automation-tools)
* [Monitoring and logs](#monitoring-and-logs)
- [Optimization](#optimization)
* [Profiling](#profiling)
* [Benchmarks](#benchmarks)
* [Caching](#caching)
* [Load balancing](#load-balancing)
- [Documentation](#documentation)
* [Markdown](#markdown)
* [Documentation inside code](#documentation-inside-code)
* [API Documentation](#api-documentation)
* [Static generators](#static-generators)
- [Building Architecture](#building-architecture)
* [Architectural Patterns](#architectural-patterns)
* [Design patterns](#design-patterns)
* [Monolithic and microservice architecture](#monolithic-and-microservice-architecture)
* [Horizontal and vertical scaling](#horizontal-and-vertical-scaling)
- [Additional and similar resources](#additional-and-similar-resources)
## Network & Internet
[Internet](https://en.wikipedia.org/wiki/Internet) is a worldwide system that connects computer networks from around the world into a single network for storing/transferring information. The Internet was originally developed for the military. But soon it began to be implemented in universities, and then it could be used by private companies, which began to organize networks of providers that provide Internet access services to ordinary citizens. By early 2020, the number of Internet users exceeded 4.5 billion.
- ### How the Internet works
Your computer does not have direct access to the Internet. Instead, it has access to your local network to which other devices are connected via a wired ([Ethernet](https://en.wikipedia.org/wiki/Ethernet)) or wireless (Wi-Fi) connection. The organizer of such a network is a special minicomputer - [router](). This device connects you to your [Internet Service Provider (ISP)](https://en.wikipedia.org/wiki/Internet_service_provider), which in turn is connected to other higher-level ISPs. Thus, all these interactions make up the Internet, and your messages always transit through different networks before reaching the final recipient.
- [Host]()
> Any device that is on any network.
- [Server]()
> A special computer on the network that serves requests from other computers.
- [Network topologies](https://en.wikipedia.org/wiki/Network_topology)
> There are several topologies (ways of organizing a network): [Point to point](), [Daisy chain](), [Bus](https://en.wikipedia.org/wiki/Bus_network), [Ring](https://en.wikipedia.org/wiki/Ring_network), [Star](https://en.wikipedia.org/wiki/Star_network) and [Mesh](https://en.wikipedia.org/wiki/Mesh_networking). The Internet itself cannot be referred to any one topology, because it is an incredibly complex system mixed with different topologies.
π References
1. π [**How does the Internet work?** β MDN](https://developer.mozilla.org/en-US/docs/Learn/Common_questions/How_does_the_Internet_work)
2. πΊ [**How does the internet work? (Full Course)** β YouTube](https://youtu.be/zN8YNNHcaZc)
3. πΊ [**What is a Server? Servers vs. Desktops Explained** β YouTube](https://youtu.be/UjCDWCeHCzY)
4. πΊ [**Network Topology** β YouTube](https://youtu.be/lnFeG4DOMcE)
5. πΊ [**Network Topologies (Star, Bus, Ring, Mesh, Ad hoc, Infrastructure, & Wireless Mesh Topology)** β YouTube](https://youtu.be/zbqrNg4C98U)
- ### What is a domain name
[Domain Names](https://en.wikipedia.org/wiki/Domain_name) are human-readable addresses of web servers available on the Internet. They consist of parts (levels) separated from each other by a dot. Each of these parts provides specific information about the domain name. For example country, service name, localization, etc.
- Who owns domain names
> [The ICANN Corporation](https://en.wikipedia.org/wiki/ICANN) is the founder of the distributed domain registration system. It gives accreditations to companies that want to sell domains. In this way a competitive domain market is formed.
- How to buy a domain name
> A domain name cannot be bought forever. It is leased for a certain period of time. It is better to buy domains from [accredited registrars](https://www.icann.org/en/accredited-registrars?filter-letter=a&sort-direction=asc&sort-param=name&page=1) (you can find them in almost any country).
π References
1. π [**What is a Domain Name?** β MDN](https://developer.mozilla.org/en-US/docs/Learn/Common_questions/What_is_a_domain_name)
2. πΊ [**A Beginners Guide to How Domain Names Work!** β YouTube](https://youtu.be/Y4cRx19nhJk)
- ### IP address
[IP address](https://en.wikipedia.org/wiki/IP_address) is a unique numeric address that is used to recognize a particular device on the network.
- Levels of visibility
> - External and publicly accessible IP address that belongs to your ISP and is used to access the Internet by hundreds of other users.
> - The IP address of your router in your ISP's local network, the same IP address from which you access the Internet.
> - The IP address of your computer in the local (home) network created by the router, to which you can connect your devices. Typically, it looks like 192.168.XXX.XXX.
> - The internal IP address of the computer, inaccessible from the outside and used only for communication between the running processes. It is the same for everyone - 127.0.0.1 or just _localhost_.
- [Port]()
> One device (computer) can run many applications that use the network. In order to correctly recognize where and which data coming over the network should be delivered (to which of the applications) a special numerical number - a port is used. That is, each running process on a computer which uses a network connection has its own personal port.
- [IPv4](https://en.wikipedia.org/wiki/IPv4)
> Version 4 of the IP protocol. It was developed in 1981 and limits the address space to about 4.3 billion (2^32) possible unique addresses.
- [IPv6](https://en.wikipedia.org/wiki/IPv6)
> Over time, the allocation of address space began to happen at a much faster rate, forcing the creation of a new version of the IP protocol to store more addresses. IPv6 is capable of issuing 2^128 (is huge number) unique addresses.
π References
1. πΊ [**IP addresses. Explained** β YouTube](https://youtu.be/7_-qWlvQQtY)
2. πΊ [**Public IP vs. Private IP and Port Forwarding (Explained by Example)** β YouTube](https://youtu.be/92b-jjBURkw)
3. πΊ [**Network Ports Explained** β YouTube](https://youtu.be/g2fT-g9PX9o)
4. πΊ [**What is IP address and types of IP address - IPv4 and IPv6** β YouTube](https://youtu.be/8npT9AALbrI)
5. πΊ [**IP Address - IPv4 vs. IPv6 Tutorial** β YouTube](https://youtu.be/ThdO9beHhpA)
6. π [**IP Address Subnet Cheat Sheet** β freeCodeCamp](https://www.freecodecamp.org/news/subnet-cheat-sheet-24-subnet-mask-30-26-27-29-and-other-ip-address-cidr-network-references/)
- ### What is DNS
[DNS (Domain Name System)](https://en.wikipedia.org/wiki/DNS) is a decentralized Internet address naming system that allows you to create human-readable alphabetical names (domain names) corresponding to the numeric [IP addresses](#ip-address) used by computers.
- Structure of DNS
> DNS consists of many independent nodes, each of which stores only those data that fall within its area of responsibility.
- [DNS Resolver](https://www.computerhope.com/jargon/d/dns-resolver.htm)
> A server that is located in close proximity to your Internet Service Provider. It is the server that searches for addresses by domain name, and also caches them (temporarily storing them for quick retrieval in future requests).
- [DNS record types](https://en.wikipedia.org/wiki/List_of_DNS_record_types)
> - A record - associates the domain name with an IPv4 address.
> - AAAA record - links a domain name with an IPv6 address.
> - CNAME record - redirects to another domain name.
> - and others - MX record, NS record, PTR record, SOA record.
π References
1. π [**What is DNS? Domain Name System explained** β freeCodeCamp](https://www.freecodecamp.org/news/what-is-dns/)
2. πΊ [**DNS (Domain Name System) explained. Types of Domain Name Servers** β YouTube](https://youtu.be/JkEYOt08-rU)
3. πΊ [**DNS as Fast As Possible** β YouTube](https://youtu.be/Rck3BALhI5c)
4. π [**All about DNS records** β Cloudflare](https://www.cloudflare.com/learning/dns/dns-records/)
5. πΊ [**DNS records explained (playlist)** β YouTube](https://youtube.com/playlist?list=PL_vyuxE-AO-DD94NKcCqd4iqwy5ah_pwq)
- ### Web application design
Modern [web applications](https://en.wikipedia.org/wiki/Web_application) consist of two parts: [Frontend and Backend](https://en.wikipedia.org/wiki/Frontend_and_backend). Thus implementing a [client-server model](https://en.wikipedia.org/wiki/Client%E2%80%93server_model).
The tasks of the Frontend are:
- Implementation of the user interface (appearance of the application)
> - A special markup language [HTML](https://en.wikipedia.org/wiki/HTML) is used to create web pages.
> - [CSS](https://en.wikipedia.org/wiki/CSS) style language is used to style fonts, layout of content, etc.
> - [JavaScript](https://en.wikipedia.org/wiki/JavaScript) programming language is used to add dynamics and interactivity.
> As a rule, these tools are rarely used in their pure form, as so-called [frameworks](https://2020.stateofjs.com/en-US/technologies/front-end-frameworks/) and [preprocessors](https://www.freecodecamp.org/news/css-preprocessors/) exist for more convenient and faster development.
- Creating functionality for generating requests to the server
> These are usually different types of input forms that can be conveniently interacted with.
- Receives data from the server and then processes it for output to the client
Tasks of the Backend:
- Handling client requests
> Checking for permissions and access, all sorts of validations, etc.
- Implementing business logic
> A wide range of tasks can be implied here: working with databases, information processing, computation, etc. This is, so to speak, the heart of the Backend world. This is where all the important and interesting stuff happens.
- Generating a response and sending it to the client
π References
1. π [**Front-End vs. Back-End explained**](https://blog.teamtreehouse.com/i-dont-speak-your-language-frontend-vs-backend)
2. πΊ [**Everything You NEED to Know About WEB APP Architecture** β YouTube](https://youtu.be/sDlCSIDwpDs)
- ### Browsers and how they work
[Browser](https://en.wikipedia.org/wiki/Web_browser) is a client which can be used to send requests to a server for files which can then be used to render web pages. In simple terms, a browser can be thought of as a program for viewing HTML files, which can also search for and download them from the Internet.
- Working Principle
> Query handling, page rendering, and the tabs feature (each tab has its own process to prevent the contents of one tab from affecting the contents of the other).
- [Extensions](https://developer.chrome.com/docs/extensions/)
> Allow you to change the browser's user interface, modify the contents of web pages, and modify the browser's network requests.
- [Chrome DevTools](https://developer.chrome.com/docs/devtools/overview/)
> An indispensable tool for any web developer. It allows you to analyze all possible information related to web pages, monitor their performance, logs and, most importantly for us, track information about network requests.
π References
1. π [**How browsers work** β MDN](https://developer.mozilla.org/en-US/docs/Web/Performance/How_browsers_work)
2. π [**How browsers work: Behind the scenes of modern web browsers** β web.dev](https://web.dev/howbrowserswork/)
3. π [**Inside look at modern web browser** β Google](https://developer.chrome.com/blog/inside-browser-part1/)
4. πΊ [**What is a web browser?** β YouTube](https://youtu.be/QzohDuGk4mM)
5. πΊ [**Anatomy of the browser 101 (Chrome University 2019)** β YouTube](https://youtu.be/PzzNuCk-e0Y)
6. πΊ [**Chrome DevTools - Crash Course** β YouTube](https://youtu.be/gTVpBbFWry8)
7. πΊ [**Demystifying the Browser Networking Tab in DevTools** β YouTube](https://youtu.be/LBgfSwX4GDI)
8. πΊ [**21+ Browser Dev Tools & Tips You Need To Know** β YouTube](https://youtu.be/TcTSqhpm80Y)
- ### VPN and Proxy
The use of VPNs and Proxy is quite common in recent years. With the help of these technologies, users can get basic anonymity when surfing the web, as well as bypass various regional blockages.
- [VPN (Virtual Private Network)](https://en.wikipedia.org/wiki/VPN)
> A technology that allows you to become a member of a private network (similar to your local network), where requests from all participants go through a single public IP address. This allows you to blend in with the general mass of requests from other participants.
>
> - Simple procedure for connection and use.
> - Reliable traffic encryption.
> - There is no guarantee of 100% anonymity, because the owner of the network knows the IP-addresses of all participants.
> - VPNs are useless for dealing with multi-accounts and some programs because all accounts operating from the same VPN are easily detected and blocked.
> - Free VPNs tend to be heavily loaded, resulting in unstable performance and slow download speeds.
- [Proxy (proxy server)](https://en.wikipedia.org/wiki/Proxy_server)
> A proxy is a special server on the network that acts as an intermediary between you and the destination server you intend to reach. When you are connected to a proxy server all your requests will be performed on behalf of that server, that is, your IP address and location will be substituted.
>
> - The ability to use an individual IP address, which allows you to work with multi-accounts.
> - Stability of the connection due to the absence of high loads.
> - Connection via proxy is provided in the operating system and browser, so no additional software is required.
> - There are proxy varieties that provide a high level of anonymity.
> - The unreliability of free solutions, because the proxy server can see and control everything you do on the Internet.
π References
1. π [**What is VPN? How It Works, Types of VPN** β kaspersky.com](https://www.kaspersky.com/resource-center/definitions/what-is-a-vpn)
2. πΊ [**VPN (Virtual Private Network) Explained** β YouTube](https://youtu.be/R-JUOpCgTZc)
3. πΊ [**What Is a Proxy and How Does It Work?** β YouTube](https://youtu.be/ayo2EUPTEkE)
4. πΊ [**What is a Proxy Server?** β YouTube](https://youtu.be/5cPIukqXe5w)
5. πΊ [**Proxy vs. Reverse Proxy (Explained by Example)** β YouTube](https://youtu.be/ozhe__GdWC8)
6. πΊ [**VPN vs. Proxy Explained Pros and Cons** β YouTube](https://youtu.be/npnqyRT77Zc)
- ### Hosting
[Hosting](https://en.wikipedia.org/wiki/Web_hosting_service) is a special [service provided](https://en.wikipedia.org/wiki/Internet_hosting_service) by hosting providers, which allows you to rent space on a server (which is connected to the Internet around the clock), where your data and files can be stored. There are different options for hosting, where you can use not only the disk space of the server, but also the CPU power to run your network applications.
- [Virtual hosting](https://en.wikipedia.org/wiki/Virtual_hosting)
> One physical server that distributes its resources to multiple tenants.
- [VPS/VDS](https://en.wikipedia.org/wiki/Virtual_private_server)
> Virtual servers that emulate the operation of a separate physical server and are available for rent to the client with maximum privileges.
- [Dedicated server](https://en.wikipedia.org/wiki/Dedicated_hosting_service)
> Renting a full physical server with full access to all resources. As a rule, this is the most expensive service.
- [Cloud hosting](https://en.wikipedia.org/wiki/Cloud_storage)
> A service that uses the resources of several servers. When renting, the user pays only for the actual resources used.
- [Colocation](https://en.wikipedia.org/wiki/Colocation_centre)
> A service that gives the customer the opportunity to install their equipment on the provider's premises.
π References
1. π [**What is Web Hosting?** β namecheap.com](https://www.namecheap.com/hosting/what-is-web-hosting-definition/)
2. πΊ [**What is Web Hosting and How Does It Work?** β YouTube](https://youtu.be/H8oAvyqQwew)
3. πΊ [**Different Hosting Types Explained** β YouTube](https://youtu.be/CtNWVmt9U1M)
4. π [**Awesome Hosting** β GitHub](https://github.com/dalisoft/awesome-hosting)
- ### OSI network model
| β | Level | Used protocols |
| --- | ------------------ | -------------------- |
| 7 | Application layer | HTTP, DNS, FTP, POP3 |
| 6 | Presentation layer | SSL, SSH, IMAP, JPEG |
| 5 | Session layer | APIs Sockets |
| 4 | Transport layer | TCP, UDP |
| 3 | Network layer | IP, ICMP, IGMP |
| 2 | Data link layer | Ethernet, MAC, HDLC |
| 1 | Physical layer | RS-232, RJ45, DSL |
[OSI (The Open Systems Interconnection model)](https://en.wikipedia.org/wiki/OSI_model) is a set of rules describing how different devices should interact with each other on the network. The model is divided into 7 layers, each of which is responsible for a specific function. All this is to ensure that the process of information exchange in the network follows the same pattern and all devices, whether it is a smart fridge or a smartphone, can understand each other without any problems.
- [Physical layer](https://en.wikipedia.org/wiki/Physical_layer)
> At this level, bits (ones/zeros) are encoded into physical signals (current, light, radio waves) and transmitted further by wire ([Ethernet](https://en.wikipedia.org/wiki/Ethernet)) or wirelessly ([Wi-Fi](https://en.wikipedia.org/wiki/Wi-Fi)).
- [Data link layer](https://en.wikipedia.org/wiki/Data_link_layer)
> Physical signals from layer 1 are decoded back into ones and zeros, errors and defects are corrected, and the sender and receiver [MAC addresses](https://en.wikipedia.org/wiki/MAC_address) are extracted.
- [Network layer](https://en.wikipedia.org/wiki/Network_layer)
> This is where traffic routing, DNS queries and [IP packet](https://en.wikipedia.org/wiki/Internet_Protocol) generation take place.
- [Transport layer](https://en.wikipedia.org/wiki/Transport_layer)
> The layer responsible for data transfer. There are two important protocols:
>
> - [TCP](https://en.wikipedia.org/wiki/Transmission_Control_Protocol) is a protocol that ensures reliable data transmission. TCP guarantees data delivery and preserves the order of the messages. This has an impact on the transmission speed. This protocol is used where data loss is unacceptable, such as when sending mail or loading web pages.
> - [UDP](https://en.wikipedia.org/wiki/User_Datagram_Protocol) is a simple protocol with fast data transfer. It does not use mechanisms to guarantee the delivery and ordering of data. It is used e.g., in online games where partial packet loss is not crucial, but the speed of data transfer is much more important. Also, requests to DNS servers are made through UDP protocol.
- [Session layer](https://en.wikipedia.org/wiki/Session_layer)
> Responsible for opening and closing communications (sessions) between two devices. Ensures that the session stays open long enough to transfer all necessary data, and then closes quickly to avoid wasting resources.
- [Presentation layer](https://en.wikipedia.org/wiki/Presentation_layer)
> Transmission, encryption/decryption and data compression. This is where data that comes in the form of zeros and ones are converted into desired formats (PNG, MP3, PDF, etc.)
- [Application layer](https://en.wikipedia.org/wiki/Application_layer)
> Allows the user's applications to access network services such as database query handler, file access, email forwarding.
π References
1. π [**Layers of OSI Model** β geeksForGeeks](https://www.geeksforgeeks.org/layers-of-osi-model/)
2. πΊ [**The OSI Model - Explained by Example** β YouTube](https://youtu.be/7IS7gigunyI)
3. πΊ [**TCP vs. UDP Crash Course** β YouTube](https://youtu.be/qqRYkcta6IE)
- ### HTTP Protocol
[HTTP (HyperText Transport Protocol)](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol) is the most important protocol on the Internet. It is used to transfer data of any format. The protocol itself works according to a simple principle: request -> response.
- [Structure of HTTP messages](https://developer.mozilla.org/en-US/docs/Web/HTTP/Messages)
> HTTP messages consist of a header section containing metadata about the message, followed by an optional message body containing the data being sent.
- [Headers](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers)
> Additional service information that is sent with the request/response.
> Common headers: [Host](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Host), [User-Agent](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent), [If-Modified-Since](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Modified-Since), [Cookie](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cookie), [Referer](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referer), [Authorization](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Authorization), [Cache-Control](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control), [Content-Type](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type), [Content-Length](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Length), [Last-Modified](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Last-Modified), [Set-Cookie](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie), [Content-Encoding](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding).
- [Request methods](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods)
> Main: [GET](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/GET), [POST](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/POST), [PUT](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/PUT), [DELETE](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/DELETE).
Others: [HEAD](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/HEAD), [CONNECT](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/CONNECT), [OPTIONS](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/OPTIONS), [TRACE](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/TRACE), [PATCH](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/PATCH).
- [Response status codes](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status)
> Each response from the server has a special numeric code that characterizes the state of the sent request. These codes are divided into 5 main classes:
> - **1Ρ
Ρ
** - Service information
> - **2Ρ
Ρ
** - Successful request
> - **3Ρ
Ρ
** - Redirect to another address
> - **4Ρ
Ρ
** - Client side error
> - **5Ρ
Ρ
** - Server side error
- [HTTPS](https://developer.mozilla.org/en-US/docs/Glossary/https)
> Same HTTP, but with encryption support. Your apps should use HTTPS to be secure.
- [Cookie](https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies)
> The HTTP protocol does not provide the ability to save information about the status of previous requests and responses. Cookies are used to solve this problem. Cookies allow the server to store information on the client side that the client can send back to the server. For example, cookies can be used to authenticate users or to store various settings.
- [CORS (Cross origin resource sharing)](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS)
> A technology that allows one domain to securely receive data from another domain.
- [CSP (Content Security Policy)](https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP)
> A special header that allows you to recognize and eliminate certain types of web application vulnerabilities.
- [Evolution of HTTP](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Evolution_of_HTTP)
> - **HTTP/1.0**: Uses separate connections for each request/response, lacks caching support, and has plain text headers.
> - **HTTP/1.1**: Introduces persistent connections, pipelining, the Host header, and chunked transfer encoding.
> - **HTTP/2**: Supports multiplexing, header compression, server push, and support a binary data.
> - **HTTP/3**: Built on [QUIC](https://developer.mozilla.org/en-US/docs/Glossary/QUIC), offers improved multiplexing, reliability, and better performance over unreliable networks.
π References
1. π [**How HTTP Works and Why it's Important** β freeCodeCamp](https://www.freecodecamp.org/news/how-the-internet-works/)
2. π [**Hypertext Transfer Protocol (HTTP)** β MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP)
3. πΊ [**Hyper Text Transfer Protocol Crash Course** β YouTube](https://youtu.be/0OrmKCB0UrQ)
4. πΊ [**Full HTTP Networking Course (5 hours)** β YouTube](https://youtu.be/2JYT5f2isg4)
5. π [**HTTP vs. HTTPS β What's the Difference?** β freeCodeCamp](https://www.freecodecamp.org/news/http-vs-https/)
6. πΊ [**HTTP Cookies Crash Course** β YouTube](https://youtu.be/sovAIX4doOE)
7. πΊ [**Cross Origin Resource Sharing (Explained by Example)** β YouTube](https://youtu.be/Ka8vG5miErk)
8. πΊ [**When to use HTTP GET vs. POST?** β YouTube](https://youtu.be/K8HJ6DN23zI)
9. πΊ [**How HTTP/2 Works, Performance, Pros & Cons and More** β YouTube](https://youtu.be/fVKPrDrEwTI)
10. πΊ [**HTTP/2 Critical Limitation that led to HTTP/3 & QUIC** β YouTube](https://youtu.be/GriONb4EfPY)
11. πΊ [**304 Not Modified HTTP Status (Explained with Code Example and Pros & Cons)** β YouTube](https://youtu.be/0QHmHR55_Lo)
12. πΊ [**What is the Largest POST Request the Server can Process?** β YouTube](https://youtu.be/0QHmHR55_Lo)
- ### TCP/IP stack
Compared to the [OSI model](#osi-network-model), the [TCP/IP](https://en.wikipedia.org/wiki/Internet_protocol_suite) stack has a simpler architecture. In general, the TCP/IP model is more widely used and practical, and the OSI model is more theoretical and detailed. Both models describe the same principles, but differ in the approach and protocols they include at their levels.
- [Link layer](https://en.wikipedia.org/wiki/Link_layer)
> Defines how data is transmitted over the physical medium, such as cables or wireless signals.
> Protocols: [Ethernet](https://en.wikipedia.org/wiki/Ethernet), [Wi-Fi](https://en.wikipedia.org/wiki/Wi-Fi), [Bluetooth](https://en.wikipedia.org/wiki/Bluetooth), [Fiber optic](https://en.wikipedia.org/wiki/Optical_fiber).
- [Internet Layer](https://en.wikipedia.org/wiki/Internet_layer)
> Routing data across different networks. It uses IP addresses to identify devices and routes data packets to their destination.
> Protocols: [IP](https://en.wikipedia.org/wiki/Internet_Protocol), [ARP](https://en.wikipedia.org/wiki/Address_Resolution_Protocol), [ICMP](https://en.wikipedia.org/wiki/Internet_Control_Message_Protocol), [IGMP](https://en.wikipedia.org/wiki/Internet_Group_Management_Protocol)
- [Transport Layer](https://en.wikipedia.org/wiki/Transport_layer)
> Data transmission between two devices. It uses protocols such as [TCP](https://en.wikipedia.org/wiki/Transmission_Control_Protocol) - reliable, but slow and [UDP](https://en.wikipedia.org/wiki/User_Datagram_Protocol) - fast, but unreliable.
- [Application Layer](https://en.wikipedia.org/wiki/Application_layer)
> Provides services to the end user, such as web browsing, email, and file transfer. It interacts with the lower layers of the stack to transmit data over the network.
> Protocols: [HTTP](#http-protocol), [FTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol), [SMTP](https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol), [DNS](https://en.wikipedia.org/wiki/Domain_Name_System), [SNMP](https://en.wikipedia.org/wiki/Simple_Network_Management_Protocol).
π References
1. π [**What is the TCP/IP Model? Layers and Protocols Explained** β freeCodeCamp](https://www.freecodecamp.org/news/what-is-tcp-ip-layers-and-protocols-explained/)
2. πΊ [**What is TCP/IP?** β YouTube](https://youtu.be/PpsEaqJV_A0)
3. πΊ [**How TCP really works. Three-way handshake. TCP/IP Deep Dive** β YouTube](https://youtu.be/rmFX1V49K8U)
- ### Network problems
The quality of networks, including the Internet, is far from ideal. This is due to the complex structure of networks and their dependence on a huge number of factors. For example, the stability of the connection between the client device and its router, the quality of service of the provider, the power, and performance of the server, the physical distance between the client and the server, etc.
- [Latency](https://developer.mozilla.org/en-US/docs/Web/Performance/Understanding_latency)
> The time it takes for a data packet to travel from sender to receiver. It depends more on the physical distance.
- [Packet loss](https://en.wikipedia.org/wiki/Packet_loss)
> Not all packets traveling over the network can reach their destination. This happens most often when using wireless networks or due to [network congestion](https://en.wikipedia.org/wiki/Network_congestion).
- [Round Trip Time (RTT)](https://en.wikipedia.org/wiki/Round-trip_delay)
> The time it takes for the data packet to reach its destination + the time to respond that the packet was received successfully.
- [Jitter](https://www.ir.com/guides/what-is-network-jitter)
> Delay fluctuations, unstable ping (for example, 50ms, 120ms, 35ms...).
- [Packet reordering](https://wiki.geant.org/display/public/EK/PacketReordering)
> The IP protocol does not guarantee that packets are delivered in the order in which they are sent.
π References
1. π [**Understanding latency** β MDN](https://developer.mozilla.org/en-US/docs/Web/Performance/Understanding_latency)
2. πΊ [**What is latency? What affects latency?** β YouTube](https://youtu.be/epAXDsq5SbE)
3. πΊ [**Basics of network bandwidth, latency, and jitter** β YouTube](https://youtu.be/WdbJdUh6W08)
4. πΊ [**Round Trip Time (RTT)** β YouTube](https://youtu.be/nT9F-USjtBg)
5. πΊ [**What Causes Packet Loss and How to Eliminate It In Your Network** β YouTube](https://youtu.be/Cg656nGbXe4)
- ### Network diagnostics
- [Traceroute](https://en.wikipedia.org/wiki/Traceroute)
> A procedure that allows you to trace to which nodes, with which IP addresses, a packet you send before it reaches its destination. Tracing can be used to identify computer network related problems and to examine/analyze the network.
- [Ping scan]()
> The easiest way to check the server for performance.
- [Checking for packet loss](https://www.dnsstuff.com/packet-loss-test)
> Due to dropped connections, not all packets sent over the network reach their destination.
- [Wireshark](https://en.wikipedia.org/wiki/Wireshark)
> A powerful program with a graphical interface for analyzing all traffic that passes through the network in real time.
π References
1. πΊ [**How does traceroute work?** β YouTube](https://youtu.be/G05y9UKT69s)
2. πΊ [**Traceroute (tracert) Explained - Network Troubleshooting** β YouTube](https://youtu.be/up3bcBLZS74)
3. πΊ [**Nmap - Host Discovery With Ping Sweep** β YouTube](https://youtu.be/LvCDaftsMwI)
4. πΊ [**Internet Troubleshooting - Pathping Packet Loss** β YouTube](https://youtu.be/VPdotNIXOgI)
5. πΊ [**Wireshark crash course (playlist)** β YouTube](https://youtube.com/playlist?list=PLBf0hzazHTGPgyxeEj_9LBHiqjtNEjsgt)
## PC device
- ### Main components (hardware)
- [Motherboard](https://en.wikipedia.org/wiki/Motherboard)
> The most important PC component to which all other elements are connected.
>
> - [Chipset](https://en.wikipedia.org/wiki/Chipset) - set of electronic components that responsible for the communication of all motherboard components.
> - [CPU socket](https://en.wikipedia.org/wiki/CPU_socket) - socket for mounting the processor.
> - [VRM (Voltage Regulator Module)](https://en.wikipedia.org/wiki/Voltage_regulator_module) β module that converts the incoming voltage (usually 12V) to a lower voltage to run the processor, integrated graphics, memory, etc.
> - Slots for RAM.
> - Expansion slots [PCI-Express](https://en.wikipedia.org/wiki/PCI_Express) - designed for connection of video cards, external network/sound cards.
> - Slots [M.2](https://en.wikipedia.org/wiki/M.2) / [SATA](https://en.wikipedia.org/wiki/SATA) - designed to connect hard disks and SSDs.
- [CPU (Central processing unit)](https://en.wikipedia.org/wiki/Central_processing_unit)
> The most important device that executes instructions (programme code). Processors only work with 1 and 0, so all programmes are ultimately a set of binary code.
>
> - [Registers](https://en.wikipedia.org/wiki/Processor_register) - the fastest memory in a PC, has an extremely small capacity, is built into the processor and is designed to temporarily store the data being processed.
> - [Cache](https://en.wikipedia.org/wiki/CPU_cache) - slightly less fast memory, which is also built into the processor and is used to store a copy of data from frequently used cells in the main memory.
> - Processors can have different [architectures](https://en.wikipedia.org/wiki/Processor_design). Currently, the most common are the [x86](https://en.wikipedia.org/wiki/X86-64) architecture (desktop and laptop computers) and [ARM](https://en.wikipedia.org/wiki/ARM_architecture_family) (mobile devices as well as the latest Apple computers).
- [RAM (Random-access memory)](https://en.wikipedia.org/wiki/Random-access_memory)
> Fast, low capacity memory (4-16GB) designed to temporarily store program code, as well as input, output and intermediate data processed by the processor.
- [Data storage](https://en.wikipedia.org/wiki/Data_storage)
> Large capacity memory (256GB-1TB) designed for long-term storage of files and installed programmes.
- [GPU (Graphics card)](https://en.wikipedia.org/wiki/Graphics_card)
> A separate card that translates and processes data into images for display on a monitor. This device is also called a discrete graphics card. Usually needed for those who do 3D modelling or play games.
[Built-in graphics card](https://en.wikipedia.org/wiki/Graphics_processing_unit#Integrated_graphics_processing_unit) is a graphics card built into the processor. It is suitable for daily work.
- [Network card](https://en.wikipedia.org/wiki/Network_interface_controller)
> A device that receives and transmits data from other devices connected to the [local network](https://en.wikipedia.org/wiki/Local_area_network).
- [Sound card](https://en.wikipedia.org/wiki/Sound_card)
> A device that allows you to process sound, output it to other devices, record it with a microphone, etc.
- [Power supply unit]()
> A device designed to convert the AC voltage from the mains to DC voltage.
π References
1. π [**Everything You Need to Know About Computer Hardware**](https://www.lifewire.com/computer-hardware-2625895)
2. π [**Putting the "You" in CPU: explainer how your computer runs programs, from start to finish**](https://github.com/hackclub/putting-the-you-in-cpu)
3. πΊ [**What does what in your computer? Computer parts Explained** β YouTube](https://youtu.be/ExxFxD4OSZ0)
4. πΊ [**Motherboards Explained** β YouTube](https://youtu.be/b2pd3Y6aBag)
5. πΊ [**The Fetch-Execute Cycle: What's Your Computer Actually Doing?** β YouTube](https://youtu.be/Z5JC9Ve1sfI)
6. πΊ [**How a CPU Works in 100 Seconds // Apple Silicon M1 vs. Intel i9** β YouTube](https://youtu.be/vqs_0W-MSB0)
7. πΊ [**Arm vs. x86 - Key Differences Explained** β YouTube](https://youtu.be/AADZo73yrq4)
- ### Operating system design
[Operating system (OS)](https://en.wikipedia.org/wiki/Operating_system) is a comprehensive software system designed to manage a computer's resources. With operating systems, people do not have to deal directly with the processor, RAM, or other parts of the PC.
OS can be thought of as an abstraction layer that manages the hardware of a computer, thereby providing a simple and convenient environment for user software to run.
- Main features
> - RAM management (space allocation for individual programs)
> - Loading programs into RAM and their execution
> - Execution of requests from user's programs (inputting and outputting data, starting and stopping other programs, freeing up memory or allocating additional memory, etc.)
> - Interaction with input and output devices (mouse, keyboard, monitor, etc.)
> - Interaction with storage media (HDDs and SSDs)
> - Providing a user's interface (console shell or graphical interface)
> - Logging of software errors (saving logs)
- Additional functions (may not be available in all OSs)
> - Organize [multitasking](https://en.wikipedia.org/wiki/Computer_multitasking) (simultaneous execution of several programs)
> - Delimiting access to resources for each process
> - [Inter-process communication](https://en.wikipedia.org/wiki/Inter-process_communication) (data exchange, synchronisation)
> - Organize the protection of the operating system itself against other programs and the actions of the user
> - Provide multi-user mode and differentiate rights between different OS users (admins, guests, etc.)
- [OS kernel]()
> The central part of the operating system which is used most intensively. The kernel is constantly in memory, while other parts of the OS are loaded into and unloaded from memory as needed.
- [Bootloader](https://en.wikipedia.org/wiki/Bootloader)
> The system software that prepares the environment for the OS to run (puts the hardware in the right state, prepares the memory, loads the OS kernel there and transfers control to it (the kernel).
- [Device drivers](https://en.wikipedia.org/wiki/Device_driver)
> Special software that allows the OS to work with a particular piece of equipment.
π References
1. π [**What is an OS? Operating System Definition for Beginners** β freeCodeCamp](https://www.freecodecamp.org/news/what-is-an-os-operating-system-definition-for-beginners/)
2. π [**Windows vs. macOS vs. Linux β Operating System Handbook** β freeCodeCamp](https://www.freecodecamp.org/news/an-introduction-to-operating-systems/)
3. πΊ [**Operating Systems: Crash Course Computer Science** β YouTube](https://youtu.be/26QPDBe-NB8)
4. πΊ [**Operating System Basics** β YouTube](https://youtu.be/9GDX-IyZ_C8)
5. πΊ [**Operating System in deep details (playlist)** β YouTube](https://youtube.com/playlist?list=PLBlnK6fEyqRiVhbXDGLXDk_OQAeuVcp2O)
6. π [**Awesome Operating System Stuff** β GitHub](https://github.com/jubalh/awesome-os)
- ### Processes and threads
- [Process]()
> A kind of container in which all the resources needed to run a program are stored. As a rule, the process consists of:
>
> - Executable program code
> - Input and output data
> - [Call stack](https://en.wikipedia.org/wiki/Call_stack) (order of instructions for execution)
> - [Heap](https://en.wikipedia.org/wiki/Memory_management#Manual_memory_management) (a structure for storing intermediate data created during the process)
> - [Segment descriptor](https://en.wikipedia.org/wiki/Segment_descriptor)
> - [File descriptor](https://en.wikipedia.org/wiki/File_descriptor)
> - Information about the set of permissible powers
> - Processor status information
- [Thread]()
> An entity in which sequences of program actions (procedures) are executed. Threads are within a process and use the same address space. There can be multiple threads in a single process, allowing multiple tasks to be performed. These tasks, thanks to threads, can exchange data, use shared data or the results of other tasks.
π References
1. πΊ [**Difference Between Process and Thread** β YouTube](https://youtu.be/O3EyzlZxx3g)
2. πΊ [**How Do CPUs Use Multiple Cores** β YouTube](https://youtu.be/S3I5WNHbnJ0)
3. πΊ [**What is Hyper Threading Technology** β YouTube](https://youtu.be/wnS50lJicXc)
- ### Concurrency and parallelism
- [Parallelism](https://en.wikipedia.org/wiki/Parallel_computing)
> The ability to perform multiple tasks simultaneously using multiple processor cores, where each individual core performs a different task.
- [Concurrency]()
> The ability to perform multiple tasks, but using a single processor core. This is achieved by dividing tasks into separate blocks of commands which are executed in turn, but switching between these blocks is so fast that for users it seems as if these processes are running simultaneously.
π References
1. π [**Concurrency, parallelism, and the many threads of Santa Claus** β freeCodeCamp](https://www.freecodecamp.org/news/concurrency-parallelism-and-the-many-threads-of-santa-claus/)
2. πΊ [**Concurrency vs. Parallelism** β YouTube](https://youtu.be/Y1pgpn2gOSg)
3. πΊ [**Concurrency is not Parallelism by Rob Pike** β YouTube](https://youtu.be/oV9rvDllKEg)
- ### Inter-process communication
A mechanism which allows to exchange data between threads of one or different processes. Processes can be run on the same computer or on different computers connected by a network. [Inter-process communication](https://en.wikipedia.org/wiki/Inter-process_communication) can be done in different ways.
- [File](https://en.wikipedia.org/wiki/Computer_file)
> The easiest way to exchange data. One process writes data to a certain file, another process reads the same file and thus receives data from the first process.
- [Signal (IPC)]()
> Asynchronous notification of one process about an event which occurred in another process.
- [Network socket](https://en.wikipedia.org/wiki/Network_socket)
> In particular, IP addresses and ports are used to communicate between computers using the TCP/IP protocol stack. This pair defines a socket (_socket_ corresponding to the address and port).
- [Semaphore]()
> A counter over which only 2 operations can be performed: increasing and decreasing (and for 0 the decreasing operation is blocked).
- [Message passing](https://en.wikipedia.org/wiki/Message_passing) & [Message queue](https://en.wikipedia.org/wiki/Message_queue)
- [Pipelines]()
> Redirecting the output of one process to the input of another (similar to a pipe).
π References
1. π [**Interprocess Communications** β Microsoft](https://learn.microsoft.com/en-us/windows/win32/ipc/interprocess-communications)
2. πΊ [**Interprocess Communication** β YouTube](https://youtu.be/dJuYKfR8vec)
3. πΊ [**Inter Process Communication** β YouTube](https://youtu.be/W0BX6geRCDQ)
## Linux Basics
Operating systems based on [Linux kernel](https://en.wikipedia.org/wiki/Linux_kernel) are the standard in the world of server development, since most servers run on such operating systems. Using Linux on servers is profitable because it is free and open source, secure and works fast on cheap hardware.
There are a huge number of [Linux distributions](https://en.wikipedia.org/wiki/Linux_distribution) (preinstalled software bundles) to suit all tastes. One of the most popular is [Ubuntu](https://en.wikipedia.org/wiki/Ubuntu). This is where you can start your dive into server development.
[Install Ubuntu](https://ubuntu.com/download/desktop) on a separate PC or laptop. If this is not possible, you can use a special program [Virtual Box](https://www.virtualbox.org/wiki/Downloads) where you can [run other OS](https://www.virtualbox.org/manual/ch01.html#create-vm-wizard) on top of the main OS. You can also run [Docker](https://www.docker.com/products/docker-desktop) [Ubuntu image container](https://hub.docker.com/_/ubuntu) (Docker is a [separate topic](#docker) that is exists in this repository).
- ### Working with shell
[Shell](https://en.wikipedia.org/wiki/Shell_(computing)) (or console, terminal) is a computer program which is used to operate and control a computer by entering special text commands. Generally, servers do not have [graphical interfaces (GUI)](https://en.wikipedia.org/wiki/Graphical_user_interface), so you will definitely need to learn how to work with shells. The are many [Unix shells](https://en.wikipedia.org/wiki/Unix_shell), but most Linux distributions come with a [Bash shell](https://en.wikipedia.org/wiki/Bash_(Unix_shell)) by default.
- Basic commands for navigating the file system
```sh
ls # list directory contents
cd [PATH] # go to specified directory
cd .. # move to a higher level (to the parent directory)
touch [FILE] # create a file
cat > [FILE] # enter text into the file (overwrite)
cat >> [FILE] # enter text at the end of the file (append)
cat/more/less [FILE] # to view the file contents
head/tail [FILE] # view the first/last lines of a file
pwd # print path to current directory
mkdir [NAME] # create a directory
rmdir [NAME] # delete a directory
cp [FILE] [PATH] # copy a file or directory
mv [FILE] [PATH] # moving or renaming
rm [FILE] # deleting a file or directory
find [STRING] # file system search
du [FILE] # output file or directory size
grep [PATTERN] [FILE] # print lines that match patterns
```
- Commands for help information
```sh
man [COMMAND] # allows you to view a manual for any command
apropos [STRING] # search for a command with a description that has a specified word
man -k [STRING] # similar to the command above
whatis [COMMAND] # a brief description of the command
```
- [Super user rights](https://en.wikipedia.org/wiki/Sudo)
> Analogue to running as administrator in Windows
```sh
sudo [COMMAND] # executes a command with superuser privileges
```
- Text editor
> Study any in order to read and edit files freely through the terminal.
> The easiest β [nano](https://en.wikipedia.org/wiki/GNU_nano).
> Something in the middle - [micro](https://micro-editor.github.io/).
> The most advanced β [Vim]().
π References
1. π [**31 Linux Commands Every Ubuntu User Should Know**](https://itsfoss.com/essential-ubuntu-commands/)
2. π [**The Linux Command Handbook** β freeCodeCamp](https://www.freecodecamp.org/news/the-linux-commands-handbook/)
3. π [**A to Z: List of Linux commands**](https://linuxhandbook.com/a-to-z-linux-commands/)
4. πΊ [**The 50 Most Popular Linux & Terminal Commands** β YouTube](https://youtu.be/ZtqBQ68cfJc)
5. πΊ [**Nano Editor Fundamentals** β YouTube](https://youtu.be/gyKiDczLIZ4)
6. πΊ [**Vim Tutorial for Beginners** β YouTube](https://youtu.be/RZ4p-saaQkc)
7. π [**Awesome Terminals** β GitHub](https://github.com/cdleon/awesome-terminals)
8. π [**Awesome CLI-apps** β GitHub](https://github.com/agarrharr/awesome-cli-apps)
- ### Package manager
The package manager is a utility that allows you to install/update software packages from the terminal.
Linux distributions can be divided into several groups, depending on which package manager they use: [apt]() (in [Debian](https://en.wikipedia.org/wiki/Debian) based distributions), [RPM](https://en.wikipedia.org/wiki/RPM_Package_Manager) (the [Red Hat](https://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux) package management system) and [Pacman](https://en.wikipedia.org/wiki/Arch_Linux#Pacman) (the package manager in [Arch-like distributions](https://en.wikipedia.org/wiki/Arch_Linux))
Ubuntu is based on Debian, so it uses apt (advanced packaging tool) package manager.
- Basic commands
```sh
apt install [package] # install the package
apt remove [package] # remove the package, but keep the configuration
apt purge [package] # remove the package along with the configuration
apt update # update information about new versions of packages
apt upgrade # update the packages installed in the system
apt list --installed # list of packages installed on the system
apt list --upgradable # list of packages that need to be updated
apt search [package] # searching for packages by name on the network
apt show [package] # package information
```
- [aptitude](https://en.wikipedia.org/wiki/Aptitude_(software))
> Interactive console utility for easy viewing of packages to install, update and uninstall them.
- Repository management
> Package managers typically work with software repositories. These repositories contain a collection of software packages that are maintained and provided by the distribution's community or official sources.
```sh
add-apt-repository [repository_url] # add a new repository
add-apt-repository --remove [repository_url] # remove a repo
# don\'t forget to update after this operations - apt update
```
```sh
/etc/apt/sources.list # a file contains a list of configured repo links
/etc/apt/sources.list.d # a directory contains files for third party repos
```
- [dpkg](https://en.wikipedia.org/wiki/Dpkg)
> Low-level tool to install, build, remove and manage Debian packages.
π References
1. πΊ [**Linux Crash Course - The apt Command** β YouTube](https://youtu.be/1kicKTbK768)
2. πΊ [**Linux Package Management | Debian, Fedora, and Arch Linux** β YouTube](https://youtu.be/lkii2cGuKao)
3. π [**sudo apt-get update vs. upgrade β What is the Difference?** β freeCodeCamp](https://www.freecodecamp.org/news/sudo-apt-get-update-vs-upgrade-what-is-the-difference)
4. π [**Repositories in Ubuntu**](https://help.ubuntu.com/community/Repositories/Ubuntu)
- ### Bash scripts
You can use scripts to automate the sequential input of any number of commands. In [Bash]() you can create different conditions (branching), loops, timers, etc. to perform all kinds of actions related to shell input.
- [Basics of Bash Scripts](https://github.com/cheatsnake/bash-scripts-by-example)
> The most basic and frequently used features such as: variables, I/O, loops, conditions, etc.
- Practice
> Solve challenges on sites like [HackerRank](https://www.hackerrank.com/domains/shell) and [Codewars](https://www.codewars.com/join?language=shell).
> Start using Bash to automate routine activities on your computer. If you're already a programmer, create scripts to easily build your project, to install settings, and so on.
- [ShellCheck](https://github.com/koalaman/shellcheck) script analysis tool
> It will point out possible mistakes and teach you best practices for writing really good scripts.
- Additional resources
> Repositories such as [awesome bash](https://github.com/awesome-lists/awesome-bash) and [awesome shell](https://github.com/alebcay/awesome-shell) have entire collections of useful resources and tools to help you develop even more skills with Bash and shell in general.
π References
1. π [**Shell Scripting for Beginners** β freeCodeCamp](https://www.freecodecamp.org/news/shell-scripting-crash-course-how-to-write-bash-scripts-in-linux/)
2. πΊ [**Bash Scripting Full Course 3 Hours** β YouTube](https://youtu.be/e7BufAVwDiM)
3. π [**HackerRank challenges for Bash with solutions**](https://github.com/Thomas-George-T/HackerRank-The-Linux-Shell-Challenges-Solutions)
- ### Users, groups, and permissions
Linux-based operating systems are multi-user. This means that several people can run many different applications at the same time on the same computer. For the Linux system to be able to "recognize" a user, he must be logged in, and therefore each user must have a unique name and a secret password.
- Working with users
```sh
useradd [name] [flags] # create a new user
passwd [name] # set a password for the user
usermod [name] [flags] # edit a user
usermod -L [name] # block a user
usermod -U [name] # unblock a user
userdel [name] [flags] # delete a user
su [name] # switch to other user
```
- Working with groups
```sh
groupadd [group] [flags] # create a group
groupmod [group] [flags] # edit group
groupdel [group] [flags] # delete group
usermod -a -G [groups] [user] # add a user to groups
gpasswd --delete [user] [groups] # remove a user from groups
```
- System files
```sh
/etc/passwd # a file containing basic information about users
/etc/shadow # a file containing encrypted passwords
/etc/group # a file containing basic information about groups
/etc/gshadow # a file containing encrypted group passwords
```
On Linux, it is possible to share privileges between users, limit access to unwanted files or features, control available actions for services, and much more. On Linux, there are only three kinds of rights - read, write and execute - and three categories of users to which they can be applied - file owner, file group and everyone else.
- Basic commands for working with rights
```sh
chown # changes the owner and/or group for the specified files
chmod # changes access rights to files and directories
chgrp # allows users to change groups
```
- Extended rights [SUID and GUID](https://en.wikipedia.org/wiki/Setuid), [sticky bit](https://en.wikipedia.org/wiki/Sticky_bit)
- [ACL (Access control list)](https://en.wikipedia.org/wiki/Access-control_list)
> An advanced subsystem for managing access rights.
π References
1. π [**Managing Users, Groups, and Permissions on Linux**](https://omarrrz-lounge.hashnode.dev/managing-users-groups-and-permissions-in-linux)
2. π [**Linux User Groups Explained** β freeCodeCamp](https://www.freecodecamp.org/news/linux-user-groups-explained-how-to-add-a-new-group-a-new-group-member-and-change-groups/)
3. πΊ [**Linux Users and Groups** β YouTube](https://youtu.be/b-9j2jiCOEA)
4. π [**An Introduction to Linux Permissions** β Digital Ocean](https://www.digitalocean.com/community/tutorials/an-introduction-to-linux-permissions)
5. π [**File Permissions on Linux β How to Use the chmod Command** β freeCodeCamp](https://www.freecodecamp.org/news/file-permissions-in-linux-chmod-command-explained/)
6. πΊ [**Understanding File & Directory Permissions** β YouTube](https://youtu.be/4e669hSjaX8)
- ### Working with processes
Linux processes can be described as containers in which all information about the state of a running program is stored. Sometimes programs can hang and in order to force them to close or restart, you need to be able to manage processes.
- Basic Commands
```sh
ps # display a snapshot of the processes of all users
top # real-time task manager
[command] & # running the process in the background, (without occupying the shell)
jobs # list of processes running in the background
fg [PID] # return the process back to the active mode by its number
# You can press [Ctrl+Z] to return the process to the background
bg [PID] # start a stopped process in the background
kill [PID] # terminate the process by PID
killall [program] # terminate all processes related to the program
```
π References
1. π [**How to Show Process Tree on Linux**](https://linuxhandbook.com/show-process-tree/)
2. π [**How to Manage Linux Processes** β freeCodeCamp](https://www.freecodecamp.org/news/how-to-manage-linux-processes/)
3. π [**How To Use ps, kill, and nice to Manage Processes on Linux** β Digital Ocean](https://www.digitalocean.com/community/tutorials/how-to-use-ps-kill-and-nice-to-manage-processes-in-linux)
4. πΊ [**Linux processes, init, fork/exec, ps, kill, fg, bg, jobs** β YouTube](https://youtu.be/TJzltwv7jJs)
- ### Working with SSH
[SSH](https://en.wikipedia.org/wiki/Secure_Shell) allows remote access to another computer's terminal. In the case of a personal computer, this may be needed to solve an urgent problem, and in the case of working with the server, remote access via SSH is an integral and regularly used practice.
- Basic commands
```sh
apt install openssh-server # installing SSH (out of the box almost everywhere)
service ssh start # start SSH
service ssh stop # stop SSH
ssh -p [port] [user]@[remote_host] # connecting to a remote machine via SSH
```
- [Passwordless login](https://www.redhat.com/sysadmin/passwordless-ssh)
```sh
ssh-keygen -t rsa # RSA key generation for passwordless login
ssh-copy-id -i ~/.ssh/id_rsa [user]@[remote_host] # copying a key to a remote machine
```
- Config files
```sh
/etc/ssh/sshd_config # ssh server global config
~/.ssh/config # ssh server local config
~/.ssh/authorized_keys # file with saved public keys
```
π References
1. π [**What the hell is SSH?**](https://codingpastor.hashnode.dev/what-the-hell-is-ssh)
2. πΊ [**Learn SSH In 6 Minutes - Beginners Guide to SSH Tutorial** β YouTube](https://youtu.be/v45p_kJV9i4)
3. πΊ [**SSH Crash Course | With Some DevOps** β YouTube](https://youtu.be/hQWRp-FdTpc)
4. π [**SSH config file for OpenSSH client**](https://www.ssh.com/academy/ssh/config)
5. π [**Awesome SSH** β GitHub](https://github.com/moul/awesome-ssh)
- ### Network utils
For Linux there are many built-in and third-party utilities to help you configure your network, analyze it and fix possible problems.
- Simple utils
```bash
ip address # show info about IPv4 and IPv6 addresses of your devices
ip monitor # real time monitor the state of devices
ifconfig # config the network adapter and IP protocol settings
traceroute # show the route taken by packets to reach the host
tracepath # traces the network host to destination discovering MTU
ping # check connectivity to host
ss -at # show the list of all listening TCP connections
dig # show info about the DNS name server
host # show the IP address of a specified domain
mtr # combination of ping and traceroute utilities
nslookup # query Internet name servers interactively
whois # show info about domain registration
ifplugstatus # detect the link status of a local Linux ethernet device
iftop # show bandwidth usage
ethtool # show detalis about your ethernet device
nmap # tool to explore and audit network security
bmon # bandwidth monitor and rate estimator
firewalld # add, configure and remove rules on firewall
ipref # perform network performance measurement and tuning
speedtest-cli # check your network download/upload speed
wget # download files from the Internet
```
- [`tcpdump`](https://en.wikipedia.org/wiki/Tcpdump)
> A console utility that allows you to intercept and analyze all network traffic passing through your computer.
- [`netcat`](https://en.wikipedia.org/wiki/Netcat)
> Utility for reading from and writing to network connections using TCP or UDP. It includes port scanning, transferring files, and port listening: as with any server, it can be used as a [backdoor]().
- [`iptables`](https://en.wikipedia.org/wiki/Iptables)
> User-space utility program that allows to configure the IP packet filter rules of the Linux kernel firewall, implemented as different Netfilter modules. The filters are organized in different tables, which contain chains of rules for how to treat network traffic packets.
- [`curl`](https://en.wikipedia.org/wiki/CURL)
> Command-line tool for transferring data using various network protocols.
π References
1. π [**21 Basic Linux Networking Commands You Should Know**](https://itsfoss.com/basic-linux-networking-commands/)
2. π [**Using tcpdump Command on Linux to Analyze Network**](https://linuxhandbook.com/tcpdump-command/)
3. πΊ [**tcpdump - Traffic Capture & Analysis** β YouTube](https://youtu.be/1lDfCRM6dWk)
4. πΊ [**tcpdumping Node.js server** β YouTube](https://youtu.be/g_tmQ5G-T2w)
5. π [**Beginnerβs guide to Netcat for hackers**](https://medium.com/@HackTheBridge/beginners-guide-to-netcat-for-hackers-55abe449991d)
6. π [**Iptables Tutorial**](https://linuxhint.com/iptables-tutorial/)
7. π [**An intro to cURL: The basics of the transfer tool**](https://blog.logrocket.com/an-intro-to-curl-the-basics-of-the-transfer-tool/)
8. πΊ [**Basic cURL Tutorial** β YouTube](https://youtu.be/7XUibDYw4mc)
9. πΊ [**Using cURL better - tutorial by cURL creator Daniel Stenberg** β YouTube](https://youtu.be/I6id1Y0YuNk)
10. π [**Awesome console services** β GitHub](https://github.com/chubin/awesome-console-services)
- ### Task scheduler
Schedulers allow you to flexibly manage the delayed running of commands and scripts. Linux has a built-in [cron](https://en.wikipedia.org/wiki/Cron) scheduler that can be used to easily perform necessary actions at certain intervals.
- Main commands
```bash
crontab -e # edit the crontab file of the current user
crontab -l # output the contents of the current schedule file
crontab -r # deleting the current schedule file
```
- Files and directories
```sh
/etc/crontab # base config
/etc/cron.d/ # a dir with crontab files used to manage the entire system
# dirs where you can store scripts that runs:
/etc/cron.daily/ # every day
/etc/cron.weekly/ # every week
/etc/cron.monthly/ # every month
```
π References
1. π [**How to schedule and manage tasks using crontab** β dev.to](https://dev.to/shaikh/how-to-schedule-and-manage-tasks-using-crontab-20dj)
2. πΊ [**Cron Jobs For Beginners | Linux Task Scheduling** β YouTube](https://youtu.be/v952m13p-b4)
3. π [**How to Check Crontab logs on Linux**](https://linuxhandbook.com/check-crontab-logs/)
- ### System logs
[Log files](https://en.wikipedia.org/wiki/Logging_(computing)) are special text files that contain all information about the operation of a computer, program, or user. They are especially useful when bugs and errors occur in the operation of a program or server. It is recommended to periodically review log files, even if nothing suspicious happens.
- Main log files
```bash
/var/log/syslog or /var/log/messages # information about the kernel,
# various services detected, devices, network interfaces, etc.
/var/log/auth.log or /var/log/secure # user authorization information
/var/log/faillog # failed login attempts
/var/log/dmesg # information about device drivers
/var/log/boot.log # operating system boot information
/var/log/cron # cron task scheduler report
```
- [lnav utility](https://lnav.org/)
> Designed for easy viewing of log files (highlighting, reading different formats, searching, etc.)
- Log rotation with [logrotate](https://github.com/logrotate/logrotate)
> Allows you to configure automatic deletion (cleaning) of log files so as not to clog memory.
- [Demon journald](https://manpages.ubuntu.com/manpages/bionic/man1/journalctl.1.html)
> Collects data from all available sources and stores it in binary format for convenient and dynamic control.
π References
1. πΊ [**Linux Crash Course - Understanding Logging** β YouTube](https://youtu.be/6uP_f_z3CbM)
2. πΊ [**Linux Monitoring and Logging** β YouTube](https://youtu.be/kZ5LhS6fThM)
3. π [**3 ways to watch logs in real time on Linux**](https://linuxhandbook.com/watch-logs-real-time/)
4. π [**Analyzing logs on Linux with journalctl command**](https://linuxhandbook.com/journalctl-command/)
5. π [**Linux File Structure Explained**](https://shubhsharma19.hashnode.dev/linux-file-structure-explained)
- ### Main issues with Linux
- Software installation and package management issues
> - [Unmet dependencies](https://fedingo.com/how-to-resolve-unmet-dependencies-in-ubuntu/) - occurs when package fails to install or update.
> - [Dependency errors and conflicts](https://ubunlog.com/en/how-to-fix-dependency-errors-in-ubuntu-and-derivatives/)
- [Problems with drivers](https://askubuntu.com/a/496654)
> All free Linux drivers are built right into its kernel. Therefore, everything should work "out of the box" after installing the system (problems may occur with brand new hardware which has just been released on the market). Drivers whose source code is closed are considered proprietary and are not included in the kernel but are installed manually (like Nvidia graphics drivers).
- File system issues
> - Check disk space availability using the `df` command and ensure that critical partitions are not full.
> - Use the `fsck` command to check and repair [file system inconsistencies](https://www3.rocketsoftware.com/rocketd3/support/documentation/d3nt/91/refman/index.htm#definitions/file_inconsistency.htm).
> - In case of data loss or accidental deletion, use data recovery tools like [`extundelete`](https://extundelete.sourceforge.net/) or [`testdisk`](https://github.com/cgsecurity/testdisk).
- Performance and resource management
> - Check system resource usage, including CPU, memory, and disk space, using `free`, `df`, or `du` commands.
> - Identify resource-intensive processes using tools like `top`, `htop`, or `systemd-cgtop`.
> - Disable unnecessary [startup services](https://askubuntu.com/questions/48321/how-do-i-start-applications-automatically-on-login) or [background processes](https://askubuntu.com/questions/636129/how-can-i-see-background-process-in-ubuntu-and-kill-unnecessary-processes) to improve performance.
- Network connectivity issues
> - Use the ping command to check network connectivity to a specific host or IP address.
> - Check the [network settings](https://ubuntu.com/server/docs/network-configuration), such as IP configuration, [DNS settings](https://unix.stackexchange.com/questions/494324/how-to-setup-dns-manually-on-linux), and [firewall rules](https://www.redhat.com/sysadmin/firewalld-linux-firewall).
- Problems with kernel
> [Kernel panic](https://en.wikipedia.org/wiki/Kernel_panic) - can occur due to an error when mounting the root file system.
> This is best helped by the skill of reading the logs to find problems (`dmesg` command).
π References
1. πΊ [**Linux Drivers Explained** β YouTube](https://youtu.be/s8t0AWmHvUM)
2. πΊ [**How Do Linux Kernel Drivers Work?** β YouTube](https://youtu.be/juGNPLdjLH4)
## General knowledge
- ### Numeral systems
[Numeral system](https://en.wikipedia.org/wiki/Numeral_system) is a set of symbols and rules for denoting numbers. In computer science, it is customary to distinguish four main number systems: binary, octal, decimal, and hexadecimal. It is connected, first of all, with their use in various branches of programming.
- [Binary number](https://en.wikipedia.org/wiki/Binary_number)
> The most important system for computing technology. Its use is justified by the fact that the logic of the processor is based on only two states (on/off, open/closed, high/low, true/false, yes/no, high/low).
- [Octal](https://en.wikipedia.org/wiki/Octal)
> It is used e.g., on Linux systems to grant access rights.
- [Decimal](https://en.wikipedia.org/wiki/Decimal)
> A system that is easy to understand for most people.
- [Hexadecimal](https://en.wikipedia.org/wiki/Hexadecimal)
> The letters A, B, C, D, E, F are additionally used for recording. It is widely used in low-level programming and computer documentation because the minimum addressable memory unit is an 8-bit byte, the values of which are conveniently written in two hexadecimal digits.
- Translation between different number systems
> You can try [online converter](https://cheatsnake.github.io/NSConverter/) for a better understanding.
π References
1. πΊ [**Number Systems Introduction - Decimal, Binary, Octal & Hexadecimal** β YouTube](https://youtu.be/FFDMzbrEXaE)
1. π [**Number System in Maths** β GeeksGorGeeks](https://www.geeksforgeeks.org/number-system-in-maths/)
- ### Logical connective
[Logical connective](https://en.wikipedia.org/wiki/Logical_connective) are widely used in programming to handle boolean types (true/false or 1/0). The result of a boolean expression is also a value of a boolean type.
AND
| a | b | a AND b |
|---|---|:-------:|
| 0 | 0 | 0 |
| 0 | 1 | 0 |
| 1 | 0 | 0 |
| 1 | 1 | 1 |
OR
| a | b | a OR b |
|---|---|:-------:|
| 0 | 0 | 0 |
| 0 | 1 | 1 |
| 1 | 0 | 1 |
| 1 | 1 | 1 |
XOR
| a | b | a XOR b |
|---|---|:-------:|
| 0 | 0 | 0 |
| 0 | 1 | 1 |
| 1 | 0 | 1 |
| 1 | 1 | 0 |
- Basic logical operations
> They are the basis of other all kinds of operations.
> There are three in total: [Operation AND (&&, Conjunction)](https://en.wikipedia.org/wiki/Logical_conjunction), [operation OR (||, Disjunction)](https://en.wikipedia.org/wiki/Logical_disjunction), [operation NOT (!, Negation)](https://en.wikipedia.org/wiki/Negation).
- Operation [Exclusive OR (XOR, Modulo 2 Addition)](https://en.wikipedia.org/wiki/Exclusive_or)
> An important operation that is fundamental to coding theory and computer networks.
- [Truth Tables](https://en.wikipedia.org/wiki/Truth_table)
> For logical operations, there are special tables that describe the input data and the return result.
- Priority of operations
> The `NOT` operator has the highest priority, followed by the `AND` operator, and then the `OR` operator. You can change this behavior using round brackets.
π References
1. πΊ [**Logical Operators β Negation, Conjunction & Disjunction** β YouTube](https://youtu.be/6kYngPvoGxU)
2. πΊ [**Logical Operators β Exclusive OR** β YouTube](https://youtu.be/m2mf6I3g2-c)
- ### Data structures
[Data structures](https://en.wikipedia.org/wiki/Data_structure) are containers in which data is stored according to certain rules. Depending on these rules, the data structure will be effective in some tasks and ineffective in others. Therefore, it is necessary to understand when and where to use this or that structure.
- [Array]()
> A data structure that allows you to store data of the same type, where each element is assigned a different sequence number.
- [Linked list](https://en.wikipedia.org/wiki/Linked_list)
> A data structure where all elements, in addition to the data, contain references to the next and/or previous element. There are 3 varieties:
>
> - A [singly linked list](https://en.wikipedia.org/wiki/Linked_list#Singly_linked_list) is a list where each element stores a link to the next element only (one direction).
> - A [doubly linked list](https://en.wikipedia.org/wiki/Doubly_linked_list) is a list where the items contain links to both the next item and the previous one (two directions).
> - A [circular linked list](https://en.wikipedia.org/wiki/Linked_list#Circular_linked_list) is a kind of bilaterally linked list, where the last element of the ring list contains a pointer to the first and the first to the last.
- [Stack]()
> Structure where data storage works on the principle of _last in - first out_ (LIFO).
- [Queue]()
> Structure where data storage is based on the principle of _first in - first out_ (FIFO).
- [Hash table](https://en.wikipedia.org/wiki/Hash_table)
> In other words, it is an associative array. Here, each of the elements is accessed with a corresponding key value, which is calculated using [hash function](https://en.wikipedia.org/wiki/Hash_function) according to a certain algorithm.
- [Tree]()
> Structure with a hierarchical model, as a set of related elements, usually not ordered in any way.
- [Heap]()
> Similar to the tree, but in the heap, the items with the largest key is the root node (max-heap). But it may be the other way around, then it is a min heap.
- [Graph]()
> A structure that is designed to work with a large number of links.
π References
1. πΊ [**10 Key Data Structures We Use Every Day** β YouTube](https://youtu.be/ouipSd_5ivQ)
2. πΊ [**CS50 2022 - Lecture about Data Structures** β YouTube](https://youtu.be/X8h4dq9Hzq8)
3. πΊ [**Data Structures Easy to Advanced Course** β YouTube](https://youtu.be/RBSGKlAvoiM)
4. π [**Free courses to learn data structures and algorithms in depth** β freeCodeCamp](https://www.freecodecamp.org/news/these-are-the-best-free-courses-to-learn-data-structures-and-algorithms-in-depth-4d52f0d6b35a/)
5. π [**Data Structures: collection of topics** β GeeksForGeeks](https://www.geeksforgeeks.org/data-structures/)
6. π [**JavaScript Data Structures and Algorithms** β GitHub](https://github.com/trekhleb/javascript-algorithms)
7. π [**Go Data Structures** β GitHub](https://github.com/emirpasic/gods)
- ### Basic algorithms
[Algorithms](https://de.wikipedia.org/wiki/Algorithmus) refer to sets of sequential instructions (steps) that lead to the solution of a given problem. Throughout human history, a huge number of algorithms have been invented to solve certain problems in the most efficient way. Accordingly, the correct choice of algorithms in programming will allow you to create the fastest and most resource-intensive solutions.
> There is a very good book about algorithms for beginners β [Grokking algorithms](https://edu.anarcho-copy.org/Algorithm/grokking-algorithms-illustrated-programmers-curious.pdf). You can start [learning a programming language](#programming-language) in parallel with reading it.
- [Binary search](https://en.wikipedia.org/wiki/Binary_search_algorithm)
> Maximum efficient search algorithm for sorted lists.
- [Selection sort](https://en.wikipedia.org/wiki/Selection_sort)
> At each step of the algorithm, the minimum element is searched for and then swapped with the current iteration element.
- [Recursion](https://en.wikipedia.org/wiki/Recursion)
> When a function can call itself and so on to infinity. On the one hand, recursion-based solutions look very elegant, but on the other hand, this approach quickly leads to Stack Overflow and is recommended to be avoided.
- [Bubble sort](https://en.wikipedia.org/wiki/Bubble_sort)
> At each iteration neighboring elements are sequentially compared, and if the order of the pair is wrong, the elements are swapped.
- [Quicksort](https://en.wikipedia.org/wiki/Quicksort)
> Improved bubble sorting method.
- [Breadth-first search](https://en.wikipedia.org/wiki/Breadth-first_search)
> Allows finding all shortest paths from a given vertex of the graph.
- [Dijkstra's algorithm](https://en.wikipedia.org/wiki/Dijkstra%27s_algorithm)
> Finds the shortest paths between all vertices of a graph and their length.
- [Greedy algorithm](https://en.wikipedia.org/wiki/Greedy_algorithm)
> An algorithm that at each step makes locally the best choice in the hope that the final solution will be optimal.
π References
1. π [**Code for the book Grokking Algorithms** β GitHub](https://github.com/egonSchiele/grokking_algorithms)
2. πΊ [**Algorithms and Data Structures Tutorial** β YouTube](https://youtu.be/8hly31xKli0)
3. π [**Largest open-source algorithm library**](https://the-algorithms.com/)
4. πΊ [**Sorting Algorithms Explained Visually** β YouTube](https://youtu.be/RfXt_qHDEPw)
- ### Algorithm complexity
In the world of programming there is a special unit of measure [Big O notation](https://en.m.wikipedia.org/wiki/Big_O_notation). It describes how the complexity of an algorithm increases with the amount of input data. Big O estimates how many actions (steps/iterations) it takes to execute the algorithm, while always showing the worst case scenario.
- Main types of complexity
> - Constant O(1) β the fastest.
> - Linear O(n)
> - Logarithmic O(log n)
> - Linearimetric O(n \* log n)
> - Quadratic O(n^2)
> - Stepwise O(2^n)
> - Factorial O(n!) β the slowest.
- [Time complexity](https://en.wikipedia.org/wiki/Time_complexity)
> When you know in advance on which machine the algorithm will be executed, you can measure the execution time of the algorithm. Again, on very good hardware the execution time of the algorithm can be quite acceptable, but the same algorithm on a weaker hardware can run for hundreds of milliseconds or even a few seconds. Such delays will be very sensitive if your application handles user requests over the network.
- [Space complexity](https://en.wikipedia.org/wiki/Space_complexity)
> In addition to time, you need to consider how much memory is spent on the work of an algorithm. It is important when you're working with limited memory resources.
π References
1. π [**Big O Algorithm Complexity cheatsheet**](https://www.bigocheatsheet.com/)
2. πΊ [**Big O Notation - Full Course** β YouTube](https://youtu.be/Mo4vesaut8g)
- ### Data storage formats
Different file formats can be used to store and transfer data over the network. Text files are human-readable, so they are used for configuration files, for example. But transferring data in text formats over the network is not always rational, because they weigh more than their corresponding binary files.
- Text formats
- [JSON (JavaScript Object Notation)](https://en.wikipedia.org/wiki/JSON)
> Represents an object in which data is stored as key-value pairs.
- [XML (eXtensible Markup Language)](https://en.wikipedia.org/wiki/XML)
> The format is closer to HTML. Here the data is wrapped in opening and closing tags.
- [YAML (Yet Another Markup Language)](https://en.wikipedia.org/wiki/YAML)
> The format is close to markup languages like HTML. Minimalist, because it has no opening or closing tags. Easy to edit.
- [TOML (Tom's Obvious Minimal Language)](https://en.wikipedia.org/wiki/TOML)
> A minimal configuration file format that's easy to read due to obvious semantics. TOML is designed to map unambiguously to a hash table. TOML should be easy to parse into data structures in a wide variety of languages.
- Binary formats
- [Message Pack](https://msgpack.org/)
> Binary analog of JSON. Allows you to pack data 15-20% more efficiently.
- [BSON (Binary JavaScript Object Notation)](https://en.wikipedia.org/wiki/BSON)
> It is a superset of JSON, including additionally regular expressions, binary data and dates.
- [ProtoBuf (Protocol Buffers)](https://en.wikipedia.org/wiki/Protocol_Buffers)
> Binary alternative to XML text format. Simpler, more compact and faster.
- Image formats
- [JPEG (Joint Photographic Experts Group)](https://en.wikipedia.org/wiki/JPEG)
> It is best suited for photographs and complex images with a wide range of colors. JPEG images can achieve high compression ratios while maintaining good image quality, but repeated editing and saving can result in loss of image fidelity.
- [PNG (Portable Network Graphics)](https://en.wikipedia.org/wiki/PNG)
> It is a lossless compression format that supports transparency. It is commonly used for images with sharp edges, logos, icons, and images that require transparency. PNG images can have a higher file size compared to JPEG, but they retain excellent quality without degradation during repeated saves.
- [GIF (Graphics Interchange Format)](https://en.wikipedia.org/wiki/GIF)
> Used for simple animations and low-resolution images with limited colors. It supports transparency and can be animated by displaying a sequence of frames.
- [SVG (Scalable Vector Graphics)](https://en.wikipedia.org/wiki/SVG)
> XML-based vector image format defined by mathematical equations rather than pixels. SVG images can be scaled to any size without losing quality and are well-suited for logos, icons, and graphical elements.
- [WebP](https://en.wikipedia.org/wiki/WebP)
> Modern image format developed by Google. It supports both lossy and lossless compression, providing good image quality with smaller file sizes compared to JPEG and PNG. WebP images are optimized for web use and can include transparency and animation.
- Video formats
- [MP4 (MPEG-4 Part 14)](https://en.wikipedia.org/wiki/MP4_file_format)
> Widely used video format that supports high-quality video compression, making it suitable for streaming and storing videos. MP4 files can contain both video and audio.
- [AVI (Audio Video Interleave)](https://en.wikipedia.org/wiki/Audio_Video_Interleave)
> Is a multimedia container format developed by Microsoft. It can store audio and video data in a single file, allowing for synchronized playback. However, they tend to have larger file sizes compared to more modern formats.
- [MOV (QuickTime Movie)](https://www.adobe.com/creativecloud/video/hub/guides/what-is-an-mov-video.html)
> Is a video format developed by Apple for use with their QuickTime media player. It is widely used with Mac and iOS devices. MOV files can contain both video and audio, and they offer good compression and quality, making them suitable for editing and professional use.
- [WEBM](https://en.wikipedia.org/wiki/WebM)
> Best for videos embedded on your personal or business website. It is lightweight, load quickly and stream easily.
- Audio formats
- [MP3 (MPEG-1 Audio Layer 3)](https://en.wikipedia.org/wiki/MP3)
> The most popular audio format known for its high compression and small file sizes. It achieves this by removing some of the audio data that may be less perceptible to the human ear. Suitable for music storage, streaming, and sharing.
- [WAV (Waveform Audio File Format)](https://en.wikipedia.org/wiki/WAV)
> Is an uncompressed audio format that stores audio data in a lossless manner, resulting in high-quality sound reproduction. WAV files are commonly used in professional audio production and editing due to their accuracy and fidelity. However, they tend to have larger file sizes compared to compressed formats.
- [AAC (Advanced Audio Coding)](https://en.wikipedia.org/wiki/Advanced_Audio_Coding)
> Is a widely used audio format known for its efficient compression and good sound quality. It offers better sound reproduction at lower bit rates compared to MP3. AAC files are commonly used for streaming music, online radio, and mobile devices, as they deliver good audio quality while conserving bandwidth and storage.
π References
1. πΊ [**Data Formats: XML, JSON, and YAML** β YouTube](https://youtu.be/JQO-x8rzNVI)
2. πΊ [**Serialization formats: JSON and Protobuf** β YouTube](https://youtu.be/uGYZn6xk-hA)
3. πΊ [**Protocol Buffers Crash Course** β YouTube](https://youtu.be/46O73On0gyI)
4. πΊ [**Explaining Image File Formats** β YouTube](https://youtu.be/WblPwVq9KnU)
5. πΊ [**What's the difference between a JPEG, PNG, GIF, etcβ¦?** β YouTube](https://youtu.be/ww12lImOJ38)
- ### Text encodings
Computers work only with numbers, or more precisely, only with 0 and 1. It is alread