https://github.com/callmeskyy111/golang-advanced
Advanced Golang concepts 🔵
https://github.com/callmeskyy111/golang-advanced
buffered-channel channels golang goroutines mutex rate-limiting signal tickers timers wait-groups worker-pools
Last synced: 8 months ago
JSON representation
Advanced Golang concepts 🔵
- Host: GitHub
- URL: https://github.com/callmeskyy111/golang-advanced
- Owner: callmeskyy111
- Created: 2025-09-26T10:50:28.000Z (9 months ago)
- Default Branch: main
- Last Pushed: 2025-10-04T18:06:01.000Z (8 months ago)
- Last Synced: 2025-10-04T19:07:35.141Z (8 months ago)
- Topics: buffered-channel, channels, golang, goroutines, mutex, rate-limiting, signal, tickers, timers, wait-groups, worker-pools
- Language: Go
- Homepage:
- Size: 64.5 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
Awesome Lists containing this project
README
# 🌱 What is a Goroutine?
A **goroutine** is a lightweight, independently executing function that runs **concurrently** with other goroutines in the same address space.
Think of it as:
* In **JavaScript**, we have an **event loop** that handles async tasks (e.g., promises, async/await).
* In **Go**, instead of a single-threaded event loop, we have **goroutines managed by the Go runtime**.
They allow us to perform tasks like handling requests, I/O operations, or computations in parallel **without manually managing threads**.
---
# ⚖️ Goroutine vs OS Thread
| Feature | Goroutine | OS Thread |
| ------------------------- | -------------------------------- | ----------------------- |
| **Size at start** | ~2 KB stack | ~1 MB stack |
| **Managed by** | Go runtime scheduler (M:N model) | OS Kernel |
| **Number you can create** | Millions | Limited (few thousands) |
| **Switching** | Very fast, done in user space | Slower, done by OS |
| **Creation cost** | Extremely cheap | Expensive |
👉 This is why we say goroutines are *lightweight threads*.
---
# ⚙️ How to Start a Goroutine
```go
package main
import (
"fmt"
"time"
)
func printMessage(msg string) {
for i := 0; i < 5; i++ {
fmt.Println(msg, i)
time.Sleep(500 * time.Millisecond)
}
}
func main() {
go printMessage("goroutine") // runs concurrently
printMessage("main") // runs in main goroutine
}
```
* The `go` keyword starts a new goroutine.
* Here:
* `main()` itself runs in the **main goroutine**.
* `go printMessage("goroutine")` starts another goroutine.
* If `main()` exits before the new goroutine finishes, the program ends immediately.
⚠️ Unlike JavaScript promises (which keep the process alive until settled), Go doesn’t wait for goroutines unless you **explicitly synchronize** them.
---
# 🧵 Go’s Concurrency Model (M:N Scheduler)
Go runtime uses an **M:N scheduler**, meaning:
* **M goroutines** are multiplexed onto **N OS threads**.
* This is different from **1:1** (like Java threads) or **N:1** (like cooperative multitasking).
The scheduler ensures:
* Goroutines are distributed across multiple threads.
* When one blocks (e.g., waiting on I/O), another is scheduled.
Think of goroutines as **tasks in a work-stealing scheduler**.
---
# 🛠️ Synchronization with Goroutines
Since goroutines run concurrently, we need synchronization tools:
### 1. **WaitGroup** – Wait for Goroutines to Finish
```go
package main
import (
"fmt"
"sync"
)
func worker(id int, wg *sync.WaitGroup) {
defer wg.Done() // signals completion
fmt.Printf("Worker %d starting\n", id)
// simulate work
fmt.Printf("Worker %d done\n", id)
}
func main() {
var wg sync.WaitGroup
for i := 1; i <= 3; i++ {
wg.Add(1) // add to wait counter
go worker(i, &wg)
}
wg.Wait() // wait for all to finish
}
```
✅ Ensures the program won’t exit before all goroutines finish.
---
### 2. **Channels** – Communication Between Goroutines
Channels are **Go’s big idea** for concurrency.
Instead of sharing memory and locking it, goroutines **communicate by passing messages**.
```go
package main
import "fmt"
func worker(ch chan string) {
ch <- "task finished" // send data into channel
}
func main() {
ch := make(chan string)
go worker(ch)
msg := <-ch // receive data
fmt.Println("Message:", msg)
}
```
👉 Think of it like JavaScript `Promise.resolve("task finished")`, but **synchronous communication** unless buffered.
---
### 3. **Buffered Channels** – Queue of Messages
```go
ch := make(chan int, 2) // capacity = 2
ch <- 10
ch <- 20
fmt.Println(<-ch)
fmt.Println(<-ch)
```
* Unbuffered channel: send blocks until receive is ready.
* Buffered channel: send doesn’t block until buffer is full.
---
### 4. **select** – Multiplexing Channels
```go
select {
case msg := <-ch1:
fmt.Println("Received", msg)
case msg := <-ch2:
fmt.Println("Received", msg)
default:
fmt.Println("No message")
}
```
Like `Promise.race()` in JS.
---
# 🔥 Key Gotchas with Goroutines
1. **Main goroutine exit kills all child goroutines**.
→ Always use WaitGroups or channels to synchronize.
2. **Race conditions** happen if goroutines write/read shared data without sync.
→ Use `sync.Mutex`, `sync.RWMutex`, or better: **channels**.
3. **Too many goroutines** can cause memory pressure, but still far cheaper than threads.
4. **Don’t block forever** – unreceived channel sends cause deadlocks.
---
# 📊 Real-World Use Cases
* **Web servers**: Each request can run in its own goroutine.
* **Scraping / Crawling**: Launch a goroutine for each URL fetch.
* **Background jobs**: Run tasks concurrently (DB writes, logging, metrics).
* **Pipelines**: Process data in multiple stages with goroutines + channels.
---
# 🧠 Mental Model (JS vs Go)
* **JavaScript** → concurrency = single-threaded event loop + async callbacks.
* **Go** → concurrency = many goroutines scheduled onto multiple OS threads.
So:
* In JS, concurrency = illusion via async.
* In Go, concurrency = real, parallel execution when multiple CPU cores exist.
---
✅ To summarize:
* Goroutines = **cheap concurrent tasks** managed by Go runtime.
* Not OS threads, but multiplexed onto threads.
* Communicate via **channels** instead of shared memory.
* Powerful with **WaitGroups, select, and synchronization tools**.
---
**concurrency vs parallelism** is a core concept in computer science and in Go (since Go was built with concurrency in mind). Let’s break it down step by step in detail.
---
## **1. The Core Idea**
* **Concurrency** = Dealing with many tasks at once (managing multiple things).
* **Parallelism** = Doing many tasks at the same time (executing multiple things simultaneously).
Both sound similar, but they’re not the same.
---
## **2. Analogy**
Imagine we’re in a restaurant kitchen:
* **Concurrency (chef multitasking):**
One chef handles multiple dishes by switching between them. He cuts vegetables for Dish A, stirs the sauce for Dish B, and checks the oven for Dish C. He’s *not doing them at the exact same time*, but he’s managing multiple tasks *in progress*.
* **Parallelism (many chefs working together):**
Three chefs cook three different dishes at the *same time*. Tasks truly happen *simultaneously*.
👉 Concurrency is about **structure** (how tasks are managed).
👉 Parallelism is about **execution** (how tasks are run in hardware).
---
## **3. Technical Definition**
* **Concurrency**:
Multiple tasks *make progress* in overlapping time periods. It doesn’t require multiple processors/cores. Even with a single CPU core, the system can *interleave execution* of tasks via context switching.
* **Parallelism**:
Multiple tasks *run at the exact same instant*, usually on different CPU cores or processors.
---
## **4. Example with Go**
Go is famous for concurrency with **goroutines**.
```go
package main
import (
"fmt"
"time"
)
func task(name string) {
for i := 1; i <= 3; i++ {
fmt.Println(name, ":", i)
time.Sleep(500 * time.Millisecond)
}
}
func main() {
go task("Task A") // run concurrently
go task("Task B")
time.Sleep(3 * time.Second)
fmt.Println("Done")
}
```
### What happens:
* **Concurrency:** Both `Task A` and `Task B` *appear to run at the same time* because Go schedules goroutines across available cores. If you run this on a single-core CPU, Go interleaves execution → that’s concurrency.
* **Parallelism:** If you run this on a multi-core CPU, `Task A` might run on Core 1 and `Task B` on Core 2 simultaneously → that’s parallelism.
---
## **5. Key Differences Table**
| Aspect | Concurrency | Parallelism |
| ------------------- | ----------------------------------- | ---------------------------------------------- |
| **Definition** | Managing multiple tasks at once | Executing multiple tasks at once |
| **Focus** | Task switching and scheduling | Simultaneous execution |
| **CPU Requirement** | Can happen on a single-core CPU | Requires multi-core CPU |
| **Analogy** | One chef multitasking across dishes | Many chefs cooking different dishes |
| **In Go** | Achieved via goroutines & channels | Achieved when goroutines run on multiple cores |
---
## **6. Visual Representation**
* **Concurrency (single-core):**
```
Time: |----A----|----B----|----A----|----B----|
^ Task A and Task B interleaved
```
* **Parallelism (multi-core):**
```
Core1: |----A----|----A----|----A----|
Core2: |----B----|----B----|----B----|
^ Tasks running truly at the same time
```
---
## **7. In Practice**
* Concurrency is **a design approach**: "How do we structure a program so that it can handle many things at once?"
* Parallelism is **an execution strategy**: "How do we use hardware to literally do many things at once?"
Go is *concurrent by design* (goroutines + channels) and *parallel by runtime* (GOMAXPROCS decides how many cores are used).
---
✅ **Final takeaway**:
* **Concurrency = composition of independently executing tasks.**
* **Parallelism = simultaneous execution of tasks.**
They are related, but not the same. A program can be concurrent but not parallel, parallel but not concurrent, or both.
---
Let’s go step by step and dive **deep into channels in Go**, because they’re one of the most powerful concurrency primitives in the language.
---
## 🔹 What are Channels in Go?
In Go, a **channel** is a **typed conduit** (pipe) through which goroutines can **communicate** with each other.
* They allow **synchronization** (ensuring goroutines coordinate properly).
* They allow **data exchange** between goroutines safely, without explicit locking (like mutexes).
👉 Think of a channel as a "queue" or "pipeline" where one goroutine can send data and another goroutine can receive it.
---
## 🔹 Syntax of Channels
### Declaring a channel
```go
var ch chan int // declare a channel of type int
```
### Creating a channel
```go
ch := make(chan int) // make allocates memory for a channel
```
Here:
* `ch` is a channel of integers.
* `make(chan int)` initializes it.
---
## 🔹 Sending and Receiving on Channels
We use the `<-` operator.
```go
ch <- 10 // send value 10 into channel
value := <-ch // receive value from channel
```
* **Send (`ch <- value`)**: Puts data into the channel.
* **Receive (`value := <-ch`)**: Gets data from the channel.
* Both operations **block** until the other side is ready (unless buffered).
---
## 🔹 Example: Simple Goroutine Communication
```go
package main
import (
"fmt"
"time"
)
func worker(ch chan string) {
time.Sleep(2 * time.Second)
ch <- "done" // send message
}
func main() {
ch := make(chan string)
go worker(ch)
fmt.Println("Waiting for worker...")
msg := <-ch // blocks until worker sends data
fmt.Println("Worker says:", msg)
}
```
✅ Output:
```
Waiting for worker...
Worker says: done
```
Here:
* `main` waits on `<-ch` until the goroutine sends "done".
* This **synchronizes** `main` and the worker.
---
## 🔹 Buffered vs Unbuffered Channels
### 1. **Unbuffered Channels** (default)
* No capacity → send blocks until a receiver is ready, and receive blocks until a sender is ready.
* Ensures **synchronization**.
```go
ch := make(chan int) // unbuffered
```
### 2. **Buffered Channels**
* Created with a capacity.
* Allows sending multiple values before blocking, up to the capacity.
```go
ch := make(chan int, 3) // capacity = 3
ch <- 1
ch <- 2
ch <- 3
// sending a 4th value will block until receiver consumes one
```
👉 Buffered channels provide **asynchronous communication**.
---
## 🔹 Closing a Channel
We can close a channel when no more values will be sent:
```go
close(ch)
```
After closing:
* Further sends → **panic**.
* Receives → still possible, but will yield **zero values** when channel is empty.
Example:
```go
package main
import "fmt"
func main() {
ch := make(chan int, 2)
ch <- 10
ch <- 20
close(ch)
for val := range ch {
fmt.Println(val)
}
}
```
✅ Output:
```
10
20
```
---
## 🔹 Directional Channels
We can restrict channels to **send-only** or **receive-only**.
```go
func sendData(ch chan<- int) { // send-only
ch <- 100
}
func receiveData(ch <-chan int) { // receive-only
fmt.Println(<-ch)
}
```
This enforces **clear contracts** between functions.
---
## 🔹 Select Statement (Channel Multiplexing)
The `select` statement is like a `switch` for channels.
It waits on multiple channel operations and executes whichever is ready first.
```go
select {
case msg1 := <-ch1:
fmt.Println("Received", msg1)
case msg2 := <-ch2:
fmt.Println("Received", msg2)
default:
fmt.Println("No messages")
}
```
👉 Useful for:
* Handling multiple channels.
* Adding **timeouts** with `time.After`.
* Preventing blocking with `default`.
---
## 🔹 Real Example: Worker Pool with Channels
Channels make it easy to build worker pools.
```go
package main
import (
"fmt"
"time"
)
func worker(id int, jobs <-chan int, results chan<- int) {
for job := range jobs {
fmt.Printf("Worker %d processing job %d\n", id, job)
time.Sleep(time.Second)
results <- job * 2
}
}
func main() {
jobs := make(chan int, 5)
results := make(chan int, 5)
// Start 3 workers
for i := 1; i <= 3; i++ {
go worker(i, jobs, results)
}
// Send jobs
for j := 1; j <= 5; j++ {
jobs <- j
}
close(jobs)
// Collect results
for r := 1; r <= 5; r++ {
fmt.Println("Result:", <-results)
}
}
```
✅ Output (order may vary):
```
Worker 1 processing job 1
Worker 2 processing job 2
Worker 3 processing job 3
Worker 1 processing job 4
Worker 2 processing job 5
Result: 2
Result: 4
Result: 6
Result: 8
Result: 10
```
This shows how channels + goroutines → powerful **concurrent systems**.
---
## 🔹 Key Takeaways
* Channels are **typed pipes** for goroutine communication.
* **Unbuffered channels** synchronize sender and receiver.
* **Buffered channels** allow limited async communication.
* Use `close()` to signal no more values.
* Directional channels (`chan<-`, `<-chan`) enforce contracts.
* `select` helps multiplex multiple channels.
* Channels + goroutines = safe, concurrent, and elegant design.
---
Now we’re going into the **guts of channels in Go**, the kind of stuff that matters if we want a *CS-level* understanding of why channels are so powerful and how they avoid race conditions. 🚀
---
# 🔬 Channels in Go: Under the Hood
Channels in Go aren’t magic — they’re implemented in the **Go runtime (part of the scheduler and memory model)**. Let’s break down their **internal structure, blocking mechanism, and scheduling behavior**.
---
## 1. Channel Data Structure (`hchan`)
Internally, every channel is represented by a structure called `hchan` (defined in Go’s runtime source, `runtime/chan.go`):
```go
type hchan struct {
qcount uint // number of elements currently in queue
dataqsiz uint // size of the circular buffer
buf unsafe.Pointer // circular buffer (for buffered channels)
elemsize uint16 // size of each element
closed uint32 // is channel closed?
sendx uint // send index (next slot to write to)
recvx uint // receive index (next slot to read from)
recvq waitq // list of goroutines waiting to receive
sendq waitq // list of goroutines waiting to send
lock mutex // protects all fields
}
```
### Key things to notice:
* **Circular Buffer** → if channel is buffered, data lives here.
* **Send/Recv Index** → used for round-robin access in buffer.
* **Wait Queues** → goroutines that are blocked are put here.
* **Lock** → ensures safe concurrent access (Go runtime manages locking, so we don’t).
---
## 2. Unbuffered Channels (Zero-Capacity)
Unbuffered channels are the simplest case:
* **Send (`ch <- x`)**:
* If there’s already a goroutine waiting to receive, value is copied directly into its stack.
* If not, sender blocks → it’s enqueued into `sendq` until a receiver arrives.
* **Receive (`<-ch`)**:
* If there’s a waiting sender, value is copied directly.
* If not, receiver blocks → it’s enqueued into `recvq` until a sender arrives.
👉 This is why unbuffered channels **synchronize goroutines**. No buffer exists; transfer happens only when both sides are ready.
---
## 3. Buffered Channels
Buffered channels add a **queue (circular buffer)**:
* **Send**:
* If buffer not full → put value in buffer, increment `qcount`, update `sendx`.
* If buffer full → block, enqueue sender in `sendq`.
* **Receive**:
* If buffer not empty → take value from buffer, decrement `qcount`, update `recvx`.
* If buffer empty → block, enqueue receiver in `recvq`.
👉 Buffered channels provide **asynchronous communication**, but when full/empty they still enforce synchronization.
---
## 4. Blocking and Goroutine Parking
When a goroutine **cannot proceed** (because channel is full or empty), Go’s runtime **parks** it:
* **Parking** = goroutine is put to sleep, removed from runnable state.
* **Unparking** = when the condition is satisfied (e.g., sender arrives), runtime wakes up the goroutine and puts it back on the scheduler queue.
This avoids **busy-waiting** (goroutines don’t spin-loop, they sleep efficiently).
---
## 5. Closing a Channel
When we `close(ch)`:
* `closed` flag in `hchan` is set.
* All goroutines in `recvq` are **woken up** and return the **zero value**.
* Any new send → **panic**.
* Receives on empty closed channel → return **zero value** immediately.
---
## 6. Select Statement Internals
`select` in Go is implemented like a **non-deterministic choice operator**:
1. The runtime looks at all channel cases.
2. If multiple channels are ready → **pick one pseudo-randomly** (to avoid starvation).
3. If none are ready → block the goroutine, enqueue it on all those channels’ `sendq/recvq`.
4. When one channel becomes available, runtime wakes up the goroutine, executes that case, and unregisters it from others.
👉 This is why `select` is **fair and efficient**.
---
## 7. Memory Model Guarantees
Channels follow Go’s **happens-before** relationship:
* A send on a channel **happens before** the corresponding receive completes.
* This ensures **visibility** of writes: when one goroutine sends a value, all memory writes before the send are guaranteed visible to the receiver after the receive.
This is similar to **release-acquire semantics** in CPU memory models.
---
## 8. Performance Notes
* Channels avoid **explicit locks** for user code — the runtime lock inside `hchan` is optimized with **CAS (Compare-And-Swap)** instructions when possible.
* For heavy concurrency, channels can become a bottleneck (due to contention on `hchan.lock`). In such cases, Go devs sometimes use **lock-free data structures** or **sharded channels**.
* But for **safe communication**, channels are much cleaner than manual locking.
---
## 9. Analogy
Imagine a **mailbox system**:
* Unbuffered channel → one person waits at the mailbox until another arrives.
* Buffered channel → mailbox has slots; sender can drop letters until it’s full.
* `select` → person waiting at multiple mailboxes, ready to grab whichever letter arrives first.
* Closing → post office shuts down; no new letters allowed, but old ones can still be collected.
---
## 🔑 Key Takeaways (CS-level)
1. Channels are backed by a **lock-protected struct (`hchan`)** with a buffer and wait queues.
2. **Unbuffered channels** → synchronous handoff (sender ↔ receiver meet at the same time).
3. **Buffered channels** → async up to capacity, but still block when full/empty.
4. Blocked goroutines are **parked** efficiently, not spin-looping.
5. **Select** allows non-deterministic, fair channel multiplexing.
6. **Closing** signals termination and wakes receivers.
7. Channels provide **happens-before memory guarantees**, making them safer than manual synchronization.
---
Let’s go deep into **unbuffered vs buffered channels in Go**, both conceptually and under the hood (CS-level).
---
# 🔹 Channels Recap
A **channel** in Go is essentially a **typed conduit** that goroutines use to communicate. Think of it like a pipe with synchronization built-in. Under the hood, Go implements channels as a **struct (`hchan`)** in the runtime, which manages:
* A **queue (circular buffer)** of values
* A list of goroutines waiting to **send**
* A list of goroutines waiting to **receive**
* Locks for synchronization
---
# 🔹 Unbuffered Channels
An **unbuffered channel** is created like this:
```go
ch := make(chan int) // no buffer size specified
```
### ✅ Key Behavior:
* **Synchronous communication.**
* A `send` (`ch <- v`) blocks until another goroutine executes a `receive` (`<-ch`).
* A `receive` blocks until another goroutine sends.
* This creates a **rendezvous point** between goroutines: both must be ready simultaneously.
### 🔍 Under the hood:
* Since the buffer capacity = 0, the channel cannot hold values.
* When a goroutine executes `ch <- v`:
1. The runtime checks if there’s a waiting receiver in the channel’s `recvq`.
2. If yes → it directly transfers the value from sender to receiver (no buffer copy).
3. If not → the sender goroutine is put to sleep and added to the `sendq`.
* Similarly, a receiver blocks until there’s a sender.
So **data is passed directly**, goroutine-to-goroutine, like a **handoff**.
### Example:
```go
func main() {
ch := make(chan int)
go func() {
ch <- 42 // blocks until receiver is ready
}()
val := <-ch // blocks until sender is ready
fmt.Println(val) // 42
}
```
This ensures synchronization — the print only happens after the send completes.
---
# 🔹 Buffered Channels
A **buffered channel** is created like this:
```go
ch := make(chan int, 3) // capacity = 3
```
### ✅ Key Behavior:
* **Asynchronous communication up to capacity.**
* A `send` (`ch <- v`) only blocks if the buffer is full.
* A `receive` (`<-ch`) only blocks if the buffer is empty.
* Acts like a **queue** between goroutines.
### 🔍 Under the hood:
* Channel has a circular buffer (`qcount`, `dataqsiz`, `buf`).
* On `ch <- v`:
1. If a receiver is waiting → value bypasses buffer, sent directly.
2. Else, if buffer is not full → value is enqueued in buffer.
3. Else (buffer full) → sender goroutine is parked in `sendq`.
* On `<-ch`:
1. If buffer has elements → dequeue and return.
2. Else, if a sender is waiting → take value directly.
3. Else → receiver goroutine is parked in `recvq`.
So buffered channels allow **decoupling**: senders and receivers don’t have to line up perfectly in time (up to buffer capacity).
### Example:
```go
func main() {
ch := make(chan int, 2)
ch <- 1 // does not block
ch <- 2 // does not block
go func() {
ch <- 3 // blocks until someone reads
}()
fmt.Println(<-ch) // frees up buffer, goroutine sends 3
fmt.Println(<-ch)
fmt.Println(<-ch)
}
```
---
# 🔹 Comparison
| Feature | Unbuffered Channel | Buffered Channel |
| ------------------- | ------------------------------------------ | --------------------------------- |
| Capacity | 0 | > 0 |
| Communication style | Synchronous (rendezvous) | Asynchronous (queue, but limited) |
| Blocking (Send) | Until a receiver is ready | Until buffer has space |
| Blocking (Receive) | Until a sender is ready | Until buffer has data |
| Use case | Enforce synchronization, handoff semantics | Decouple producer/consumer timing |
---
# 🔹 Analogy
* **Unbuffered channel**: Like passing a note **hand-to-hand** — both people must meet at the same time.
* **Buffered channel**: Like a mailbox — sender can drop the note and leave, receiver can pick it up later (until the mailbox is full).
---
# 🔹 CS-Level Insight
* The Go runtime (`runtime/chan.go`) maintains:
* `hchan` struct:
```go
type hchan struct {
qcount uint // number of elements in queue
dataqsiz uint // size of circular buffer
buf unsafe.Pointer // actual buffer
sendx uint // next send index
recvx uint // next receive index
recvq waitq // list of waiting receivers
sendq waitq // list of waiting senders
lock mutex
}
```
* **Unbuffered**: `dataqsiz == 0`, always uses `sendq/recvq`.
* **Buffered**: Uses circular buffer for queuing, only parks goroutines when full/empty.
---
👉 So in short:
* **Unbuffered** channels couple goroutines tightly in time (synchronous handoff).
* **Buffered** channels decouple them (asynchronous, but limited by capacity).
---
Perfect 👌 — this is where Go really shines. Let’s break down how the **Go runtime** orchestrates **goroutines + channels** together, in detail, both conceptually and at the runtime (CS-level implementation).
---
# 🔹 The Go Runtime Model
Go doesn’t rely on the OS to schedule lightweight tasks. Instead, it implements its **own scheduler** inside the runtime. This allows goroutines and channels to work smoothly together.
---
## 1. **Goroutines in the Runtime**
* A **goroutine** is a lightweight thread of execution, managed by the Go runtime (not OS).
* Under the hood:
* Each goroutine is represented by a `g` struct.
* Each has its own **stack** (starts tiny, grows/shrinks dynamically).
* Thousands (even millions) of goroutines can run inside one OS thread.
### Scheduler: **M:N model**
* **M** = OS threads
* **N** = Goroutines
* The runtime maps N goroutines onto M OS threads.
* **Key runtime structs:**
* **M (Machine)** → OS thread
* **P (Processor)** → Logical processor, responsible for scheduling goroutines on an M
* **G (Goroutine)** → A goroutine itself
* Scheduling is **cooperative + preemptive**:
* Goroutines yield at certain safe points (e.g., blocking operations, function calls).
* Since Go 1.14, preemption also works at loop backedges.
So: goroutines are not OS-level threads — they’re scheduled by Go’s own runtime.
---
## 2. **Channels in the Runtime**
Channels are the **synchronization primitive** between goroutines.
Runtime implementation: `runtime/chan.go`.
Struct:
```go
type hchan struct {
qcount uint // # of elements in queue
dataqsiz uint // buffer size
buf unsafe.Pointer // circular buffer
sendx uint // next send index
recvx uint // next receive index
recvq waitq // waiting receivers
sendq waitq // waiting senders
lock mutex
}
```
### Core idea:
* Channels are **queues with wait lists**:
* If buffered → goroutines enqueue/dequeue values.
* If unbuffered → goroutines handshake directly.
* Senders & receivers that cannot proceed are **parked** (suspended) into the `sendq` or `recvq`.
---
## 3. **How Goroutines & Channels Interact**
### Case A: Unbuffered channel
```go
ch := make(chan int)
go func() { ch <- 42 }()
val := <-ch
```
1. Sender (`ch <- 42`):
* Lock channel.
* Check `recvq` (waiting receivers).
* If receiver waiting → value copied directly → receiver wakes up → sender continues.
* If no receiver → sender is **parked** (blocked) and added to `sendq`.
2. Receiver (`<-ch`):
* Lock channel.
* Check `sendq` (waiting senders).
* If sender waiting → value copied → sender wakes up → receiver continues.
* If no sender → receiver is parked and added to `recvq`.
This ensures **synchronous handoff**.
---
### Case B: Buffered channel
```go
ch := make(chan int, 2)
```
1. Sender (`ch <- v`):
* Lock channel.
* If `recvq` has waiting receivers → skip buffer, deliver directly.
* Else if buffer has space → enqueue value → done.
* Else (buffer full) → park sender in `sendq`.
2. Receiver (`<-ch`):
* Lock channel.
* If buffer has values → dequeue → done.
* Else if `sendq` has waiting senders → take value directly.
* Else → park receiver in `recvq`.
So buffered channels act as a **mailbox** (async up to capacity).
---
## 4. **Parking & Resuming Goroutines**
When goroutines can’t make progress (blocked send/recv), the runtime:
* **Parks** them: puts them in channel queues (`sendq` or `recvq`) and removes them from the scheduler’s run queue.
* Stores a `sudog` (suspended goroutine) object in the queue with metadata (which goroutine, element pointer, etc.).
When the condition is satisfied (buffer space, sender arrives, etc.):
* The runtime **wakes up** a waiting goroutine by moving it back into the scheduler’s run queue.
* The scheduler later assigns it to a P (processor) → M (thread) → resumes execution.
This is why Go channels feel seamless: the runtime transparently parks and wakes goroutines.
---
## 5. **Select & Channels**
`select` is also handled in runtime:
* The runtime checks multiple channels in random order to avoid starvation.
* If one is ready → proceeds immediately.
* If none are ready → goroutine is parked, attached to all involved channels’ queues, and woken up when one becomes available.
---
## 6. **Performance & Efficiency**
* Channel operations are protected by **mutex + atomic ops** → very efficient.
* Goroutines are cheap (KB stack, small structs).
* Parking/waking is implemented in pure runtime → no heavy syscalls unless all goroutines block (then Go hands thread back to OS).
---
# 🔹 Visual Summary
### Unbuffered
```
G1: ch <- 42 <-----> G2: val := <-ch
(synchronous handoff, both must rendezvous)
```
### Buffered
```
G1: ch <- 42 ---> [ buffer ] ---> G2: val := <-ch
(asynchronous until buffer full/empty)
```
### Runtime scheduling
```
[M:OS Thread] <----> [P:Logical Processor] <----> [G:Goroutine Queue]
```
---
# 🔹 Big Picture
* **Goroutines** = cheap lightweight threads managed by Go runtime.
* **Scheduler** = M:N model with P (processor) abstraction.
* **Channels** = safe queues with wait lists.
* **Interaction** = senders/receivers park & wake, enabling CSP-style concurrency.
* **Runtime magic** = efficient, cooperative scheduling + lightweight context switching.
---
👉 So: goroutines are like "actors," channels are "mailboxes," and the Go runtime is the "stage manager" that schedules actors and delivers their messages efficiently.
---
Let’s build a **step-by-step execution timeline** for how the Go runtime handles **goroutines + channels**.
Two cases: **unbuffered** and **buffered** channels.
---
# 🔹 Case 1: Unbuffered Channel
Code:
```go
ch := make(chan int)
go func() {
ch <- 42
fmt.Println("Sent 42")
}()
val := <-ch
fmt.Println("Received", val)
```
---
### Execution Timeline (runtime flow)
1. **Main goroutine (G_main)** creates channel `ch` (capacity = 0).
* Runtime allocates an `hchan` struct with empty `sendq` and `recvq`.
2. **Spawn goroutine (G1)** → scheduled by runtime onto an M (OS thread) via some P.
3. **G1 executes `ch <- 42`:**
* Lock channel.
* Since `recvq` is empty, no receiver is waiting.
* Create a `sudog` for G1 (stores goroutine pointer + value).
* Add `sudog` to `sendq`.
* **G1 is parked (blocked)** → removed from run queue.
4. **Main goroutine executes `<-ch`:**
* Lock channel.
* Sees `sendq` has a waiting sender (G1).
* Runtime copies `42` from G1’s stack to G_main’s stack.
* Removes G1 from `sendq`.
* Marks G1 as runnable → puts it back in the scheduler’s run queue.
* G_main continues with value `42`.
5. **Scheduler resumes G1** → prints `"Sent 42"`.
**Main goroutine prints `"Received 42"`.
---
🔸 **Key point**: In unbuffered channels, send/recv must rendezvous. One goroutine blocks until the other arrives.
---
# 🔹 Case 2: Buffered Channel
Code:
```go
ch := make(chan int, 2)
go func() {
ch <- 1
ch <- 2
ch <- 3
fmt.Println("Sent all")
}()
time.Sleep(time.Millisecond) // give sender time
fmt.Println(<-ch)
fmt.Println(<-ch)
fmt.Println(<-ch)
```
---
### Execution Timeline (runtime flow)
1. **Main goroutine (G_main)** creates channel `ch` (capacity = 2).
* Runtime allocates buffer (circular queue), size = 2.
2. **Spawn goroutine (G1)**.
3. **G1 executes `ch <- 1`:**
* Lock channel.
* Buffer not full (0/2).
* Enqueue `1` at `buf[0]`.
* Increment `qcount` = 1.
* Return immediately (non-blocking).
4. **G1 executes `ch <- 2`:**
* Lock channel.
* Buffer not full (1/2).
* Enqueue `2` at `buf[1]`.
* `qcount` = 2.
* Return immediately.
5. **G1 executes `ch <- 3`:**
* Lock channel.
* Buffer is full (2/2).
* No receivers waiting (`recvq` empty).
* Create `sudog` for G1.
* Put it in `sendq`.
* Park G1 (blocked).
6. **Main goroutine executes `<-ch`:**
* Lock channel.
* Buffer has elements (`qcount` = 2).
* Dequeue `1`.
* `qcount` = 1.
* Since there’s a blocked sender in `sendq` (G1 with value `3`), runtime:
* Wakes G1.
* Copies `3` into buffer (at freed slot).
* G1 resumes later.
7. **Main goroutine executes `<-ch` again:**
* Dequeue `2`.
* `qcount` = 1 (still has `3`).
8. **Main goroutine executes `<-ch` final time:**
* Dequeue `3`.
* `qcount` = 0 (buffer empty).
9. **Scheduler resumes G1** → `"Sent all"` printed.
---
🔸 **Key point**: Buffered channels decouple sender/receiver timing. G1 only blocked when the buffer was full.
---
# 🔹 Visual Snapshot
### Unbuffered
```
G1: send(42) ---- waits ----> G_main: recv()
<--- wakes ----
```
### Buffered (capacity = 2)
```
Buffer: [ 1 ][ 2 ] <- send 1, send 2
Buffer: full <- send 3 blocks
Recv 1 → slot frees <- wakes sender, puts 3 in
Recv 2, Recv 3 <- empties buffer
```
---
👉 In both cases, the **Go runtime orchestrates this**:
* `sendq` & `recvq` hold waiting goroutines (`sudog` objects).
* Blocked goroutines are **parked** (suspended).
* When conditions change (buffer frees, peer arrives), goroutines are **woken** and put back into the scheduler’s run queue.
---
# Buffered channels in Go — deep dive 🔎
A **buffered channel** is a channel with capacity > 0:
```go
ch := make(chan int, 3) // capacity 3
```
It provides a small queue (a circular buffer) between senders and receivers. A send (`ch <- v`) only blocks when the buffer is **full**; a receive (`<-ch`) only blocks when the buffer is **empty** — *unless* there are waiting peers, in which case the runtime can do a direct handoff.
Use it when we want to **decouple producer and consumer timing** (allow short bursts) but still bound memory and concurrency.
---
# Creation & introspection
* Create: `ch := make(chan T, capacity)` where `capacity >= 1`.
* Zero value is `nil`: `var ch chan int` → nil channel (send/recv block forever).
* Inspect: `len(ch)` gives number of queued elements, `cap(ch)` gives capacity.
---
# High-level send/receive rules (precise)
**When sending (`ch <- v`)**:
1. If there is a *waiting receiver* (parked on `recvq`) → **direct transfer**: runtime copies `v` to receiver and wakes it (no buffer enqueue).
2. Else if the buffer has free slots (`len < cap`) → **enqueue** the value into the circular buffer and return immediately.
3. Else (buffer full and no receiver) → **park the sender** (sudog) on the channel's `sendq` and block.
**When receiving (`<-ch`)**:
1. If buffer has queued items (`len > 0`) → **dequeue** an item and return it.
2. Else if there is a *waiting sender* (in `sendq`) → **direct transfer**: take the sender’s value and wake the sender.
3. Else (buffer empty and no sender) → **park the receiver** on `recvq` and block.
> Important: the runtime prefers delivering directly to a waiting peer if one exists — it avoids unnecessary buffer operations and wake-ups.
---
# Under-the-hood (simplified runtime view)
Channels are implemented by the runtime in a structure conceptually like:
```go
// simplified conceptual fields
type hchan struct {
qcount uint // number of elements currently in buffer
dataqsiz uint // capacity (buffer size)
buf unsafe.Pointer // pointer to circular buffer memory
sendx uint // next index to send (enqueue)
recvx uint // next index to receive (dequeue)
sendq waitq // queue of waiting senders (sudog)
recvq waitq // queue of waiting receivers (sudog)
lock mutex // protects the channel's state
}
```
* The buffer is a circular array indexed by `sendx`/`recvx` modulo `dataqsiz`.
* `sendq` and `recvq` are queues of parked goroutines (sudog objects) waiting for a send/receive.
* Operations lock the channel, check queues and buffer, then either enqueue/dequeue or park/unpark goroutines.
* Parked goroutines are moved back to the scheduler run queue when woken.
---
# Example — behavior & output
```go
package main
import (
"fmt"
"time"
)
func main() {
ch := make(chan int, 2) // capacity 2
go func() {
ch <- 1 // does NOT block
fmt.Println("sent 1")
ch <- 2 // does NOT block
fmt.Println("sent 2")
ch <- 3 // blocks until receiver consumes one
fmt.Println("sent 3")
}()
time.Sleep(100 * time.Millisecond) // let sender run
fmt.Println("recv:", <-ch) // receives 1; this will unblock sender for 3
fmt.Println("recv:", <-ch) // receives 2
fmt.Println("recv:", <-ch) // receives 3
}
```
Expected printed sequence (order may vary slightly with scheduling, but logically):
```
sent 1
sent 2
recv: 1
sent 3 // unblocks here after first recv frees slot
recv: 2
recv: 3
```
---
# Closing a buffered channel
* `close(ch)`:
* Makes the channel no longer accept sends. Any sends to a closed channel Panic.
* Receivers can still drain buffered items.
* Once buffer is empty, subsequent receives return the zero value and `ok == false`.
* Example:
```go
ch := make(chan int, 2)
ch <- 10
ch <- 20
close(ch)
v, ok := <-ch // v==10, ok==true
v, ok = <-ch // v==20, ok==true
v, ok = <-ch // v==0, ok==false (channel drained and closed)
```
* Closing is normally done by the **sender/owner** side. Closing from multiple places or closing when other senders still send is dangerous.
---
# `select` + buffered channels (non-blocking tries)
We often use a `select` with `default` to attempt a non-blocking send/recv:
```go
select {
case ch <- v:
// succeeded
default:
// buffer full — do alternate action
}
```
This is how we implement try-send / try-receive semantics.
---
# Typical patterns & idioms
1. **Bounded buffer / producer-consumer**
* Buffer provides smoothing for bursts.
2. **Worker pool (task queue)**
* `tasks := make(chan Task, queueSize)` — spawn worker goroutines that `for t := range tasks { ... }`.
3. **Semaphore / concurrency limiter**
```go
sem := make(chan struct{}, N) // allow N concurrent active tasks
sem <- struct{}{} // acquire (blocks when N reached)
<-sem // release
```
4. **Pipelines**
* Stage outputs into buffered channels to decouple stages.
---
# Synchronization & memory visibility
* A successful **send** on a channel *synchronizes with* the corresponding **receive** that receives the value. That means the receive sees all memory writes that happened before the send (happens-before guarantee).
* Using channels for signalling is safe: if we send after setting fields, the receiver will see those fields set.
---
# Performance considerations
* Buffered channels improve throughput where producers and consumers are not tightly synchronized.
* Too large buffers:
* Consume more memory.
* Increase latency for consumers (items may sit in buffer).
* Mask backpressure (producers can outrun consumers).
* Too small buffers:
* Lead to frequent blocking and context switching.
* Tuning:
* Choose `cap` to match burst size / acceptable queueing.
* For heavy throughput, benchmark channels vs other concurrency primitives (e.g., pools, atomics) — channels are convenient and fast but not free.
---
# Common pitfalls & gotchas
* **Deadlock**: If producers fill the buffer and nobody consumes, they block. If blocked sends prevent the program from progressing, deadlock occurs.
* **Send on closed channel**: panic — avoid by ensuring only the owner closes the channel.
* **Nil channel**: `var ch chan T` without make is `nil` — send/recv block forever.
* **Large struct values**: sending large values copies them into the buffer; prefer pointers or smaller structs if copying is expensive.
* **Mixing close and multiple senders**: close only from a single owner to avoid races/panics.
---
# FIFO & fairness
* The runtime enqueues waiting senders/receivers (sudogs) and generally wakes them in FIFO order — so waiting goroutines are served in roughly the order they arrived. For `select` across multiple channels, selection is randomized among ready cases to avoid starvation.
---
# Quick cheatsheet
* `make(chan T, n)` → buffered channel with capacity `n`.
* `len(ch)` → items queued now.
* `cap(ch)` → total capacity.
* `close(ch)` → no more sends; readers drain buffer then get `ok==false`.
* `select { case ch<-v: default: }` → non-blocking send attempt.
---
# When to use buffered channels
* When producers produce in bursts and consumers are slower but able to catch up.
* When you want some decoupling but still bounded memory/queueing.
* When you need a simple concurrency limiter (semaphore style).
---
Channel Synchronization is one of the most important and elegant parts of Go’s concurrency model.
---
# 🔹 What is Channel Synchronization?
* In Go, **channels are not just for communication** (passing values between goroutines).
* They are also a **synchronization primitive**: they coordinate execution order between goroutines.
Think of it like:
👉 **Send blocks until the receiver is ready** (unbuffered)
👉 **Receive blocks until the sender provides data**
👉 This mutual blocking acts as a synchronization point.
---
# 🔹 Case 1: Synchronization with **Unbuffered Channels**
Unbuffered channels enforce **strict rendezvous synchronization**:
* When goroutine A sends (`ch <- x`), it is **blocked** until goroutine B executes a receive (`<- ch`).
* Both goroutines meet at the channel, exchange data, and continue.
### Example:
```go
package main
import (
"fmt"
"time"
)
func worker(done chan bool) {
fmt.Println("Worker: started")
time.Sleep(2 * time.Second)
fmt.Println("Worker: finished")
// notify main goroutine
done <- true
}
func main() {
done := make(chan bool)
go worker(done)
// wait for worker to finish
<-done
fmt.Println("Main: all done")
}
```
🔎 Here:
* `done <- true` **synchronizes** the worker with the main goroutine.
* Main will **block** on `<-done` until the worker signals.
* No explicit `mutex` or condition variable is needed — the channel ensures correct ordering.
---
# 🔹 Case 2: Synchronization with **Buffered Channels**
Buffered channels allow **decoupling** between sender and receiver, but can still be used for synchronization.
Rules:
* Sending blocks **only if buffer is full**.
* Receiving blocks **only if buffer is empty**.
### Example:
```go
package main
import (
"fmt"
"time"
)
func worker(tasks chan int, done chan bool) {
for {
task, more := <-tasks
if !more {
fmt.Println("Worker: all tasks done")
done <- true
return
}
fmt.Println("Worker: processing task", task)
time.Sleep(500 * time.Millisecond)
}
}
func main() {
tasks := make(chan int, 3)
done := make(chan bool)
go worker(tasks, done)
for i := 1; i <= 5; i++ {
fmt.Println("Main: sending task", i)
tasks <- i
}
close(tasks) // signals no more tasks
<-done // wait for worker
fmt.Println("Main: worker finished")
}
```
🔎 Here:
* Buffer allows **temporary queuing** of tasks.
* Synchronization happens when `tasks` is full (main blocks) or empty (worker blocks).
* Closing the channel signals the worker to stop.
---
# 🔹 How the Go Runtime Synchronizes with Channels
Now let’s peek **under the hood**.
### 1. Each channel (`hchan`) has:
* A **buffer** (circular queue, if buffered).
* Two wait queues:
* `sendq` → goroutines waiting to send.
* `recvq` → goroutines waiting to receive.
### 2. Unbuffered channel (capacity = 0):
* A send operation checks `recvq`:
* If a goroutine is waiting to receive → direct handoff (value copied, receiver resumed).
* If not → sender parks itself in `sendq` (blocked).
* A receive operation checks `sendq`:
* If a goroutine is waiting to send → direct handoff.
* If not → receiver parks itself in `recvq`.
This ensures **synchronous rendezvous**.
---
### 3. Buffered channel (capacity > 0):
* Send:
* If buffer is **not full** → enqueue value, return immediately.
* If buffer is **full** → block in `sendq`.
* Receive:
* If buffer is **not empty** → dequeue value, return immediately.
* If buffer is **empty** → block in `recvq`.
---
### 4. Synchronization = parking and unparking goroutines
* When a goroutine blocks, the runtime:
* Saves its state (stack, registers).
* Moves it off the run queue.
* Adds it to the channel’s wait queue.
* When the opposite operation happens, the runtime:
* Wakes a goroutine from the wait queue.
* Puts it back on the scheduler run queue.
* This is how Go **synchronizes goroutines without explicit locks**.
---
# 🔹 Real-world Patterns of Channel Synchronization
1. **Signaling** (done channels, as in worker example).
2. **Worker pools** (tasks + done channels).
3. **Bounded queues** (buffered channels to control throughput).
4. **Fan-in / Fan-out** (multiple producers and consumers).
5. **Rate limiting** (token buckets using buffered channels).
---
✅ **Summary**
* Channels synchronize goroutines naturally: **send blocks until receive, receive blocks until send** (with buffering rules).
* Runtime uses **wait queues (sendq, recvq)** and **goroutine parking/unparking** for this.
* This synchronization mechanism replaces the need for explicit mutexes in many cases.
---
Great 🚀 — let’s deep-dive into **channel synchronization in Go**, because this is one of the core reasons channels exist: **coordinating goroutines safely without explicit locks**.
We’ll go step by step, from simple usage all the way to **CS-level synchronization semantics**.
---
# 🔹 What is Synchronization?
**Synchronization** means making sure multiple concurrent goroutines operate in a **coordinated, predictable way**, without stepping on each other’s work or causing race conditions.
In Go, channels synchronize goroutines by enforcing rules on when data can be sent and received.
---
# 🔹 1. How Channels Synchronize
Channels synchronize via **blocking semantics**:
* **Send (`ch <- value`)**:
* Blocks until a receiver is ready (on unbuffered channel).
* On buffered channel, blocks if buffer is full.
* **Receive (`<-ch`)**:
* Blocks until a sender sends.
* On buffered channel, blocks if buffer is empty.
👉 This blocking ensures **coordination**: the sending goroutine knows the receiver has received (or will eventually receive) the value.
---
# 🔹 2. Synchronization with Unbuffered Channels
Unbuffered channels are the **purest form of synchronization**.
They act like a **handshake**: both goroutines must be ready at the same time.
Example:
```go
package main
import (
"fmt"
"time"
)
func worker(done chan bool) {
fmt.Println("Working...")
time.Sleep(2 * time.Second)
fmt.Println("Done work")
// notify main
done <- true
}
func main() {
done := make(chan bool)
go worker(done)
// main waits for signal
<-done
fmt.Println("Main exits")
}
```
✅ Explanation:
* `worker` sends `true` into `done`.
* `main` is blocked on `<-done` until the worker finishes.
* This ensures **main only exits after worker is done**.
This is pure **synchronization without shared memory**.
---
# 🔹 3. Synchronization with Buffered Channels
Buffered channels add a **queue** (limited capacity), which changes synchronization rules:
```go
ch := make(chan int, 2)
ch <- 1 // does not block
ch <- 2 // still fine
// ch <- 3 would block until someone reads
```
* Buffered channels let sender and receiver **work asynchronously** (up to the buffer capacity).
* Still provide synchronization when buffer is full (sender waits) or empty (receiver waits).
Use case: **producer-consumer pattern**.
---
# 🔹 4. Synchronization via Closing a Channel
Closing channels is another synchronization signal:
```go
package main
import "fmt"
func main() {
ch := make(chan int)
go func() {
for i := 1; i <= 3; i++ {
ch <- i
}
close(ch) // signal: no more data
}()
// range until channel closes
for v := range ch {
fmt.Println("Received:", v)
}
fmt.Println("All done")
}
```
✅ Here:
* `close(ch)` synchronizes **end of data stream**.
* Receivers know exactly when producer is finished.
---
# 🔹 5. Synchronization with `select`
`select` synchronizes across **multiple channels**.
Example: timeout synchronization
```go
select {
case msg := <-ch:
fmt.Println("Got:", msg)
case <-time.After(2 * time.Second):
fmt.Println("Timeout")
}
```
👉 This synchronizes **channel communication with time constraints**.
---
# 🔹 6. Under the Hood (CS-Level Synchronization)
At runtime:
* Every channel (`hchan`) has a **mutex lock** and **wait queues** (`sendq`, `recvq`).
* When a goroutine sends and no receiver is ready, it’s **parked** (blocked) in `sendq`.
* When a goroutine receives and no sender is ready, it’s **parked** in `recvq`.
* When a match happens (send & receive ready), the Go runtime:
1. Locks the channel.
2. Transfers the value directly (or via buffer).
3. **Unparks** the waiting goroutine (wakes it up).
4. Releases the lock.
This mechanism guarantees:
* **No busy-waiting** (goroutines don’t spin, they sleep).
* **FIFO fairness** (waiting goroutines handled in queue order).
* **Memory safety**: A send happens-before a corresponding receive completes.
👉 This “happens-before” guarantee ensures **synchronization of memory writes** (data visible to sender before send is visible to receiver after receive).
---
# 🔹 7. Patterns of Synchronization with Channels
1. **Signal Notification**
* Use a channel just to notify completion (`done chan struct{}`).
2. **Worker Pools**
* Workers consume jobs from a channel, producer feeds jobs in.
3. **Fan-in / Fan-out**
* Multiple goroutines send to one channel (fan-in).
* One producer sends to multiple consumers (fan-out).
4. **Pipeline**
* Stages of computation connected by channels, synchronized at each stage.
---
# 🔹 8. Comparison with Mutex Synchronization
* **Mutex**: Protects shared memory by locking. Synchronization is about *exclusive access*.
* **Channel**: Passes ownership of data. Synchronization is about *handover of values/events*.
👉 Go’s philosophy: “**Do not communicate by sharing memory; instead, share memory by communicating**.”
This makes channel-based synchronization **less error-prone** than locks (no risk of forgetting `Unlock()` or deadlock chains).
---
# 🔹 Key Takeaways
1. Channels synchronize goroutines by **blocking semantics** (send/receive waits until possible).
2. **Unbuffered channels** → strongest synchronization, like a handshake.
3. **Buffered channels** → allow async work but still block when full/empty.
4. **Closing channels** synchronizes termination/end of data.
5. **Select** multiplexes synchronization across many events.
6. Under the hood → `hchan`, wait queues, goroutine parking, **happens-before memory model guarantees**.
7. Channels are safer than mutexes because they transfer ownership instead of sharing memory.
---
**In depth into multiplexing with `select` in Go**, because this is where channels + concurrency really shine.
---
# 🔹 What is Multiplexing?
**Multiplexing** means handling multiple communication channels (inputs/outputs) at the same time **without blocking on just one**.
In Go, this is done with the `select` statement, which works like a `switch` but for channel operations.
👉 With `select`, we can **wait on multiple channels simultaneously** and let Go decide which case is ready.
---
# 🔹 Syntax of `select`
```go
select {
case val := <-ch1:
fmt.Println("Received", val, "from ch1")
case ch2 <- 42:
fmt.Println("Sent value to ch2")
default:
fmt.Println("No channel is ready")
}
```
* Each `case` must be a **send** (`ch <- v`) or **receive** (`<-ch`) on a channel.
* `default` executes if none of the channels are ready (non-blocking).
* If multiple cases are ready → **Go chooses one at random** (to avoid starvation).
---
# 🔹 1. Basic Multiplexing Example
```go
package main
import (
"fmt"
"time"
)
func main() {
ch1 := make(chan string)
ch2 := make(chan string)
// Goroutines producing messages at different times
go func() {
time.Sleep(1 * time.Second)
ch1 <- "Message from ch1"
}()
go func() {
time.Sleep(2 * time.Second)
ch2 <- "Message from ch2"
}()
// Listen on both channels
for i := 0; i < 2; i++ {
select {
case msg1 := <-ch1:
fmt.Println("Received:", msg1)
case msg2 := <-ch2:
fmt.Println("Received:", msg2)
}
}
}
```
✅ Output (order depends on timing):
```
Received: Message from ch1
Received: Message from ch2
```
👉 This shows **multiplexing**: instead of waiting only on `ch1` or only on `ch2`, we wait on both.
---
# 🔹 2. Using `default` (Non-Blocking Multiplexing)
```go
select {
case msg := <-ch:
fmt.Println("Received:", msg)
default:
fmt.Println("No message, moving on")
}
```
* If `ch` has no data, it won’t block → it immediately runs `default`.
* Useful for **polling channels** or preventing deadlocks.
---
# 🔹 3. Adding Timeouts with `time.After`
`time.After(d)` returns a channel that sends a value after duration `d`.
We can use it to **timeout channel operations**.
```go
select {
case msg := <-ch:
fmt.Println("Got message:", msg)
case <-time.After(2 * time.Second):
fmt.Println("Timeout after 2s")
}
```
👉 If no message arrives in 2 seconds, the timeout triggers.
This is essential for **robust synchronization** in real systems.
---
# 🔹 4. Multiplexing Multiple Producers
Imagine multiple goroutines producing values at different speeds:
```go
package main
import (
"fmt"
"time"
)
func producer(name string, delay time.Duration, ch chan string) {
for i := 1; i <= 3; i++ {
time.Sleep(delay)
ch <- fmt.Sprintf("%s produced %d", name, i)
}
}
func main() {
ch1 := make(chan string)
ch2 := make(chan string)
go producer("Fast", 1*time.Second, ch1)
go producer("Slow", 2*time.Second, ch2)
for i := 0; i < 6; i++ {
select {
case msg := <-ch1:
fmt.Println("ch1:", msg)
case msg := <-ch2:
fmt.Println("ch2:", msg)
}
}
}
```
✅ Output (interleaved, depending on goroutine timing):
```
ch1: Fast produced 1
ch1: Fast produced 2
ch2: Slow produced 1
ch1: Fast produced 3
ch2: Slow produced 2
ch2: Slow produced 3
```
👉 Multiplexing lets us **interleave messages from multiple sources**.
---
# 🔹 5. Closing Channels in Multiplexing
When channels close, `select` cases still work:
```go
for {
select {
case val, ok := <-ch:
if !ok {
fmt.Println("Channel closed")
return
}
fmt.Println("Got:", val)
}
}
```
👉 Using `ok` ensures we detect channel closure cleanly.
---
# 🔹 6. Internals of `select` (CS-Level)
Under the hood:
* `select` compiles into runtime calls that check all channel states.
* If **one is ready**: Go executes it immediately.
* If **multiple are ready**: Go picks one randomly (fairness).
* If **none are ready**:
* With `default`: executes immediately.
* Without `default`: goroutine **parks** and gets queued on all channels in that `select`. When one becomes available, runtime wakes it up and removes it from the other queues.
👉 This makes `select` an efficient **multiplexer**, similar to `epoll` or `select()` in OS networking.
---
# 🔹 7. Real-World Use Cases
1. **Network Servers**
* Multiplexing multiple connections without blocking.
* Each connection’s data is a channel.
2. **Worker Pools**
* Gather results from many workers on a single loop.
3. **Timeouts/Heartbeats**
* Synchronize goroutines with `time.After` or `time.Tick`.
4. **Fan-in Pattern**
* Combine multiple producers into one consumer loop.
---
# 🔹 Key Takeaways
1. `select` allows **waiting on multiple channels simultaneously**.
2. If multiple cases are ready → one chosen at random.
3. `default` makes `select` **non-blocking**.
4. Can integrate with `time.After` or `time.Tick` for **timeouts & heartbeats**.
5. Used in **multiplexing, cancellation, worker pools, fan-in/fan-out pipelines**.
6. Internally, `select` **registers goroutines on multiple channels** and runtime wakes it up when one is ready.
---
**Closing Channels in Go**. 🚀
This is a super important concept, because channels are not just for passing values, but also for **signaling lifecycle events** between goroutines.
---
# 🔹 1. What Does Closing a Channel Mean?
When we call `close(ch)` on a channel:
* We tell all receivers: **“No more values will ever be sent on this channel.”**
* The channel itself is not destroyed — it can still be read from.
* Sending to a closed channel causes a **panic**.
* Receiving from a closed channel **never blocks**:
* If buffer has values → those are drained first.
* Once empty → it returns the **zero value** of the channel’s type, plus a boolean `ok=false` (if using the `comma-ok` idiom).
---
# 🔹 2. Rules of Closing a Channel
1. **Only the sender should close a channel.**
* Receivers should never close a channel they didn’t create.
* This avoids race conditions where receivers might close while senders are still writing.
2. **Closing is optional.**
* Not all channels need to be closed.
* You only close channels when you want to **signal that no more data is coming**.
3. **You can’t reopen a channel once closed.**
* Channels are single-lifecycle objects.
---
# 🔹 3. Receiving from a Closed Channel
Let’s break it down:
```go
ch := make(chan int, 2)
ch <- 10
ch <- 20
close(ch)
fmt.Println(<-ch) // 10
fmt.Println(<-ch) // 20
fmt.Println(<-ch) // 0 (zero value, because channel is closed + empty)
```
👉 After draining, receivers **get zero value** (`0` for int, `""` for string, `nil` for pointers/maps/etc).
---
# 🔹 4. The `comma-ok` Idiom
To check if a channel is closed:
```go
val, ok := <-ch
if !ok {
fmt.Println("Channel closed!")
} else {
fmt.Println("Got:", val)
}
```
* `ok = true` → value was received successfully.
* `ok = false` → channel is closed and empty.
---
# 🔹 5. Ranging Over a Channel
When using `for range` with a channel:
```go
for v := range ch {
fmt.Println(v)
}
```
* The loop ends automatically when the channel is **closed and empty**.
* This is the most idiomatic way to consume from a channel until sender is done.
---
# 🔹 6. Closing in Synchronization
Closing channels is often used as a **signal**:
```go
done := make(chan struct{})
go func() {
// do some work
close(done) // signal completion
}()
<-done // wait until goroutine signals done
fmt.Println("Worker finished")
```
👉 Here, the **empty struct channel** is just a signal — no values, just closure.
---
# 🔹 7. Closing Multiple Producers Case
⚠️ **Important rule**:
If multiple goroutines send to a channel, none of them should close it, unless you carefully coordinate. Otherwise → race conditions.
Instead, use a **separate signal** to stop them, or let the main goroutine close after all producers finish.
Example with `sync.WaitGroup`:
```go
ch := make(chan int)
var wg sync.WaitGroup
for i := 0; i < 3; i++ {
wg.Add(1)
go func(id int) {
defer wg.Done()
ch <- id
}(i)
}
go func() {
wg.Wait()
close(ch) // only close once all senders are done
}()
for v := range ch {
fmt.Println("Received:", v)
}
```
---
# 🔹 8. Closing an Unbuffered Channel
* Closing an **unbuffered channel** wakes up **all receivers** waiting on it.
* Each receiver gets the zero value.
* This is often used in **broadcast signals** (e.g., cancel all workers).
Example: cancellation
```go
stop := make(chan struct{})
go func() {
<-stop // wait for signal
fmt.Println("Worker stopped")
}()
close(stop) // broadcast stop
```
---
# 🔹 9. Internals (CS-Level)
When `close(ch)` is called:
1. Runtime sets the `closed` flag in the channel’s internal `hchan` struct.
2. All goroutines waiting in the **recvq** (blocked receivers) are awakened:
* They return immediately with **zero value** and `ok=false`.
3. Any goroutine waiting in the **sendq** panics → "send on closed channel".
4. Future receives still succeed (zero + `ok=false`).
👉 Closing is therefore a **one-way synchronization primitive**:
* Wake up all receivers.
* Forbid new sends.
* Allow safe draining of buffered values.
---
# 🔹 10. Common Mistakes
❌ Sending to a closed channel → **panic**.
❌ Closing a nil channel → **panic**.
❌ Closing the same channel twice → **panic**.
❌ Receivers closing a channel → race conditions.
---
# 🔹 11. Real-World Use Cases
1. **Signaling completion** (`done` channel pattern).
2. **Fan-out workers** stop when channel is closed.
3. **Pipelines**: closing signals no more input → downstream stages terminate.
4. **Graceful shutdowns**: broadcaster closes a `quit` channel to stop all goroutines.
---
# 🔑 Key Takeaways
1. `close(ch)` signals **no more values** will be sent.
2. Only **senders** should close channels.
3. Receiving from closed channels:
* Drain buffered values first.
* Then return zero + `ok=false`.
4. `for range ch` stops when channel is closed + empty.
5. Closing is a **synchronization signal**, not just an end-of-life marker.
6. Internally → wakes receivers, panics senders.
---
Let’s go very deep into **closing channels in Go**, with both **practical examples** and **under-the-hood (CS-level) details**.
---
# 🔹 Why Do We Need to Close Channels?
A **channel** in Go is like a **concurrent queue** shared between goroutines. Closing a channel signals that:
* **No more values will be sent** into this channel.
* Receivers can safely finish reading remaining buffered values and stop waiting.
Think of it like an **EOF (End Of File)** signal for communication between goroutines.
---
# 🔹 How to Close a Channel
We use the built-in function:
```go
close(ch)
```
* Only the **sender** (the goroutine writing into the channel) should close it.
* Closing a channel multiple times → **panic**.
* Reading from a closed channel:
* If there are buffered values → still gives values until buffer is empty.
* Once empty → always returns **zero-value** of the type immediately.
---
# 🔹 Behavior of a Closed Channel
1. **Sending to a closed channel → panic**
```go
ch := make(chan int)
close(ch)
ch <- 1 // ❌ panic: send on closed channel
```
2. **Receiving from a closed channel**
```go
ch := make(chan int, 2)
ch <- 10
ch <- 20
close(ch)
fmt.Println(<-ch) // 10
fmt.Println(<-ch) // 20
fmt.Println(<-ch) // 0 (int zero-value, since closed and empty)
```
After it’s drained, receives are **non-blocking** and return **zero value**.
3. **Checking if channel is closed**
Go provides a **comma-ok** idiom:
```go
v, ok := <-ch
if !ok {
fmt.Println("Channel closed")
}
```
* `ok == true` → received valid value.
* `ok == false` → channel is closed **and empty**.
---
# 🔹 Real-World Use Case: Fan-in Pattern
```go
package main
import (
"fmt"
"sync"
)
func main() {
ch := make(chan int)
var wg sync.WaitGroup
// Multiple senders
for i := 1; i <= 3; i++ {
wg.Add(1)
go func(id int) {
defer wg.Done()
for j := 1; j <= 2; j++ {
ch <- id*10 + j
}
}(i)
}
// Closer goroutine
go func() {
wg.Wait()
close(ch) // Sender closes the channel
}()
// Receiver
for v := range ch {
fmt.Println("Received:", v)
}
}
```
### 🔍 What’s happening?
* `for v := range ch` **automatically stops** when the channel is closed and drained.
* Only the **sending side closes** (`wg.Wait()` ensures no sender is active).
---
# 🔹 Under the Hood (CS Level)
Inside Go’s **runtime** (`src/runtime/chan.go`), a channel is represented by `hchan`:
```go
type hchan struct {
qcount uint // number of data in the queue
dataqsiz uint // size of circular buffer
buf unsafe.Pointer // circular buffer
sendx uint // send index
recvx uint // receive index
recvq waitq // list of recv waiters
sendq waitq // list of send waiters
closed uint32 // is channel closed?
lock mutex
}
```
When we `close(ch)`:
1. The **closed flag** is set (`closed = 1`).
2. All **waiting receivers** in `recvq` are woken up → they receive zero-values.
3. All **waiting senders** in `sendq` → panic if they try to send.
4. Future sends → panic.
5. Future receives:
* If buffer still has values → values are dequeued normally.
* If buffer is empty → returns zero-value immediately.
This mechanism is **lock-protected** to ensure no race condition when closing while goroutines are waiting.
---
# 🔹 Rules of Thumb
✅ Close channels **only from sender side**.
✅ Use `for range ch` to receive until closed.
✅ Use `v, ok := <-ch` when you need to explicitly detect closure.
❌ Never close a channel from the **receiver side**.
❌ Don’t close the same channel multiple times.
---
# 🔹 Mental Model
Think of a **channel** as a **pipeline**:
* `close(ch)` = cutting off the source.
* Water (values) still inside the pipe will flow out.
* Once drained → only “empty flow” (zero value).
* Trying to pour (send) more into a cut pipe → explosion (panic).
---