Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/xiaolin/winterfresh

Home AI assistant
https://github.com/xiaolin/winterfresh

Last synced: 6 months ago
JSON representation

Home AI assistant

Awesome Lists containing this project

README 

          
# Winterfresh 🌨️



A fast, concise voice assistant for Raspberry Pi with wake word detection, conversation memory, and barge-in support.



## Features



- 🎤 Local wake word detection ("winterfresh") - no API cost for listening

- 💬 Conversational memory (5-minute timeout)

- 🔇 Barge-in support (interrupt while speaking)

- 🎵 Audio feedback with elegant chimes

- 🔄 Auto-restart on inactivity

- 🎙️ Hardware echo cancellation (with USB speakerphone)



## Hardware Requirements



- Raspberry Pi 5 (8GB recommended) or Raspberry Pi 4 (4GB+)

- Raspberry Pi 5 27W USB-C Power Supply (recommended for Pi 5 with USB peripherals)

- SanDisk 64GB Extreme microSDXC UHS-I Memory Card (fast boot and read/write)

- USB Speakerphone with echo cancellation (recommended):

  - EMEET Conference Speakerphone M0 Plus



## Prerequisites



- Node.js v20+

- Python 3.11+

- OpenAI API key

- Sox audio tools



## Installation



### 1. System Setup (Raspberry Pi)



```bash

# Update system

sudo apt-get update

sudo apt-get upgrade -y



# Install Node.js v20

curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -

sudo apt-get install -y nodejs



# Install Python 3.11+ and pip

sudo apt-get install -y python3 python3-pip python3-venv



# Install sox for audio recording/playback

sudo apt-get install -y sox libsox-fmt-all



# Install portaudio (required for sounddevice)

sudo apt-get install -y portaudio19-dev



# Install git

sudo apt-get install -y git



# Verify versions

node --version    # Should be v20+

python3 --version # Should be 3.11+

```



### 2. Clone & Setup Project



```bash

# Clone repository

git clone https://github.com/xiaolin/winterfresh.git ~/winterfresh

cd ~/winterfresh



# Install Node.js dependencies

npm install



# Build TypeScript

npm run build

```



### 3. Python Virtual Environment Setup



```bash

# Create virtual environment

python -m venv .venv



# Activate it

source .venv/bin/activate



# Install Python dependencies

pip install -r requirements.txt



# Verify installation

python -c "import numpy, sounddevice, vosk; print('✅ All Python packages installed')"



# Deactivate (optional, the app will use .venv/bin/python directly)

deactivate

```



### 4. Download Vosk Model (for local wake word detection)



```bash

# Create models directory

mkdir -p models

cd models



# Download small English model (~50MB)

wget https://alphacephei.com/vosk/models/vosk-model-small-en-us-0.15.zip

unzip vosk-model-small-en-us-0.15.zip

rm vosk-model-small-en-us-0.15.zip



cd ..

```



### 5. Environment Configuration



```bash

# Create environment file

vi .env

```



Environment variables in `.env`:



| Variable                | Default                  | Description                            |

| ----------------------- | ------------------------ | -------------------------------------- |

| `OPENAI_API_KEY`        | (required)               | Your OpenAI API key                    |

| `CHAT_MODEL`            | `gpt-4o-mini`            | OpenAI chat model                      |

| `TRANSCRIBE_MODEL`      | `gpt-4o-mini-transcribe` | OpenAI transcription model             |

| `TTS_MODEL`             | `gpt-4o-mini-tts`        | OpenAI text-to-speech model            |

| `WINTERFRESH_MAX_TURNS` | `20`                     | Max conversation turns before trimming |

| `SAMPLE_RATE`           | `24000`                  | Audio sample rate for recording        |

| `ARECORD_DEVICE`        | `mic_share`              | ALSA device for recording (Linux)      |

| `ARECORD_CHANNELS`      | `2`                      | Audio channels for recording (Linux)   |



Example `.env` file:



```bash

OPENAI_API_KEY=your_openai_api_key_here

CHAT_MODEL=gpt-4o-mini

TRANSCRIBE_MODEL=gpt-4o-mini-transcribe

TTS_MODEL=gpt-4o-mini-tts

WINTERFRESH_MAX_TURNS=20

ARECORD_DEVICE=mic_share

ARECORD_CHANNELS=2

```



### 6. Audio Device Setup (Raspberry Pi)



#### 6.1 Check USB Audio Device



```bash

# Plug in your USB speakerphone



# Check if detected

cat /proc/asound/cards

# Should show your USB device (e.g., card 2: EMEET OfficeCore M0 Plus)



arecord -l  # List capture devices

aplay -l    # List playback devices

```



#### 6.2 Configure ALSA for Shared Microphone Access



Create `~/.asoundrc` to allow multiple processes to share the microphone:



```bash

cat > ~/.asoundrc << 'EOF'

pcm.mic_share {

    type dsnoop

    ipc_key 12345

    slave {

        pcm "hw:2,0"

        rate 16000

        channels 2

    }

}

EOF

```



**Note:** Replace `hw:2,0` with your actual card number from `arecord -l` if different.



#### 6.3 Configure PulseAudio



```bash

# Check PulseAudio is running

systemctl --user status pulseaudio



# If not running, start it

systemctl --user start pulseaudio



# List available sinks (output) and sources (input)

pactl list sinks short

pactl list sources short



# Set your USB device as default (use actual device names from above)

# Example for EMEET speakerphone:

pactl set-default-sink alsa_output.usb-EMEET_EMEET_OfficeCore_M0_Plus_.analog-stereo

pactl set-default-source alsa_input.usb-EMEET_EMEET_OfficeCore_M0_Plus_.analog-stereo

```



#### 6.4 Make PulseAudio Settings Persistent



```bash

mkdir -p ~/.config/pulse

cat > ~/.config/pulse/default.pa << 'EOF'

.include /etc/pulse/default.pa

set-default-sink alsa_output.usb-EMEET_EMEET_OfficeCore_M0_Plus_.analog-stereo

set-default-source alsa_input.usb-EMEET_EMEET_OfficeCore_M0_Plus_.analog-stereo

EOF

```



**Note:** Replace the device names with your actual device names from `pactl list sinks short` and `pactl list sources short`.



#### 6.5 Test Audio



```bash

# Test recording with shared device

arecord -D mic_share -f S16_LE -c 2 -r 16000 -d 3 /tmp/test.wav



# Test playback

aplay -D plughw:2,0 /tmp/test.wav



# Test simultaneous recording (both should work without "Device busy" error)

arecord -D mic_share -f S16_LE -c 2 -r 16000 -d 10 /tmp/test1.wav &

arecord -D mic_share -f S16_LE -c 2 -r 16000 -d 5 /tmp/test2.wav



# Test Python audio

source .venv/bin/activate

python -c "import sounddevice as sd; print(sd.query_devices())"

deactivate

```



### 7. Test the Setup



```bash

# Test wake word detection first

source .venv/bin/activate

python wake.py

# Say "winterfresh" - should print "WAKE" and exit

deactivate



# Test full app

npm run dev

# Or: npx tsx src/app.ts

```



### 8. Install PM2 for Production



```bash

# Install PM2 globally

sudo npm install -g pm2



# Build TypeScript

npm run build



# Start winterfresh using ecosystem config

pm2 start pm2.config.cjs



# Or start directly

pm2 start dist/app.js --name winterfresh



# Save PM2 configuration

pm2 save



# Setup auto-start on boot

pm2 startup

# Follow the command it outputs (copy/paste and run it)



# Verify it's running

pm2 status

pm2 logs winterfresh --lines 50

```



## Usage



### Voice Commands



1. **Wake the assistant**: Say "winterfresh", "winter fresh", or "hey winterfresh"

2. **Speak your request**: After hearing the wake chime

3. **Interrupt anytime**: Start speaking to interrupt long responses

4. **Wait for timeout**: 7 seconds of inactivity returns to wake word mode



### Audio Feedback



- 🎵 **Wake chime** - Wake word detected, ready to listen

- 🎵 **Processing chime** - Processing your request, stops when assistant speaks



### PM2 Commands



```bash

# View live logs

pm2 logs winterfresh



# Monitor resource usage

pm2 monit



# Restart after code changes

pm2 restart winterfresh



# Stop the assistant

pm2 stop winterfresh



# Start the assistant

pm2 start winterfresh

```



## Development



### Local Testing (Mac)



**Note:** For Mac testing, use headphones to avoid echo (built-in speakers will be heard by built-in mic).



```bash

# Install system dependencies (Mac)

brew install sox portaudio



# Setup Python venv

python3 -m venv .venv

source .venv/bin/activate

pip install -r requirements.txt

deactivate



# Download Vosk model

mkdir -p models && cd models

curl -LO https://alphacephei.com/vosk/models/vosk-model-small-en-us-0.15.zip

unzip vosk-model-small-en-us-0.15.zip

cd ..



# Install Node dependencies

npm install



# Run

npm run dev

```



### Update Script



Create `update.sh` for easy updates:



```bash

#!/bin/bash

cd ~/winterfresh

git pull

npm install

npm run build

source .venv/bin/activate

pip install -r requirements.txt 2>/dev/null || pip install numpy sounddevice vosk

deactivate

pm2 restart winterfresh

echo "✅ Winterfresh updated!"

```



Make it executable:



```bash

chmod +x update.sh

```



## Troubleshooting



### Wake word not detecting



```bash

# Test Python script directly

source .venv/bin/activate

python wake.py

# Speak and watch the audio level bar - it should move

# Say "winterfresh" clearly

deactivate



# If no audio level, check your mic:

python -c "import sounddevice as sd; print(sd.query_devices())"

```



### "No module named numpy" error



```bash

# Make sure packages are in the venv, not global Python

.venv/bin/pip install numpy sounddevice vosk



# Verify

.venv/bin/python -c "import numpy, sounddevice, vosk; print('OK')"

```



### Audio device not found



```bash

# List available devices

arecord -l

aplay -l



# Check Python sees the device

source .venv/bin/activate

python -c "import sounddevice as sd; print(sd.query_devices())"

```



### PulseAudio connection refused



```bash

# Check if PulseAudio is running

systemctl --user status pulseaudio



# Restart PulseAudio

systemctl --user restart pulseaudio



# Wait a moment then test

sleep 2

pactl info

```



### PulseAudio sink missing after reboot



```bash

# Check current sinks

pactl list sinks short



# If your USB device isn't the default, set it again

pactl set-default-sink alsa_output.usb-EMEET_EMEET_OfficeCore_M0_Plus_.analog-stereo



# Make sure ~/.config/pulse/default.pa exists with correct settings

cat ~/.config/pulse/default.pa

```



### "Device or resource busy" error



This happens when multiple processes try to access the microphone without using the shared `dsnoop` device.



```bash

# Verify ~/.asoundrc exists

cat ~/.asoundrc



# If missing, recreate it

cat > ~/.asoundrc << 'EOF'

pcm.mic_share {

    type dsnoop

    ipc_key 12345

    slave {

        pcm "hw:2,0"

        rate 16000

        channels 2

    }

}

EOF



# Make sure .env uses mic_share

grep ARECORD .env

# Should show: ARECORD_DEVICE=mic_share

```



### No audio output / "no default audio device"



```bash

# Check PulseAudio sink

pactl list sinks short

pactl get-default-sink



# If shows auto_null or missing, your USB device isn't detected

# Replug the USB device and restart PulseAudio

systemctl --user restart pulseaudio



# Verify your USB device is now the default

pactl list sinks short

pactl set-default-sink alsa_output.usb-EMEET_EMEET_OfficeCore_M0_Plus_

```



### Permission issues (Raspberry Pi)



```bash

# Add user to audio group

sudo usermod -a -G audio $USER



# Logout and login again

```



### Mac microphone permission



Go to **System Settings → Privacy & Security → Microphone** and enable for:



- Terminal (or iTerm)

- Visual Studio Code (if running from VS Code)



Quit and reopen the app after enabling.



## Performance



- Wake word detection: Local (Vosk) - no API cost

- Speech-to-text: OpenAI Whisper API

- Chat: OpenAI GPT-4o-mini

- Text-to-speech: OpenAI TTS



**Raspberry Pi 5 (8GB):**



- Memory usage: ~150-200MB

- CPU usage: ~5-10% during wake word listening

- Wake word latency: ~0.3-0.5 second

- Response latency: ~1-2 seconds (API dependent)



**Raspberry Pi 4 (4GB):**



- Memory usage: ~150-200MB

- CPU usage: ~10-20% during wake word listening

- Wake word latency: ~0.5-1 second

- Response latency: ~1-3 seconds (API dependent)



## License



MIT