7 min read Advanced 25.0k

Prometheus + Grafana monitoring stack

Published on May 17, 2026 at 04:02 PM

Set up the reference open-source monitoring: Prometheus to collect metrics, Node Exporter on each VPS, Grafana to visualize. The most widely used stack in the world for sysadmin monitoring.

Introduction

Prometheus has become the de facto standard for monitoring: used by SoundCloud (its creators), CNCF, Kubernetes-native, and 90% of tech startups. The winning combo:

Prometheus: scrapes metrics every 15s, efficient TSDB storage
Node Exporter: system agent (CPU, RAM, disk, network) on each VPS
Grafana: visualization and alerting
Alertmanager: alert routing (email, Discord, Slack)

Benefits vs paid SaaS tools (Datadog, New Relic):

Free, self-hosted, at your place
No metric limits
Sovereign data

Prerequisites

1 "central" VPS for Prometheus + Grafana (2 GB RAM)
N VPS to monitor (Node Exporter)
Docker on the central VPS
Network between VPS (WireGuard recommended)

Step 1: Architecture

   ┌──────────────────┐  ┌──────────────────┐
   │ VPS Web 1        │  │ VPS DB           │
   │  + Node Exporter │  │  + Node Exporter │
   │  + Nginx Exporter│  │  + MySQL Exporter│
   └────────┬─────────┘  └────────┬─────────┘
            │                     │
            └──────────┬──────────┘
                       │
              ┌────────▼────────┐
              │  Central VPS    │
              │  ┌───────────┐  │
              │  │ Prometheus│  │
              │  └───────────┘  │
              │  ┌───────────┐  │
              │  │  Grafana  │  │
              │  └───────────┘  │
              │  ┌───────────┐  │
              │  │Alertmanager│ │
              │  └───────────┘  │
              └─────────────────┘

Step 2: Deploy the central stack

sudo mkdir -p /opt/monitoring
cd /opt/monitoring
sudo nano docker-compose.yml

services:
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    restart: unless-stopped
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - ./alerts.yml:/etc/prometheus/alerts.yml:ro
      - prometheus_data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus'
      - '--storage.tsdb.retention.time=90d'
      - '--web.console.libraries=/usr/share/prometheus/console_libraries'
      - '--web.console.templates=/usr/share/prometheus/consoles'
    networks:
      - monitoring

  grafana:
    image: grafana/grafana:latest
    container_name: grafana
    restart: unless-stopped
    ports:
      - "127.0.0.1:3000:3000"
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=AStrongPassword
      - GF_USERS_ALLOW_SIGN_UP=false
    volumes:
      - grafana_data:/var/lib/grafana
    networks:
      - monitoring
    depends_on:
      - prometheus

  alertmanager:
    image: prom/alertmanager:latest
    container_name: alertmanager
    restart: unless-stopped
    ports:
      - "9093:9093"
    volumes:
      - ./alertmanager.yml:/etc/alertmanager/alertmanager.yml:ro
    networks:
      - monitoring

  node_exporter:
    image: prom/node-exporter:latest
    container_name: node_exporter
    restart: unless-stopped
    pid: host
    network_mode: host
    volumes:
      - /:/host:ro,rslave
    command:
      - '--path.rootfs=/host'

volumes:
  prometheus_data:
  grafana_data:

networks:
  monitoring:

Step 3: Prometheus config

sudo nano /opt/monitoring/prometheus.yml

global:
  scrape_interval: 15s
  evaluation_interval: 15s

alerting:
  alertmanagers:
    - static_configs:
        - targets:
            - alertmanager:9093

rule_files:
  - "alerts.yml"

scrape_configs:
  # Prometheus itself
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']

  # Central VPS (Node Exporter)
  - job_name: 'node_central'
    static_configs:
      - targets: ['host.docker.internal:9100']
        labels:
          host: 'srv-central'

  # VPS Web 1
  - job_name: 'node_web1'
    static_configs:
      - targets: ['10.66.66.2:9100']
        labels:
          host: 'srv-web-01'

  # VPS Web 2
  - job_name: 'node_web2'
    static_configs:
      - targets: ['10.66.66.3:9100']
        labels:
          host: 'srv-web-02'

  # VPS DB
  - job_name: 'node_db'
    static_configs:
      - targets: ['10.66.66.4:9100']
        labels:
          host: 'srv-db-01'
  
  # Nginx Exporter
  - job_name: 'nginx'
    static_configs:
      - targets: ['10.66.66.2:9113']
        labels:
          host: 'srv-web-01'

  # MySQL Exporter
  - job_name: 'mysql'
    static_configs:
      - targets: ['10.66.66.4:9104']
        labels:
          host: 'srv-db-01'

Step 4: Basic alert rules

sudo nano /opt/monitoring/alerts.yml

groups:
  - name: system_alerts
    interval: 30s
    rules:
      - alert: HighCPU
        expr: 100 - (avg by(host) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 90
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "CPU > 90% on {{ $labels.host }}"
          description: "CPU at {{ $value }}% for 5 minutes."

      - alert: HighMemory
        expr: (node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes) / node_memory_MemTotal_bytes * 100 > 90
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "RAM > 90% on {{ $labels.host }}"

      - alert: DiskFull
        expr: (node_filesystem_size_bytes{mountpoint="/"} - node_filesystem_avail_bytes{mountpoint="/"}) / node_filesystem_size_bytes{mountpoint="/"} * 100 > 90
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "Disk > 90% on {{ $labels.host }}"

      - alert: NodeDown
        expr: up{job=~"node_.*"} == 0
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: "Host {{ $labels.host }} is DOWN"

      - alert: HighLoad
        expr: node_load5 > 4
        for: 10m
        labels:
          severity: warning
        annotations:
          summary: "Load > 4 on {{ $labels.host }}"

Step 5: Alertmanager config (Discord/email)

sudo nano /opt/monitoring/alertmanager.yml

global:
  resolve_timeout: 5m

route:
  group_by: ['alertname', 'host']
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 12h
  receiver: 'discord'

receivers:
  - name: 'discord'
    webhook_configs:
      - url: 'https://discord.com/api/webhooks/YOUR_WEBHOOK'
        send_resolved: true

  - name: 'email'
    email_configs:
      - to: '[email protected]'
        from: '[email protected]'
        smarthost: 'smtp.example.com:587'
        auth_username: '[email protected]'
        auth_password: 'password'
        require_tls: true

Step 6: Start the stack

sudo docker compose up -d
sudo docker compose ps

Check:

Prometheus: http://localhost:9090
Grafana: http://localhost:3000 (via SSH tunnel)
Alertmanager: http://localhost:9093

Step 7: Install Node Exporter on each source VPS

On every VPS to monitor:

# Download
wget https://github.com/prometheus/node_exporter/releases/download/v1.8.1/node_exporter-1.8.1.linux-amd64.tar.gz
tar xvf node_exporter-1.8.1.linux-amd64.tar.gz
sudo mv node_exporter-1.8.1.linux-amd64/node_exporter /usr/local/bin/
sudo chown root:root /usr/local/bin/node_exporter

# Dedicated user
sudo useradd --no-create-home --shell /bin/false node_exporter
sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter

Systemd service:

sudo nano /etc/systemd/system/node_exporter.service

[Unit]
Description=Node Exporter
After=network.target

[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter \
    --collector.systemd \
    --collector.processes \
    --web.listen-address=0.0.0.0:9100

Restart=always

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now node_exporter

Test:

curl http://localhost:9100/metrics | head -30

Must show metrics node_cpu_seconds_total, node_memory_MemTotal_bytes, etc.

Step 8: Secure Node Exporter

⚠️ Don't expose port 9100 publicly (reveals sensitive data). Solutions:

WireGuard between VPS (recommended)
UFW: sudo ufw allow from CENTRAL_VPS_IP to any port 9100
Reverse proxy with basic auth on the central

Step 9: Add Grafana

SSH tunnel:

ssh -L 3000:127.0.0.1:3000 user@CENTRAL_VPS_IP

http://localhost:3000 → admin / AStrongPassword

Prometheus datasource

Configuration → Data sources → Add data source
Prometheus
URL: http://prometheus:9090
Save & test

Node Exporter dashboard

Dashboards → Import
ID: 1860 (Node Exporter Full, most popular)
Datasource: Prometheus
Import

You instantly get CPU, RAM, disk, network, load, processes for all your VPS.

Step 10: Additional exporters

Based on your services:

Nginx Exporter

docker run -d \
  --restart unless-stopped \
  --name nginx-exporter \
  -p 9113:9113 \
  nginx/nginx-prometheus-exporter \
  -nginx.scrape-uri=http://localhost/nginx_status

Enable stub_status in Nginx:

server {
    listen 127.0.0.1:80;
    location /nginx_status {
        stub_status;
        allow 127.0.0.1;
        deny all;
    }
}

Grafana dashboard: ID 12708.

MySQL Exporter

docker run -d \
  --restart unless-stopped \
  --name mysql-exporter \
  -p 9104:9104 \
  -e DATA_SOURCE_NAME='exporter:password@(localhost:3306)/' \
  prom/mysqld-exporter

Create a dedicated MySQL user:

CREATE USER 'exporter'@'localhost' IDENTIFIED BY 'password' WITH MAX_USER_CONNECTIONS 3;
GRANT PROCESS, REPLICATION CLIENT, SELECT ON *.* TO 'exporter'@'localhost';
FLUSH PRIVILEGES;

Grafana dashboard: ID 7362.

Redis Exporter

docker run -d \
  --restart unless-stopped \
  --name redis-exporter \
  -p 9121:9121 \
  oliver006/redis_exporter \
  --redis.addr=redis://localhost:6379 \
  --redis.password=YOUR_REDIS_PASSWORD

Dashboard: ID 763.

Blackbox Exporter (HTTP/TCP probes)

To monitor external URLs (your site's uptime):

docker run -d \
  --name blackbox-exporter \
  -p 9115:9115 \
  prom/blackbox-exporter

Step 11: Recommended dashboards

Import in Grafana via Dashboards → Import:

1860: Node Exporter Full
12708: Nginx Exporter
7362: MySQL Overview
763: Redis Dashboard
13639: Loki + Promtail (logs)
893: Docker and Host
9628: PostgreSQL

Step 12: Alerting via Grafana (Alertmanager alternative)

Grafana 10+ has its own native alerting:

Alerting → Alert rules → New alert rule
Query: 100 - (avg by(host) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 90
Add contact point: Discord, Slack, Telegram, etc.
Notification policy: grouping, throttling

More practical than Alertmanager for simple setups.

Troubleshooting

Targets DOWN in Prometheus

http://localhost:9090/targets → check connectivity:

curl http://VPS_IP:9100/metrics

Causes: firewall, wrong IP in prometheus.yml, node_exporter not started.

No metrics in Grafana

Check datasource (Configuration → Data sources → test). Then in Explore, test up{job="node_web1"}.

Prometheus eats too much space

Limit retention:

command:
  - '--storage.tsdb.retention.time=30d'  # instead of 90d

Alerts not sent

sudo docker compose logs alertmanager

Test Discord webhook manually:

curl -X POST -H "Content-Type: application/json" \
  -d '{"content": "test"}' \
  https://discord.com/api/webhooks/YOUR_WEBHOOK

Useful commands

# Targets state
curl http://localhost:9090/api/v1/targets

# CLI query Prometheus
curl "http://localhost:9090/api/v1/query?query=up"

# Reload config without restart
curl -X POST http://localhost:9090/-/reload

# Test alerts
curl http://localhost:9090/api/v1/alerts

# Check Prometheus config
sudo docker run --rm -v $(pwd)/prometheus.yml:/etc/prometheus/prometheus.yml \
    prom/prometheus promtool check config /etc/prometheus/prometheus.yml

# Check alert rules
sudo docker run --rm -v $(pwd)/alerts.yml:/etc/prometheus/alerts.yml \
    prom/prometheus promtool check rules /etc/prometheus/alerts.yml

Conclusion

With this stack you get:

CPU/RAM/disk/network/load monitoring on all your VPS
Automatic alerting on problems
Visual dashboards for fast debugging
Long-term metric storage (90 days default)

Going further:

Add Loki for logs (already covered in another guide)
Add Tempo for distributed tracing
Migrate to Thanos or Cortex for HA Prometheus
Use Prometheus Operator if you move to Kubernetes

Resources

Official site: https://prometheus.io
Prometheus documentation: https://prometheus.io/docs/
Grafana dashboards: https://grafana.com/grafana/dashboards/
VeryCloud guide — Loki + Promtail logs: /docs/article/loki-promtail
VeryCloud guide — Docker Compose prod: /docs/article/docker-compose-prod