📘 PCA EXAM — DOMAIN 5: DASHBOARDING & VISUALIZATION (8%)

Complete One-Stop Guide: Theory → Examples → Exam Questions

TABLE OF CONTENTS

5.1   Visualization in the Prometheus Ecosystem
5.2   Prometheus Native UI (Expression Browser)
5.3   Grafana — Overview & Architecture
5.4   Adding Prometheus as a Data Source in Grafana
5.5   Grafana Panel Types
5.6   Building Dashboards with PromQL
5.7   Variables & Templating
5.8   Dashboard Best Practices
5.9   Grafana vs Prometheus UI — When to Use Which
5.10  Real-World Dashboard Examples
5.11  EXAM-STYLE QUESTIONS (20 Questions with Answers)

5.1 📖 VISUALIZATION IN THE PROMETHEUS ECOSYSTEM

The Visualization Landscape

Prometheus itself is primarily a metrics collection and storage system.
For visualization, you have two main options:

┌─────────────────────────────────────────────────────────┐
│                                                         │
│  Option 1: Prometheus Native UI (Built-in)              │
│    → Simple expression browser at http://localhost:9090 │
│    → Good for quick queries and debugging               │
│    → Limited visualization capabilities                 │
│    → No persistent dashboards                           │
│                                                         │
│  Option 2: Grafana (External, Most Popular)             │
│    → Rich, interactive dashboards                       │
│    → Multiple data sources (not just Prometheus)        │
│    → Alerting, annotations, variables                   │
│    → Industry standard for Prometheus visualization     │
│                                                         │
│  Other Options (less common):                           │
│    → Console Templates (Prometheus built-in, advanced)  │
│    → Perses (CNCF sandbox project, Prometheus-native)   │
│    → Custom dashboards using Prometheus HTTP API        │
│                                                         │
└─────────────────────────────────────────────────────────┘

💡 Key Takeaway for Exam

Prometheus is NOT a visualization tool — it’s a metrics collection/storage system Grafana is the de facto standard for Prometheus visualization Prometheus has a basic built-in UI for ad-hoc queries

5.2 📖 PROMETHEUS NATIVE UI (Expression Browser)

Accessing the UI

URL: http://<prometheus-server>:9090

Default port: 9090
No authentication by default (add reverse proxy for production!)

Key Pages in the Prometheus UI

┌─────────────────────────────────────────────────────────┐
│  Prometheus UI Navigation                               │
├─────────────────────────────────────────────────────────┤
│                                                         │
│  /graph     → Expression Browser (PromQL queries)       │
│  /alerts    → Active alerting rules and their state     │
│  /status    → Runtime & Build Information               │
│  /targets   → Scrape targets and their health           │
│  /rules     → Loaded recording and alerting rules       │
│  /service-discovery → Active SD configurations          │
│  /flags     → Command-line flags                        │
│  /config    → Current prometheus.yml configuration      │
│  /tsdb-status → TSDB storage statistics                 │
│  /metrics   → Prometheus's OWN metrics (self-monitoring)│
│                                                         │
└─────────────────────────────────────────────────────────┘

The Expression Browser (/graph)

Two modes:

1. Table View (Instant Query):
   → Enter a PromQL expression
   → Click "Execute"
   → Shows the latest value for each matching time series
   → Returns an instant vector
   → Example: up{job="web-app"}
   
   Result:
   Element                                    Value
   up{instance="app1:8080", job="web-app"}    1
   up{instance="app2:8080", job="web-app"}    1
   up{instance="app3:8080", job="web-app"}    0

2. Graph View (Range Query):
   → Enter a PromQL expression
   → Set time range (e.g., last 1 hour)
   → Click "Execute"
   → Shows a time-series graph
   → Returns a range of values over time
   → Example: rate(http_requests_total[5m])
   
   Features:
   → Adjustable time range
   → Resolution control (step size)
   → Stacked/Unstacked toggle
   → Can embed graphs via URL

The Targets Page (/targets)

Shows all configured scrape targets and their current status:

Endpoint                          State   Labels                         Last Scrape
http://app1:8080/metrics          UP      job="web", instance="app1:8080"  2.345s ago
http://app2:8080/metrics          UP      job="web", instance="app2:8080"  1.234s ago
http://app3:8080/metrics          DOWN    job="web", instance="app3:8080"  15.678s ago
http://db:9104/metrics            UP      job="mysql", instance="db:9104"  0.567s ago

Useful for:
  → Checking which targets are healthy (UP) vs down (DOWN)
  → Seeing the last scrape time and duration
  → Verifying labels applied to each target
  → Debugging scrape failures (error messages shown)

Limitations of the Prometheus UI

❌ No persistent dashboards (queries are lost on page refresh)
❌ No multi-panel layouts
❌ No variables or templating
❌ No alerting visualization (beyond /alerts page)
❌ No user management or access control
❌ Limited graph customization
❌ No annotation support
❌ Cannot combine multiple data sources

✅ Good for: Quick ad-hoc queries, debugging, checking target health

5.3 📖 GRAFANA — OVERVIEW & ARCHITECTURE

What is Grafana?

Grafana is an open-source visualization and analytics platform that allows you to query, visualize, alert on, and understand your metrics from multiple data sources.

Key Facts

Website:    https://grafana.com
Default Port: 3000
Default Login: admin / admin (change immediately!)
License:    AGPLv3 (open source)
Maintainer: Grafana Labs
Data Sources: Prometheus, InfluxDB, Elasticsearch, MySQL, 
              PostgreSQL, CloudWatch, Loki, Tempo, Jaeger, etc.

Architecture

┌─────────────────────────────────────────────────────────┐
│                    GRAFANA ARCHITECTURE                  │
│                                                         │
│  ┌─────────────────────────────────────────────────┐    │
│  │              Grafana Server                      │    │
│  │                                                  │    │
│  │  ┌──────────┐  ┌──────────┐  ┌──────────┐      │    │
│  │  │ Dashboard│  │ Alerting │  │ Plugin   │      │    │
│  │  │ Engine   │  │ Engine   │  │ System   │      │    │
│  │  └────┬─────┘  └────┬─────┘  └────┬─────┘      │    │
│  │       │              │              │            │    │
│  │  ┌────┴──────────────┴──────────────┴─────┐      │    │
│  │  │         Data Source Plugins             │      │    │
│  │  │  Prometheus │ InfluxDB │ Elasticsearch  │      │    │
│  │  └────┬──────────────┬──────────────┬─────┘      │    │
│  └───────┼──────────────┼──────────────┼────────────┘    │
│          │              │              │                 │
│          ▼              ▼              ▼                 │
│  ┌──────────────┐ ┌──────────┐ ┌──────────────┐         │
│  │  Prometheus  │ │ InfluxDB │ │ Elasticsearch│         │
│  │  Server      │ │          │ │              │         │
│  └──────────────┘ └──────────┘ └──────────────┘         │
│                                                         │
│  Users access Grafana via web browser (port 3000)       │
│  Grafana queries data sources and renders dashboards    │
└─────────────────────────────────────────────────────────┘

Key Grafana Concepts

Organization (Org):
  → Top-level grouping (e.g., "My Company")
  → Each org has its own dashboards, data sources, users

Data Source:
  → Connection to a backend system (Prometheus, InfluxDB, etc.)
  → Must be configured before creating dashboards
  → Can have multiple data sources of the same type

Dashboard:
  → A collection of panels organized in a grid layout
  → Can have variables, annotations, and time range controls
  → Stored as JSON (can be version-controlled!)

Panel:
  → A single visualization within a dashboard
  → Contains a query (PromQL), visualization type, and settings
  → Types: Graph, Stat, Gauge, Table, Heatmap, etc.

Folder:
  → Organizes dashboards into logical groups
  → Supports permissions

User/Team:
  → Users can be assigned roles (Admin, Editor, Viewer)
  → Teams allow group-based permissions

5.4 📖 ADDING PROMETHEUS AS A DATA SOURCE IN GRAFANA

Step-by-Step Configuration

1. Log into Grafana (http://grafana:3000)
2. Go to Configuration → Data Sources → Add data source
3. Select "Prometheus"
4. Configure:

   Name:     Prometheus-Production  (any descriptive name)
   URL:      http://prometheus:9090  (Prometheus server URL)
   Access:   Server (default) — Grafana server queries Prometheus
             Browser — User's browser queries Prometheus directly
   
   HTTP Settings:
     Timeout: 60s
     Max concurrent queries: 10
   
   Custom HTTP Headers: (if needed for auth)
     Authorization: Bearer <token>
   
   Query Settings:
     Min time interval: 15s  (should match scrape interval)
     Max data points: 11000
   
   HTTP Method: POST (recommended for large queries)
                GET  (default, simpler)

5. Click "Save & Test"
   → Should show "Data source is working" ✅

Important Settings

Min Time Interval:
  → Should match your Prometheus scrape_interval (e.g., 15s)
  → Prevents Grafana from querying at higher resolution than data exists
  → If set too low, queries will return empty gaps

Max Data Points:
  → Controls the maximum number of data points returned per query
  → Grafana automatically adjusts the step size based on this
  → Default: 11000 (matches typical screen pixel width)

Access Mode:
  → Server (recommended): Grafana backend queries Prometheus
    ✅ Works behind firewalls
    ✅ Credentials stay on server
  → Browser: User's browser queries Prometheus directly
    ❌ Requires Prometheus to be accessible from user's browser
    ❌ Exposes Prometheus URL to users

5.5 📖 GRAFANA PANEL TYPES

Complete Panel Type Reference

Detailed Panel Examples

1. Time Series Panel (Most Common)

Purpose: Show how metrics change over time

Configuration:
  Query: rate(http_requests_total[5m])
  Legend: {{method}} - {{status}}
  Unit: requests/sec (ops)
  
  Display options:
    → Lines, Bars, or Points
    → Stacked or Unstacked
    → Line width, fill opacity
    → Gradient mode
    → Thresholds (color changes at values)
  
  Axes:
    → Y-axis: auto-scale or fixed range
    → X-axis: time (auto)
  
  Standard options:
    → Min/Max
    → Decimals
    → Unit (seconds, bytes, percent, etc.)

Best for: Rate, latency trends, resource usage over time

2. Stat Panel

Purpose: Display a single big number (current value)

Configuration:
  Query: sum(rate(http_requests_total[5m]))
  Calculation: Last (not mean, not sum!)
  Unit: requests/sec
  
  Thresholds:
    → Green: < 1000
    → Yellow: 1000-5000
    → Red: > 5000
  
  Color mode: Background or Value
  Graph mode: None, Area, or Line (sparkline)

Best for: KPIs, current values, SLI indicators
Example: "Current Error Rate: 0.02%" (green)

3. Gauge Panel

Purpose: Show a value within a min-max range (like a speedometer)

Configuration:
  Query: (1 - node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes) * 100
  Min: 0
  Max: 100
  Unit: percent
  
  Thresholds:
    → Green: 0-60%
    → Yellow: 60-80%
    → Red: 80-100%

Best for: Utilization metrics (CPU, memory, disk)
Example: CPU Usage gauge showing 75% (yellow zone)

4. Table Panel

Purpose: Display data in rows and columns

Configuration:
  Query A: up
  Query B: time() - process_start_time_seconds
  
  Transform: Join by field (instance)
  
  Columns:
    → Instance
    → Status (up/down)
    → Uptime (formatted as duration)
  
  Overrides:
    → Color "Status" column: 1=green, 0=red
    → Format "Uptime" as "d hh:mm:ss"

Best for: Inventory lists, target status, SLA reports

5. Heatmap Panel

Purpose: Show distribution of values over time (2D histogram)

Configuration:
  Query: sum(increase(http_request_duration_seconds_bucket[5m])) by (le)
  Format: Heatmap
  Legend: {{le}}
  
  Display:
    → Color scheme: Green-Yellow-Red
    → Y-axis: latency buckets
    → X-axis: time

Best for: Latency distributions, spotting patterns
Example: "Most requests are < 100ms, but at 2 PM there's a spike to 2s"

5.6 📖 BUILDING DASHBOARDS WITH PromQL

Dashboard Structure

┌─────────────────────────────────────────────────────────────┐
│  📊 Production API Dashboard          [Last 6 hours ▼] [🔄]│
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐      │
│  │  Total   │ │  Error   │ │  p99     │ │  Active  │      │
│  │  Req/s   │ │  Rate %  │ │  Latency │ │  Users   │      │
│  │  12,450  │ │  0.02%   │ │  234ms   │ │  3,456   │      │
│  │  (Stat)  │ │  (Stat)  │ │  (Stat)  │ │  (Stat)  │      │
│  └──────────┘ └──────────┘ └──────────┘ └──────────┘      │
│                                                             │
│  ┌─────────────────────────┐ ┌─────────────────────────┐   │
│  │  Request Rate by Method │ │  Error Rate Over Time   │   │
│  │  (Time Series)          │ │  (Time Series)          │   │
│  │  📈 GET, POST, PUT      │ │  📈 5xx rate %         │   │
│  └─────────────────────────┘ └─────────────────────────┘   │
│                                                             │
│  ┌─────────────────────────┐ ┌─────────────────────────┐   │
│  │  Latency Distribution   │ │  CPU & Memory Usage     │   │
│  │  (Heatmap)              │ │  (Time Series)          │   │
│  │  🟩🟨🟥                 │ │  📈 per instance        │   │
│  └─────────────────────────┘ └─────────────────────────┘   │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Common Dashboard Queries

# Row 1: Stat Panels
# Total Request Rate
sum(rate(http_requests_total[5m]))

# Error Rate %
sum(rate(http_requests_total{status=~"5.."}[5m])) 
/ sum(rate(http_requests_total[5m])) * 100

# p99 Latency
histogram_quantile(0.99, 
  sum(rate(http_request_duration_seconds_bucket[5m])) by (le)
)

# Active Users (gauge)
app_active_users

# Row 2: Time Series
# Request Rate by Method
sum(rate(http_requests_total[5m])) by (method)

# Error Rate Over Time
sum(rate(http_requests_total{status=~"5.."}[5m])) 
/ sum(rate(http_requests_total[5m])) * 100

# Row 3: Heatmap + Resources
# Latency Heatmap
sum(increase(http_request_duration_seconds_bucket[5m])) by (le)

# CPU Usage per Instance
100 - (avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)

5.7 📖 VARIABLES & TEMPLATING

What are Variables?

Variables allow you to create dynamic, reusable dashboards where users can select different values from dropdown menus.

Types of Variables

1. Query Variable (Most Common)

Purpose: Populate dropdown from a PromQL query

Example: Create an "instance" dropdown
  Name: instance
  Type: Query
  Data source: Prometheus
  Query: label_values(up, instance)
  Result: Dropdown with [app1:8080, app2:8080, app3:8080]

Example: Create a "job" dropdown
  Name: job
  Type: Query
  Query: label_values(up, job)
  Result: Dropdown with [web-app, database, cache]

Example: Create an "endpoint" dropdown (filtered by job)
  Name: endpoint
  Type: Query
  Query: label_values(http_requests_total{job="$job"}, endpoint)
  Result: Dynamically updates when $job changes!

2. Custom Variable

Purpose: Manually define a list of values

Example:
  Name: environment
  Type: Custom
  Values: production, staging, development
  
  Usage in query:
  sum(rate(http_requests_total{env="$environment"}[5m]))

3. Interval Variable

Purpose: Let users select the time resolution

Example:
  Name: interval
  Type: Interval
  Values: 1m, 5m, 15m, 30m, 1h
  
  Usage in query:
  rate(http_requests_total[$interval])

4. Textbox Variable

Purpose: Free-text input from the user

Example:
  Name: search
  Type: Textbox
  Default: .*
  
  Usage in query:
  http_requests_total{endpoint=~"$search"}

5. Datasource Variable

Purpose: Let users switch between data sources

Example:
  Name: datasource
  Type: Datasource
  Type filter: Prometheus
  
  Usage: Select data source for panels dynamically

6. Constant Variable

Purpose: Hidden constant value (useful for templating)

Example:
  Name: cluster
  Type: Constant
  Value: us-east-1

Using Variables in Queries

# Single value variable
rate(http_requests_total{job="$job"}[5m])

# Multi-value variable (use regex matching!)
rate(http_requests_total{job=~"$job"}[5m])
# If user selects "web" and "api", this becomes:
# rate(http_requests_total{job=~"web|api"}[5m])

# All values
rate(http_requests_total{instance=~"$instance"}[5m])
# If "All" is selected: {instance=~".*"}

# Nested variables
rate(http_requests_total{job="$job", instance="$instance"}[5m])

Variable Settings

Multi-value: Allow selecting multiple values (use =~ in PromQL!)
Include All: Add an "All" option (maps to .* in regex)
Refresh: When to re-query the variable values
  → On Dashboard Load
  → On Time Range Change
Sort: Alphabetical, Numerical, Alphabetical (case-insensitive)
Hide: Hide the variable from the UI (useful for constants)

💡 Key Takeaway for Exam

Variables make dashboards dynamic and reusable label_values(metric, label) populates dropdowns from Prometheus Use =~ (regex match) for multi-value variables $variable syntax references variables in queries Query variables are the most common type

5.8 📖 DASHBOARD BEST PRACTICES

1. Organize by Purpose

Good:
  → "Production API Overview" (high-level KPIs)
  → "Production API - Detailed Latency" (deep dive)
  → "Infrastructure - Node Metrics" (servers)
  → "Database - MySQL Performance" (database)

Bad:
  → "Everything Dashboard" (too many panels, too slow)

2. Use the Right Panel Type

Trend over time?     → Time Series
Single KPI number?   → Stat
Utilization 0-100%?  → Gauge
Distribution?        → Heatmap
Tabular data?        → Table
Comparison?          → Bar Chart

3. Set Appropriate Units

Always set the correct unit in Grafana:
  → Latency: seconds (s), milliseconds (ms)
  → Throughput: requests/sec (reqps), ops/sec (ops)
  → Memory/Disk: bytes (decbytes or binbytes)
  → CPU: percent (0-100) or percentunit (0-1)
  → Network: bytes/sec (Bps) or bits/sec (bps)

⚠️ Prometheus uses base units (seconds, bytes)
   Grafana can auto-convert for display (e.g., bytes → GB)

4. Use Meaningful Legends

Good:
  Legend: {{method}} {{status}}    → "GET 200", "POST 500"
  Legend: {{instance}}             → "app1:8080"

Bad:
  Legend: {A}                      → Meaningless!
  Legend: (empty)                  → Can't distinguish lines!

5. Set Thresholds

Use color thresholds to highlight problems:
  → CPU: Green < 60%, Yellow 60-80%, Red > 80%
  → Error Rate: Green < 0.1%, Yellow 0.1-1%, Red > 1%
  → Latency: Green < 200ms, Yellow 200ms-1s, Red > 1s

6. Dashboard JSON Model

Grafana dashboards are stored as JSON!
This means you can:
  → Version-control dashboards in Git
  → Deploy dashboards via CI/CD (Grafana provisioning)
  → Share dashboards via grafana.com/dashboards
  → Export/Import dashboards between Grafana instances

Provisioning:
  Place JSON files in /etc/grafana/provisioning/dashboards/
  Grafana loads them automatically on startup

5.9 📖 GRAFANA vs PROMETHEUS UI — WHEN TO USE WHICH

💡 Key Takeaway for Exam

Prometheus UI = Debugging, ad-hoc queries, checking targets Grafana = Production dashboards, visualization, alerting, sharing

5.10 📖 REAL-WORLD DASHBOARD EXAMPLES

Example 1: Node Exporter Dashboard

Row 1 (Stats): CPU Usage | Memory Usage | Disk Usage | Network I/O
Row 2 (Graphs): CPU by Mode | Memory Breakdown
Row 3 (Graphs): Disk I/O | Network Traffic
Row 4 (Table): Filesystem Usage per Mount Point

Variables: $instance (from label_values(node_uname_info, instance))

Key queries:
  CPU: 100 - (avg(rate(node_cpu_seconds_total{mode="idle",instance="$instance"}[5m])) * 100)
  Mem: (1 - node_memory_MemAvailable_bytes{instance="$instance"} / node_memory_MemTotal_bytes{instance="$instance"}) * 100
  Disk: (1 - node_filesystem_avail_bytes{instance="$instance",mountpoint="/"} / node_filesystem_size_bytes{instance="$instance",mountpoint="/"}) * 100

Example 2: HTTP Service Dashboard (RED Method)

Row 1 (Stats): Request Rate | Error Rate | p50 Latency | p99 Latency
Row 2 (Graphs): Rate by Endpoint | Error Rate by Status
Row 3 (Graphs): Latency Heatmap | Duration Percentiles
Row 4 (Graphs): Traffic by Method | Saturation (CPU/Mem)

Variables: $job, $instance, $endpoint

Key queries:
  Rate: sum(rate(http_requests_total{job="$job"}[5m])) by (endpoint)
  Errors: sum(rate(http_requests_total{job="$job",status=~"5.."}[5m])) / sum(rate(http_requests_total{job="$job"}[5m])) * 100
  p99: histogram_quantile(0.99, sum(rate(http_request_duration_seconds_bucket{job="$job"}[5m])) by (le))

5.11 📝 EXAM-STYLE QUESTIONS (20 Questions)

Question 1

What is the primary visualization tool used with Prometheus in production environments?

A) Prometheus Native UI B) Kibana C) Grafana D) Nagios

Question 2

What is the default port for Grafana?

A) 9090 B) 3000 C) 8080 D) 9100

Question 3

What is the default port for the Prometheus server web UI?

A) 3000 B) 8080 C) 9090 D) 9100

Question 4

Which page in the Prometheus UI shows the health status of all scrape targets?

A) /graph B) /alerts C) /targets D) /status

Question 5

When configuring Prometheus as a data source in Grafana, what should the “Min time interval” be set to?

A) 1 second B) It should match the Prometheus scrape interval (e.g., 15s) C) 1 hour D) It doesn’t matter

Question 6

Which Grafana panel type is best suited for displaying the current CPU usage as a single number with color-coded thresholds?

A) Time Series B) Heatmap C) Stat D) Table

Question 7

Which Grafana panel type would you use to visualize the distribution of HTTP request latencies over time?

A) Stat B) Gauge C) Heatmap D) Pie Chart

Question 8

What Grafana function is used to populate a variable dropdown with label values from Prometheus?

A) values(metric, label) B) label_values(metric, label) C) get_labels(metric) D) query_labels(metric, label)

Question 9

When using multi-value variables in Grafana with Prometheus, which PromQL matcher should you use?

A) = (exact match) B) != (not equal) C) =~ (regex match) D) !~ (regex not match)

Question 10

How are Grafana dashboards stored internally?

A) As binary files in a proprietary format B) As JSON documents C) As YAML files D) As SQL database records only

Question 11

What is the purpose of the /graph endpoint in the Prometheus UI?

A) To display pre-built dashboards B) To serve as an expression browser for running ad-hoc PromQL queries C) To configure scrape targets D) To manage alerting rules

Question 12

Which of the following is a limitation of the Prometheus native UI?

A) It cannot run PromQL queries B) It does not support persistent dashboards or multi-panel layouts C) It cannot display time series graphs D) It requires a separate license

Question 13

What is the correct way to reference a Grafana variable named “instance” in a PromQL query?

A) {{instance}} B) ${instance} or $instance C) #instance D) @instance

Question 14

Which Grafana panel type is best for showing a value within a defined min-max range, like a speedometer?

A) Stat B) Time Series C) Gauge D) Bar Chart

Question 15

What does the “Access” setting control when configuring a Prometheus data source in Grafana?

A) The authentication method (Basic Auth, Token, etc.) B) Whether Grafana server or the user’s browser makes the query to Prometheus C) The read/write permissions on the Prometheus data D) The network firewall rules

Question 16

Which Prometheus UI page would you check to verify that your recording rules are loaded correctly?

A) /graph B) /targets C) /rules D) /config

Question 17

What is the purpose of Grafana dashboard provisioning?

A) To automatically create Prometheus scrape targets B) To load dashboards from JSON files on disk automatically when Grafana starts C) To provision new Grafana user accounts D) To configure Prometheus alerting rules

Question 18

In Grafana, what is the purpose of the “Legend” field in a panel query?

A) To set the panel title B) To customize how each time series is labeled in the graph using template variables like {{method}} C) To add a description to the dashboard D) To define the Y-axis unit

Question 19

Which of the following is NOT a valid Grafana variable type?

A) Query B) Custom C) Interval D) PromQL

Question 20

What is the recommended approach for creating a production monitoring dashboard for a Prometheus-monitored application?

A) Use the Prometheus native UI and bookmark your queries B) Use Grafana with Prometheus as a data source, creating persistent dashboards with variables and thresholds C) Export all Prometheus data to CSV and use a spreadsheet D) Use the Prometheus /graph page and take screenshots

✅ DOMAIN 5 REVISION SUMMARY CHEAT SHEET

┌──────────────────────────────────────────────────────────────────────┐
│              DOMAIN 5: DASHBOARDING & VISUALIZATION CHEAT SHEET      │
├──────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  PROMETHEUS UI (Port 9090):                                         │
│  /graph   → Expression browser (ad-hoc PromQL)                      │
│  /targets → Scrape target health (UP/DOWN)                          │
│  /alerts  → Active alerting rules                                    │
│  /rules   → Recording + alerting rules                               │
│  /config  → Current prometheus.yml                                   │
│  /status  → Runtime info                                             │
│  /metrics → Self-monitoring                                          │
│  Limitations: No persistent dashboards, no variables, no multi-panel │
│                                                                      │
│  GRAFANA (Port 3000):                                                │
│  Default login: admin/admin                                          │
│  Data source: Add Prometheus (URL: http://prometheus:9090)           │
│  Min time interval = scrape_interval (e.g., 15s)                     │
│  Access: Server (recommended) or Browser                             │
│  Dashboards stored as JSON (version-controllable!)                   │
│  Provisioning: Load dashboards from JSON files on startup            │
│                                                                      │
│  PANEL TYPES:                                                        │
│  Time Series → Trends over time                                      │
│  Stat        → Single KPI number with thresholds                     │
│  Gauge       → Value in min-max range (speedometer)                  │
│  Heatmap     → Distribution over time                                │
│  Table       → Tabular data                                          │
│  Bar Chart   → Category comparison                                   │
│  Pie Chart   → Proportions                                           │
│                                                                      │
│  VARIABLES:                                                          │
│  Query    → label_values(metric, label) — most common                │
│  Custom   → Manual list of values                                    │
│  Interval → Time resolution selector (1m, 5m, 15m)                  │
│  Textbox  → Free-text input                                          │
│  Syntax:  $variable or ${variable} in PromQL queries                 │
│  Multi-value: Use =~ (regex match) in PromQL                         │
│  Include All: Maps to .* in regex                                    │
│                                                                      │
│  BEST PRACTICES:                                                     │
│  ✅ Set correct units (seconds, bytes, percent)                      │
│  ✅ Use meaningful legends ({{method}}, {{instance}})                │
│  ✅ Set color thresholds (green/yellow/red)                          │
│  ✅ Organize dashboards by purpose                                   │
│  ✅ Version-control dashboard JSON in Git                            │
│  ✅ Use variables for reusability                                    │
│                                                                      │
│  PROMETHEUS UI vs GRAFANA:                                           │
│  Debugging/ad-hoc queries → Prometheus UI                            │
│  Production dashboards    → Grafana                                  │
│                                                                      │
└──────────────────────────────────────────────────────────────────────┘

🎯 NEXT STEPS

Reply with “Domain 5 Complete” and I’ll provide the guides for the final two domains:

Domain 6: Service Discovery (6%)

• Static, File, DNS, Kubernetes, Consul, EC2 SD
• Relabeling deep dive (relabel_configs, metric_relabel_configs)

Domain 7: Alerting & Alertmanager (4%)

• Alerting rules syntax, Alertmanager routing, grouping, silencing, inhibition

These are the last two domains — only 10% combined! We’re almost there! 🚀