Abstract ( Time Series Clustering )

In order to understand Time Series Clustering we need to understand the time series data, characterized by sequential observations over time, which is ubiquitous in domains such as system monitoring, finance, and IoT. While forecasting is a common analytical goal, understanding inherent patterns across multiple time series is equally critical. Time series clustering, an unsupervised machine learning technique, groups similar temporal behaviors, enabling pattern discovery and anomaly detection without prior labels. This blog post, tailored for an academic lab session, explores time series clustering using Dynamic Time Warping (DTW) in R to analyze endpoint telemetry data. We present a case study on detecting anomalous CPU usage patterns across 50 endpoints, providing a reproducible workflow with code, visualizations, and interpretations. Additionally, we reference related time series resources from DataNizant to deepen understanding and provide complementary tools for forecasting and analysis.

Introduction

Time series data, such as CPU usage, sensor readings, or user activity logs, captures temporal dynamics critical to system observability. Traditional forecasting methods like ARIMA assume known signal characteristics, but real-world scenarios often involve hundreds or thousands of series with unknown behaviors. Time series clustering addresses three key challenges:

1. Grouping Behavioral Cohorts: Identifying similar temporal patterns across multiple series.
2. Anomaly Detection: Detecting deviations without labeled data.
3. Prioritizing Analysis: Segmenting series for targeted forecasting or investigation.

This post demonstrates how to use the dtwclust package in R to cluster time series data, focusing on a simulated dataset of endpoint CPU usage. The methodology is grounded in Dynamic Time Warping (DTW), a distance metric that aligns temporal sequences robustly against shifts and distortions.

Why Cluster Time Series?

Clustering time series data offers several advantages:

• Scalability: Manages large volumes of telemetry data by grouping similar behaviors.
• Anomaly Detection: Identifies outliers (e.g., malware-induced spikes or offline endpoints) without requiring labeled data.
• Preprocessing for Forecasting: Segments data into homogeneous groups, improving model performance for methods like ARIMA or ETS.
• Interpretability: Reveals structural patterns, such as cyclic behaviors or trends, across systems.

This approach is particularly valuable in domains like cybersecurity, where anomalous endpoint behavior may indicate threats, or in operations, where it can signal hardware issues.

Case Study: Detecting Anomalous Endpoints Using DTW Clustering

We simulate a scenario involving 50 endpoints monitored for CPU usage every 30 minutes over two days (96 time points). The endpoints exhibit four behaviors:

• Normal Oscillation: Regular CPU cycles, modeled as a sinusoidal pattern with low noise.
• Flat Usage: Constant low usage, indicating potential offline status.
• Spiky Usage: High variance, suggestive of malware or external interference.
• Drifting Trend: Gradual increase, indicating issues like memory leaks.

Prerequisites

To follow this lab session, ensure the following:

• Software: R (version 4.0 or higher), RStudio recommended.
• Packages: Install tibble, dtwclust, ggplot2, and dplyr using:

  install.packages(c("tibble", "dtwclust", "ggplot2", "dplyr"))
  

• Dataset: Simulated data generated in R (code provided below).

Step-by-Step Workflow

Step 1: Simulate the Data

We generate synthetic time series data to mimic real-world endpoint telemetry. Each series consists of 96 observations (2 days at 30-minute intervals). Normal endpoints follow a sinusoidal pattern with low noise, while anomalous ones exhibit high noise, flat lines, or trends.

library(tibble)
library(dtwclust)
library(ggplot2)
library(dplyr)

set.seed(42)

# Function to simulate time series
simulate_ts <- function(n, noise=1, type="normal") {
  t <- seq(0, 10, length.out = 96)
  if (type == "normal") {
    ts <- sin(t) + rnorm(96, sd=noise)
  } else if (type == "flat") {
    ts <- rep(0, 96) + rnorm(96, sd=0.1)
  } else if (type == "spiky") {
    ts <- sin(t) + rnorm(96, sd=noise * 3)
  } else if (type == "drift") {
    ts <- sin(t) + cumsum(rnorm(96, sd=0.1))
  }
  ts
}

# Generate 50 time series with varied behaviors
data <- lapply(1:50, function(i) {
  type <- case_when(
    i %% 10 == 0 ~ "spiky",   # High noise every 10th endpoint
    i %% 8 == 0 ~ "flat",     # Flat line every 8th endpoint
    i %% 12 == 0 ~ "drift",   # Drifting trend every 12th endpoint
    TRUE ~ "normal"           # Normal oscillation otherwise
  )
  simulate_ts(i, noise=1, type=type)
})
names(data) <- paste0("endpoint_", 1:50)

# Convert to data frame for visualization
data_df <- as.data.frame(do.call(rbind, data))
data_df$time <- 1:96
data_long <- reshape2::melt(data_df, id.vars="time", variable.name="endpoint", value.name="cpu_usage")

Explanation:

• The simulate_ts function generates a time series based on the endpoint type:
  • Normal: Sinusoidal with low noise (sd=1).
  • Spiky: Sinusoidal with high noise (sd=3).
  • Flat: Constant near zero with minimal noise.
  • Drift: Sinusoidal with a cumulative random walk.
• The dataset includes 50 series, with specific indices assigned to anomalous behaviors for diversity.
• The data is reshaped into a long format for visualization using ggplot2.

Step 2: Visualize the Raw Data

Visualizing the time series helps understand their diversity before clustering.

ggplot(data_long, aes(x=time, y=cpu_usage, color=endpoint)) +
  geom_line() +
  theme_minimal() +
  labs(title="CPU Usage Across 50 Endpoints", x="Time (30-min intervals)", y="CPU Usage") +
  theme(legend.position="none")

Output: This plot displays 50 overlaid time series, highlighting varied behaviors (e.g., flat lines, high variance, or trends).

Step 3: Cluster Using DTW

Dynamic Time Warping (DTW) measures similarity between time series by aligning them to account for temporal shifts and distortions. We use dtwclust for partitional clustering with the PAM (Partitioning Around Medoids) centroid method.

# Perform DTW clustering
cluster_result <- tsclust(data, type="partitional", k=4, distance="dtw", centroid="pam", seed=42)

# Plot cluster results
plot(cluster_result, type="series", clus=1:4)

Explanation:

• Parameters:
  • type="partitional": Uses k-medoids clustering.
  • k=4: Specifies four clusters based on expected behaviors.
  • distance="dtw": Uses DTW as the distance metric.
  • centroid="pam": Uses Partitioning Around Medoids for robust centroids.
• The plot function visualizes each cluster’s time series and their centroid (representative series).

Step 4: Interpret the Clusters

The clustering results group endpoints into four categories:

To assign endpoints to clusters and summarize:

# Extract cluster assignments
cluster_assignments <- data.frame(
  endpoint = names(data),
  cluster = cluster_result@cluster
)

# Summarize cluster sizes
cluster_summary <- cluster_assignments %>%
  group_by(cluster) %>%
  summarise(count=n(), endpoints=list(endpoint))

print(cluster_summary)

Output: A table showing the number of endpoints per cluster and their names, e.g., Cluster 1 (Normal) may contain 35 endpoints, Cluster 3 (Spiky) may contain 5, etc.

Step 5: Visualize Clusters

Visualize each cluster separately to confirm interpretations.

# Add cluster labels to data
data_long$cluster <- rep(cluster_result@cluster, each=96)

# Plot by cluster
ggplot(data_long, aes(x=time, y=cpu_usage, color=endpoint)) +
  geom_line() +
  facet_wrap(~cluster, scales="free_y") +
  theme_minimal() +
  labs(title="Time Series by Cluster", x="Time (30-min intervals)", y="CPU Usage") +
  theme(legend.position="none")

Output: Faceted plots showing distinct behaviors per cluster, confirming the separation of normal, flat, spiky, and drifting series.

Step 6: Anomaly Detection

Endpoints in clusters 2, 3, and 4 are potential anomalies (offline, spiky, or drifting). To quantify anomalies, compute the distance of each series to its cluster centroid:

# Calculate distances to centroids
distances <- cluster_result@dists
anomaly_scores <- data.frame(
  endpoint = names(data),
  cluster = cluster_result@cluster,
  distance = apply(distances, 2, max)
)

# Flag outliers (e.g., top 10% distances within each cluster)
anomaly_threshold <- quantile(anomaly_scores$distance, 0.9)
anomaly_scores$anomaly <- anomaly_scores$distance > anomaly_threshold

# Display anomalies
anomalies <- anomaly_scores %>% filter(anomaly)
print(anomalies)

Output: A table listing endpoints with high distances, flagged as anomalies, e.g., endpoint_10 in Cluster 3 with a high distance score.

Takeaways

This case study illustrates:

• System Health Monitoring: Clustering identifies normal vs. anomalous endpoints at scale.
• Unsupervised Anomaly Detection: No labeled data is required, making DTW clustering versatile.
• Preprocessing for Forecasting: Clusters can guide the application of models like ARIMA to homogeneous groups.

When to Use What?

Related Time Series Resources

The following resources from DataNizant provide complementary insights into time series analysis, forecasting, and related techniques, enhancing the clustering approach presented here:

• ARIMA in Python: This guide details the implementation of ARIMA (AutoRegressive Integrated Moving Average) models in Python for univariate time series forecasting. It covers stationarity checks, parameter selection (p, d, q), and model evaluation. Usability: Ideal for students transitioning from clustering to forecasting, as ARIMA can be applied to individual clusters identified in this case study to predict future CPU usage. Importance: ARIMA’s interpretability and statistical foundation make it a cornerstone for time series forecasting, particularly in economics and system monitoring.
• Time Series Analysis Techniques: This article explores various time series analysis methods, including decomposition, smoothing, and advanced models like SARIMA. Usability: Provides a broad overview to contextualize clustering within the spectrum of time series techniques, helping students understand preprocessing steps like detrending before clustering. Importance: Offers a foundational understanding of time series components (trend, seasonality, noise), critical for interpreting clustering results.
• LSTM Time Series Forecasting: This post demonstrates how to use Long Short-Term Memory (LSTM) neural networks for time series forecasting in Python, focusing on capturing complex temporal dependencies. Usability: Complements clustering by enabling forecasting on clustered groups, especially for non-linear patterns like spiky usage. Importance: LSTMs excel in modeling high-dimensional, non-linear data, making them suitable for advanced analysis of telemetry data.
• How LSTM Became the Forecasting Workhorse: This article explains why LSTMs are widely adopted for time series forecasting, covering their architecture and advantages over traditional models. Usability: Provides theoretical context for students interested in deep learning approaches post-clustering. Importance: Highlights LSTMs’ ability to handle long-term dependencies, relevant for forecasting drifting trends identified in clusters.
• Machine Learning Mastery: This resource offers a comprehensive introduction to machine learning techniques, including those applicable to time series, such as ensemble methods and neural networks. Usability: Useful for students exploring alternative machine learning methods (e.g., XGBoost) for time series analysis after clustering. Importance: Broadens the toolkit for handling complex datasets, bridging clustering and predictive modeling.
• Covariance Matrix Calculator: This tool explains how to compute covariance matrices in Python, useful for multivariate time series analysis. Usability: Relevant for analyzing relationships between multiple time series (e.g., CPU and memory usage) before clustering. Importance: Understanding covariance aids in feature engineering, which can enhance clustering or forecasting accuracy in multivariate settings.
• Python Topic Modeling: While focused on text data, this post introduces unsupervised learning techniques like LDA, which share conceptual similarities with time series clustering. Usability: Provides a comparative perspective on unsupervised methods, useful for students exploring clustering in other domains. Importance: Reinforces the value of unsupervised learning for pattern discovery, aligning with the clustering focus of this case study.

These resources collectively provide a robust framework for students to extend their analysis from clustering to forecasting and multivariate modeling, enhancing their ability to tackle real-world time series problems.

Download the RMarkdown

Reproduce this case study using the following RMarkdown script:

---
title: "Time Series Clustering for Anomaly Detection in Endpoint Telemetry"
author: "Kinshuk Dutta"
output: html_document
---

{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
library(tibble)
library(dtwclust)
library(ggplot2)
library(dplyr)
library(reshape2)

{r data_simulation}
set.seed(42)
simulate_ts <- function(n, noise=1, type="normal") {
  t <- seq(0, 10, length.out = 96)
  if (type == "normal") {
    ts <- sin(t) + rnorm(96, sd=noise)
  } else if (type == "flat") {
    ts <- rep(0, 96) + rnorm(96, sd=0.1)
  } else if (type == "spiky") {
    ts <- sin(t) + rnorm(96, sd=noise * 3)
  } else if (type == "drift") {
    ts <- sin(t) + cumsum(rnorm(96, sd=0.1))
  }
  ts
}
data <- lapply(1:50, function(i) {
  type <- case_when(
    i %% 10 == 0 ~ "spiky",
    i %% 8 == 0 ~ "flat",
    i %% 12 == 0 ~ "drift",
    TRUE ~ "normal"
  )
  simulate_ts(i, noise=1, type=type)
})
names(data) <- paste0("endpoint_", 1:50)
data_df <- as.data.frame(do.call(rbind, data))
data_df$time <- 1:96
data_long <- melt(data_df, id.vars="time", variable.name="endpoint", value.name="cpu_usage")

{r visualize_raw}
ggplot(data_long, aes(x=time, y=cpu_usage, color=endpoint)) +
  geom_line() +
  theme_minimal() +
  labs(title="CPU Usage Across 50 Endpoints", x="Time (30-min intervals)", y="CPU Usage") +
  theme(legend.position="none")

{r clustering}
cluster_result <- tsclust(data, type="partitional", k=4, distance="dtw", centroid="pam", seed=42)
plot(cluster_result, type="series", clus=1:4)

{r cluster_summary}
cluster_assignments <- data.frame(
  endpoint = names(data),
  cluster = cluster_result@cluster
)
cluster_summary <- cluster_assignments %>%
  group_by(cluster) %>%
  summarise(count=n(), endpoints=list(endpoint))
print(cluster_summary)

{r visualize_clusters}
data_long$cluster <- rep(cluster_result@cluster, each=96)
ggplot(data_long, aes(x=time, y=cpu_usage, color=endpoint)) +
  geom_line() +
  facet_wrap(~cluster, scales="free_y") +
  theme_minimal() +
  labs(title="Time Series by Cluster", x="Time (30-min intervals)", y="CPU Usage") +
  theme(legend.position="none")

{r anomaly_detection}
distances <- cluster_result@dists
anomaly_scores <- data.frame(
  endpoint = names(data),
  cluster = cluster_result@cluster,
  distance = apply(distances, 2, max)
)
anomaly_threshold <- quantile(anomaly_scores$distance, 0.9)
anomaly_scores$anomaly <- anomaly_scores$distance > anomaly_threshold
anomalies <- anomaly_scores %>% filter(anomaly)
print(anomalies)

Download the RMarkdown file from this link (replace with actual URL hosted on DataNizant).

Final Thoughts

Time series clustering, exemplified by DTW-based methods, transforms raw telemetry into actionable insights. By grouping similar behaviors and flagging anomalies, it enables scalable system monitoring and informed preprocessing for forecasting. The referenced DataNizant resources extend this framework by providing tools for forecasting (ARIMA, LSTM), multivariate analysis, and broader machine learning applications, making this approach applicable across domains—finance (market trends), cybersecurity (threat detection), and operations (equipment health). As observability becomes central to data-driven decision-making, tools like dtwclust and the accompanying resources empower analysts to uncover patterns in complex, high-dimensional data.