Artifact: PSMark: A Distributed IoT Benchmark for Publish/Subscribe Under Domain-Based Workloads

Abstract

This artifact paper presents a guide for PSMark, a distributed benchmarking framework to evaluate Publish/Subscribe (pub/sub) systems against real-world representative IoT workloads. PSMark addresses limitations in existing pub/sub benchmarks by supporting heterogeneous device behaviors, distributed multi-node deployments, and cross-protocol evaluation across MQTT and DDS. PSMark consists of Managers that initialize, coordinate, and terminate executions and result storage across distributed nodes and their pub/sub clients; Protocol Adapters that translate high-level pub/sub operations into system-specific commands; Metric and Client Interface Plugins that facilitate the inclusion of new metric calculations, advanced pub/sub logic, and pub/sub client implementations; and Workload and Scenario Configurations providing 12 built-in, customizable workloads derived from 7 real-world datasets across 4 domains. Each workload specifies device types, publication rates, payload sizes, and connection stability parameters, and PSMark provides ready-to-run scenarios for the built-in workloads.