Skip to content

Historians and Data Archives

Cloud-Native DCS pushes process data out over MQTT, and it serves the same data back over REST and SQL. A plant historian takes delivery by subscribing to the broker, by polling the gateway, or by querying the historian database directly. What the product does not offer is an OPC UA server on the northbound side, so a historian connector that works by browsing an address space and subscribing to nodes has nothing to connect to. That connector is the one most historian vendors ship, which makes this the first thing to settle in an integration design.

Push and pull

What moves Surface Direction Contract
Live tag values, alarm events, ISA-88 state transitions, batch lifecycle MQTT, JSON payloads on the dcs/ topic tree DCS publishes MQTT Telemetry
Historical tag values, alarms, state transitions Gateway REST, /api/v1/historian/... Consumer polls Historian
The same history at rest PostgreSQL with TimescaleDB, ordinary tables Consumer queries Historian
Live values and alarms for an operator screen Gateway WebSocket or SSE Consumer subscribes SCADA Integration
Batch production records and the audit trail Gateway REST, plus the archive bundles in object storage Consumer polls Batch Records, Backup and Recovery

Three properties of the push path decide what a historian sees at the other end of it. Function block output values republish on a fixed interval whether or not the value moved, so there is no deadband or exception reporting on our side to tune. Alarm events, control module health, and runtime status are retained on the broker. A subscriber that connects mid-batch is therefore handed the current state at once. The payloads are JSON on a hierarchical topic tree, and no part of the feed speaks Sparkplug B. The rates, the retained flags, and the payload keys are specified in MQTT Telemetry.

The pull path is paged, and a historian backfill is therefore a loop over offsets. Server-side aggregation over a bucket interval is available on the same route and is the cheaper way to move a long window. Historian carries the parameters and the row caps.

The northbound limit

OPC UA in Cloud-Native DCS is southbound only. The unit runtime acts as an OPC UA client toward field devices, and the gateway exposes no server for anything upstream to browse. There is no address space to walk and no Sparkplug B birth certificate to parse, so a historian gains no tag list by discovery.

Budget the integration work that follows, because it is real and it lands on the consumer's side. Somebody writes a subscriber on the broker or a poller on the REST API, maps our tag addresses onto the historian's tag names, and then operates that process like any other production component. A site that already runs an integration platform or an ETL host usually puts the mapping there, since that host is already covered by its change control.

AVEVA PI is the exception. The exception is to the work, and the limit stands. There is still no address space to browse. The product ships omf-egress, which posts OMF to the site's PI Web API endpoint and creates the AF elements and PI Points itself. See OMF Egress to AVEVA PI. The work it replaces is the subscriber and the mapping. The work it does not replace is agreeing the equipment and tag names before the first publish, since those names become the ones a PI user builds against.

Archive or source

The historian ships disabled (historian.enabled: false). A plant whose system of record is somebody else's historian can leave it that way and take the MQTT feed instead. Three things change when it is off, and each of them is visible.

  • The /data Trends and Alarm History views report that no historian is configured, since both are historian-backed. See Production Records Interface for the per-view fallback.
  • The consolidated batch production record still assembles from the Batch resource and the audit trail. It returns carrying a historian not configured warning and without its process-value section, so the record loses the measurements taken during the batch.
  • The audit archival CronJob has no database to archive into, and audit records then live only in their hot tier in etcd.

Running both historians is the common outcome, and it needs a written answer to which one is authoritative for what. Ours is scoped to the batch. It is where the consolidated record finds the tag values, alarms, and state transitions belonging to a lot, and it is where archived audit records land under a signed manifest. Theirs holds the plant-wide process history over years, at whatever compression its own engine applies.

Retention is set in four independent places on our side, and a deployment that has agreed which historian is the system of record should set all four deliberately.

What it governs Setting Default Reference
Time-series data in the historian database historian.retention.days 365 days Historian
Audit records before they leave etcd historian.audit.archival.activeRetentionDays 90 days Backup and Recovery
Archived audit records in PostgreSQL historian.audit.archival.archiveRetentionDays 1095 days Backup and Recovery
Archived audit bundles in the Object Lock mirror historian.audit.archival.immutable.retentionDays 2555 days Backup and Recovery

Set the first of those against the retention the plant historian already guarantees. Paying twice for the same decade of process data is the thing a plant tends to discover a year in.

The named systems

None of the integrations below has been tested by us against the named product. Each path is traced to that product's own documentation, and the glue between it and one of our surfaces is written by the deploying integrator. AVEVA PI is the one row where the glue ships as product code instead, and it is equally untested against a real PI Server.

Historian What its usual connector expects Path with today's surfaces
AVEVA PI An OPC UA endpoint, or Sparkplug B over MQTT A shipped component posts OMF to the site's PI Web API endpoint
Aspen InfoPlus.21 A Cim-IO device interface, commonly OPC UA Write through the IP.21 client interface the site operates
Canary OPC UA, Sparkplug B, or JSON over MQTT Subscribe to our broker and map the JSON in the collector
InfluxDB A Telegraf input plugin Subscribe to our broker with mqtt_consumer
TimescaleDB SQL Query or replicate our tables

AVEVA PI

PI is the one product in this table that does not need glue from the deploying integrator. A shipped component, omf-egress, subscribes to the broker and posts OMF over HTTPS to the site's PI Web API endpoint, creating the AF elements and PI Points as it goes. The contract it honours (the type set, the names it creates, authentication, buffering, and what a rename does) is OMF Egress to AVEVA PI. Read that page before enabling it, because the names are a one-way door once a PI user has bound a display to them.

The connector routes a PI site would normally reach for are both closed, which is why the product ships its own path instead. The OPC UA route needs a server endpoint to add as a data source, and AVEVA's PI Connector for OPC UA has nothing here to browse. The MQTT route is built around Sparkplug B data sources in both the PI Connector for MQTT Sparkplug and the newer AVEVA Adapter for MQTT, and our topics carry plain JSON.

OMF is AVEVA's own ingress format, so the same component reaches Edge Data Store and AVEVA Data Hub as well. Pointing it at any of the three is a configuration entry. Two constraints are worth checking against the site before a design commits. The projection targets OMF 1.2, which needs PI Web API 2021 or later. Authentication is HTTP Basic over TLS, and Kerberos is not implemented.

Aspen InfoPlus.21

IP.21 collects through Cim-IO device interfaces, and the one a site is most likely to have licensed is OPC UA. AspenTech publishes its interface documentation behind the AspenTech Support portal, so this page cannot state what a given Cim-IO version accepts from a custom publisher. Confirm that with your AspenTech contact before a design commits to it. The shape that holds whatever the answer turns out to be is a subscriber on our broker that writes into IP.21 through a client interface the site already operates.

Canary

Canary is the closest fit to what the product publishes today. Its MQTT collector reads Sparkplug B 2.2 and 3.0 and also plain JSON, and for JSON it offers a script parser that receives the message and the published topic and returns tag names with timestamped values. The MQTT Collector overview documents both parsers and the qualities they accept. A mapping written there needs no additional process to operate. Canary also documents a Sender Web API for writing values over REST, which suits a poller against the gateway instead.

InfluxDB

Telegraf's mqtt_consumer input plugin subscribes to broker topics and parses payloads with any of its input formats, including json_v2. Its topic_parsing block lifts segments of the topic into tags. That is how the namespace, unit, and tag address in our topics become queryable dimensions. Both halves are Telegraf configuration and need no code from either side.

TimescaleDB and other time-series databases

A plant standardizing on TimescaleDB is standardizing on what our historian already writes. The tables are ordinary PostgreSQL hypertables and the historian writes them as an ordinary database user. A BI tool, a dashboard, or a replication job can therefore read them where they sit. The section Querying from other tools shows the query the built-in Trends view runs, which is the query anyone else would write against the same table.

Batch records and events

Production records leave the system by pull. The gateway serves the consolidated batch production record as JSON or BatchML XML and the audit trail as REST, and the archived audit records sit in object storage as signed bundles. Nothing in the product posts a record to an external endpoint when a batch finishes. The one outbound HTTP call it makes is the ChangeRequest webhook backend, which carries a change-control approval to an eQMS and carries no production data.

MQTT closes the polling gap. The batch lifecycle topic publishes every batch's terminal state, so a consumer subscribes there for the event and pulls the finished record over REST once it arrives. That pairing delivers a record within seconds of completion without a scheduled sweep over every batch in the site.