Skip to content

OMF Egress to AVEVA PI

Cloud-Native DCS posts plant data northbound to an OMF endpoint over HTTPS. OSIsoft Message Format is the ingress AVEVA's own products accept. The same component therefore reaches PI Web API, Edge Data Store, and AVEVA Data Hub. Pointing it from one to another is a configuration entry, with no second integration to build.

This page owns the OMF contract: what the endpoint receives, what it is named, how the connection is authenticated and trusted, and what happens when the endpoint is unreachable. The decision behind the naming is ADR 0056, and the model being projected is ADR 0044. Where this page and either ADR disagree, the ADR is correct and this page has a bug.

Nobody has yet run this against a real PI Server. Everything below is written against the published OMF 1.2 specification, AVEVA's own OMF samples, and the PI naming rules Fledge's open-source north plugin applies. Treat the first deployment at a site as a commissioning exercise, and never as a mere configuration change.

The one-way door

Read this section before the first enable, because it is the only part of the integration that cannot be undone by changing a value.

The projection creates AF elements and PI Points inside the customer's PI System, and it names them after the equipment declarations in this cluster. A PI user then binds displays, reports, and calculations to those names, usually within hours. Renaming a ControlModule, moving one under a different parent, or changing the naming scheme afterwards does not move that work. The renamed equipment arrives in PI as new elements and new points, and the old ones stop receiving data and stay exactly where they are, because they are the customer's objects and their history is in them.

Two consequences follow, and both belong in the commissioning plan.

Equipment and tag names become a customer-visible contract the moment a site first publishes. Settle them before that first publish.

A rename leaves orphaned elements and points behind. The egress never sends action: delete. Retiring them is a decision the PI administrator makes deliberately in AF, with the displays that reference them in front of them.

What is published, and what turns it on

A site publishes nothing until its own Site CR says so. The northbound block is absent by default, and every reader treats an absent block as "nothing about this site leaves the cluster". What a Site Publishes Upstream is the administrator-facing page for that declaration. The short form is this:

spec:
  northbound:
    publish: true
    excludeUnits: [toll-fermenter-04]   # optional

Enabling the chart component and forgetting the declaration produces a running egress that publishes nothing. That is the deliberate direction of the failure: data withheld by accident is noticed by the engineer waiting for it, and data published by accident is noticed after it has left.

What travels, for a site that publishes, is the content of ADR 0044: declared tag values, ISA-88 unit state and mode, the occupying batch and the active procedural element, control-module health and NE 107 device health, runtime state, and the ISA-18.2 alarm lifecycle with its shelve state. Only declared tags are published. The runtime's raw-fallback value stream is not on this surface, and neither is anything the MQTT feed carries about a site that has not declared publication.

The occupying batch and the running procedural element reach AF as context on the unit. Neither becomes an element of its own. A unit gets one Batch element, reused for every batch that ever runs there, carrying BatchID and BatchState. Under it sits one ActiveElement element carrying the Kind, Name, and State of the Phase or UnitProcedure executing. Which batch was on a unit at a given moment is therefore read by time-correlating those streams with the tag streams, the way a PI user reads any other stepped point. The alternative would be an element per batch. This projection never deletes what it has created, so an AF database would then grow one permanent element per lot forever.

Two consequences of that shape are worth knowing. BatchID is not cleared when a batch finishes: the terminal word on BatchState is what says the unit is free, and BatchID stays on the batch that last ran there. And a Batch element gives a unit a child name that a control module or an equipment module could also carry, so a resource named Batch under a unit is refused. See Refused names.

How that batch is decided is worth a note, because it is not obvious from the topic tree. The egress stamps the batch on a record at the moment the message arrives, well before the record is projected. Everything it knows at that moment came off MQTT. The binding between a batch and the equipment it runs on is carried by the batch lifecycle topic and nothing else. That binding is tracked per unit, so a site running two batches at once attributes each unit's records to the batch that actually holds it (#1549). The lifecycle topic is retained, so an egress that restarts mid-batch learns the binding on connect. Without retention it forwarded every record unattributed until the batch next transitioned (#1582).

A record whose unit cannot be named carries no batch at all. That is deliberate. An unattributed record reads as unattributed and stays recoverable from the batch record, while one carrying a batch that did not produce it reads as fact inside a customer's PI Server, where the attribute is one-way. The full chain is in MQTT Telemetry.

The contract at the endpoint

Transport

Property Value
Protocol HTTPS POST to the configured OMF endpoint
OMF version 1.2, sent in the omfversion header
Minimum PI Web API 2021
Message types type, container, data
Delivery At-least-once
Ordering Preserved; one post in flight at a time
Direction Outbound only. Nothing is read back, and PI cannot write here

The PI Web API floor comes from OMF 1.2 itself. propertyoverrides arrived in 1.2, and without it engineering units and range can only be declared on the type. An OMF type is immutable. A 1.1 projection would therefore need one type per distinct combination of unit and range, created permanently inside the customer's PI Server. A site on an earlier PI Web API release is refused with that reason stated. A degraded projection is never served.

The type set

Three dynamic types carry every value, and each one carries Timestamp, Quality, and a single Value property.

Type id Value Carries
CNDCS.Numeric number / float64 Analog tag values
CNDCS.Integer integer / int64 Integer tag values
CNDCS.Text string Boolean and string tags, ISA-88 unit state, unit mode, runtime state, control-module health, NE 107 device health, alarm state, alarm severity, alarm message, shelve state, batch identity, active element

CNDCS.Numeric is interpolated continuously and the other two are stepped. A state or a counter therefore holds its value between samples, and no ramp is drawn through values it never had.

Which of the three a tag lands on comes from its declared dataType. Boolean and String take CNDCS.Text. The integer spellings (Int16, Int32, Int64, their unsigned forms, and the IEC 61131-3 names such as DINT and WORD) take CNDCS.Integer. Everything else takes CNDCS.Numeric, which is the same reading of the field the rest of the product already applies. A value that does not fit the type its declaration chose is skipped and counted. Forcing it would cost the whole batch, because the endpoint rejects an entire message carrying a string where the type declares a number.

The static types are the AF Element Templates a PI engineer sees in the Template field: CNDCS.Enterprise, CNDCS.Site, CNDCS.Area, CNDCS.ProcessCell, CNDCS.Unit, CNDCS.Equipment, CNDCS.ControlModule, CNDCS.Alarms, CNDCS.Alarm, CNDCS.Batch, CNDCS.ActiveElement, and CNDCS.Folder.

CNDCS.Folder appears only when a site configures an AF root path. It is the template for the ancestor elements named in that path, which have to exist before the Enterprise element can be linked beneath them. ADR 0056 §4 requires those elements and its §1 table listed no template for them, which its 2026-08-14 amendment corrects. A deployment that leaves the AF root path empty never instantiates it.

The type set is declared to the endpoint with the first thing that uses it. It is never declared on its own. A type message creates AF Element Templates inside your database, so a deployment that is connected but publishing nothing (every site still opted out) leaves no trace there at all.

Type ids carry no version, and the version keyword is metadata per the specification. Changing the shape of a type takes a new type id, which recreates every PI Point on it. The type set is therefore deliberately small enough that growth in the published model does not force one. A new context leaf is a new Container on CNDCS.Text, and a control module that gains a tag gains a Container.

Alarm severity is one of these strings and not a number. AlarmSeverity is a closed set of four words (Low, Medium, High, Critical), and the product carries no numeric severity anywhere. Publishing a number would mean inventing a scale here. A number also invites a PI engineer to threshold on it, and a severity level added later would then have nowhere to sit between two existing values. ADR 0056 §1's table listed severity under CNDCS.Integer, and its 2026-08-14 amendment records the word as what ships.

Two further shapes are worth knowing because they look like omissions. A boolean tag publishes its declared trueLabel and falseLabel as a string. It never becomes a PI digital point, because a digital state set is configuration inside the customer's PI Server that OMF cannot create. An ISA-88 state transition is not a separate event type. It is the point at which a CNDCS.Text stream changes value, and PI stores that as a stepped string point whose history is the transition history.

Quality carries Good, Raw, or Bad verbatim and does not map onto PI's system digital states, because Raw has no member there and a partial mapping would discard it. Raw deserves a read. It says the value came straight off the I/O card, unscaled, because the control module's program was not running. Those are the three words the surface uses. A payload that did not parse at all is reported as Bad, so your endpoint never meets a fourth word it has no reading for.

Names

A Container id is the ADR 0044 canonical path of the leaf it carries, with its segments joined by /. For the tag PV on control module tt-101:

acme/plant-01/fermentation/cell-01/fermenter-01/tt-101/PV

The endpoint derives the PI Point name from that id. An AF element takes the name of the CR it comes from, and an AF attribute takes the tag name or context leaf name verbatim. Declared tag metadata rides on the Container through OMF 1.2's propertyoverrides: engUnits becomes uom, and engMin and engMax become minimum and maximum. Each falls back to the control module's eng_units, eng_min and eng_max parameter when the tag declares none, and a bound that is not a literal number is left unset, and never guessed at. The Container's name is the declared tag name. A tag carries no label field of its own, so there is nothing else to put there. The declarations OMF has no slot for (role, dataType, accessLevel, and the declared publish mode) ride as Container metadata key-value pairs, each present only when it was declared.

The separator is /, and two reasons rule out .. A metadata.name is a DNS-1123 subdomain and may contain . itself, so joining on . would leave a PI user unable to read a point name back to the equipment that produced it. PI Point names also take . as their own suffix convention, and the endpoint may append one.

Refused names

PI's naming rules are narrower than ours, and a name PI cannot carry is refused at the declaration that produced it, and never rewritten. Five conditions refuse a stream or an element. Each refusal is logged once at error level, naming the canonical path and the declaration responsible, and counted on dcs_omf_egress_refused_total, while dcs_omf_egress_refused_streams says how many are still outstanding. The conditions are:

  • A name carrying *, ?, ;, {, }, [, ], |, \, `, ', ", or a control character, all of which PI rejects.
  • A projected Container id or element name longer than 200 characters. The Container id prefix, if a site sets one, counts against that limit.
  • A case-insensitive collision within one parent, because PI names are not case sensitive. Three shapes reach this. A tag declared as state collides with the State context attribute every control-module element carries. The context attribute keeps the name because it means the same thing at every site, so the tag is refused. Two tags on one module that differ only by case (Level and LEVEL) are both refused, because neither is more entitled to the name and publishing either would put a value in PI under a name whose other meaning is invisible. And an equipment resource whose metadata.name folds onto a context element its parent carries is refused with it: a control module or equipment module named Batch or Alarms under a unit, or named Alarms under an equipment module. The context element keeps the name for the same reason the context attribute does, and the resource is the one an engineer can rename.
  • A name beginning with __, which PI reserves for its own system names and which OMF's predefined messages use.
  • A name that is empty, or that carries leading or trailing whitespace. An empty name addresses nothing, and PI trims the whitespace around a padded one. PV would therefore land on the same object as PV. Only the padded name is refused here. Unlike a case collision, one of the two is plainly what the engineer wrote.

Two of those come from the deployment itself, and the component checks them at startup, before the first record could discover them. The AF root path and the Container id prefix are the parent of everything it sends, so an endpoint that rejects them rejects every batch. A value PI cannot carry is named in the startup error.

Substituting _ for an illegal character or truncating a long name is what Fledge does, and it is rejected here. Both collapse two distinct declarations onto one name, and two streams then write one PI Point with neither side visible. Fledge substitutes because its asset names arrive from arbitrary south plugins and it has nobody to ask. Ours are declared in a reviewed CRD by an engineer who can rename them, so a refusal on the day the declaration is written costs less than a silent merge discovered later.

Equipment names come from metadata.name and are DNS-1123, which intersects none of the rejected set. A refusal in practice is therefore nearly always a tag name. The context-element collision above is the one exception. batch and alarms are legal DNS-1123 names. A resource carrying one is refused for where it sits in the tree.

Authentication and trust

HTTP Basic over TLS is the supported scheme, and it is what PI Web API expects. Kerberos is what some PI sites prefer and it is painful from a container, so it is not implemented. A deployment that requires Kerberos cannot use this component today.

The producertoken header is available for Edge Data Store and older endpoints. OMF 1.2 obsoletes it. Authentication against a current PI Web API endpoint is therefore whatever that endpoint itself requires, which in practice is Basic.

The endpoint's certificate is verified against the PEM bundle named by omfEgress.tls.caSecret. Leaving it empty falls back to the container's system roots, which is usually wrong: a PI Web API host normally sits behind an enterprise CA that no public root chains to. Verification can be disabled for development, and doing so sends both the plant's data and the credential to an unverified peer.

Rotating the credential is Secret and Certificate Rotation.

When the endpoint is unreachable

An endpoint that is down or slow is the normal case for a historian feed, and most of the component's behaviour is about that case.

Records buffer in a bounded in-memory queue while posting fails, and posting is retried with exponential backoff. A failed batch is retried from the same position and is never overtaken by a later one, because PI reads an out-of-order arrival as a data quality problem. When the queue reaches its retention cap the oldest records are dropped, counted on dcs_omf_egress_dropped_total{reason="capacity"}, and logged at error level. A silent drop here would be indistinguishable from a quiet plant.

That log report is rate-limited (#1609). The egress writes one line when the queue reaches the cap, one line per retry-backoff interval for as long as it stays there, and one line when it drains below the cap again. Each carries the drops it accounts for and the running total for the episode. A line per dropped record would instead run at the plant's tag rate. That rate buries the post failure naming which endpoint refused and why, and that failure is the line to act on during an outage. The exact per-record count is on the counter.

Size omfEgress.buffer.queueLimit against the longest endpoint outage the site expects to ride out, multiplied by its record rate. It is a retention decision before it is a memory one.

Two things the buffer does not do. It does not survive a pod restart, so a restart during an outage leaves a gap that is not backfilled. The egress announces the window it lost, and it never pretends to continuity it does not have. It also does not read the historian tables to fill such a gap. The historian is optional and ships disabled, so a backfill path through it is one most deployments do not have.

Duplicates are possible in one direction. A post that times out may still have landed, and the retry is safe because a data message at an existing (container, timestamp) overwrites in place, with nothing appended.

Readiness follows the MQTT broker and not the OMF endpoint. Reporting unready for an unreachable endpoint would have Kubernetes restart the one process holding the queue that survives the outage.

A batch is bounded by bytes as well as by records. OMF caps a message body at 192 KB. What a record costs in bytes is decided by how long the site's equipment names are, because every OMF message carries canonical paths instead of opaque handles. Measured against the default of 500 records, a plant with ordinary industrial names produces a first batch whose largest message is over that cap.

The egress measures a batch before posting it and splits one that does not fit. So this needs no attention until dcs_omf_egress_batch_splits_total{trigger="preflight"} rises steadily. That is the sign to lower omfEgress.buffer.batchSize and stop paying for the split. A count on trigger="endpoint" means something rarer. There the endpoint is enforcing a smaller body than the 192 KB the specification publishes, and no one has yet confirmed that number against a real PI Web API.

Watching it

The dcs_omf_egress_* metrics are documented in Monitoring and Metrics, and the chart ships a ServiceMonitor for them when monitoring.serviceMonitor.enabled is on. Three readings answer most questions.

Question Read
Is the endpoint accepting data? dcs_omf_egress_endpoint_up
Is the plant falling behind it? dcs_omf_egress_oldest_pending_seconds against dcs_omf_egress_queue_depth
Has anything been lost? dcs_omf_egress_dropped_total, which is the loss record and the only one of the three that does not recover on its own
Is the batch size wrong for this plant's names? dcs_omf_egress_batch_splits_total

A slow endpoint and a lost one look alike for the first few seconds and then diverge: post duration climbing while dcs_omf_egress_endpoint_up stays at 1 is an endpoint under load, and endpoint_up at 0 with the queue depth climbing is an outage being ridden out as designed.

Deploying it

The component ships disabled. Enabling it is a Helm value, and the endpoint, the credential, and the egress destination are all required. The chart refuses the render, so a component that cannot reach anything never installs.

omfEgress:
  enabled: true
  endpoint: https://pi.plant.example.com/piwebapi/omf
  auth:
    mode: basic
    existingSecret: pi-omf-credential   # keys: username, password
  tls:
    caSecret: plant-enterprise-ca       # key: ca.crt
  af:
    rootPath: ""                        # empty = root of the configured AF database
    containerPrefix: ""                 # empty = no prefix on PI Point names
  egress:
    destinationCIDRs:
      - 10.20.30.40/32
    ports:
      - 443

omfEgress.egress.destinationCIDRs has no default and the chart fails the render while it is empty. A NetworkPolicy cannot resolve a hostname, so the endpoint's address has to be written down. An empty list would install cleanly and deny the traffic. The component would then report an endpoint that is down, indistinguishable from the endpoint actually being down. Writing 0.0.0.0/0 is available, and it is a decision somebody made on the record. This is the only rule in the chart that sends plant data outside the cluster. Network Requirements carries the firewall side of the same statement.

Three further properties of the deployment are worth knowing before the first install.

It runs as exactly one replica, and there is no value that changes that. Ordering is held by a single flusher with one post in flight inside one process, and a second replica would post the same plant concurrently with nothing ordering the two. They would also share an MQTT client id, which makes the broker disconnect each pod as the other connects.

It reads the Kubernetes API. AF structure is the declared equipment hierarchy, and the MQTT topic tree does not carry it. The chart therefore grants the component a read-only ClusterRole over the equipment tree, alarms, and batches. Nothing about the component writes to the cluster.

It has its own broker identity, dcs-omf-egress, with an ACL that reads the dcs/ tree and publishes nothing. It is separate from the historian's identity even though the two grants are identical, so that revoking one does not take the other's feed with it.

That identity has two credential slots, dcs-omf-egress and dcs-omf-egress-b, carrying the same ACL rules (ADR 0061). The second account exists only while its password is being rotated, and rotating it moves this component alone. The MQTT credential is separate from the OMF endpoint credential in every way that matters: the broker password rotates with no disconnect, and the endpoint password cannot, because the overlap would have to exist in the PI Server's own account store.

What this is not

  • A path for PI to write into the DCS. Nothing is read back from the endpoint. Setpoint changes and commands from a supervisory system go through SCADA Integration.
  • A replacement for the built-in historian. The two answer different questions, and Historians and Data Archives covers which is authoritative for what.
  • A general OPC UA or Sparkplug B northbound surface. Neither exists. The reasons, and what each historian's usual connector expects instead, are in Historians and Data Archives.
  • Tested against a real PI Server. Getting access to one is tracked separately, and the specific list of things a real endpoint has to confirm is in ADR 0056's consequences.