The Fault-to-Fix Trace: One Pump Failure, Every Digital Step, and the Three That Went Wrong

A mechanical seal leak on P-204, Headworks influent pump 4, moves through six stages between the alarm and a closed work order: FAULT, UNDERSTAND, PREPARE, REPAIR, RETURN, LEARN. This guide traces the digital steps through EQUA AIMMS, naming what was read, what was produced and who decided at each one. Three of the first answers were wrong. Two of those three ended in a human decision rather than an automated recovery, and that is the design doing its job.

This is a pre-authored scenario built on synthetic data, and that is the point. P-204 is not a real asset at a real utility. Every record, revision, stock count, supplier response, approval route and duration below was written so you can check each step against the one before it, which is exactly what a redacted customer incident will never let you do. None of it is customer data. Nothing here is a product screenshot. No measured metric appears anywhere in this article, and no duration below is a result, a benchmark or a guarantee. Nothing on this page is dressed up as something that happened to somebody.

Key takeaways

• The trace, not the outcome, is the evidence. A closed work order tells you the pump runs. It says nothing about which steps before the close were done, skipped or guessed.
• Match the part to the installed configuration, not the model. P-204 was rebuilt in 2021 to a different seal arrangement, so the kit the catalogue named could not be fitted.
• A system read can be contradicted by a shelf. Inventory said one on hand. The bin was empty. AIMMS records the contradiction and treats usable stock as zero. It escalates rather than resolves.
• An approval gate with no available approver is a recorded open gate, not a workaround. The clock kept running and the record says so.
• Detection is a read. Control is a write. The boundary is on the write. Across the whole trace: 0 writes to a PLC, DCS or SCADA system, and 0 authority gates bypassed.
• Two kinds of duration appear in this scenario, both of the kind AIMMS records: how long an AIMMS action took, and how long a human gate stayed open. Both are authored, not measured. No maintenance work times appear at all.

What is P-204, and why is that not a standards-issued instrument tag?

P-204 is Headworks influent pump 4: a dry-pit non-clog centrifugal pump lifting raw influent at the front of the plant. The tag is a disclosed site convention, not a standard. It reads as area 200, equipment class P, sequence 04, written down by this utility for its own rotating equipment register.

Site registers and instrument tags get conflated constantly, so name the difference. There is a published ISA standard covering instrumentation and control symbols and identification, ANSI/ISA-5.1-2024. It is a paid standard, named here by number and title only, and this article does not characterise its contents or reproduce any part of it. What matters for P-204 does not need the standard opened at all: this tag was written by a utility for its own equipment register, so presenting it as a standards-issued instrument tag would be wrong.

Identity resolution is still real work. The same machine is P-204 in the maintenance system, a different point name in SCADA, a nameplate serial on the pump, and “number 4 influent” in a decade of technician notes.

Why a seal leak and not a ragging event?

Ragging is the failure mode a wastewater reader thinks of first, and it is the wrong choice here. Clearing an obstruction rarely needs configuration-specific procurement: isolate, open, pull the rag ball, close, return to service. A bearing fault does not justify buying a seal kit. A mechanical seal leak on a dry-pit non-clog influent pump carries every element the trace needs at once.

Element the trace needs	Why the seal leak supplies it
A configuration-dependent part	Seal arrangements change at rebuild. The model number does not say what is inside the pump.
A stock result that can be wrong	Seal kits are stocked, consumed and mis-recorded.
A supplier comparison	Unusable stock means buying.
A governed release	The purchase crosses a spend threshold.
An isolation procedure	The pump is de-energized before anyone opens it.
A return record	A person authorizes return to service after the fit is verified.

What does AIMMS read, and what can it write?

Direction	Systems	What happens
Read only	SCADA and the historian	Alarm state, trends, operating context. Consumed, never commanded.
Read and write back	CMMS, inventory, procurement	Under the permissions and approval rules the utility sets.
No path at all	PLC, DCS, SCADA control	No setpoint, restart, interlock or actuator command, at any permission level.

AIMMS does not replace the CMMS. The maintenance system stays the system of record. AIMMS keeps its own operating record of the case and writes back under rules the customer configures. Autonomy and approval thresholds are the customer’s to define.

AIMMS detects anomalies and deviations in the read-only data it is connected to, raises the fault itself, alerts the responsible people, and surfaces slower drifts as recommended scheduled maintenance. That is detection on process and equipment data, and it is not a condition monitoring programme. Vibration analysis, infrared thermography, oil and wear debris analysis and motor current signature analysis are a separate discipline with their own instrumentation and specialists. A utility that needs condition monitoring still needs it.

What does the trace look like, stage by stage?

Stage 1: FAULT

What AIMMS read	What it did	Artifact produced	Who decided	What was recorded
Seal chamber leak switch, discharge pressure, motor amps, read-only	Detected the deviation against the operating envelope and raised the fault	Time-stamped fault record, evidence attached	Operations, on process impact and whether P-204 came off line	Trigger, evidence snapshot, acknowledgement
Asset register, SCADA tag map, work order text	Resolved four identities to one asset: register tag, SCADA point, nameplate serial, free-text aliases	One identity, each alias named	AIMMS proposed, the technician confirmed	Which aliases matched, which were low confidence
Standby availability and plant flow, read-only	Captured the impact of losing one influent pump at current flow	Impact statement on the record	Operations	Redundancy state at the fault

Scenario AIMMS action time to open the case with evidence attached: 00:38. That figure is authored, and so is every other duration on this page. Now look at what is absent. AIMMS did not stop the pump, transfer to standby, or change an alarm limit. It has no path to any of those.

Stage 2: UNDERSTAND

What AIMMS read	What it did	Artifact produced	Who decided	What was recorded
Historian trends, 3 prior work orders, the O&M manual section, the 2021 rebuild entry, a night-shift note held elsewhere	Assembled one working context, every line naming its source and age	Working context, source-linked	AIMMS assembled. A qualified person accepted the likely cause.	Every claim with its source and age
Gaps in the above	Marked what was missing or stale instead of filling it from a substitute	Uncertainty list, including a manual revision that could not be confirmed	The technician, on proceeding without it	The unresolved items, by name

Scenario AIMMS action time: 02:41. The uncertainty list is the part worth arguing about. An answer built partly on a document the system could not verify is worse than a shorter answer that names what it could not reach.

Stage 3: PREPARE

This is where the first answer went wrong, twice. The next section takes both apart.

What AIMMS read	What it did	Artifact produced	Who decided	What was recorded
Catalogue entry for the model, configuration record, 2021 rebuild entry, stores	Matched the part to the installed rebuild, not the catalogue	1 on shelf. Rev A. Usable 0	AIMMS matched, the planner accepted	Catalogue answer, the rebuild that superseded it, why usable stock is zero
Inventory line for the correct arrangement	Read one on hand at the outlying store, then took a contradicting physical count	Disputed stock record, escalated	The storeroom supervisor, that the record was wrong	Both readings, the contradiction, the escalation, and that the record stayed the storeroom’s
Approved supplier list, terms, sourcing rules	Sent 3 requests under policy and compared the 2 that returned	Supplier comparison	Procurement, on selection	Requests, responses, comparison, selection and reason
Delegation table and approval thresholds	Routed the purchase release with its evidence	Approval request, above the delegation limit	The role holding that spend band	The gate, the role, the threshold crossed, the time it opened
Energy control procedure for P-204 and its MCC bucket	Assembled the isolation requirements into the work package	Safe work path, attached	Qualified personnel isolate and hold the locks	Which procedure applies, and which revision

Scenario AIMMS action times: 00:55 to match the part to the installed rebuild, 00:52 to source it under policy and compare what came back.

On isolation the requirement is public, so name it exactly. 29 CFR 1910.147, “The control of hazardous energy (lockout/tagout)”, states at (c)(4)(i) that procedures shall be developed, documented and utilized for the control of potentially hazardous energy when employees are engaged in the activities covered by the section, subject to a narrow exception in the same paragraph that lets an employer skip documenting the procedure only where every one of several listed conditions is met. At (d)(4)(i) it states that lockout or tagout devices shall be affixed to each energy isolating device by authorized employees. AIMMS assembles the documented procedure into the package. It does not isolate anything, hang a lock, or release one.

One jurisdictional note, because municipal readers are routinely told the wrong thing. The Department of Labor puts it plainly: workers at state and local government agencies are not covered by federal OSHA, but are protected under the OSH Act if they work in a state with an OSHA-approved programme. OSHA currently lists 22 State Plans covering both private sector and state and local government workers, and seven covering state and local government workers only. That settles who may inspect and cite a municipal employer. It does not settle whether the same substantive requirements reach a given utility, which turns on the law of the state the utility sits in. Check the regime that covers you rather than assuming a lockout standard does not apply to a public works crew.

Stage 4: REPAIR

What AIMMS read	What it did	Artifact produced	Who decided	What was recorded
The released work package	Kept parts, supplier, notification and record tasks moving. No physical work.	Running work record	Qualified people did the repair and held every physical action	Who did what, against which approval
A field report that the shaft sleeve was scored, outside planned scope	Reopened the affected step rather than closing over it	Scope change on the open case	The planner and technician	The change, the time, who authorized it

No duration is published for this stage. Isolation, strip, fit and alignment times vary by pump size, access and site procedure, and asserting them would need a superintendent’s signoff.

Stage 5: RETURN

What AIMMS read	What it did	Artifact produced	Who decided	What was recorded
Post-repair readings and return-to-service criteria	Assembled the return evidence in the form the procedure asks for	Return record, incomplete until every check is present	Utility personnel verified and authorized return to service	Each check, its result, the authorizing person
One outstanding check at first pass	Held the case open and named the missing item	Open item, named	The operator, on when it was satisfied	That the case was held, and for what

Scenario AIMMS action time to assemble the return evidence: 01:12. AIMMS never returned the asset to service. It cannot.

Stage 6: LEARN

What AIMMS read	What it did	Artifact produced	Who decided	What was recorded
Everything the case accumulated	Attached it to the asset so the next crew opens it rather than rebuilding it	Asset memory on P-204	The utility’s systems of record stay authoritative	The corrected seal arrangement, now the default match
The three exceptions below	Kept them in the record rather than tidying them out	Exception record	The planner reviewed it	What went wrong, when it surfaced, who decided

Scenario AIMMS action time: 00:44. Case counters at close: writes to PLC, DCS or SCADA: 0. Authority gates bypassed: 0.

FIG. 1

Where the interval goes in this scenario

1. 1 Detect and raise
2. 2 Resolve the asset
3. 3 Assemble evidence
4. 4 Identify installed part
5. 5 Verify real stock
6. 6 Source and quote
7. 7 Route approval
8. 8 Wait for the part
9. 9 Physical repair
10. 10 Verified closeout

Hands on the equipment Everything else Proportional. No units, no total, no measured duration.

Scroll sideways to see the whole drawing.

Figure 1. Where the interval goes in this scenario. A single horizontal bar running from the fault to the pump back in service, divided into proportional segments in this order: detect and raise, resolve the asset, assemble evidence, identify the installed part, verify real stock, source and quote, route approval, wait for the part, physical repair, and verified closeout. The physical repair segment is drawn solid and narrow. Every other segment is hatched. A key notes the bar is proportional and unitless, carries no measured duration, and describes the authored scenario in this article rather than any customer deployment.

Three places the first answer was wrong

A trace with no exceptions in it is not a trace. It is a brochure with timestamps.

1. The part is superseded

The wrong first answer. The catalogue lists a seal kit against the pump model. Matched on model alone, that kit is the answer, and stores holds one. Order it, book the crew, and find out at the pump.

How the mismatch surfaced. P-204 was rebuilt in 2021 to a different seal arrangement. That rebuild sits in the asset configuration record, entered at the time, and nothing ever carried it back into the catalogue mapping. AIMMS matched against the installed configuration first and the catalogue second, so the two answers were compared instead of one being assumed. The comparison produced the artifact: 1 on shelf. Rev A. Usable 0. The unit is present. Usable stock is zero, because the arrangement in the pump is not the one that kit serves.

What happened next. The planner accepted the match. The Rev A unit stayed on the shelf for the two identical pumps that were never rebuilt, and sourcing opened for the correct arrangement.

What the record shows afterwards. Both candidate parts, the rebuild entry that decided between them, and the reason usable stock is zero. The corrected arrangement is now the default match for P-204, so the next crew inherits the answer instead of repeating the mistake. Matching to the installed rebuild rather than the catalogue is the whole job. A part number that matches the model is a guess dressed as a fact.

2. Stores says one on the shelf and the shelf is empty

The wrong first answer. With the Rev A kit ruled out, AIMMS read the inventory line for the correct arrangement: one on hand at the outlying pump-station store. On that read, no purchase is needed and the job is a day away from done.

How it surfaced. A storekeeper walked to the bin. It was empty. The unit had been consumed on an earlier job and never issued against a work order.

What happened next, and it is not an automated recovery. AIMMS has no way to see a shelf. Its record was contradicted by a person standing in front of the stock, and the person wins. The system did three things and no more: marked the inventory line disputed rather than overwriting it, treated usable stock as zero and reopened sourcing, and escalated to the storeroom supervisor, who owns the record. In the scenario that human gate stayed open 00:26 until the supervisor confirmed the count.

What the record shows afterwards. Both readings, system and physical, with times and names. The escalation. That the corrected count was entered by the supervisor, not by AIMMS. This exception escalates rather than resolves, and it should: the storeroom’s record belongs to the storeroom. A vendor claiming their system reconciles inventory against physical reality with nobody in the loop is describing a cycle count, which a person also performs.

3. The approver is on leave

The wrong first answer. The purchase release sat above the planner’s delegation limit, so AIMMS routed it to the role holding that band in the utility’s delegation table. The routing was correct against the configured rule. The role holder was on approved leave.

How it surfaced. Non-response, then an automatic reply. An out-of-office message is not a delegation of authority, and AIMMS did not treat it as one.

What happened next. The configured escalation path named the next authority for that band. The request went there with its evidence attached, and a person in that role approved it. AIMMS did not approve anything, lower the threshold, act on an assumed delegation, or split the purchase into two orders to fall under the limit. Name that last one out loud. It is a tempting failure mode, it is easy to automate, and in public procurement it is a serious one.

What the record shows afterwards, and this is the part most systems omit. In the scenario the gate is recorded as open for 04:00:34. Not as a delay attributed to nobody, not as a status that quietly cleared, and not backdated to the approval. The record names the role, the threshold crossed, the time it opened, the escalation, and the role that released it. If a gate carries no duration, nobody can improve it, and nobody can tell a board or a council where the four hours went.

What the three have in common

Exception	Detected by	Resolved by	Ends in
Superseded part	The system, comparing installed configuration against catalogue	The system, with the planner accepting the match	An automated correction, confirmed by a person
Empty shelf	A person, contradicting the system’s own read	The storeroom supervisor	A human decision. The system escalates.
Absent approver	Non-response against a configured gate	The escalation path	A human decision. The system waits and records.

Two of the three end in a human decision. That is the correct design, and it is the behaviour to insist on. A bin and an approval are facts about the physical and organizational world that no amount of connected data can observe, so a system that produces an answer for them anyway is producing fiction. What a system can do is notice fast, refuse to guess, route to the person who actually holds the record, and keep an honest clock while it waits. When your phone rings at 2am, you want the one that escalates.

The data for every step above existed before the alarm did. The press release for the Black and Veatch 2026 Water Report, 9 June 2026, drawing on more than 600 United States water industry stakeholders, states it in one line: “seven in 10 (70%) say they collect sufficient data, but only 19% say they leverage it effectively”. That figure is quoted from the release rather than from the report body, which is where the exact question wording lives.

Ask for the trace, including the exceptions

Who is asking. EQUA AI builds a product that every question below applies to. Put the list to every vendor on your shortlist, this one included, and let each of them answer no.

If you are evaluating a system that claims to move maintenance work, the demonstration to ask for is not a dashboard. It is a trace of one incident that went wrong somewhere.

Ask for	What a real answer looks like
One complete incident, end to end	Every step: what was read, what was produced, who decided, what was recorded. Not a summary.
The exceptions in that same incident	At least one point where the first answer was wrong, shown in the record rather than described in the pitch.
How a part is matched	Model or installed configuration, what it does when the two disagree, and who may correct a record a person contradicts.
The gate record	Every gate with a role, a threshold, an open time and a close time. If gates carry no duration, they are not measured.
What it does when an approver is unavailable	Delegation and escalation as configured rules, and whether it can ever act without the gate.
The write boundary, in writing	Which systems it writes to, which it only reads, and whether any permission level changes that.
The list of things it refuses to do	A vendor who cannot name a limit has not thought about one.

The last one tells you the most. Ask what the system does when it does not know. A system that degrades quietly, filling a gap with a plausible answer, is more dangerous in a plant than one that stops and names the record it could not reach.

Sources

• OSHA, “State Plans” (which states operate approved plans and whom each covers; 22 plans covering both private sector and state and local government workers, seven covering state and local government workers only): osha.gov/stateplans
• U.S. Department of Labor, elaws Employment Law Guide, “Occupational Safety and Health” (federal OSHA coverage of state and local government workers): webapps.dol.gov/elaws/elg/osha.htm
• 29 CFR 1910.147, “The control of hazardous energy (lockout/tagout)”, paragraphs (c)(4)(i) and (d)(4)(i), GovInfo: govinfo.gov
• ISA, ANSI/ISA-5.1-2024, instrumentation and control symbols and identification (paid standard, named by number and title only, contents not characterised here): isa.org
• Black and Veatch, “Evolving water challenges drive innovation: 15th annual Black and Veatch 2026 Water Report highlights path forward”, 9 June 2026, more than 600 U.S. water industry stakeholders. Figure cited, verbatim from the release: “Seven in 10 (70%) say they collect sufficient data, but only 19% say they leverage it effectively.” bv.com

Who wrote this

EQUA AI builds EQUA AIMMS. When a fault comes in, AIMMS reads what your systems already hold, works out which asset it belongs to, pulls the history and the drawings that matter, matches the part to what is actually installed rather than what the catalogue says, checks real stock, gets quotes moving, routes the approval to whoever holds that authority, and writes the job back into your CMMS as it goes.

The trace above is what that produces. Every step names its source, every gate names its owner, and the exceptions stay in the record instead of being tidied out of it. That is the difference between knowing the pump runs again and being able to show what was done to it.