Inductive Automation Blog

In 2022, Döhler South Africa realized they were reaching capacity in their production facility, an impasse that countless growing companies have reached. The question always becomes: make do with too-cramped quarters or invest in a long-term solution?

Döhler — a multinational company that manufactures fruit juices, compounds, flavors, emulsions, and concentrates — chose the latter for their new facility, which specializes in blending and secondary transformation. Located in Paarl, South Africa, the new plant adds tenfold capacity while remaining scalable. “We had to really plan for the future and really give ourselves the capability to grow,” said Dirk Brand, Head of Engineering at Döhler South Africa. “Not only for the next three years, but for the next ten years.”

Integration for a project of this size and complexity has the potential to become a game of compromise, but with the help of INTEG System Integrators, Döhler South Africa successfully implemented a massive manufacturing system built in Ignition — the unlimited industrial automation platform for SCADA, MES, IIoT, and more — that balances a staggering amount of functionality with an intuitive interface.

Greenfield Development

While the sheer scale of this greenfield project might make some blanch, INTEG viewed this as an opportunity to build from a clean slate. “We could start from the beginning and develop the project like we wanted to, in essence,” said Brian Cooper, Managing Director at INTEG. “On a brownfield project, it's difficult to do that. You've got existing standards, you've got existing equipment, and it makes it difficult.”

While the project requirements remained fairly consistent throughout, on a more granular level there were, inevitably, changes every day. “How we thought it was going to work versus how it actually worked at the end were two very different processes,” said Brand.

“During commissioning, it became apparent how easy it is to make changes on the fly and build new functionality within the SCADA system very quickly,” said Tean Butler, Technical Manager at INTEG.

Now complete, the Ignition system — built from the core combination of Ignition’s Perspective, Tag Historian, and Reporting modules along with Sepasoft’s Batch Procedure Module and the Canary Historian — boasts approximately 1.2 million tags on two gateways, as well as sophisticated batching, navigation, interlocking, and ID generation systems.

Architecture

INTEG originally considered a standard Ignition architecture on a single gateway. However, it quickly became evident that the system — which encompasses around 200 routes, 2,000 phases, and 400 equipment modules for 1,800 control modules — required some additional architectural complexity.

The system features a load-sharing, bifold structure to match the plant’s two sections, each with their own separate gateway, tag provider, and Siemens S7-1500 1518 PLC. The first gateway acts as a hub, running Perspective and housing over 600,000 tags, while the second, a “headless” spoke gateway, has over 500,000 tags. Even with Ignition's unlimited licensing model, a tag count this high requires certain considerations. All of those tags are document text exposed within the User-Defined Type (UDT) structures to allow easy access and increased flexibility with the Siemens PLCs.

ISA 88-Compliant MES

Batching integrates everything in the plant. Using Sepasoft’s Batch Procedure Module as the engine, INTEG developed the MES solution to the exacting standards of the food and beverage industry. “We were quite adamant on using a strict ISA 88 standard,” said Brand.

With an ISA 88-compliant project of this size, it was vital to build the plant according to a set standard, which led INTEG to develop an ID generation system. “The ID generation system was built using Ignition and a database system that helped us to always provide a unique code or unique identifier to each and every single component,” said Butler.

The ID generation system works based on an ownership model to ensure proper interlocking throughout the plant, from general capabilities down to phases, then equipment, and finally to control modules. Because every component in the system has a discrete ID, when an operator runs a batch, the system makes all associated equipment unavailable for any other processes.

Döhler’s plant does not function conventionally via SAP recipes; operators have the flexibility to run batches when needed. Each order comprises several batches as it progresses through the plant, the last of which being the transfer to the shipping truck for delivery. “In order to proceed from one station to another, a batch has to be completed,” said Butler. “One new batch cannot start before a previous batch has been executed.”

Every batch must pass quality control (QC) before moving onto the next process, and equipment must be Clean-In-Place (CIP) before it can be used again. To prevent any batches from starting without proper QC and CIP, INTEG used Ignition’s host of scripting functionalities to implement a sophisticated interlocking system.

“As part of the batching standard that we've developed, we've also implemented a unit state,” said Cooper. “The unit state determines whether the equipment is dirty or clean or being washed or CIP'd, and that is used to interlock specific systems or specific equipment.” The batching system makes cleaning in place exceptionally easy because Döhler can create a recipe to run against the CIP process. The Batch Procedure Module collects this information, recognizing which recipes are linked, and then displays the correct recipe for the operator. This, along with the interlocking that extends down to the control-module level, provides the operator with full visibility of the entire plant from the batching engine.

“Everything is linkable. Everything is trackable,” said Brand. “It really does make their life easier.”

Navigation

Döhler’s staff needed an intuitive method for navigating a system this large and complex. INTEG developed the interface with the simple conceit that an operator should be able to reach any part of the system in three clicks. To accomplish this, they implemented what they dubbed the “breadcrumb” system. “The breadcrumb is an easy way of taking you exactly to that specific area,” said Cooper. “It's developed in a logical way, so if you just know the plant and you know where you wanna go, the breadcrumb would easily take you there.”

This type of quick navigation lends itself naturally to acknowledging alarms. Integrating the breadcrumb system with Ignition’s Alarm Notification Module, INTEG created an easy way for operators and maintenance staff to pinpoint alarms. From the system homepage, operators can simply click on the notification in the header, then filter down through the sections of the plant, following visual indicators to specific areas and process cells.

Alternatively, if operators already know what they’re looking for, they can use the search function. The system follows ISA 95 naming conventions, so by either entering the control module prefix or copying and pasting parameters, operators can quickly scan through the available equipment.

High-Performance HMI

During the commissioning phase, Döhler was unsure whether the plant would have a dedicated control room or field-mounted stations. Ultimately, they opted to forgo a large control room, a decision that greatly influenced the HMI design.

With operators not tied to a central area, INTEG aimed to heighten situational awareness with a high-performance HMI. The grayscale palette reserves color for events or notifications that require immediate attention. “It's easy to build a lot of P&ID pictures and confuse the operators,” said Cooper. “So we developed different layers.” As operators drill down through the layers, they go from overview tiles and simple routes to detailed P&ID-style views, giving individual operators the ability to decide which style works best for them.

“You have buttons and options and menus for everything, but the learning curve was a lot shorter than what I imagined it would be,” said Brand. “Operators pick it up quite quickly.” The Ignition HMI standardizes the look and feel across the whole plant, allowing operators to move from one process to another easily. That flexibility extends to process improvement, letting Döhler add new screens to the Ignition application without restarting or reinstalling the system.

Historical Data And Trending

As with any MES application, historical data and traceability are paramount. Using Ignition, Döhler has complete visibility of their data and the tools to perform root-cause analysis. “It's quite easy now to, within the click of a few buttons, know the exact origin of the problem, the resolution of the problem, and what was done to fix it,” said Brand.

This historical data can also be put to more immediate uses like trending. “We realized that it is necessary for operators to be able to pull up trends easily from the user interface,” said Butler. The system allows operators to access the trending tool from any control module faceplate and easily add values from the same pop-up.

Linguistic Diversity And Symbols

South Africa is host to a huge amount of linguistic diversity, with twelve official languages and at least thirty-five languages spoken in the region. In Paarl and the Western Cape area, there are only three dominant languages, but English — which is used exclusively in Döhler’s Ignition system — would still likely be a second language to much of the company’s staff.

Even without this linguistic hurdle, there is an inherent learning curve for a system with this many moving pieces (not to mention routes for moving those pieces). INTEG sidestepped this potential issue by including symbols throughout the screens to clearly indicate each piece of equipment’s capabilities. “Symbols really helped ease the transition for a lot of the operators,” said Brand. “Something like a mixed proof valve might not be something that's familiar to all the operators, but a picture of a valve makes it clear.”

Maintaining consistent symbols from screen to screen has made it simple for operators to connect the physical plant floor with the representation on the HMI. “The operators picked up on a complex system very easily,” said Cooper. “They, in a short period of time, were able to operate the plant and get product out the door.”

A Communal Success

The success of Döhler’s new facility was truly a group effort. “What really impressed me about the INTEG project team is their product knowledge. The experience with similar plants and similar processes, how proactive they were in identifying potential problems and solving [them] before it becomes a problem,” said Brand.

Likewise, INTEG appreciated the two-way communication during development. As Cooper said of Döhler, “They are knowledgeable people. They gave us feedback and input on control philosophies, on control narratives, and on the methodologies that we used to develop this batching system.”

In addition to technical assistance from Inductive Automation, the project also received support from Element8, the Authorized Ignition Distributor for Sub-Saharan Africa. Butler said, “Any project such as this will encounter technical difficulties, but they had our back all the way.”

With scalability built into the system, Döhler is already looking toward future improvements. “There's almost no limit to what Ignition can offer a client,” concluded Butler, adding that Ignition “makes it easy for developers to express themselves more deeply within the SCADA systems.”

Project Scope

• Start Date: March 2023
• Deploy Date: Phase 1: December 2023, Phase 2: May 2024
• Tags: 1,200,000+
• Screens: 80
• Clients: 7
• Alarms: 6,400
• Devices used:
  • Two Siemens S7-1500 1518 CPU
  • 600+ ASi-Valves
  • 200+ IO-Link Instruments
  • 200+ Profinet Remote IO
  • 70+ Flowmeters and Drives
• Architectures used: Hub & Spoke
• Databases used: PostgreSQL for access control, trending tool and materials library. NoSQL for historical data logged.
• Historical data logged: 30,000

When Castle & Key took ownership of the long-idled Old Taylor Distillery in Frankfort, Kentucky, and revitalized it as a modern facility, they opted to leave some of the 140-year-old buildings’ wear as is. The property, which features an honest-to-goodness castle, a sunken garden, and the world’s longest rickhouse, had languished in disrepair. Prior to the sale in 2014, there was even talk of deconstructing the castle itself and selling the limestone bricks. Leaving the patinaed brass and occasional cracked tile is an aesthetic choice, one that nods to the site’s history as the birthplace of bourbon hospitality in the 1890s. But step inside, under the original Old Taylor sign, and it’s clear that Castle & Key is equal parts tradition and innovation.

Castle & Key implemented Ignition — an industrial automation platform for SCADA, HMI, IIoT, and more — with the help of Gray AES to replace an outdated FactoryTalk system. Headquartered in nearby Lexington, Gray AES is a professional services company offering architecture, engineering, and automation solutions across a wide range of industries, including major greenfield or brownfield expansions. “Supporting distilled spirits producers and bourbon distillers, being headquartered in Lexington, Kentucky, is very important for us, not just as a systems integrator, but as a corporate citizen of our home state,” said Taylor Sawyer, Director of Business Development at Gray AES.

The two companies have a long history together; before the Ignition implementation, Gray AES had redone Castle & Key’s control cabinet, running new terminals and updating the PLC programming. After that success, Gray AES was the clear choice to upgrade Castle & Key’s SCADA system. As Elliott Schmitz, Distillery Manager at Castle & Key, put it, “Our goals were aligned from the get-go.”

Improving Operational Efficiency

The biggest issue with the old system was the lack of historical data. While Castle & Key could see current production numbers, there was little context. For a company with multiple products that require years to properly mature, this inability to look back made looking forward increasingly difficult. “Distilleries are in a unique position. They have to make decisions on a five-to-ten-year spectrum. They can't make a product today and sell it tomorrow,” said Sawyer.

There is a limit to throughput as well; spirits like bourbon require physical space to age. To remain competitive in the market, Castle & Key needed to use the space they already had to its full potential.

Speaking about operational efficiency, Sawyer said, “It's not so much a buzzword or a euphemism within the industry. It's just the nature of where the industry is heading. How do you do what you're doing today, albeit more efficiently?” This is especially true for a registered historic site that doubles as a production floor.

“We are a historic distillery, but the things we like to modernize aren't necessarily traditional whiskey practices. It’s more [about] improved automation techniques, better data and analytics, correlation and connectivity, and then just constantly being neurotic about scientific advancement of our craft,” said Castle and Key’s Brett Connors, whose formal job title is Whiskey Wizard, a position that encompasses the duties of head blender, product strategy, hospitality, and sales support.

High-Performance HMI

This implementation was Gray AES’ first large-scale project using Ignition’s mobile-responsive Perspective Module. “The main request was to make it a more modern look and feel. Perspective was a natural fit, taking advantage of the CSS and the style sheets,” said Jeremy Plunkett, Digital Transformation Engineer at Gray AES.

Gray AES designed a new SCADA system featuring a high-performance HMI, leveraging modern frameworks closer to website design than traditional SCADA visualization. The goal was to modernize the system to run mostly on iPads while maintaining a close visual resemblance so that operators could easily transition to Ignition with minimal training.

When developing the system, Gray AES utilized DevOps principles. “We would pull down a stack into our local environment, spin it up with Docker, have code reviews using GitHub, and we'd create pull requests for any changes. Then we deploy it to our test server, test out changes before we'd actually deploy it to the production environment,” said Plunkett.

The design process was collaborative between both companies, not just between Gray AES and Castle & Key upper management, but with plant-floor staff as well. One request from operators was to keep the application’s primary color the same. While the new HMI still adheres to the fundamentals of high-performance screens — heavy use of gray, bright colors like red reserved for alarms, minimal clutter — the background is a bright, inviting teal. When activated, all valves, pumps, and motors turn white, in line with the high-performance standard. “It felt very unfamiliar until we had the teal background. And it makes the white stand out a little bit more too,” said Schmitz.

More than anything, this gave operators a sense of ownership of the Ignition system. “At the beginning, operators were a little resistant, just because change is difficult,” said Schmitz. “However, there have been some really good features in Ignition. It’s very easy to navigate and it's been well-received among the team.”

Being able to access the Ignition system through mobile devices has greatly improved operator workflows, and the application’s “crisp” response provides immediate feedback.

UI/UX

The Ignition system defaults to an overview of Castle & Key’s still, giving operators an immediate view of the high-priority functions. The system is logically connected for the process flow, allowing operators to navigate between screens without having to return to the main overview like they did in the previous system.

The side menu provides quick access to other screens within the application. The operator can select the screen, monitor key metrics like temperature, and control distillation to hit target proofs and ensure consistency. With so much information on the screens, operators can move pop-ups out of the way, monitoring flow rate while adjusting the aperture of a valve. Additionally, Gray AES designed a variety of standard faceplates so that anytime Castle & Key needs to add another pump, agitator, or valve, they can reuse assets.

For processes like grain intake, the application shows the operator the entire path flow, including all conveyors and valves, as well as any information required to bring the grain from the truck to one of the silos. In addition to individual mobility, the Ignition system saves time by giving operators access to every part of the facility at any time. For example, when one operator is getting ready to unload a grain truck and does not have an iPad, they can radio for assistance, and another operator can help remotely.

Historian

Beyond the visuals, the Ignition system includes the Tag Historian Module. This allows Castle & Key to easily view historical data, identify trends, and make projections. Perhaps most importantly, this access to historical data allows them to identify anomalies, which can have far-reaching consequences for a process that is still as much art as science.

“On the product strategy and research side, we love the historian functionality of [Ignition] because it allows us to really integrate our product quality to historical records and data to be able to improve our overall strategy and processes,” said Connors. “We're excited about the data being able to come from the historian and to correlate that into our production methodology.”

Alarming

The system also features Ignition’s Alarm Notification Module, which provides two locations where operators can access alarms: current overview and alarm history. The former allows operators to see any active alarms in the system, while the latter gives Castle & Key long-term data, similar to the historian, to make continual improvements and better dial in preventative maintenance.

Prior to Ignition, acknowledging alarms was a highly manual endeavor, requiring an operator to physically walk to a machine (sometimes on a separate level of the production floor) and press a button to stop the process. Now, with Perspective’s mobile-responsive capabilities, operators have a convenient, and immediate, method for responding to alarms.

Momentary Push Buttons

Distillation involves a great deal of sensory response during production; operators monitor the smell, taste, and visual clarity of product as it travels through the system. Fittingly, part of the process is dependent on another sense — touch — more specifically, the need to hold down buttons. “The FactoryTalk application had a lot of momentary push buttons, and that was a bit tricky with Perspective because there's not a one-to-one component that mimics a momentary push button,” said Plunkett. “We didn't want to have to rewrite all the logic in the PLC, so we pretty much created our own custom momentary push button.”

Gray AES’ solution was to let operators tap a button in the Ignition application to “bump” open a valve or open it completely instead of holding down and releasing. Accurate grain weight is a critical component of spirit production, so the ability to slightly open, throttle, and completely close a gate with a tap or two instead of continually holding the button down has been just as, if not more, effective.

Parallel Deployment

As with most manufacturers, Castle & Key could not afford to halt production while Gray AES installed the new system. “We took a gateway backup and dropped it into their Ignition gateway on site. We had a simulation PLC as well, so we had already had all the tags mapped,” said Plunkett.

That last point was originally considered an issue because all labels were stored in the FactoryTalk HMI and could not be directly collected from the PLC. While this task seemed daunting at first, Ignition’s scripting capability allowed Gray AES to write a script that parsed through all of the tags from the HMI, then populated them in Ignition, avoiding what would have been an arduous conversion process.

Deploying the Ignition system was fast (“It took five minutes to deploy,” said Plunkett) but to ensure there was absolutely no downtime, Castle & Key ran the old and new systems in parallel. That way, if there were any unexpected changes after the fact, the plant floor could continue moving, not to mention bubbling, agitating, and distilling. “You can connect either to the control panel on the iPad on the same internal network or be connected to the PLC and Ignition at the same time,” said Schmitz.

“The ability to continuously run on our daily basis as we're developing this new programming system was really paramount because it caused us to not have to lose any production time as we were planning and eventually now transferred onto the new system,” said Connors.

Process Refinement

Shift after shift, Castle & Key is collecting data to further refine their processes. They have found that the Ignition system opens up a whole new path forward. As Schmitz put it, Ignition allows them “to get that access to identify trends and make improvements either to our programming or to mechanical aspects in order to drive consistency.”

Even as Castle & Key plans to expand the system by incorporating more processes, they intend to retain the human element that makes their operation unique. “Our industry relies a lot on organoleptic and sensory data. How do you correlate a human experience to your automated experience?” asked Connors. “To be able to actually take that data and scientific overlay and then apply that to your organoleptic program is incredibly uncommon. That's kind of the way our industry merges the balance of science and art, where we're still very human, but the more we move towards automation, we're not trying to remove that human element from the actual tasting and enjoying of whiskey, but we're trying to imbue that into our scientific ideology.”

Sawyer echoed the sentiment succinctly. “Technology is seen as it's going to replace human beings,” he said. “Here it augments, it enhances.”

Project Scope

• 27,505 Tags
• 27 Screens
• 16 Pop-ups
• 10 Clients
• 736 Alarms
• 1 Devices
• Basic Architecture
• 1 Database
• 62 Tags logging historical data