Check out the AR/VR mini series
Safety, training, productivity and efficiency are key factors for optimal operations. Thus, technologies that improve these areas are proliferating across industries. Among them, artificial intelligence (AI), virtual reality (VR) and augmented reality (AR) are gaining traction for their benefits while also providing savings.
"The semiconductor industry is going through a major transformation," says Dr. DP Prakash, global head of innovation in the CIO division at GlobalFoundries in Santa Clara, Calif. "First, growing demand for product mixes comes with an associated rise in manufacturing complexity. Second, as we approach quantum limits on how small microprocessors can go, Moore’s Law scaling is fading out. Productivity improvements through innovative technologies including machine learning, artificial intelligence (AI), virtual reality (VR), AR and robotic process automation are more important than ever."
Prakash reports that demonstrations on the potential value of enterprise AR by PTC inspired GlobalFoundries to pursue it. "We were convinced AR and VR will improve employee safety, training and productivity," he says.
Prakash adds that AR can deliver value in four primary areas, and that GlobalFoundries is exploring and integrating all of them. They include:
- Documentation that's simpler and 10 times faster with AR headsets and software because maintenance workflows can be quickly captured by experts, rapidly edited, and pushed to a cloud-computing service.
- Training by projecting digital displays on plant-floor equipment to supplement classroom instruction, which can cut training time in half.
- Operations with voice-activated AR allows quick access to the right data at the right time, reducing cognitive-load errors.
- Analytics for predictive and proactive maintenance before manufacturing tools and process applications experience problems or shutdowns.
Following recent pilot projects, GlobalFoundries enabled all its factories with Vuforia Expert Capture AR software and Vuforia Engine training software from PTC. "Now, Vuforia helps us capture all that content on the RealWear or Microsoft HoloLens headset, edit, organize, and redeploy it wherever it's needed," adds Prakash. "We've already cut training time by 50%."
Enrique Herrera, industry principal for manufacturing at OSIsoft, adds: "AR is definitely useful on the plant floor. Imagine you’re sent to evaluate a boiler or some other piece of machinery that’s making unusual noises. With AR, you can serve up a 3D image of its internal workings complete with data streams telling you its current operating status. Suddenly, you have a much better understanding of what is going on without having to shut down the process or take things apart. Plus, you only have two hands, so anytime you need information, but can’t use a phone or a laptop, is a potential opportunity for AR on a wearable device.
"Call AR on the plant floor the 'real meets the ideal.' You could use AR to compare, in real time, real production or processes with an optimized model, so you could see drift and manage it before it becomes a problem. With AR, you can get live data feeds, a simulation of what’s being produced and a simulation of what should be produced according to production and quality metrics set earlier. Could you get this same information on a laptop? Yes, but served up on an AR screen can make it more accessible and more useful."
AR = HMI + digital twin
If potential users find 3D graphics floating in front of physical equipment disorienting, it may help to think of AR is as a natural evolution and extension of the human-machine interfaces (HMI) that came before. Just as paper and clipboards gave way to display panels, PCs, tablets and smart phones, today's screens are giving way to headsets, glasses and other screen-less displays, but the basic data delivery mission of all these HMI formats remains the same. (Logically, delivering data right into biological brains will likely be inevitable in the future, but that's an interface story for another day.) The twist now is AR's flexibility allows data, models and digital twins to be added to interfaces for more efficient responses.
"The biggest added value for AR may come from augmenting work processes, maintenance and particularly safety," says Rashmi Kasat, vice president and head of digital business development at Metso, who also developed its new Digital Garage program for rapid innovation and feedback. "Today, we have maintenance manuals with step-by-step instructions on how to maintain machines, but we constantly get younger people onboard, and not everyone has the time and patience to thoroughly read pages of printed instructions, which can lead to safety concerns. One solution could be digitalizing manuals and making them available via AR to guide workers step-by-step in real-time at the machine. The guided instructions will provide all safety instructions step by step, and this will ensure that safety is taken into account throughout the maintenance operation."
Ken Adamson, vice president for PlantSight at Bentley Systems Inc., adds that: "A digital twin is virtual information that reflects what's physically there, so it must be continually updated, and bring new data to the model from different sources like spreadsheets. The other side is AR with most whiz-bang examples putting 3D pictures on the real world, but it can use any type of data on top of the model, and show it in a real-world context. We're calling AR 'visual operations' because it superimposes data on real items, including video feeds to headsets and hardhats, so users can work hands-free. Another benefit of AR is it can integrate 3D models into reality models to show processes that usually can't be seen, such as pipes underground or wiring behind walls."
Some pictorial history
While AR, VR, mixed reality (MR) and fill-in-the-blank X reality (XR) are getting plenty of attention lately, they're based on 3D graphics and display technologies that have been around for many years. First-person, videogame-style, walkthrough simulations were expected to take over about 10 years ago, and made headway in some industries, but they mostly came and went in process applications because they didn't achieve widespread implementation.
"In 2008, we started working with first-person gaming software, and using 3D graphics to show real-time SCADA information. But now, more use cases have emerged, including the use of building information system (BIM) software like Autodesk's Revit for 3D simulation and walkthroughs," says Russ Agrusa, president and CEO of Iconics, now part of Mitsubishi Electric Corp. "Different industries adopt AR at different rates."
Figure 1: iQagent software scans QR codes and other points of interest (POI), links with a server via wireless, and calls up real-time operating data, documents, video and other resources about the equipment it's viewing on tablet PCs, smart phones and wearable devices. Source: iQagent
Agrusa reports that one of AR's primary advances is how it can perform object recognition using stored device types or barcodes, QR codes or tags on equipment, and use that recognition to superimpose physical images with graphics showing documentation, manuals, operations or other useful information.
"A tank can be overlaid with images showing status, contents and flows. All these elements can appear in the user's field of view, right on top of the physical equipment it represents, which is more efficient than calling back and forth to a control room on a walkie-talkie, or even looking at a regular tablet PC," adds Agrusa. "We're going to see a wave of innovation in this area over the next five years, and just as we moved from PLCs to PCs and the Internet, AR is the next step for interfaces. For example, AR can be used with facial recognition or biometrics for improving security or audit trails. It can also work with global positioning systems (GPS) to deploy the closest people to fix problems faster."
Bob Meads, CEO of iQagent, reports his company and its AR software grew out of his 20-year-old system integration firm, iQuest, which was established to develop software integration with Siemens Simatic WinCC HMI software. "When we saw the iPad 2 come out with its camera in 2011, we knew we wanted get it onto the plant floor quickly, and create something new. Our idea was to create an app that could recognize equipment and display relevant information," says Meads. "We bootstrapped our iQagent software, and launched in 2012 as one the first AR solutions to make relevant data and resources appear immediately on a mobile device, which in turn, allows them to be more efficient."
From the beginning, Meads reports AR was intended to put digital information into the context of the real world. However, to provide data, AR systems needs to know where:
- Users are located in a facility;
- Equipment is situated on the plant floor, and what other devices are in the same area; and
- Sensors and instruments are oriented, and how their data will be captured and visualized.
"We started with QR and barcodes to create points of interest (POI) because they were more efficient and cheaper than using algorithms for object recognition," explains Meads. "Now, devices like Microsoft HoloLens can map an environment, and add information about the devices in it via a wireless network. Users can see what's happening where they're at, and view data just like seeing the POIs that show up in Google Maps. They can also run iQagent on RealWear headsets if they need to be hands-free."
