You are currently on IBM Systems Media’s archival website. Click here to view our new website.

POWER > Infrastructure > Systems Management

Optimize Your AI Journey

AI Infrastructure
Illustration by Andy Potts

Artificial intelligence (AI) has rapidly become an essential tool to drive business efficiencies, productivity and new business models. Enterprises use it to glean insights from the massive amounts of data coming from social media, the Internet of Things and myriad other data sources. IBM has streamlined the process of realizing business value with AI through hardware and software optimizations—a combination not available elsewhere. This enables enterprises to more easily reap the benefits afforded by the AI era.

To date, enterprises have focused on classical machine learning with many evolving to leverage deep learning, a subset of machine learning, says Adel El-Hallak, director of product management, IBM Cognitive Systems. Popular use cases include computer vision, which enables the computer to see images or video for predictive maintenance or drone surveillance, anomaly detection for fraud prevention and natural language processing for humanlike interactions in speech and sentiment analysis. IBM’s offerings enable clients to gain speedier insights by harnessing the IBM Power Systems* platform’s faster compute capabilities with optimized software that addresses the challenges in each stage of the process.

Improving Client Experiences

Traditional process-automated analytics can’t keep up with the flood of data coming into enterprises today. A different, cognitive approach is needed, says Dylan J. Boday, offering manger, Cognitive Infrastructure. Such an approach uses algorithms that continually learn to sift quickly through the information and derive value from it, just as humans do.

Enterprises use AI to improve product design and boost sales. “Using the massive amounts of data available, you can make the right, higher quality product the end user really wants and generate the insights to recommend it when the customer is searching,” Boday notes.

Embarking on the AI journey is imperative for business success today. “If you don’t incorporate AI in your business, you will get disrupted,” says El-Hallak.

Applying deep learning adds intelligence to the analytics process. It also prevents enterprise amnesia, which is the gap between the data created and insights that are gleaned from the data. “Throughout this process, you are getting to know your clients better and improving client experiences,” he adds.

Applications for leveraging AI exist across market sectors due to the tremendous value it provides by improving business outcomes. Use cases are endless, including a bank in China using computer vision to prevent physical ATM theft by detecting and signaling an alarm when a person approaches the ATM wearing a mask; in media where entertainment companies use computer vision to assemble trailers in hours as opposed to watching 30 days of footage; and large steel manufacturers using deep learning to identify errors early in the production process, says El-Hallak.

Threat detection, where AI applications can search for either objects or persons of interest, is a particular use case that will continue to grow in importance. With the value of PowerAI and IBM Power Systems AC922, these models can be made more accurate and iterated upon much faster, Boday says.

Enterprises use many AI models to improve customer experiences. A bank employs natural language processing to assess the tone and sentiment when a customer calls. Based on that data, AI uses another model to serve up the next best action based on the customer’s response, reducing turnaround times and improving customer satisfaction. Financial institutions also use deep learning to identify and prevent threats through anomaly detection.

It takes a team effort upfront to build, deploy and manage deep-learning models. A typical AI team is comprised of several people, including a data engineer to extract data and a data scientist to help prepare the data as well as build and train machine and deep-learning models. A business analyst evaluates the model to ensure it meets the business line’s needs before the model is turned over to DevOps for deployment and an apps developer to create the customer-facing application. Finally, the model must be monitored for accuracy.

“This is a team sport that involves a lot of different stakeholders and enterprises need to break new ground in terms of learning how to work with new stakeholders,” El-Hallak explains.

Bottlenecks Removed

As AI projects expand in scope and size, deployments require hardware and software that can scale and handle larger, more complex models to deliver insights more quickly and accurately. The IBM Power Systems platform is an industry leading solution for AI, providing differentiated cognitive infrastructure and co-optimized deep learning frameworks that are easy to deploy with tools that make them simpler to use.

Enterprises can begin the AI journey today with the latest POWER9* processor that’s designed for the AI era. The POWER9 chip has a number of advantages geared toward AI, including the ability to handle massive amounts of data via industry-leading innovations that include advanced I/O interfaces and microprocessor architectures.

The AI era is driving compute demands that CPU alone can’t adequately handle including data volume and the speed required for moving the data within a server or a cluster. The POWER9 processor delivers next generation NVIDIA NVLink between the processor and the GPU, PCIe 4.0 and coherence with system memory.

The next generation NVLink bus between the POWER9 processor and the NVIDIA V100 GPU delivers 5.6x more bandwidth than competitive x86 architectures. The interface not only provides game-changing data movement, but also is coherent, enabling the GPUs to access system memory coherently as if they were directly connected. “Coherence enables clients to tackle larger models as the accelerators have access to system memory and no longer are solely dependent on GPU memory,” says Boday.

Another key feature only available on POWER9 is the PCIe Gen 4.0 bus, which yields 2x greater data throughput over PCIE Gen 3.0 found on x86 machines. Leveraging PCIe 4.0 with InfiniBand, an interconnect technology for clusters, clients will be able to interconnect nodes faster and accelerate data transfer in and out of the network more efficiently than ever before.

The innovations only available on POWER9 enable data to flow freely within a node and a cluster, allowing for larger, more complex models to be tackled with ease with unprecedented speeds of insight. “These innovations allow more accurate models around deep learning frameworks or the ability to train those models faster, which the combination of POWER9 in AC922 and co-optimized frameworks demonstrate a nearly 4x improvement in performance over x86 alternatives,” says Boday. “Advanced I/O bus NVLink, PCIE Gen 4.0 and coherence are game-changing technologies for the end user of AI today and we are seeing the performance measurements that demonstrate it.”

Achieving Business Value

IBM’s PowerAI software addresses the challenges across the different stages of a deep learning workflow. The software includes popular open-source deep learning frameworks, such as Tensorflow and Caffe, which are easy to deploy on-premises. El-Hallak regularly challenges his team to use a few command lines to deploy the frameworks with all the dependencies, drivers and libraries required, rather than spend weeks or months configuring the software.

“Advanced I/O bus NVLink, PCIE Gen 4.0 and coherence are game-changing technologies for the end user of AI today and we are seeing the performance measurements that demonstrate it.”
—Dylan J. Boday, offering manager, IBM Cognitive Infrastructure

PowerAI and its modules also make data wrangling less cumbersome. That’s important as 80 percent of a data scientist’s time is spent cleaning data—reformatting it, sizing it and labeling it. “You don’t just dump the data into a repository and the algorithm picks it up and runs with it,” El-Hallak says.

Thanks to IBM’s efforts to make AI consumable, clients don’t need a huge staff of data scientists spending time preprocessing data. PowerAI Vision streamlines the work of prelabeling the data and preconditioning it for computer vision use cases. Further, subject matter experts, such as doctors and quality control engineers, can take control of labeling images by using intuitive user interfaces provided in PowerAI Vision. Many of these transformative functions are run in parallel to save time.

AI is compute intensive and requires a robust and powerful system such as the Power Systems platform. Deep learning training is an iterative process, so new data points or hyper-parameter changes require the models to be retrained. Through distributed deep learning, PowerAI with the POWER difference can scale to hundreds of GPUs efficiently. For example, in one instance, training was reduced from nine days to just four hours, making data scientists’ time more productive. PowerAI also provides tools to speed optimization through automated hyper-parameter tuning.

Flexible deployment options enable the algorithm to be consumed in the inference or scoring stage—the stage at which AI gets monetized. “Realizing business value with AI can be difficult and we’re looking to alleviate the difficulties all the way through inference and maintaining model accuracy,” notes El-Hallak.

Clients can use IBM Watson* technology to add complementary AI services like pretrained APIs, which enable functions like natural language processing. A Watson API can be used to understand the tone and sentiment of a banking customer, for instance, which enables the model to offer the most appropriate subsequent action.

Partners for the Journey

IBM is available to assist clients at every stage of the AI journey, from education through deployment.

IBM gets clients started on the AI path with a discovery workshop through IBM Lab Services. IBM ensures the right stakeholders are engaged, looks at their use cases and identifies one or two projects to start. IBM makes sure all the prerequisites are in place for these projects, including stakeholder commitment and sufficient data for the algorithms. The next step is to execute on a design workshop to enable the use cases identified.

Finally, when the clients are ready to deploy, they want a partner that will be there for the duration with enterprise support, which is very important to IBM. “You can talk to someone at IBM if it’s a hardware or software problem and you can even speak with a data scientist at IBM about AI questions,” Boday says. “The combination of differentiated hardware, optimized software, ease of use tools and services and support enables IBM to serve as that partner for our clients along the journey to AI, which is unique to IBM,” he says.

IBM offers Level 1 through Level 3 support across the stack, meaning clients can contact IBM with any issues. If clients encounter problems, IBM will also reach out to partners such as NVIDIA, Mellanox and the open-source community to help find solutions, deliver a fix pack to get you back up and running again and then work to upstream the fix back in to the open-source community.

Any journey goes better with a knowledgeable partner. IBM is ready to assist clients to launch their AI initiatives and help them prosper.

Shirley S. Savage is a Maine-based freelance writer. Shirley can be reached at



2019 Solutions Edition

A Comprehensive Online Buyer's Guide to Solutions, Services and Education.


Integrate SAP HANA into your Power Systems Environment


Optimize Your AI Journey

IBM Systems Magazine Subscribe Box Read Now Link Subscribe Now Link iPad App Google Play Store