The Ultimate Metric of Reliability: Data Storage
The Prime View got the unique opportunity to interview Nathan Hall, VP of Systems Engineering – Americas at Pure Storage, a global company that builds all-flash data storage products and services for enterprises around the globe.
The interview digs into the notion of Modern Data Experience, the challenges of building the right innovation culture at a tech company, and the future of the data storage industry.
Nathan, you have an impressive technology background from working at some world-famous brands, companies like Dell and Hewlett Packard. And now, for almost three years, you’re part of Pure Storage. What are some of the most valuable lessons that you’ve learned from working at these companies?
My career has been in technology pretty much since I got out of college, and there has been a repeated lesson for me throughout my career in how important a company’s culture is to the degree of a company’s success, that it took a while for me to fully digest. Firstly, it is critical to protect a great culture by hiring the best talent you can access. Secondly, more than anything, you must keep your people passionate and keep them inspired. Then it almost doesn’t matter what your current initiatives are because you can do anything, and the talent you hire will help you determine the right initiatives and direction.
Obviously, market dynamics, market cycles, and timing play a significant role in any company’s success. But I’ve seen companies that performed poorly despite the fact they had the right technology in the right place and at the right time. The outcomes were mediocre because they allowed their culture and their level of talent to erode from great to good. That’s the major lesson that has taken me a while to learn over my career.
Nathan Hall
VP of Systems Engineering at Pure Storage
When I first started evaluating new job opportunities in my career, I prioritized technology over the specific role, targeting the tech I wanted to get into over the role I wanted to get into. It’s only recently that I’ve begun to prioritize the company’s culture over all other aspects. And that’s ultimately why I’m here today, at Pure Storage.
On the website, Pure promotes the Modern Data Experience. What exactly is that notion, and how has it evolved over the years?
It’s a relatively new notion, we haven’t used that phrase for most of our ten years as a company, but it captures what we’ve been focused on since day one. Many people think of Pure as the company that mainstreamed flash into the storage market. In reality, that’s only part of what we were focused on at the beginning. We looked at the major storage vendors and the products they sold. And virtually all of them came into prominence at the beginning of the Dotcom boom in the 90s. Since then, we’ve seen the decline of Unix, the rise of Linux, virtualization, cloud, and containerization. Yet, all of the major storage platforms and the large vendors associated with them had been born before those major architectural shifts in IT, and their architectures and business models had largely remained the same since prior to those major shifts.
At Pure, we saw this opportunity to rethink the way companies store and manage their data in the context of modern app and platform architectures that have taken place in those multiple cycles since.
So the Modern Data Experience is really the kind of experience that we’re providing that results from having the opportunity to “clean sheet” the architecture, as opposed to mitigate and maintain the decades of tech debt that a lot of other architectures have. And also, a “clean sheet” a business model that’s focused on what companies are trying to do with their data today and how we can best support that as a data storage company, and that’s what we call the Modern Data Experience.
Could you please provide a specific example of how you could apply this concept to life?
The data storage industry was built on a high degree of expected tech debt – every three to five years, you had to purchase a new storage array because of the growing maintenance costs and other factors. That migration is risky; it’s painful, and it takes much time. You get the burden of administration, and this hamster wheel continues on and on and only gets worse as your data footprint grows.
Another thing is that data storage administration prior to Pure had become almost like the mainframe, an industry where you have to have storage specialists that know the secret storage commands to run those systems in a world where everything else was moving towards DevOps, where the business or an IT generalist is able to run the infrastructure.
We thought we can figure out how to make sure people continue to evolve and improve their storage technology without ever migrating. So, instead of every three years rebuying a storage array and migrating data to the new array, we have a 30-minute process that keeps all the data in place while upgrading all the technology that services the data; it makes the array faster, better, more cloud compatible.
Also, we’ve plumbed up the storage array to operate it 100% in an as-a-service model. This allows you to request storage from it merely via APIs instead of some storage admin-specific proprietary command lines. That is true whether that’s a physical storage array that we have in a physical data center or a software-only array that we have up in AWS or Azure. It all looks the same to the customer, and they never have to worry about re-migrating every three to five years. It’s a cloud-like experience, regardless if you’re in the public cloud or on-premises.
That’s a profound explanation. My next question is about the future of reliable, scalable, and high-performance storage. Where do you think this market is going in the next 5-10 years?
I think within five years, almost 100% of storage will become as-a-service. It won’t be the end-user responsible for administering the arrays, doing capacity management, or the plumbing work that’s often associated with it. Instead, it will be customers who will consume storage-as-a-service, and it won’t matter where it is because it’ll all look functionally the same to them. It will have the same capability of scaling up or scaling down, and it will be consumed mainly by non-IT generalists in a business.
How does the innovation process look like at Pure?
Innovation is something we started the conversation off with – getting the right culture. It’s an equation that begins with being passionate about getting the best talent in the industry and then preserving that talent in that culture. That is the biggest thing in driving innovation – hire the best developers and strategists and keep them motivated. And if we put them together to allow for cross-pollination, it does magical things for innovation.
It’s a concept that isn’t new, but it’s rarely practiced. It can be expensive; it can be time-consuming because everybody wants to hire fast instead of having the patience to wait for the best candidate. There’s a concept of the 10x engineer – if you can get more of those 10x engineers capable of developing faster and developing better, and then put them together, so they play off each other, it creates a lab for innovation.
At Pure, we devote a large percentage of our revenue to R&D. It allows our strategists and our developers to focus on the long run because we think that’s where we have the opportunity to win a tremendous amount of market share.
Finally, we’re convinced that there will be tremendous growth in containerized applications two to three years from now. To develop something that’s far ahead of the competition, we’ve recently acquired the leader in the space, a company called Portworx that’s mainly about data services in the Kubernetes space.
How have you managed to build a variety of hardware and software products around data storage while keeping a laser focus on quality and reliability?
Obviously, hiring the best people into your organization matters. But you have to have a process that enforces quality. It can’t merely rely on people as no quality process says, “Hey, just hire great people, and everything will work out.”
We have consistent processes embedded with automation. As much as possible, we automate everything. If you look at our CI/CD pipeline for developing our unified fast file and object platform, FlashBlade, we’ve automated test failure root cause analysis. That’s where in a lot of organizations, they have large dedicated teams of QA engineers working on that failure identification task. You’ll often see 20-30% of a platform’s development workforce devoted to QA and specifically triaging test failures. But we’ve automated the testing and failure analysis in our CI/CD pipeline, which allows us to consistently test throughout development and get a fast result on failure and get the reason for the failure to the developer that just wrote that code, often within seconds. And that ability to discover failure is essential as you want to be able to fix whatever you messed up while it’s fresh in your mind, versus somebody comes to you three weeks later and says, “Hey, there was a regression found here. Do you remember what you wrote three weeks ago?”
The other part of product development is testing in the field, getting empirical results on product reliability from your install base. That’s something we’re also maniacal about to the point that we measure and issue public statements on it, based upon our aggregate statistics. One of them is that our customers receive over seven 9’s of uptime across their installed base. We’re the only storage company that has released a public statement disclosing that high of a level of availability across their entire install base. That’s the ultimate metric of quality.
Pure works with many independent software vendors to bring new integrated offerings to the market. What are some examples of innovative solutions that resulted from those partnerships?
We have platforms largely at the infrastructure layer, and as such, we get to see the fascinating uses that our customers in automotive, healthcare, and other industries build upon it. Those are things that our customers built on top of us – they took our unique functionality that lets them get to their business outcomes faster.
An example of an intentional partnership we had was with Nvidia. We’ve developed a product called AIRI, the first AI converged infrastructure platform brought to market. Partnering with Nvidia made sense; by combining their obvious leadership in the AI space with their GPU Silicon with our FlashBlade platform, it removed all of the traditional bottlenecks of scaling on-premises AI platforms. With that strong partnership, we released a product to market together into a category that did not exist, AI Converged Infrastructure, until we came together to create it.
Cloud providers offer new and more efficient data-centric services each year. Do you consider it as a threat, or maybe you see some growth opportunities?
At Pure, we see it as a huge opportunity. If you look at our existing on-premises market share, it’s around 5% versus 15%, 20%, or 30% of some of our competitors. Simultaneously, we are the fastest-growing company in the data storage field today by capturing an additional percent or two of the storage market share every year, while most of the other major vendors are flat or in decline.
We’re increasing our R&D spend to build partnerships with organizations such as Amazon AWS and Google to help deliver their cloud-like experiences to on-premises. We have also taken the software that runs our storage in our on-premises appliances and made that run in AWS and Azure natively. So it makes it easier for customers to move non-cloud native apps to the cloud.
Many customers have traditionally had challenges around monolithic apps that have been more difficult or even incompatible to run in the cloud in terms of availability and performance. We took our existing software and extended it to the cloud to make cloud storage more robust, allowing many traditional apps that couldn’t run there before to run well there.
Also, there’s the data gravity problem. We have a single global data fabric that can make data more mobile. And so, the classic problem customers have in terms of “I want to get it up there, but how do I get it up there? We can accelerate that for them significantly. And that’s a huge opportunity for us.
And, of course, or acquisition of Portworx. Kubernetes is increasingly becoming the operating system of the cloud, and no one can deny the rise of Kubernetes at this point. But Kubernetes only orchestrates the apps – it does not manage data and storage. That’s what our Portworx platform does today, and multiple analysts have positioned Portworx as the leader in this space. As cloud and Kubernetes adoption continues at this rapid pace, that’s a really exciting place for us to be here at Pure Storage.
Nathan, thank you for sharing your grand vision on the innovative approaches to data storage.
Stay tuned for the next interviews!