Leading Industry with Advanced Memory Solutions
Today we’re delighted to welcome Young Paik to the show. Young is a Senior Director of Product Planning at Samsung Electronics, where he leads the development of advanced memory products.
In this interview, we talked about the way high performant memory improves database performance, the challenge of scaling computer memory innovations, the importance of the industry partnerships, and the business post-COVID recovery.
Watch and Listen to the full interview episode.
Young, can you talk about your work and what makes it successful and what makes it hard?
My work personally draws me into AI/ML systems. I have been in data analytics for many years, and a severe limitation was that while the data was there, it was just too hard to go through it in a meaningful way. Too often it took too long to process it. Now, new techniques such as Deep Learning and new hardware such as GPUs are opening exciting new avenues for processing that data. We are finding that many companies are now successfully deploying AI/ML projects and finally making use of all the data they have been collecting over many years.
The challenge is still there, because even if you find a solution that can work at a small scale, moving to a large scale is non-trivial. An experiment that works fantastically with a single GPU and local SSDs does not easily translate over many GPUs and larger storage systems.
Young Paik
Senior Director of Product Planning at Samsung Electronics
At Samsung, you’re working on advanced memory products. How Samsung’s high performant memory can improve database performance, both SQL and NoSQL?
SQL and NoSQL databases have some important differences. The key problem with SQL databases is the need for ACID properties that often need locks and logs to ensure you don’t lose data. These methods often limited performance and meant that memory was not the main bottleneck for many of these systems.
In recent years, new techniques for dealing with such things have appeared. And NoSQL databases are one such response. For each use case, it is up to the architect to determine what trade-offs make sense. So now, we are looking at database servers where memory bandwidth is much more important.
There are a few different technologies that can help improve performance:
· DDR5 – This is the next version of standard memory on servers today in the DIMM form factor. We will see a doubling of memory bandwidth. Coupled with faster CPUs and higher capacities, we will see increases in performance and capacity.
· HBM/GDDR – There are new database companies that can run SQL queries on GPU. They make great use of the parallel processing of the GPUs and the huge memory bandwidth to process queries much faster than on the CPU. Often the limitation here is how fast you can get the data into the GPUs.
· SmartSSD – Another way to increase query speed is to move some data processing closer to the storage. Samsung has been working on the SmartSSD, allowing the SSD to process some of the data and allow for much better parallel processing in a shorter period.
What sort of AI workloads can be done more efficiently with the latest Samsung memory products?
Samsung products are not directly used in AI/ML today. Instead, they are a major constituent of products like GPUs. Most AI/ML workloads can use Samsung memory in some form.
We have found that the most popular workload for our customers are:
· For Deep Learning workloads, the memory bandwidth is the most expedient way to increase speed. We are working with many different companies that make AI/ML products to make the most efficient memory use in HBM or GDDR. Deep Learning workloads are typically used for computer vision and NLP. Note that this is biased toward larger-scale problems since these are the ones I most often see.
· Currently, many companies are making ASICS that will use Samsung memory (especially HBM) that will be more efficient for specific models. We are very interested in seeing how well these do in the marketplace.
In addition, one of the problems I mentioned before was the scaling problem. We are finding more and more use cases where companies doing AI/ML projects at scale realise they need much faster, much larger storage. So one of the activities that keep me most busy is working on the storage solutions for these customers and partners. In general, these storage systems do not fit comfortably in the same usage model as traditional storage systems. Their requirements often must be tailored to not only the type of ML but the IT environment of the customer.
With the computer memory field innovations, do you think we’ve achieved the physical limits of what is possible? Do further advancements require radically new approaches?
It is hard to say if we have reached the physical limits. Every time we think we are close, some breakthrough allows for better scaling and performance. At Samsung, we are doing everything to find better ways to create memory products.
I noticed you’re looking for partners to build new memory products. What is the added value of partners for the products you create? And what are the benefits for partners to work with Samsung?
Perspective is always critical in this industry. Anyone who makes a product without acquiring enough different perspectives is at risk of creating a product that will not suit the market. In working with partners, we at Samsung, understand whether we are making the correct product for the right market. Another company gains a partner that understands how to deal with the solution at scale. If you are working on niche products, Samsung will likely not be interested in the partnership. But if you want a product that will significantly impact the industry, then Samsung is an excellent partner.
How do you think the pandemic has affected corporations’ ability to innovate longer-term?
The pandemic has entered a fair amount of uncertainty into the marketplace. When the world went into lockdown, most people expected businesses to close down. However, we saw shifts in the market – while some parts of the market did reduce their orders, some of the customers had to ramp up orders.
As we look to a post-COVID recovery, it is hard to say what the impact will be. Keep in mind this is coinciding with a shortage in the semiconductor supply, as well. However, I believe this is an excellent time for innovations in business – AI/ML and advanced analytic technology are maturing, and the ability to deploy these solutions is not just a differentiator but a powerful weapon. Companies that can more efficiently operate today and predict the future will have a massive advantage over their competitors.
What type of individuals are you looking to bring into your company to establish the right culture and foundation for the next five years?
One of the typical considerations for people nowadays is adaptability. Technology is changing very fast, so keeping up with that is a must. However, Samsung also needs people who can think about how these changes will impact the marketplace at a large scale and over time. As a small company, making products that make $100M/year is a significant achievement. For Samsung, this is a minor fluctuation.