“The ‘Internet of Things’ means sensors connected to the Internet and behaving in an Internet-like way by making open, ad hoc connections, sharing data freely and allowing unexpected applications, so computers can understand the world around them and become humanity’s nervous system.” Kevin Ashton (inventor of the term)
In our last article, we considered the potential of IoT – also called the “Internet of Everything” – to drive NAND sales and thus Micron (NASDAQ:MU) profits over the course of the next decade. This article will take a more granular look at the architecture and its intersection with memory and storage.
A recent address entitled “Computing of The Future” by IBM’s (NYSE:IBM) Jeffrey Welser, Director of Research at IBM Research Almaden, highlighted just how important the Internet of Things (hereafter IoT) will be in creating the data explosion at the edge. I encourage you to listen to the YouTube video, but here are some of the key points he made, all of which are relevant to the need for NAND (and other forms of NVM) at the edge:
“90% of the data collected in the last 10 years was never captured or analyzed – we just threw it away.”[…] we are creating data at twice the rate we are creating bandwidth. […and ] even if we wanted to we couldn’t move the data to the cloud.” “60% of valuable sensor data loses its value in milliseconds […]” “(…In 2017 for the first time in history) the amount of storage and computing power in our cell phones worldwide exceeds what exists in data centers.”
Given these trends, Welser’s conclusion is that the challenge for computing is to move computing to the data because scale and latency constraints will make it impossible to do anything else. If he is right, what is the implication for storage technology? He doesn’t spend time on this in his talk, but perhaps we can take a moment to explore it.
The first and most obvious takeaway from the trends Welser outlined above is that NAND is essential to the solution space. Spinning rust simply doesn’t fit here. It simply cannot handle the latency requirements of an environment in which large streams of machine data are being generated. Does this mean that NAND is necessarily the only alternative? No, certainly not. Critics of the NAND rightly point out that NAND isn’t a great choice for real time environments. They are certainly correct about that. Writing to a NAND device is much slower than reading from it. Most importantly, due to constraints of the media, writing speed is non-deterministic and power hungry. This makes NAND less than ideal for a real-time sensor environment where system performance in response to inputs must be precisely executed and where power budgets are strictly rationed.
One of the current entrants in the market, for example, Everspin Technologies (NASDAQ:MRAM), is selling its MRAM product into real-time embedded applications right now. IoT sensor devices will undoubtedly contain diverse types of memory that is best suited to their function. But what of the enormous amount of data that these edge sensors generate? This is where NAND comes in. The point is that while NAND isn’t the perfect memory it will still be the workhorse in terms of bits and dollars because of its low cost and density advantage. Indeed, the new Persistent Memory computing architecture that enables IoT will incorporate a wide variety of memory types across the entire solution domain, including 3D XPoint, MRAM, ReRAM, and others.
So, where specifically will NAND be used in the emerging IoT architecture? Here’s a slide that gives us a good picture.
(Source: Hewlett Packard Enterprise, LLC)
NAND will populate stages 2 and 3 where dense, relatively low power, and inexpensive memory is needed to process, collect, archive, and analyze the sensor data. Consider, for a moment, the case of an aircraft management system and the myriad of sensors attached to the engines and the airframe. Data flows could be immense – GB/sec requirements are not uncommon in industrial environments – and the stage 2 Internet gateway and data acquisition systems must be able to collect this data and preprocess it. Even an hour of data in such an environment can aggregate into petabytes of storage when inputs from the engines in combination with the entire airframe are considered.
And, that is just a medium-sized example. Indeed, it is difficult to grasp the sheer magnitude of some of the largest projects underway. The largest of all may be the Square Kilometer Array, which is basically a software-defined telescope being constructed during the last part of this decade. It is being built in South Africa and Australia and is scheduled to begin service in 2021. It is projected that just in its first phase the telescope will produce some 160 terabytes of raw data per second that will need to be stored and analyzed by exascale class supercomputers. That’s the equivalent of more than 10 Exabytes (EB)/day of data, all of which needs to be processed and distributed. When fully built out in 2028, the quantity of data generated daily is projected to be 200 EB. Massively large local storage will be a fundamental requirement of the architecture. It is obvious that any computing architecture capable of processing this data stream must focus immense computing and memory capabilities at the edge.
Let’s take a moment to explore the practical issues that drive edge computing with the example we have presented above. Remember our IBMer’s statement above that data is growing twice as fast as bandwidth? So, how big is just one of those 10 exabytes that the SKA will generate in 2020? (1 EB is 1 billion TB.)
Here’s a shorthand way of looking at the problem of moving that much data. Let’s say we want to send 1,000 terabytes (TB) of data over a 1 Gbps link. That would take over 80 days. We’re still a long way from our Exabyte, so let’s bump the data to 10,000 TB and the network to a 10Gbps link. That would take over three months. You no doubt see where this is going. There is simply no way that any kind of centralized cloud architecture can accommodate all the but most trivial data requirements on the edge. Will 5G save us? Sorry, no – current specs are for 5G upload bandwidth to reach 10Gbps per mobile base station, a total bandwidth that will be shared with potentially 1 million connected devices. Computing and analytics simply have to come to the edge, and that implies gateways and analytics servers that will have to accommodate large volumes of data that will need to be analyzed and reduced.
This conclusion is reflected in current studies of the physical limitations of the World Wide Web in terms of bandwidth and data traffic. Cisco currently estimates that total web traffic will increase from today’s 1.1 ZB figure to 2.3 ZB by 2020. Contrast this now with IDC’s latest estimate that by 2025 a total of 163 ZB of data will be generated annually. Clearly, gigantic applications like big science and big surveillance will overwhelm any architecture that is not focused on edge data capture and real-time processing and analytics. This is the realm of NAND flash.
Let’s consider one more case that will serve as a great example of the data torrent that IoT generates. Below, you’ll see a graphic from this Cisco study that addresses the data generated by the connected, “smart” city. (There are 53 SMSAs in the US with a population of over 1 million.)
Just one of the large set of applications that comprise the “smart grid” is street lighting. GE (NYSE:GE) recently sold an integrated system to the City of San Diego that will reduce by 60% the city’s energy costs, thereby saving $2.4m/year. And, that’s just the beginning, according to the city’s press release:
The deployment of 3,200 smart sensors will be the largest city-based deployment of an “Internet of Things” platform in the world. […] The City will be installing smart nodes that can use real-time anonymous sensor data to do things such as direct drivers to open parking spaces, help first responders during emergencies, track carbon emissions and identify intersections that can be improved for pedestrians and cyclists. The information can be used to support San Diego’s “Vision Zero” strategy to eliminate traffic fatalities and severe injuries.
It should be clear that these applications are fiscally attractive and beneficial, with compelling ROIs. As the early innovators like the City of San Diego gain experience during the last few years of this decade, we can easily foresee why demand for all the components of IoT will grow exponentially through time. NAND flash is essential to the solution space, and demand for NAND will grow accordingly. Take a look at the graphic below, where Cisco’s consultants mapped the implementation challenges of IoT by use case.
Note how the “aggregation” level space expands for the use cases on the right-hand side. As use case complexity increases, more and more data is being generated that must be analyzed and preprocessed before the refined data can be sent to the data center. This area is the realm of NAND. Another key takeaway is that, even for the data that needs to be resident in the cloud, rapid (if not real time) data analysis must be done so that system-wide objectives can be obtained. Think, for example, of an autonomous car fleet application. The edge system in the car may have responded to an external event (say, a rock slide blocking a highway) taking the car out of service. The edge system would respond by stopping the vehicle, updating status information (including a dense forensic package that includes images of the area surrounding the vehicle, passenger data, vehicle system information, etc.) to the gateway system on the vehicle that would in turn perform the appropriate analytics and generate a relatively terse update package for the cloud. Based on the data and recommendations received from the gateway, the cloud-based fleet management system would promptly reconfigure and rebalance its fleet availability database so that customer service objectives could be met as best as possible.
Several questions arise, but the most pertinent is the following. What does this oncoming torrent of demand for NAND flash mean for the memory industry? Here are the top four takeaways.
- Machine-generated data is only just beginning to make an impact on the overall demand for storage. Because of the rapidly sloping character of the demand curve as implementations develop and mature over the balance of the current decade, the industry will be hard-pressed to build fabs soon enough to meet demand. Because of the two-year-long lead time to bring up a NAND fab, suppliers have a built-in bias to delay capacity additions until the shape and scope of demand are well understood. (Especially with the specter of Chinese supply looming.)
- IoT is in its infancy, and there remain several important technological underpinnings that need to be built. Most of these are sensor and edge networking related. For example, sensors need to be self-configuring and context-aware so that no implementation planning is required to deploy them. Work in this area is progressing, but no standards are currently in place. The current working consensus is that IoT implementations during the balance of this decade will evolve as standards are codified and instantiated. Translation – the IoT train has just left the station, and deployments will start accelerating rapidly toward the end of the decade.
- The smartphone is increasingly an IoT edge device. This is a direct function of diagnostic and performance applications that are collecting vital medical data from either on phone or connected devices like smart watches and performance bands. Exhibit one in this area could be the Apple (NASDAQ:AAPL) project that is developing a blood sugar sensor for diabetes patients.
- Industrial deployments have been and will continue to drive the early adoption of IoT because of the compelling return on investment cases that are being made. Right now, most of these use cases are implemented with sensor-direct-to-cloud architectures and are thus driving data center demand rather than edge. This will change as IoT standards mature and the applications evolve. By 2022, there are estimates that this market will be worth close to $200B alone.
The bottom line? The flood of IoT data will drive the memory business well into the decade of the 2020s. NAND suppliers will be the single biggest beneficiaries of emerging IoT wave because the wave as it crests leaves data in its wake. As we documented, the scale of this so-called edge data is several multiples of data center data – Cisco/IDC has estimated 5x. For at least the next 10 years, applications will be coming online that will result in large increments of NAND demand that are not currently being projected by many industry analysts. At the very least, this massive demand will push the industry to the very limit of its ability to find enough capex to build the needed fabs (with or without China). The most probable scenario in my models is that the industry is entering a period – possibly by next year and certainly by 2019 – where NAND is chronically undersupplied, with consequences that are hard to fully comprehend at this point. One thing for sure – pricing will be strong. Here’s what my models show.
Just to be clear, the 2020 “IoT demand” is roughly 80 EB in excess of production, which is roughly 100k wpm of 3D Gen 4 production. That’s two fabs. What about the Chinese? Don’t count on the Chinese cavalry riding to the rescue. If they are producing viable 3D product at all, they will be at Gen 1 or Gen 2, producing product that is one-fourth as dense as state-of-the-art Gen 4 product. That’s 400k wpm for China to make up the difference. Not likely.
What about DRAM? Given the 3D XP Gen2 and Gen3 roadmap and the expected commercialization of other resistive memories, DRAM will not suffer for substitutes that will better serve the needs of an IoT device. (For example, the new memories are non-volatile.) Bottom line, IoT is not likely to drive incremental DRAM demand unless the new memories we expect to be coming online in the early 2020s don’t deliver as expected. Otherwise, I expect DRAM demand to start subsiding in the 2021 time frame as these new memories, led by 3D XPoint Gen2/3, take big bites out of what would formerly have been DRAM configurations. Here’s Micron’s positioning of these new memories from its last Analyst Day.
Note how this positioning envisions broad swathes of the market TAM for IM’s XP product line and for Micron’s “New Memory,” albeit at different ends of the market. As you can see, future generations of XP are both faster and less expensive than the still nascent ReRAM technologies we expect to see in the 2020s. Even so, Micron makes it clear in this slide that XP will never be a NAND replacement – it will remain too expensive. It will instead displace and augment DRAM with a cost midway between NAND and DRAM. The yet-to-be-announced “New Memory” comes in at a cost that slightly less than DRAM but is not quite as fast. Why use it? It’s persistent and, presumably, would be significantly more attractive for IoT applications because its power-loading coefficient should be much lower than DRAM.
Where does this leave Micron? In two words – growing and profitable. True, the company will have to manage the DRAM decline as the 2020s progress, but 3D XPoint and its upcoming Next Generation memory should more than rise in tandem with DRAM’s fall. With the big questions in terms of demand resolved by IoT, pricing power for the manufacturers will be strong. So, where is the risk in this scenario? In a word – execution. The central question that remains for Micron is execution. Specifically, when will we see the announcement and ramp of XP Gen 2? When will we see Micron raise the curtain on the “New Memory?” We don’t know the answer to those questions, and until we do and in so doing gain confidence in the market TAM available to these memories, the jury will remain out, however good the NAND business is.
Disclosure: I am/we are long MU, WDC.
I wrote this article myself, and it expresses my own opinions. I am not receiving compensation for it (other than from Seeking Alpha). I have no business relationship with any company whose stock is mentioned in this article.