Hard drive

Typical deep learning applications required large sets of data that is in 100s of GB. Since this data cannot be set in any RAM, there is an ongoing data pipeline is constructed. A deep learning application loads the mini-batch data from GPU RAM, which in turns keeps on reading data from CPU RAM, which loads data directly from the hard drive. Since GPU's have a larger number of cores and each of these cores have a mini-batch of their data, they constantly need to be reading large volumes of data from the disk to allow for high data parallelism.

For example, in AlexNet's Convolutional Neural Network (CNN) based model, roughly 300 MB of data needs to be read every second. This can often cripple the overall application performance. Hence, a solid state driver (SSD) is often the right choice for most deep learning application developers.