Storage: HDD,SSD,file systems and how data is stored

While receiving an alert from Google cloud monitoring as disk usage:98% or no space left on the device. The application suddenly crashes, users cannot upload files, logs stop being written and also the database stops working. As a cloud engineer, we need to understand where the files are stored, why disks become full, how storage affects performance, when to increase disk size, how cloud storage differs from local storage. Storage is one of the most common causes of production issues.

What is storage?

Storage is the place where data is kept permanently. Unlike the RAM, which is volatile memory. It tempoerarily holds data and programs while they are in use.

🌟 Examples: windows,ubuntu,photos,videos,PDFs, applications, databases. Everything is stored on a storage device.

Types of storage?

There are three major types you'll encounter.

🔷 HDD(Hard disk drive): An HDD stores data using spinning magnetic disks. Inside an HDD we can see platters, Read/write heads, motor. The read/write head moves across the spinning disk to read or write data. It is very cheap and have large capacity, but it is slow, mechanical parts can wear out and more sensitive to shocks

🔷SSD (solid state drive): A SSD has no moving parts. It stores data in flash memory chips. It is much faster with lower power consumption and more resistant to physical shock but it's higher cost per GB than HDDs. Modern laptops and cloud platforms primarily use SSDs.

🔷NVMe SSD: NVMe SSDs also use flash memory but communicate over the PCIe bus, allowing much higher throughput and lower latency than SATA SSDs.

How files are stored?

When your saving an image as 'batman.jpg'. You don't choose where each byte is written. The file system decides where the data is placed on the storage device. The operating system keeps track of file name, location, size, permissions, timestamps.

What is a file system?

A file system organizes data on a storage device. Without one, the operating system wouldn't know where files begin or end.

common file systems

| Operating System | File System     |
| ---------------- | --------------- |
| Windows          | NTFS            |
| Linux            | ext4 (commonly) |
| USB Drives       | FAT32 or exFAT  |

cloud engineers often work with NTFS(windows VMs) and ext4(Linux VMs)

common file systems

| Operating System | File System     |
| ---------------- | --------------- |
| Windows          | NTFS            |
| Linux            | ext4 (commonly) |
| USB Drives       | FAT32 or exFAT  |

cloud engineers often work with NTFS(windows VMs) and ext4(Linux VMs)

What is a partition?

A partition divides one physical storage device into separate logical sections.

Example:
500 GB SSD

C: Drive → 250 GB

D: Drive → 250 GB

windows treats each partition like a separate drive.

Example:
500 GB SSD

C: Drive → 250 GB

D: Drive → 250 GB

windows treats each partition like a separate drive.

Boot disk: The boot disk contains operating system, bootloader, system files. Without a valid boot disk, the computer cannot start. In Google cloud, every compute engine VM needs a boot disk.

Data disk: A computer can also have additional disks for storing application data.

Disk 1:

Windows
   |
Boot Disk

Disk 2:

Project Files
    |
Data Disk

In cloud environments, spreading the operating system from application data
is a common practice.

Disk 1:

Windows
   |
Boot Disk

Disk 2:

Project Files
    |
Data Disk

In cloud environments, spreading the operating system from application data
is a common practice.

Read and write operations

Two fundamental storage operations are:

Read: copy data from storage into memory, like for example opening a pdf.

Write: Save data to storage, like for example saving a word document. Everytime you save a file, a write operation occurs.

IOPS(Input/output operations per second)

It measures how many read/write operations a storage device can perform each second. Higher IOPS generally means better perforamance for workloads with many small storage operations such as dataases.

Storage in Google cloud

When you create a compute engine VM, you select a boot disk.

Ubuntu 24.04

Boot Disk

50 GB

Ubuntu 24.04

Boot Disk

50 GB

Google cloud also allows you to attach additional persistent disks for application data.

Checkout the previous blog

Happy Learning!!!😊

🔰 If you like the article plz..do clap(click on the clap option below)😃 it would be a support 😁and also do share it 😊.

Contents