Storage scheme
The S1 system is a general-purpose single storage engine that can manage all storage resources for all use cases. It combines cutting-edge storage technology with maximum flexibility, providing customized storage options at the logical volume level. S1 has its own storage scheme that may differ from other storage systems. The following provides a quick description of S1 storage scheme components.
Pools
Physical drives are located inside your disk enclosure and cannot be accessed directly. To use these drives, you need to create a pool and specify which drives the pool can use.
Application instance
An application instance is simply a namespace for logical volumes. Any volume has to be mapped to one (and only one) application instance. However, an application instance may contain more than one logical volume.
Logical volumes
A logical volume is a virtual storage drive. It allows you to create multiple virtual partitions on a single physical device or to create a single large volume that spans multiple physical devices. The entire logical volume can be managed as a single entity. This is useful for a variety of purposes, such as creating separate partitions on hard drives for different purposes (for example, one for the operating system, one for application files, and one for user data).
Logical volumes are thin provisioned. Thin provisioning of a logical volume allows you to allocate storage as needed, rather than being fully allocated up front.
A logical volume is a fundamental storage unit of the S1 system. When creating a new logical volume, you must specify the pools that the logical volume contains. If the logical volume has run out of space, you can increase the storage capacity by approving additional drives for its pools and editing the volume to utilize the additional storage.
Consistency groups
A consistency group is a basic logical storage entity that consists of one or more logical volumes. Every volume must be mapped to a consistency group. This mapping is not one to one, hence multiple volumes may be mapped to the same consistency group. When you create a logical volume, you can map it to an existing consistency group. If you do not map your newly created volume to an existing consistency group, then S1 system creates a consistency group automatically.
How to create a consistency group
To create a new consistency group, you need to create a logical volume that is not already a part of an existing consistency group. This automatically creates the consistency group and adds the logical volume to it. There is no separate command for creating consistency groups.
Why we need consistency groups
There are several reasons why it may be useful to group volumes together into consistency groups.
-
Data consistency: By grouping volumes together into a consistency group, you can ensure that the data on all the volumes is consistent with each other. This is especially important in applications where multiple volumes may be accessed and modified simultaneously. For example, if a database is spread across multiple volumes, it is essential that the data within the database is consistent across all volumes. This is because the data in the database represents a single logical view of the information and any inconsistencies between the data on different volumes could result in errors or data loss.
-
Data protection: Consistency groups can help protect against data loss or corruption by ensuring that data is consistently written to multiple volumes. If one volume in the consistency group fails or becomes corrupted, the data can be recovered from the other volumes in the group.
-
Management: Consistency groups can also make it easier to manage volumes by allowing you to apply policies, such as replication and snapshot policies, to multiple volumes at once.
Filesystem
A volume is simply a storage space, similar to a logical volume on your local machine. To use the storage space within these volumes, you must create a filesystem on them, just as you would need to partition and create a filesystem (such as NTFS, fat32, ext2, or ext4) on a physical disk in order to use it.
There are two options for formatting a filesystem on your volumes:
- Using the S1 system: This involves using either the command-line interface or a graphical-user interface to create a new filesystem.
- Mapping your volume: This option involves connecting your volume to a remote machine via a storage-area network (SAN). This allows the remote machine to access the disk as if it were a local disk and format it in the same way it would format a local disk.
Storage architecture
File storage architecture may divide into three types.
Object storage architecture
Object storage architecture stores data as individual units called objects. Each object contains all of the data associated with it and is assigned a unique identifier. The objects are stored in a flat address space, rather than in a hierarchical file system.
When to use object storage architecture
Object storage is a type of data storage architecture that is designed for storing and accessing large volumes of data, such as video, audio, documents, and other types of binary data. It is a good choice for storing data that does not need to be accessed frequently and does not require the low latency or high performance of a traditional block or file storage system.
Some common use cases for object storage include:
- Backup and archiving: Object storage can be used to store large amounts of data that need to be retained for a long period of time, such as backups and archives.
- Media and entertainment: Object storage is often used to store and distribute media content, such as videos, music, and images.
- Scientific research: Object storage can be used to store and manage large volumes of data generated by scientific research projects.
- Cloud storage: Object storage is often used as the underlying storage layer for cloud-based applications and services.
Object storage is well-suited where data durability and scalability are important considerations.
File storage architecture
File storage architecture stores files hierarchically, with folders and files arranged in a tree-like structure. Each file has a unique name and is stored in a specific location within the file system.
When to use file storage architecture
File storage is typically accessed over a network using a protocol such as Network File System (NFS) or Server Message Block (SMB).
Some common use cases for file storage include:
- Collaboration and sharing: File storage can be used to store and share files with team members or other users, allowing multiple people to access and edit the same files.
- File servers: File storage can be used to store and manage files on a centralized server, providing a single point of access for users.
- Network-attached storage (NAS): File storage can be used to store and access files on a device that is attached to a NAS.
- Home directories: Home directories are commonly used in organizations to provide each user with a designated space to store their personal files, making it easier for users to access and share their files with others. File storage can be used to store and manage personal files, such as documents, photos, and music.
In general, file storage is a good choice for frequent access data that requires low latency. File storage is a suitable solution for scenarios that prioritize data organization and access control, particularly when multiple users require access to and editing capabilities for the same files.
Block storage architecture
Block storage architecture organizes data into fixed-size blocks, typically ranging in size from 512 bytes to 64 kilobytes. Each block is assigned a unique identifier and can be accessed individually.
Block storage is typically used to store data that needs to be accessed quickly, such as operating system files and application data. It is often used in conjunction with file storage systems, with the file storage system providing a logical organization of the data and the block storage system providing fast access to the actual data blocks.
One of the main benefits of block storage is its high performance, as data can be accessed quickly and directly. However, a block storage system (SAN) can be more expensive to implement and maintain compared to other storage architectures.
When to use block storage architecture
Block storage is a type of data storage architecture that is designed for storing and accessing data at the block level, rather than as files or objects. It is typically used to store data that requires fast, low-latency access, such as databases, virtual machines (VM) images, and other types of transactional data.
Some common use cases for block storage include:
- Databases: Block storage is often used to store and access data for transactional databases, such as MySQL, Oracle, and Microsoft SQL Server.
- Virtual machine (VM) images: Block storage is often used to store VM images, which are used to create and run virtual machines in a cloud or on-premise environment.
- Applications with high performance requirements: Block storage can be used to store data for applications that require fast, low-latency access to data, such as financial trading systems and online gaming platforms.
- Cloud storage: Block storage is often used as the underlying storage layer for cloud-based applications and services.
In general, block storage is a good choice for storing and accessing data that requires fast, low-latency access, and is well-suited for use cases where data performance and scalability are more important considerations than cost.