How to Improve The Performance of Data Storage Systems in The Data Center
According to Cisco, by 2019, one user will generate 1.6 GB of data on cloud storage per month. Increasing the volume of information is a major factor in the growth of the IT market and at the same time a serious challenge for the infrastructure. Therefore, in this article, we’ll see how to improve the performance and efficiency of SSD and HDD in the data center.
What affects the performance of HDD and SSD in data centers
Performance problems related to storage are limited by the size of the data blocks, regardless of the storage architecture – be it traditional infrastructure or distributed storage. For example, operations with the same amount of data when working with blocks require different amounts of resources: writing 100 bytes into one block or several takes different times.
Another factor is how the system accesses the files and how they are stored on the disk. Procedures for locating information lead to processing delays in the case of HDD. At the same time, large data, over the size of a petabyte, is converted into a huge amount of metadata, which not every file system will cope with.
- Also, the physical limitations of technology affect the performance of storage systems. There is an opinion that with the growth of performance, the speed of development of HDD parameters does not follow Moore’s law. SSDs are developing predictably: the capacity along with the capacity is growing, as the industry expects.
However, the HDD still has room for maneuver – work is underway to reduce the size of discs and increase the recording density, aimed at increasing the efficiency of storage devices. For example, only in early October this year, WD company introduced a new technology of microwave magnetic recording (MAMR), which, according to the calculations of the developers, will create 40 terabytes HDD.
How to improve the performance of storage systems
Identify bottlenecks
Valdis Filks, research director at Gartner Inc., compares the search for a “leak” of performance in the storage system with a treasure hunt. And notes that this requires special tools. Today, the industry offers both built-in systems for assessing the health of storage arrays, as well as third-party software for managing storage resources (SRM) and monitoring.
For example, Dell EMC offers built-in tools for managing storage resources and reporting. They allow you to generate statistics on the capacity, configuration and performance of the arrays used. Note that similar solutions are provided by IaaS-providers if we are talking about monitoring storage performance in a virtual infrastructure.
Optimize data
Increase the performance of storage is possible if you correctly build the drives into pools. One option is to combine disks with low performance, which store the bulk of the data and use dynamic memory allocation and deduplication.
There are a lot of modern solutions for these functions. In addition, they provide protection for data and virtual servers. One example is the ClearSky Data solution, which eliminates duplicate data and offers backup and recovery capabilities after a disaster.
- The idea of the project is as follows: the cache with business-critical information is stored on the client’s site, the “warm” data is placed in the local cloud, and the “cold” data is taken out to the outer cloud. The company has developed software that automatically manages data by moving it between storage points. Algorithms analyze the intensity of the use of information and the requirements of the customer.
Another project is a start-up Datrium. The idea of developers is that deduplication is performed twice: on the local flash memory before sending to the long-term storage device, and after it. You can read about other intelligent deduplication and backup systems in one of our materials.
Optimize your cache
If the amount of data used by the system is too large for the cache, the performance may be significantly reduced. The virtualization of storage helps to solve the problem. Virtualization of mount points, in which the storage system communicates with the server and the OS, allows you to use DRAM to create a shared cache.
Subsequently, it can be used to speed up the operation of applications and perform buffering. Experts from the analytical company StorageSwiss say that this approach shows itself well when working with real-time transaction processing systems.
Optimize network performance
Network upgrades help when performance suffers from disadvantages at the level of the host bus adapter, network switch or cable infrastructure. Cable connections are often overlooked as a factor in reducing the efficiency of data storage.
- Another reason is ineffective traffic management. If the network has limited bandwidth, then part of the channel can be assigned to critical services, “decreased” the work of secondary (say, e-mail).
To improve the quality of traffic management, experts of the consulting group Ashton Metzler & Associates Inc. advise to monitor the network and prepare reports in order to change the model of data distribution in a timely manner.
As noted by the staff of Viavi Solutions, one of the most effective ways of traffic management is a hierarchical quality of service (H-QoS) system. It combines network policies and filtering policies. H-QoS reduces latency, without requiring an increase in network bandwidth.
Use storage features
Although SSD and HDD perform the same function, they are based on different technologies. Therefore, these devices require a different approach. For example, in the case of SSD, fragmentation does not affect performance in any way. This means that defragmentation does not give any advantages in the optimization issue. Therefore, the team PC Magazine Labs notes that sometimes it is useful to turn off this function for solid-state drives.
Also note that the internal implementation of a number of operations, for example, file deletion, in SSD and HDD differs. And the performance of some standard functions on the SSD (for example, formatting) even leads to a deterioration in their performance. And to counter this effect, it is worth using the TRIM command, which reduces the effect of “garbage collection” on the SSD device.
There are situations when TRIM is not supported by the OS. For such cases, special tools have been developed that make it possible to send the command “manually”. These utilities form a list of free blocks and transmit it to the disk as a sequence of TRIM commands. In this case, there are both universal tools (for example, hdparm), and specialized software for discs of individual manufacturers (Intel SSD Toolbox).
