https://lwn.net/ml/all/cover.1747070147.git.anand.jain@oracle.com/ LWN.net Logo LWN .net News from the source LWN * Content + Weekly Edition + Archives + Search + Kernel + Security + Events calendar + Unread comments + ------------------------------------------------------------- + LWN FAQ + Write for us User: [ ] Password: [ ] [Log in] | [Subscribe] | [Register] Subscribe / Log in / New account [PATCH RFC 00/10] btrfs: new performance-based chunk allocation using device roles Thread information [Search the all archive] Anand Jain [this message] ` [PATCH 01/10] btrfs: fix thresh scope in should_alloc_chunk() Anand Jain ` [PATCH 02/10] btrfs: refactor should_alloc_chunk() arg type Anand Jain ` [PATCH 03/10] btrfs: introduce btrfs_split_sysfs_arg() for argument parsing Anand Jain ` [PATCH 04/10] btrfs: introduce device allocation method Anand Jain ` [PATCH 05/10] btrfs: sysfs: show " Anand Jain ` [PATCH 06/10] btrfs: skip device sorting when only one device is present Anand Jain ` [PATCH 07/10] btrfs: refactor chunk allocation device handling to use list_head Anand Jain ` [PATCH 08/10] btrfs: introduce explicit device roles for block groups Anand Jain ` [PATCH 09/10] btrfs: introduce ROLE_THEN_SPACE device allocation method Anand Jain ` [PATCH 10/10] btrfs: pass device roles through device add ioctl Anand Jain ` [PATCH RFC 00/14] btrfs-progs: add support for device role-based chunk allocation Anand Jain ` [PATCH 01/14] btrfs-progs: minor spelling correction in the list-chunk help text Anand Jain ` [PATCH 02/14] btrfs-progs: refactor devid comparison function Anand Jain ` [PATCH 03/14] btrfs-progs: rename local dev_list to devices in btrfs_alloc_chunk Anand Jain ` [PATCH 04/14] btrfs-progs: mkfs: prepare to merge duplicate if-else blocks Anand Jain ` [PATCH 05/14] btrfs-progs: mkfs: eliminate duplicate code in if-else Anand Jain ` [PATCH 06/14] btrfs-progs: mkfs: refactor test_num_disk_vs_raid - split data and metadata Anand Jain ` [PATCH 07/14] btrfs-progs: mkfs: device argument handling with a list Anand Jain ` [PATCH 08/14] btrfs-progs: import device role handling from the kernel Anand Jain ` [PATCH 09/14] btrfs-progs: mkfs: introduce device roles in device paths Anand Jain ` [PATCH 10/14] btrfs-progs: sort devices by role before using them Anand Jain ` [PATCH 11/14] btrfs-progs: helper for the device role within dev_item::type Anand Jain ` [PATCH 12/14] btrfs-progs: mkfs: persist device roles to dev_item::type Anand Jain ` [PATCH 13/14] btrfs-progs: update device add ioctl with device type Anand Jain ` [PATCH 14/14] btrfs-progs: disable exclusive metadata/data device roles Anand Jain ` [PATCH RFC 00/14] btrfs-progs: add support for device role-based chunk allocation David Sterba ` [PATCH RFC 0/2] fstests: btrfs: add functional verification for device roles Anand Jain ` [PATCH 1/2] fstests: common/btrfs: add _require_btrfs_feature_device_roles Anand Jain ` [PATCH 2/2] fstests: btrfs/366: add test for device role-based chunk allocation Anand Jain ` [PATCH RFC 00/10] btrfs: new performance-based chunk allocation using device roles Forza ` Anand Jain ` Zygo Blaxell ` Anand Jain ` Zygo Blaxell ` waxhead ` Anand Jain ` waxhead ` Ferry Toth ` Anand Jain ` Ferry Toth ` Anand Jain ` Jani Partanen ` Anand Jain From: Anand Jain To: linux-btrfs-AT-vger.kernel.org Subject: [PATCH RFC 00/10] btrfs: new performance-based chunk allocation using device roles Date: Tue, 13 May 2025 02:07:06 +0800 Message-ID: In host hardware, devices can have different speeds. Generally, faster devices come with lesser capacity while slower devices come with larger capacity. A typical configuration would expect that: - A filesystem's read/write performance is evenly distributed on average across the entire filesystem. This is not achievable with the current allocation method because chunks are allocated based only on device free space. - Typically, faster devices are assigned to metadata chunk allocations while slower devices are assigned to data chunk allocations. Introducing Device Roles: Here I define 5 device roles in a specific order for metadata and in the reverse order for data: metadata_only, metadata, none, data, data_only. One or more devices may have the same role. The metadata and data roles indicate preference but not exclusivity for that role, whereas data_only and metadata_only are exclusive roles. Introducing Role-then-Space allocation method: Metadata allocation can happen on devices with the roles metadata_only, metadata, none, and data in that order. If multiple devices share a role, they are arranged based on device free space. Similarly, data allocation can happen on devices with the roles data_only, data, none, and metadata in that order. If multiple devices share a role, they are arranged based on device free space. Finding device speed automatically: Measuring device read/write latency for the allocaiton is not good idea, as the historical readings and may be misleading, as they could include iostat data from periods with issues that have since been fixed. Testing to determine relative latency and arranging in ascending order for metadata and descending for data is possible, but is better handled by an external tool that can still set device roles. On-Disk Format changes: The following items are defined but are unused on-disk format: btrfs_dev_item:: __le64 type; // unused __le64 start_offset; // unused __le32 dev_group; // unused __u8 seek_speed; // unused __u8 bandwidth; // unused The device roles is using the dev_item::type 8-bit field to store each device's role. Anand Jain (10): btrfs: fix thresh scope in should_alloc_chunk() btrfs: refactor should_alloc_chunk() arg type btrfs: introduce btrfs_split_sysfs_arg() for argument parsing btrfs: introduce device allocation method btrfs: sysfs: show device allocation method btrfs: skip device sorting when only one device is present btrfs: refactor chunk allocation device handling to use list_head btrfs: introduce explicit device roles for block groups btrfs: introduce ROLE_THEN_SPACE device allocation method btrfs: pass device roles through device add ioctl fs/btrfs/block-group.c | 11 +- fs/btrfs/ioctl.c | 12 +- fs/btrfs/sysfs.c | 130 ++++++++++++++++++++-- fs/btrfs/volumes.c | 242 +++++++++++++++++++++++++++++++++-------- fs/btrfs/volumes.h | 35 +++++- 5 files changed, 366 insertions(+), 64 deletions(-) -- 2.49.0 Copyright (c) 2025, Eklektix, Inc. Comments and public postings are copyrighted by their creators. Linux is a registered trademark of Linus Torvalds