starlingx/kernel
Code Issues Proposed changes

Debian: kernel: Parallelize xz archive compression

Browse Source 
Without this commit, a 12-CPU build VM used by the author takes between
9 and 10 minutes to compress the standard/non-preempt-rt kernel's
linux-source-5.10.tar.xz source archive.

This commit parallelizes the compression operation, and this reduces the
compression time to about 1 minute, saving about 8 minutes during the
build procedure.

During the preparation of this commit, reduction of the compression
ratio was also experimented with, but was not opted for due to the
increased need for disk space.

Some of the measurements that were acquired with the same 12-CPU build
VM can be seen in the table below:

			Archive size	Compression time
			(Bytes)
baseline		120687336	9~10 min
--threads=0		"		~55 sec
--threads=0 -3		142357736	~21 sec
--threads=0 -0		174808508	~ 7 sec

Verification
- "build-pkgs -c -b rt,std -p linux,linux-rt" rebuilds the kernel
  successfully, but takes a shorter duration of time.
- This commit has been in the author's local repository for more than a
  month, and no issues were observed during the kernel rebuilds carried
  out during the author's day-to-day work.

Closes-Bug: 1988442
Change-Id: Ib7c701f3f690d6120dc5aca1d0d42862151a2fc0
Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
This commit is contained in:
M. Vefa Bicakci
2022-07-29 19:11:55 +00:00
parent b68f1aabab
commit da523dc2d0
4 changed files with 70 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
kernel-rt/debian/deb_patches/0017-debian-rules.real-Parallelize-xz-compression.patch Normal file
View File

@@ -0,0 +1,34 @@
From 90afea5c8af57d90c0db6ff4330e5d9a5f120648 Mon Sep 17 00:00:00 2001
From: "M. Vefa Bicakci" <vefa.bicakci@windriver.com>
Date: Fri, 29 Jul 2022 19:10:11 +0000
Subject: [PATCH 17/18] debian/rules.real: Parallelize xz compression
This commit enables the multi-threading feature of the xz program so
that it uses all of the available CPUs for the compression of the kernel
source archive.
In a 12-core virtual machine, this reduces the compression time from
about 9 minutes to a little less than 1 minute.
Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
---
debian/rules.real | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/debian/rules.real b/debian/rules.real
index ffd75e75fc6e..2e6f636f70af 100644
--- a/debian/rules.real
+++ b/debian/rules.real
@@ -134,7 +134,8 @@ $(BUILD_DIR)/$(SOURCE_BASENAME)-source-$(UPSTREAMVERSION).tar.xz: $(STAMPS_DIR)/
-print0 | \
sort -z | \
tar --owner=root --group=root --numeric-owner \
- --no-recursion --null -T - -caf '$(SOURCE_BASENAME)-source-$(UPSTREAMVERSION).tar.xz'
+ --no-recursion --null -T - -I 'xz --threads=0' \
+ -cf '$(SOURCE_BASENAME)-source-$(UPSTREAMVERSION).tar.xz'
rm -rf '$(DIR)'
$(BUILD_DIR)/linux-patch-$(UPSTREAMVERSION)-%.patch.xz: $(STAMPS_DIR)/source_none $(STAMPS_DIR)/source_%
--
2.25.1

1
kernel-rt/debian/deb_patches/series
View File

@@ -14,3 +14,4 @@
0014-kernel-rt-fix-proc-version-content.patch
0015-kernel-rt-clean-patches-from-debian-release.patch
0016-Debian-Added-apparmor-security-module.patch
0017-debian-rules.real-Parallelize-xz-compression.patch

34
kernel-std/debian/deb_patches/0016-debian-rules.real-Parallelize-xz-compression.patch Normal file
View File

@@ -0,0 +1,34 @@
From 3fc9ffca7fe8d12ff3ad74b879bc30f8b205cbc0 Mon Sep 17 00:00:00 2001
From: "M. Vefa Bicakci" <vefa.bicakci@windriver.com>
Date: Fri, 29 Jul 2022 19:10:11 +0000
Subject: [PATCH 16/17] debian/rules.real: Parallelize xz compression
This commit enables the multi-threading feature of the xz program so
that it uses all of the available CPUs for the compression of the kernel
source archive.
In a 12-core virtual machine, this reduces the compression time from
about 9 minutes to a little less than 1 minute.
Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
---
debian/rules.real | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/debian/rules.real b/debian/rules.real
index ffd75e75fc6e..2e6f636f70af 100644
--- a/debian/rules.real
+++ b/debian/rules.real
@@ -134,7 +134,8 @@ $(BUILD_DIR)/$(SOURCE_BASENAME)-source-$(UPSTREAMVERSION).tar.xz: $(STAMPS_DIR)/
-print0 | \
sort -z | \
tar --owner=root --group=root --numeric-owner \
- --no-recursion --null -T - -caf '$(SOURCE_BASENAME)-source-$(UPSTREAMVERSION).tar.xz'
+ --no-recursion --null -T - -I 'xz --threads=0' \
+ -cf '$(SOURCE_BASENAME)-source-$(UPSTREAMVERSION).tar.xz'
rm -rf '$(DIR)'
$(BUILD_DIR)/linux-patch-$(UPSTREAMVERSION)-%.patch.xz: $(STAMPS_DIR)/source_none $(STAMPS_DIR)/source_%
--
2.25.1

1
kernel-std/debian/deb_patches/series
View File

@@ -13,3 +13,4 @@
0013-kernel-std-fix-proc-version-content.patch
0014-kernel-std-clean-patches-from-debian-release.patch
0015-Debian-Added-apparmor-security-module.patch
0016-debian-rules.real-Parallelize-xz-compression.patch