cerbero package performance on android-universal: bzip2
During the package
step on android bzip2 is used on a 1.8GB input tarfile/stream. This pegs a single core for multiple minutes.
I wonder how difficult it would be to use another bzip2 compression tool like lbzip2
here, which can make use of all cores, and still retains the same compression ratio:
- on
gst-gitlab-htz-runner1
(32 vcpus):bzip2 -9
2m5s vs.lbzip2 -9
6.7s (on 1.8GBgstreamer-1.0-android-universal-1.15.2.tar
) - on
gst-gitlab-htz-runner2
(48 cores):bzip2 -9
2m21s vs.lbzip2 -9
3.7s (on 1.8GBgstreamer-1.0-android-universal-1.15.2.tar
)
Or maybe we should move to another compression like xz
. xz --threads=48 -9 gstreamer-1.0-android-universal-1.15.2.tar
takes just as long as the single-threaded bzip2
(ca. 2mins), but compresses better (364MB vs. 558MB). xz
doesn't seem to want to make use of more than 8 cores though.
Not sure if it's worth it, just filing a bug for discussion.