Date: 2023-03-15
(Duplicate alert: This article was also published for the company I work for, CIQ: https://ciq.co/blog/rocky-linux-security-and-CVE-tools/ . I thought I'd put it here as well because I've been really lazy, to put it bluntly, about publishing new things!)
We get many requests, both at CIQ and in the community at large, about Rocky Linux security practices. Especially when it comes to particular vulnerabilities (CVEs) on a particular installed system or set of systems. This page will serve as a kind of guide, with many links and resources to help answer those questions.
The Rocky Linux goal is to compile and match RHEL, version-for-version, for every package and release in the distro (3000+ packages). The Rocky release schedule is therefore the same as RHEL. Rocky releases are identified by a major version number, and a minor version number. For example, in “8.7”, the major version is 8, and minor is 7.
Major versions are supported with updates for 10 years , minor versions are supported for 6 months. Every 6 months a new minor version is released. For example, Rocky 8.7 was released in November 2022, so Rocky 8.8 will be released in May of 2023. Rocky 8.9 will be released in November of 2023, etc. A new major version is released every 3 years. Rocky 9 was released in 2022 (and will receive updates until 2032). Rocky 10 will be released in 2025, and receive updates until 2035.
RHEL/Rocky generally receives at least a few package updates every week during the course of a minor version. These are almost always very small - little patches to fix bugs, and especially security issues. New features or newer versions of software packages are released with a new minor version. For example, Rocky 9.0 had PHP 8.0.13, with minor security patches during the 9.0 lifecycle (8.0.13-1, 8.0.13-2, etc.). Rocky 9.1 introduced PHP 8.0.27 . a more significant upgrade with newer features. Future minor releases may bump PHP to 8.1 or 8.2.
Here are some important factoids about the way major/minor releases work:
Once the public project releases a new minor version, the previous minor version is put into the “vault”, and will not receive updates anymore. When Rocky 8.8 is released, 8.7 is frozen in the vault and never updated again. Exception exists for CIQ’s “Long Term Support” offering - will be discussed below
By default, users will automatically get each new minor release by simply running dnf update . No other action is required - minor release upgrades are seamless from their perspective.
Upgrades between major versions (7 → 8, 8 → 9) are not supported . The recommended path is to set up configuration management and/or automated scripts, create a new machine, and redeploy software on the newer major version. (This may be subject to change as CentOS/RHEL 7 nears end of life in 2024)
Excellent guide (with pictures) for understanding the RHEL release cycle: https://access.redhat.com/support/policy/updates/errata
Here’s a kind of ticket comes in often at CIQ support, as well as the Rocky public project:
We ran a security scan on our Rocky systems with <SECURITY_PRODUCT>. Rocky 8 has an
important unpatched OpenSSL vulnerability: CVE-2022-0778 Rocky 8 is running the old
and unsupported OpenSSL 1.1.1k!
Please tell us how to upgrade our Rocky 8 to version 1.1.1t, or 3.0.2. These
are the latest from openssl.org, and have the security fixes we need!
This is urgent!
-Concerned User
This ticket (and the customer’s SECURITY_PRODUCT) pre-supposes some things that are false. It’s true that the current OpenSSL in Rocky 8 is version 1.1.1k, and that version is now a few years old as of this writing. What needs explaining is the concept of security backports.
To maintain software stability, RHEL (and therefore Rocky) take selected “small fixes” from newer versions of software and bring them back (called a “backport”) to the version in the distro. The modified version contains the fix, but the user is not forced into a major upgrade in order to get it. This concept is super important, and is a big reason behind the appeal of RHEL/Rocky in the enterprise. Many packages are compiled against and depend on the OpenSSL library - if Rocky 8 was kept on the bleeding edge of OpenSSL versions (1.1.1t, or the even newer 3.0.2 series), other software would break! The OpenSSL project modifies their API when new versions are released. Rocky Linux packages that rely on the old version (like Apache, SSH, curl, and Python, to name a few) would also need to be rebuilt, or they would break. This backporting strategy allows “little fixes” to be released without breaking workflow or software compatibility, while keeping the software secure and bug-free
Excellent read about RHEL backport policy: https://access.redhat.com/security/updates/backporting .
To investigate this example security issue (OpenSSL + CVE-2022-0778), we have several resources readily available:
https://errata.rockylinux.org/ : This is the Rocky Linux security and update errata site. I can search for “openssl” or “CVE-2022-0778” to try and find out if this security issue has been addressed. Filter functionality is also available to locate the proper update errata. I can browse until I find the proper openssl security update. Here it is on the errata site: https://errata.rockylinux.org/RLSA-2022:1065 . This is the fix! Looking at “Affected Packages”, it appears this was fixed in openssl-1.1.1k-6. The -6 indicates minor patches/backports that have been done, modifying it from the original openssl-1.1.1k. I also see this issue was fixed in March 2022, shortly after this vulnerability was discovered. (note: The Rocky errata search feature is non-functional, but being fixed as of this writing. Filter options work as normal)
https://access.redhat.com/errata : If I filter and search the RHEL errata, I can also confirm that the fix was issued in RHEL: https://access.redhat.com/errata/RHSA-2022:1065 . That page has a link to the CVE in question as well here: https://access.redhat.com/security/cve/CVE-2022-0778 . Note that Rocky uses the same numeric indicators for security issues as RHEL (“RHSA” vs. “RLSA”). Rocky security data is derived from RHEL’s, as Rocky is a downstream clone. We see that the RHEL update came in March 2022, and Rocky faithfully followed suit.
RPM/DNF on the Rocky System Itself:
On user’s system with RPM: rpm -q openssl --changelog | less . This command will get me the changelog of the currently installed openssl on my Rocky system. I can search through this changelog text, CVE references are almost always included with their corresponding version listing.
If I want to check the changelog of a package in the repository but not installed, I can also do that: dnf repoquery -q --changelog openssl | less . This might be slightly different than my installed version, if a newer version is available in the Rocky Repos.
DNF Auto-update to fix a particular CVE: dnf update --cve CVE-2022-0778 . If my installed openssl is vulnerable, this command will update to the fixed one. If it’s not vulnerable, the command does nothing. I can also check on an update before I install it with dnf updateinfo info --cve CVE-2022-0778 .
Going one step further, I can browse through updateinfo for ALL potential updates simply with: dnf updateinfo (summary), or details: dnf updateinfo info | less . Or I can filter so I just see packages with security implications: dnf updateinfo info --security | less . CVEs are easily searchable within this text.
Rocky Linux Source Code and Build Systems:
https://koji.rockylinux.org : This is the build system for Rocky 8. If I search “openssl” in the top right (Packages search), I see that it shows openssl-1.1.1k-6 was built on 2022-03-28. Clicking on that version, I can check the RPM changelog at the very bottom, and sure enough, the top entry has our security issue in it:
* Wed Mar 23 2022 Clemens Lang <cllang@redhat.com> - 1:1.1.1k-6
- Fixes CVE-2022-0778 openssl: Infinite loop in BN_mod_sqrt() reachable when parsing certificates
- Resolves: rhbz#2067144
I’m also able to view the compilation logs and download the RPM artifacts for this (or any other) package version. Here is the openssl-1.1.1k-6 link: https://koji.rockylinux.org/koji/buildinfo?buildID=14039
https://peridot.build.resf.org/ : This is the build system for Rocky 9. Its interface is wildly different from Koji, but you can still browse builds (and code imports) that the project has done. Logs and package versions can be investigated via the web interface.
https://git.rockylinux.org/ : This is where all Rocky package source code is kept. OpenSSL in particular is in https://git.rockylinux.org/staging/rpms/openssl .
In the “r8” branch (for Rocky 8), we can click “Commits”, and find a list of versions imported. Clicking on our desired fixed version commit, (openssl-1.1.1k-6, remember?) we can see the exact backported code that fixed this CVE-2022-0778 issue: https://git.rockylinux.org/staging/rpms/openssl/-/commit/cb48d82142792160a7fe9f1fdfdd8a0b72e0477f . This is useful for traceability: this (public) link shows exactly what code was used to fix the problem. And users can verify the fixed update made it onto their system.
This information is all public and freely available! We strongly encourage their use when researching Rocky Linux issues, security or otherwise. These links and the relevant changelogs should make it clear exactly what is in Rocky Linux, where it comes from, and how it’s been modified from the original upstream source.
CIQ offers an additional paid service for customers: Rocky Linux LTS backports. This is designed for organizations who want to remain on a previous minor release of Rocky Linux (like 8.6), which is no longer supported by the public project. CIQ staff continue backporting security and bugfixes into the supported minor releases for a period of time.
Subscribed customers who value stability can remain on their desired release longer, while still enjoying minor security and bugfix updates to their packages. As of this writing, CIQ is supporting non-zero, even-numbered point releases (8.6, 8.8, 9.2, 9.4, etc.) with LTS offerings. LTS support lasts for 18 months after the release is retired from the public project. For example, Rocky 8.6 was retired in November 2022. CIQ’s LTS-8.6 support will last 18 months from that, or May 2024.