Rocky Linux Build Lab Part 2: Let's build Bind!

Rocky Linux build lab troubleshooting

UPDATE: (This article had a technical issue that has been fixed. I forgot to give instructions on modifying the Mock config to pull in non-default modules! Skip G., 2021-11-13)

Bind is the world's most popular DNS server, and we'll see that it's a bit harder to compile than bash. We'll find out first-hand about "hidden dependencies" and how we deal with them!

Note: this article assumes you have Mock and tools set up, and have completed the previous lab successfully (cloning and building Bash with Mock). If you haven't, then please do that first, or be familiar with how to do it! We'll be building on its concepts.


The start: much like Bash

We begin our work very similar to the bash package above. First, clone the git repository for bind:

cd ~/rockysrc
git clone  https://git.rockylinux.org/staging/rpms/bind

cd bind/

Next, We need to grab the binary source files, listed in that .bind.metadata file. There are 2 of them this time, which is a little different, but still easy:

[skip@RpiRockyDev bind]$ cat .bind.metadata 
14064c865920842e48f444be2bda9dc91770e439 SOURCES/bind-9.11.26.tar.gz
a164fcad1d64d6b5fab5034928cb7260f1fa8fdd SOURCES/random.data


curl -o SOURCES/bind-9.11.26.tar.gz  https://rocky-linux-sources-staging.a1.rockylinux.org/14064c865920842e48f444be2bda9dc91770e439

curl -o SOURCES/random.data  https://rocky-linux-sources-staging.a1.rockylinux.org/a164fcad1d64d6b5fab5034928cb7260f1fa8fdd

Now, let's build our SRPM file from these sources. Again, works just like last time:

mkdir ~/rockybuild/bind

mock -v --resultdir=~/rockybuild/bind   --buildsrpm --spec=~/rockysrc/bind/SPECS/bind.spec  --sources=~/rockysrc/bind/SOURCES/


Tricky part: Let's attempt a build and see what happens!

time mock -v --resultdir=~/rockybuild/bind   ~/rockybuild/bind/bind-9.11.26-4.el8.src.rpm

You'll notice that this build command errors out! Ruh Roh Raggy! We'll have to put on our Troubleshooting Pants to fix it:


Fixing Bind

The sharp-eyed among you will notice this dnf error message from your mock output:

No matching package to install: 'kyua'

As I explained in the previous article, not all build-time dependencies are included in the Rocky / RHEL 8 repositories, and this is one of them! We need this kyua package to proceed!

Fortunately, all dependencies are located in git, including kyua. So we can roll up our sleeves and compile it! First, the source code and SRPM generation:

git clone  https://git.rockylinux.org/staging/rpms/kyua

[skip@RpiRockyDev kyua]$ cat .kyua.metadata 
2e437ffdb96dfc6325e283e531a1a3e11bebe4d7 SOURCES/kyua-0.13.tar.gz



# Note that kyua has no SOURCES/ folder included, because there are no patches in the repository.  That's ok, we'll just create one before downloading the tar.gz file:

mkdir SOURCES

curl -o SOURCES/kyua-0.13.tar.gz  https://rocky-linux-sources-staging.a1.rockylinux.org/2e437ffdb96dfc6325e283e531a1a3e11bebe4d7



mkdir ~/rockybuild/kyua

mock -v --resultdir=~/rockybuild/kyua  --buildsrpm --spec=~/rockysrc/kyua/SPECS/kyua.spec  --sources=~/rockysrc/kyua/SOURCES/


Then, the compile step with Mock:

time mock -v --resultdir=~/rockybuild/kyua   ~/rockybuild/kyua/kyua-0.13-1.el8.src.rpm


More Trouble : Another failure when trying to build kyua! This time we get:

No matching package to install: 'pkgconfig(lutok) >= 0.4'

Uh oh....

We need to go.... DEEPER!


Dependencies and Local Repos

I'm going to save us a little time and "give away the ending" to this rabbit hole we've been tumbling down. The full dependency chain we need to compile bind looks like this:

bind ---depends on ---> kyua ----depends on ---> lutok ---depends on ----> atf

Only the final product, bind, is in the Rocky/RHEL repositories. The rest must be fetched and compiled from source.

But here's a conundrum: what do we do with these packages, even after we compile them? Let's say we successfully produce the atf RPM packages, for example. When we go to compile the next in the chain, lutok, that mock command won't automatically be able to see our atf packages and pull them in as dependencies! We must make it available somehow via DNF repo.

The answer is simple: we have to create our own (local) DNF repository! It's easier than you might think:

# We'll put our local repository in our home folder, under local_mock/ :

mkdir ~/local_mock

createrepo -v ~/local_mock

# (you installed "createrepo" when  you installed mock.  Did you notice?  ;-) )


Now that we have a repository (even though it's empty), we need to edit our mock config to include it in builds. Append this option to /etc/mock/default.cfg:

# Be sure to swap USERNAME with your own user!  
# (or just specify anywhere on the filesystem you want to put your repo)

config_opts['dnf.conf'] += """
[local_mock]
baseurl=file:///home/USERNAME/local_mock
gpgcheck=0
enabled=1
"""


Now we're ready! As we build dependencies, we can simply copy (cp) the produced RPMs to our repo folder, and run createrepo -v ~/local_mock. The next build should then "see" the dependencies we built. We are maintaining our own little repository!

Skip's Note: We could easily do this with a web server as well. Simply run the createrepo command pointing to a folder being served by your apache/nginx/whatever, and make the baseurl line start with http:// instead. You can share repositories with others this way. Do this enough, and next thing you know you'll have created your own Linux distro ;-) .


Example: atf

I've cloned my source into ~/rockysrc/atf , build the SRPM, and then built the binary RPMs into ~/rockybuild/atf. Adding the products to my local repo is simple. I copy RPMs, and run createrepo:

cp  ~/rockybuild/atf/*.rpm   ~/local_mock/


createrepo -v ~/local_mock


Now the next build in the chain (lutok) can "see" the results of this build! It won't fail, because I've given it atf package powers(!)

We rinse and repeat, following our chain all the way up to bind. We build lutok, copy those RPMs, run createrepo. Then build kyua, copy those RPMs, and again run createrepo. Now that we have all the RPMs from the atf -> lutok -> kyua "chain" in our repository, we are (finally) ready to build bind. We've collected every dependency that we need!



Getting Bind to Compile

Once we get to bind, if you try it, we'll uncover yet another issue:

Problem: conflicting requests

package softhsm-2.6.0-5.module+el8.4.0+429+6bd33fea.x86_64 is filtered out by modular filtering  

Doh! We'll fix this, and it will act as a kind of "preview" for when modules are covered on this blog.

Basically, the issue is this: Bind requires a certain package: "softhsm" to build successfully. Softhsm is available in Rocky (and RHEL), but not as part of a "default" module. It is part of the idm module, specifically the DL1 stream for that module. We can check this with dnf commands ("dnf module list idm")

[skip@RpiRockyDev ~]$ dnf module list idm
Last metadata expiration check: 0:00:19 ago on Sun 14 Nov 2021 11:32:12 PM EST.
Rocky Linux 8 - AppStream
Name                               Stream                                   Profiles                                                              Summary                                                                                    
idm                                DL1                                      adtrust, client, common [d], dns, server                              The Red Hat Enterprise Linux Identity Management system module                             
idm                                client [d]                               common [d]                                                            RHEL IdM long term support client module                                                   

Hint: [d]efault, [e]nabled, [x]disabled, [i]nstalled

We see that the "idm:client" module stream is the default, but idm:DL1 is another option. Let's check the packages available in each one:

dnf module info idm

This will spit out a list of all the versions of these modules ever released, as well as which packages are included. If you scroll to some of the idm:DL1 entries, you'll see "softhsm" packages listed. These are not available in the default idm:client entries.


The (module) solution:

The solution is to tell our Mock config that we desire the idm:DL1, and NOT the default idm:client module enabled when we build Bind. Fortunately, there's an easy way to do this. Append this text to the end of your /etc/mock/default.cfg:

# This is used for enabling a proper non-default module for compiling bind
config_opts['module_setup_commands'] = [
('disable', 'idm'),
('enable',  'idm:DL1'),
]

Here we are telling Mock to first disable the idm module, and enable the non-default "idm:DL1" stream. Now the DNF inside the Mock build can "see" all the packages in idm:DL1. And your build can now pull in softhsm as a dependency.

Give bind a try again, and it should build this time! I recommend commenting out the those module_setup_commands by default in future builds though, and only stray from default modules when you need to. We'll dive more into modules later, but this gives you a way to enable/disable them in your builds, and to explore which packages are a part of each one.



Conclusion

Hope you learned something about package build chains and dependency management from this article. I'm encouraging engagement - feel free to chat with questions, tips, comments, etc. in the Reddit thread (under r/RockyLinux/), forums.rockylinux.org, or especially chat.rockylinux.org. I'm around decently often, and am usually down to chat about this stuff!


Next article, we'll leave the lab, but enter the factory! We'll take a (public) tour around the tools we use to trigger and manage builds at scale, mostly Koji and Distrobuild. We'll see what they're all about, and check out how much build activity is publicly available for all to see (hint: it's a bunch!).


As always, thanks for reading!

-Skip