Running FreeBSD as a virtual machine with UEFI on ARM64 came to the point that it just works. I have to use QEMU with u-boot to get FreeBSD up and running on the Raspberry Pi as a virtual machine with older FreeBSD versions: https://stafwag.github.io/blog/blog/2021/03/14/howto_run_freebsd_as_vm_on_pi/.
But with the latest versions of FreeBSD ( not sure when it started to work, but it works on FreeBSD 14) you can run FreeBSD as a virtual machine on ARM64 with UEFI just like on x86 on GNU/Linux with KVM.
UEFI on KVM is in general provided by the open-source tianocore project.
I didn’t find much information on how to run OpenBSD with UEFI on x86 or ARM64.
So I decided to write a blog post about it, in the hope that this information might be useful to somebody else. First I tried to download the OpenBSD 7.4 ISO image and boot it as a virtual machine on KVM (x86). But the iso image failed to boot on a virtual with UEFI enabled. It looks like the ISO image only supports a legacy BIOS.
ARM64 doesn’t support a “legacy BIOS”. The ARM64 download page for OpenBSD 7.4 doesn’t even have an ISO image, but there is an install-<version>.img image available. So I tried to boot this image on one of my Raspberry Pi systems and this worked. I had more trouble getting NetBSD working as a virtual machine on the Raspberry Pi but this might be a topic for another blog post :-)
You’ll find my journey with my installation instructions below.
Download the latest OpenBSD installation ARM64 image from: https://www.openbsd.org/faq/faq4.html#Download
The complete list of the mirrors is available at https://www.openbsd.org/ftp.html
Download the image.
[staf@staf-pi002 openbsd]$ wget https://cdn.openbsd.org/pub/OpenBSD/7.4/arm64/install74.img
--2024-02-13 19:04:52-- https://cdn.openbsd.org/pub/OpenBSD/7.4/arm64/install74.img
Connecting to xxx.xxx.xxx.xxx:3128... connected.
Proxy request sent, awaiting response... 200 OK
Length: 528482304 (504M) [application/octet-stream]
Saving to: 'install74.img'
install74.img 100%[===================>] 504.00M 3.70MB/s in 79s
2024-02-13 19:06:12 (6.34 MB/s) - 'install74.img' saved [528482304/528482304]
[staf@staf-pi002 openbsd]$
Download the checksum and the signed checksum.
2024-02-13 19:06:12 (6.34 MB/s) - 'install74.img' saved [528482304/528482304]
[staf@staf-pi002 openbsd]$ wget https://cdn.openbsd.org/pub/OpenBSD/7.4/arm64/SHA256
--2024-02-13 19:07:00-- https://cdn.openbsd.org/pub/OpenBSD/7.4/arm64/SHA256
Connecting to xxx.xxx.xxx.xxx:3128... connected.
Proxy request sent, awaiting response... 200 OK
Length: 1392 (1.4K) [text/plain]
Saving to: 'SHA256'
SHA256 100%[=============================>] 1.36K --.-KB/s in 0s
2024-02-13 19:07:01 (8.09 MB/s) - 'SHA256' saved [1392/1392]
[staf@staf-pi002 openbsd]$
[staf@staf-pi002 openbsd]$ wget https://cdn.openbsd.org/pub/OpenBSD/7.4/arm64/SHA256.sig
--2024-02-13 19:08:01-- https://cdn.openbsd.org/pub/OpenBSD/7.4/arm64/SHA256.sig
Connecting to xxx.xxx.xxx.xxx:3128... connected.
Proxy request sent, awaiting response... 200 OK
Length: 1544 (1.5K) [text/plain]
Saving to: 'SHA256.sig'
SHA256.sig 100%[=============================>] 1.51K --.-KB/s in 0s
2024-02-13 19:08:02 (3.91 MB/s) - 'SHA256.sig' saved [1544/1544]
[staf@staf-pi002 openbsd]$
OpenBSD uses signify to validate the cryptographic signatures. signify
is also available for GNU/Linux (at least on Debian GNU/Linux and Arch Linux).
More details on how to verify the signature with signify is available at: https://www.openbsd.org/74.html
This blog post was also useful: https://www.msiism.org/blog/2019/10/20/authentic_pufferfish_for_penguins.html
Download the signify public key from: https://www.openbsd.org/74.html
[staf@staf-pi002 openbsd]$ wget https://ftp.openbsd.org/pub/OpenBSD/7.4/openbsd-74-base.pub
--2024-02-13 19:14:25-- https://ftp.openbsd.org/pub/OpenBSD/7.4/openbsd-74-base.pub
Connecting to xxx.xxx.xxx.xxx:3128... connected.
Proxy request sent, awaiting response... 200 OK
Length: 99 [text/plain]
Saving to: 'openbsd-74-base.pub'
openbsd-74-base.pub 100%[=============================>] 99 397 B/s in 0.2s
2024-02-13 19:14:26 (397 B/s) - 'openbsd-74-base.pub' saved [99/99]
[staf@staf-pi002 openbsd]$
I run Debian GNU/Linux on my Raspberry Pi’s, let see which signify
packages are available.
[staf@staf-pi002 openbsd]$ sudo apt search signify
sudo: unable to resolve host staf-pi002: Name or service not known
[sudo] password for staf:
Sorting... Done
Full Text Search... Done
chkrootkit/stable 0.57-2+b1 arm64
rootkit detector
elpa-diminish/stable 0.45-4 all
hiding or abbreviation of the mode line displays of minor-modes
fcitx-sayura/stable 0.1.2-2 arm64
Fcitx wrapper for Sayura IM engine
fcitx5-sayura/stable 5.0.8-1 arm64
Fcitx5 wrapper for Sayura IM engine
signify/stable 1.14-7 all
Automatic, semi-random ".signature" rotator/generator
signify-openbsd/stable 31-3 arm64
Lightweight cryptographic signing and verifying tool
signify-openbsd-keys/stable 2022.2 all
Public keys for use with signify-openbsd
[staf@staf-pi002 openbsd]$
There’re two OpenBSD signify packages available on Debian 12 (bookworm);
signify-openbsd/
: The OpenBSD signify
tool.signify-openbsd-keys
: This package contains the OpenBSD release public keys, installed in /usr/share/signify-openbsd-keys/
. Unfortunately, the OpenBSD 7.4 release isn’t (yet) included in Debian 12 (bookworm).[staf@staf-pi002 openbsd]$ sudo apt install signify-openbsd signify-openbsd-keys
sudo: unable to resolve host staf-pi002: Name or service not known
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
signify-openbsd signify-openbsd-keys
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 70.4 kB of archives.
After this operation, 307 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bookworm/main arm64 signify-openbsd arm64 31-3 [62.3 kB]
Get:2 http://deb.debian.org/debian bookworm/main arm64 signify-openbsd-keys all 2022.2 [8020 B]
Fetched 70.4 kB in 0s (404 kB/s)
Selecting previously unselected package signify-openbsd.
(Reading database ... 94575 files and directories currently installed.)
Preparing to unpack .../signify-openbsd_31-3_arm64.deb ...
Unpacking signify-openbsd (31-3) ...
Selecting previously unselected package signify-openbsd-keys.
Preparing to unpack .../signify-openbsd-keys_2022.2_all.deb ...
Unpacking signify-openbsd-keys (2022.2) ...
Setting up signify-openbsd-keys (2022.2) ...
Setting up signify-openbsd (31-3) ...
[staf@staf-pi002 openbsd]$
Verify the checksum.
[staf@staf-pi002 openbsd]$ sha256sum install74.img
09e4d0fe6d3f49f2c4c99b6493142bb808253fa8a8615ae1ca8e5f0759cfebd8 install74.img
[staf@staf-pi002 openbsd]$
[staf@staf-pi002 openbsd]$ grep 09e4d0fe6d3f49f2c4c99b6493142bb808253fa8a8615ae1ca8e5f0759cfebd8 SHA256
SHA256 (install74.img) = 09e4d0fe6d3f49f2c4c99b6493142bb808253fa8a8615ae1ca8e5f0759cfebd8
[staf@staf-pi002 openbsd]$
Execute the signify
command to verify the checksum. See the OpenBSD signify manpage for more information.
You’ll find a brief list of the arguments that are used to verify the authenticity of the image.
-C
: Will verify the Checksum.-p <path>
: The path to the Public key.-x <path>
: The path to the signature file.Verify the image with signify
.
[staf@staf-pi002 openbsd]$ signify-openbsd -C -p openbsd-74-base.pub -x SHA256.sig install74.img
Signature Verified
install74.img: OK
[staf@staf-pi002 openbsd]$
The Debian UEFI package for libvirt ovmf
is based on https://github.com/tianocore/tianocore.github.io/wiki/OVMF.
Debian Bookworm comes with the following UEFI BIOS settings:
/usr/share/AAVMF/AAVMF_CODE.ms.fd
This is with secure boot enabled./usr/share/AAVMF/AAVMF_CODE.fd
This is without secure boot enabled.The full description is available at /usr/share/doc/ovmf/README.Debian
on a Debian system when the ovmf
package is installed.
To install OpenBSD we need to disable secure boot.
I first started a test boot.
Logon to the Raspberry Pi.
[staf@vicky ~]$ ssh -X -CCC staf-pi002
Warning: untrusted X11 forwarding setup failed: xauth key data not generated
Linux staf-pi002 6.1.0-17-arm64 #1 SMP Debian 6.1.69-1 (2023-12-30) aarch64
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Wed Feb 14 06:08:45 2024 from xxx.xxx.xxx.xxx
[staf@staf-pi002 ~]$
Start virt-manager
and click on the [ Create on new VM ] icon.
This will bring up the new vm window. Select ( ) Import existing disk image, you review the architecture option by selecting the \/ Architecture options. The defaults are fine. Click on [ Forward ].
This will open the “import vm” window. Click on [ Browse ] to select the OpenBSD installation image or just copy/paste the path.
At the bottom of the screen, you’ll see Choose the operating system you are installing. Starting type openbsd
and select [ X ] include end-of-life operating systems Debian 12 (bookworm) doesn’t include support for OpenBSD 7.4 (yet) so we need to set it to “OpenBSD 7.0”. Click on [ Forward ].
In the next windows keep the default Memory and CPU settings as we’re just verifying that we can boot from the installation image.
Debian uses “secure boot” by default. We need to disable secure boot. Select [ X ] Customize configuration before install, this allows us to set the UEFI boot image.
Set the Firmware to: /usr/share/AAVMF/AAVMF_CODE.fd
to disable secure boot and click on [ Begin Installation ].
Let’s check if OpenBSD can boot. Great, it works!
I prefer to use the command line to install as this allows me to make the installation reproducible and automated.
I used ZFS on my Raspberry Pi’s, this makes it easier to create snapshots etc when you’re testing software etc.
root@staf-pi002:/var/lib/libvirt/images# zfs create staf-pi002_pool/root/var/lib/libvirt/images/openbsd-gitlabrunner001
root@staf-pi002:/var/lib/libvirt/images#
root@staf-pi002:/var/lib/libvirt/images/openbsd-gitlabrunner001# pwd
/var/lib/libvirt/images/openbsd-gitlabrunner001
root@staf-pi002:/var/lib/libvirt/images/openbsd-gitlabrunner001#
os-variant
To get the operating system settings you can execute the command virt-install --osinfo list
root@staf-pi002:/var/lib/libvirt/images/openbsd-gitlabrunner001# virt-install --osinfo list | grep -i openbsd7
openbsd7.0
root@staf-pi002:/var/lib/libvirt/images/openbsd-gitlabrunner001#
We’ll use openbsd7.0
as the operating system variant.
Create a destination disk image.
root@staf-pi002:/var/lib/libvirt/images/openbsd-gitlabrunner001# qemu-img create -f qcow2 openbsd-gitlabrunner001.qcow2 50G
Formatting 'openbsd-gitlabrunner001.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=53687091200 lazy_refcounts=off refcount_bits=16
root@staf-pi002:/var/lib/libvirt/images/openbsd-gitlabrunner001#
virt-install
Run virt-install
to import the virtual machine.
#!/bin/bash
virt-install --name openbsd-gitlabrunner001 \
--noacpi \
--boot loader=/usr/share/AAVMF/AAVMF_CODE.fd \
--os-variant openbsd7.0 \
--ram 2048 \
--import \
--disk /home/staf/Downloads/isos/openbsd/install74.img \
--disk /var/lib/libvirt/images/openbsd-gitlabrunner001/openbsd-gitlabrunner001.qcow2
If everything goes well the virtual machine gets booted.
BdsDxe: loading Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x1,0x3)/Pci(0x0,0x0)
BdsDxe: starting Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x1,0x3)/Pci(0x0,0x0)
disks: sd0*
>> OpenBSD/arm64 BOOTAA64 1.18
boot>
cannot open sd0a:/etc/random.seed: No such file or directory
booting sd0a:/bsd: 2861736+1091248+12711584+634544 [233295+91+666048+260913]=0x13d5cf8
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
Copyright (c) 1995-2023 OpenBSD. All rights reserved. https://www.OpenBSD.org
OpenBSD 7.4 (RAMDISK) #2131: Sun Oct 8 13:35:40 MDT 2023
deraadt@arm64.openbsd.org:/usr/src/sys/arch/arm64/compile/RAMDISK
real mem = 2138013696 (2038MB)
avail mem = 2034593792 (1940MB)
random: good seed from bootblocks
mainbus0 at root: linux,dummy-virt
psci0 at mainbus0: PSCI 1.1, SMCCC 1.1
efi0 at mainbus0: UEFI 2.7
efi0: EDK II rev 0x10000
smbios0 at efi0: SMBIOS 3.0.0
smbios0:
sd1 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, >
sd1: 51200MB, 512 bytes/sector, 104857600 sectors
virtio35: msix per-VQ
ppb5 at pci0 dev 1 function 5 vendor "Red Hat", unknown product 0x000c rev 0x00: irq
pci6 at ppb5 bus 6
ppb6 at pci0 dev 1 function 6 vendor "Red Hat", unknown product 0x000c rev 0x00: irq
pci7 at ppb6 bus 7
ppb7 at pci0 dev 1 function 7 vendor "Red Hat", unknown product 0x000c rev 0x00: irq
pci8 at ppb7 bus 8
ppb8 at pci0 dev 2 function 0 vendor "Red Hat", unknown product 0x000c rev 0x00: irq
pci9 at ppb8 bus 9
ppb9 at pci0 dev 2 function 1 vendor "Red Hat", unknown product 0x000c rev 0x00: irq
pci10 at ppb9 bus 10
ppb10 at pci0 dev 2 function 2 vendor "Red Hat", unknown product 0x000c rev 0x00: irq
pci11 at ppb10 bus 11
ppb11 at pci0 dev 2 function 3 vendor "Red Hat", unknown product 0x000c rev 0x00: irq
pci12 at ppb11 bus 12
ppb12 at pci0 dev 2 function 4 vendor "Red Hat", unknown product 0x000c rev 0x00: irq
pci13 at ppb12 bus 13
ppb13 at pci0 dev 2 function 5 vendor "Red Hat", unknown product 0x000c rev 0x00: irq
pci14 at ppb13 bus 14
pluart0 at mainbus0: rev 1, 16 byte fifo
pluart0: console
"pmu" at mainbus0 not configured
agtimer0 at mainbus0: 54000 kHz
"apb-pclk" at mainbus0 not configured
softraid0 at root
scsibus2 at softraid0: 256 targets
root on rd0a swap on rd0b dump on rd0b
WARNING: CHECK AND RESET THE DATE!
erase ^?, werase ^W, kill ^U, intr ^C, status ^T
Welcome to the OpenBSD/arm64 7.4 installation program.
(I)nstall, (U)pgrade, (A)utoinstall or (S)hell?
Continue with the OpenBSD installation as usual. Make sure that you select the second disk during the installation process.
To fully automate the installation we need a system that executes the post-configuration at the first boot. On GNU/Linux is normally done by cloud-init while there are solutions to get cloud-init working on the BSDs. I didn’t look into this (yet).
Have fun!
In the next blog posts will add code to GitLab and set up GitLab runners on different Operating systems.
There was an initial root user with a password created during the installation, the initial root password is saved - or was saved as GitLab will delete the initial password after 24H - to /etc/gitlab/initial_root_password
.
To update the root password, log in to GitLab with your web browser click on your “avatar” icon and click on [ Edit profile ]
Select [ Password ] to update the root password.
You need to re-login after the password is updated.
In this section we’ll set up an additional administrator user, I usually try to have a backup admin account. It isn’t recommended to use an admin account directly so we’ll set up a regular user that we can use for your daily development tasks.
The full documentation about GitLab users is available at: https://docs.gitlab.com/ee/user/profile/account/create_accounts.html#create-users-in-admin-area
In the admin area select users on the left-hand side.
In the New user window, fill in the Account Name / Username and Email Address. Please note that you can’t use admin as this is a reversed user name in GitLab. At the Access level select Administrator.
If everything goes well you or the user will receive an email to reset his or her password.
Repeat the same steps to create a normal user and keep the Access level to “Regular”.
To access your git repositories using ssh you’ll need to upload your ssh public key. I use hardware tokens ( smartcard-hsm ) to protect my private keys.
You can find more information on how to use SmartCard-HSM with ssh in a previous blog post: https://stafwag.github.io/blog/blog/2015/12/05/protecting-your-ssh-keys-with-smartcard-hsm/
You can find information on how to generate a ssh key pair for GitLab at:https://docs.gitlab.com/ee/user/ssh.html
If you don’t mind having your keypair on your filesystem you can use the steps below ( but I advise you to look at the better options ).
Execute ssh-keygen
to generate a ssh key-pair.
I recommend using a passphrase so the private key is encrypted; this prevents an attacker from copying it as plain text from the filesystem. To prevent that you need to type in a passphrase during the development you can use a ssh-agent so you don’t need to type in your password each time you push a commit to your git repository (see below).
The unencrypted key is still stored in memory if you use an ssh-agent unless you store your private key on a hardware token (HSM).
Please note that we use a non-default ssh key location in the command below ~/.ssh/testgitlab.key
, we update your ssh
client config to use this private key.
[staf@vicky ~]$ ssh-keygen -f ~/.ssh/testgitlab.key -t ed25519 -C "test gitlab key"
Generating public/private ed25519 key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/staf/.ssh/testgitlab.key
Your public key has been saved in /home/staf/.ssh/testgitlab.key.pub
The key fingerprint is:
SHA256:m4TpKmOjvwkoY2H3arG3tTLTOgYeA03BJoACD86Gkr0 test gitlab key
The key's randomart image is:
+--[ED25519 256]--+
|B ... |
|** + |
|=+B |
|o. o o |
| oE. o S |
|o o=.. . o |
|=.. *.o.o |
|oo==.O... |
|.+=*+oB. |
+----[SHA256]-----+
[staf@vicky ~]$
Edit your profile.
Goto SSH Keys and select [ Add new key ].
[staf@vicky ~]$ cat ~/.ssh/testgitlab.key.pub
ssh-ed25519 <snip> test gitlab key
[staf@vicky ~]$
Add the public key to GitLab. If you don’t want to set an Expiration date, clear the field.
And click on [ Add key ]
When you upload your public key to the GitLab GUI, it will update ~/.ssh/authorized_keys
for the git
user. The git
user was created during the installation.
Let’s quickly review it.
Log on to your GitLab server.
[staf@vicky ~]$ ssh staf@gitlab.stafnet.local
Linux gitlab 6.1.0-13-arm64 #1 SMP Debian 6.1.55-1 (2023-09-29) aarch64
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Sat Nov 25 08:02:53 2023 from 192.168.1.10
staf@gitlab:~$
Get details about the git
user.
staf@gitlab:~$ cat /etc/passwd | grep -i git
gitlab-www:x:999:996::/var/opt/gitlab/nginx:/bin/false
git:x:996:995::/var/opt/gitlab:/bin/sh
gitlab-redis:x:995:994::/var/opt/gitlab/redis:/bin/false
gitlab-psql:x:994:993::/var/opt/gitlab/postgresql:/bin/sh
gitlab-prometheus:x:993:992::/var/opt/gitlab/prometheus:/bin/sh
staf@gitlab:~$
Display the ~/.ssh/authorized_keys
contents.
staf@gitlab:~$ sudo cat /var/opt/gitlab/.ssh/authorized_keys
[sudo] password for staf:
command="/opt/gitlab/embedded/service/gitlab-shell/bin/gitlab-shell key-1",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty <snip>
command="/opt/gitlab/embedded/service/gitlab-shell/bin/gitlab-shell key-2",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty <snip>
command="/opt/gitlab/embedded/service/gitlab-shell/bin/gitlab-shell key-3",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty <snip>
staf@gitlab:~$
Check the connection with the command below. Please note that we use the git
user. GitLab will use the matching public key to determine (map to) the real GitLab user.
[staf@vicky ~]$ ssh -T git@gitlab.stafnet.local -i ~/.ssh/testgitlab.key
Enter passphrase for key '/home/staf/.ssh/testgitlab.key':
Welcome to GitLab, @stafwag!
[staf@vicky ~]$
The -T
option disables pseudo-terminal allocation (pty
). Without it you’ll get a warning.
We’ll update the ssh config to use our private key.
Open your ssh config with your favourite editor.
[staf@vicky ~]$ vi ~/.ssh/config
Add a section for our GitLab host and set the IdentityFile
set to our private key.
Host gitlab.stafnet.local
PreferredAuthentications publickey
IdentityFile ~/.ssh/testgitlab.key
Test the SSH access.
[staf@vicky ~]$ ssh -T git@gitlab.stafnet.local
Welcome to GitLab, @stafwag!
[staf@vicky ~]$
We’ll add our private key to the ssh-agent
to avoid the need to type in our password each time we perform a git action.
Most modern Unix desktop systems ( GNOME, KDE, Xfce ) will set up a ssh-agent
automatically.
Some GNU/Linux distributions will even add to the key automatically when you use it.
My FreeBSD desktop (Xfce) only starts the ssh-agent
.
When your Unix desktop doesn’t set up a ssh-agent
you’ll need to start it.
When a ssh-agent is configured the SSH_AGENT_PID
and the SSH_AUTH_SOCK
environment variables are set.
Check if there is already a ssh-agent configured.
staf@fedora39:~$ set | grep SSH
SSH_CLIENT='192.168.122.1 49618 22'
SSH_CONNECTION='192.168.122.1 49618 192.168.122.96 22'
SSH_TTY=/dev/pts/2
staf@fedora39:~$
The easiest way to start a ssh-agent
and set up the environment variables is to execute the command eval $(ssh-agent)
.
staf@fedora39:~$ eval $(ssh-agent)
Agent pid 3542
staf@fedora39:~$
This starts the ssh-agent
, eval
will execute the ssh-agent output and set the required environment variables.
staf@fedora39:~$ set | grep SSH
SSH_AGENT_PID=3542
SSH_AUTH_SOCK=/tmp/ssh-XXXXXXcQhGER/agent.3541
SSH_CLIENT='192.168.122.1 60044 22'
SSH_CONNECTION='192.168.122.1 60044 192.168.122.96 22'
SSH_TTY=/dev/pts/2
staf@fedora39:~$
Add the private key to the ssh-agent
.
staf@fedora39:~$ ssh-add /home/staf/.ssh/testgitlab.key
Enter passphrase for /home/staf/.ssh/testgitlab.key:
Identity added: /home/staf/.ssh/testgitlab.key (test gitlab key)
staf@fedora39:~$
Test the connection.
staf@fedora39:~$ ssh -T git@gitlab.stafnet.local
Welcome to GitLab, @stafwag!
staf@fedora39:~$
With everything set up we’re ready to start to use GitLab. In a further blog post we will go start with adding code to GitLab, add runners and create artifacts.
Have fun!
When you want or need to use CI/CD you have a lot of CI/CD platforms where you can choose from. As with most “tools”, the tool is less important. What (which flow, best practices, security benchmarks, etc) and how you implement it, is what matters.
One of the most commonly used options is Jenkins.
I used and still use Jenkins and created a jenkins build workstation to build software and test in my homelab a couple of years back.
Jenkins started as Hudson at Sun Microsystem(RIP). Hudson is one of the many open-source projects that were started at Sun and killed by Oracle. Jenkins continued as the open-source fork of Hudson.
Jenkins has evolved. If you need to do more complex things you probably end up creating a lot of groovy scripts, nothing wrong with groovy. But as with a lot of discussions about programming, the ecosystem (who is using it, which libraries are available, etc) is important.
Groovy isn’t that commonly used in and known in the system administration ecosystem so this is probably something you need to learn if you’re coming for the system administrator world ( as I do, so I learnt the basics of Groovy this way ).
The other option is to implement CI/CD using the commonly used source hosting platforms; GitHub and GitLab.
Both have offerings that you can use on-premise;
There are other CI/CD systems available. Other examples are Tekon - used by RedHat on OpenShift - and Drone which have gained popularity. Just to name a few.
I started to use GitLab in my lab to build/test/and deploy software. Why GitLab? with Gitlab you have the option to use the same pipelines as on the Gitlab source hosting platform on your on-premise installation.
Gitlab comes in two versions;
You can find more information about the difference between the two versions at: https://about.gitlab.com/handbook/marketing/brand-and-product-marketing/product-and-solution-marketing/tiers/.
In general, I don’t like this approach, having a paid non-open source version doesn’t help that end-users (developers) and different companies work together to improve the software. Companies that use the software will pay for support and consultancy services in the long run.
But it is what it is. If you want to use GitLab CI/CD on GitLab in your homelab environment GitLab-CE is the option to go.
In this blog series, we will go over the installation of GitLab CE and its basic features, as always my blog posts are my installation instructions/notes that I took during my setup in the hope that it is useful to somebody else.
All actions are executed on Debian GNU/Linux 12 (bookworm) on X86 and ARM64 as I migrated all my systems that are running constantly on a cluster running on Raspberry Pi’s 4 to save power.
I mainly follow the official installation instructions at https://about.gitlab.com/install/#debian and the blog post at: https://www.linuxtechi.com/how-to-install-gitlab-on-ubuntu/.
As always it’s a good idea to update your software (regularly).
Update the package database.
staf@tstgitlab:~$ sudo apt update -y
[sudo] password for staf:
Get:1 http://security.debian.org/debian-security bookworm-security InRelease [48.0 kB]
Get:2 http://deb.debian.org/debian bookworm InRelease [151 kB]
Get:3 http://deb.debian.org/debian bookworm-updates InRelease [52.1 kB]
Get:4 http://deb.debian.org/debian bookworm-backports InRelease [56.5 kB]
Get:5 http://security.debian.org/debian-security bookworm-security/main Sources [57.5 kB]
Get:7 http://security.debian.org/debian-security bookworm-security/main amd64 Packages [95.7 kB]
Get:8 http://security.debian.org/debian-security bookworm-security/main Translation-en [54.4 kB]
Get:9 http://deb.debian.org/debian bookworm-backports/main Sources.diff/Index [63.3 kB]
Get:10 http://deb.debian.org/debian bookworm-backports/main amd64 Packages.diff/Index [63.3 kB]
Hit:6 https://packages.gitlab.com/gitlab/gitlab-ce/debian bookworm InRelease
Get:11 http://deb.debian.org/debian bookworm-backports/main Sources T-2023-11-03-1405.27-F-2023-11-03-1405.27.pdiff [836 B]
Get:11 http://deb.debian.org/debian bookworm-backports/main Sources T-2023-11-03-1405.27-F-2023-11-03-1405.27.pdiff [836 B]
Get:12 http://deb.debian.org/debian bookworm-backports/main amd64 Packages T-2023-11-03-2011.07-F-2023-11-03-2011.07.pdiff [1,216 B]
Get:12 http://deb.debian.org/debian bookworm-backports/main amd64 Packages T-2023-11-03-2011.07-F-2023-11-03-2011.07.pdiff [1,216 B]
Fetched 644 kB in 1s (793 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.
staf@tstgitlab:~$
Upgrade the packages.
staf@tstgitlab:~$ sudo apt upgrade -y
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
staf@tstgitlab:~$
Install the required dependencies.
staf@tstgitlab:~$ sudo apt-get install -y curl openssh-server ca-certificates perl
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
curl is already the newest version (7.88.1-10+deb12u4).
curl set to manually installed.
openssh-server is already the newest version (1:9.2p1-2+deb12u1).
ca-certificates is already the newest version (20230311).
ca-certificates set to manually installed.
perl is already the newest version (5.36.0-7).
perl set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
staf@tstgitlab:~$
If you want your system to send out emails for GitLab, you’ll need to set up a local mail server like Postfix or you can reconfigure GitLab to use another email server. See below for references on how to configure this on GitLab.
GitLab provides a script to set up the GitLab repositories for both GitLab CE and GitLab EE. We’ll use GitLab CE. As always it’s a good idea to not run a script that you pulled for the internet blindly; we’ll download the script, review it and execute it.
Create a directory.
staf@tstgitlab:~$ mkdir gitlab
staf@tstgitlab:~$
Download the repository setup script.
staf@tstgitlab:~/gitlab$ wget https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh
--2023-11-03 08:11:58-- https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh
Resolving packages.gitlab.com (packages.gitlab.com)... 172.64.148.245, 104.18.39.11, 2606:4700:4400::6812:270b, ...
Connecting to packages.gitlab.com (packages.gitlab.com)|172.64.148.245|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6865 (6.7K) [text/install_instructions]
Saving to: ‘script.deb.sh’
script.deb.sh 100%[===================>] 6.70K --.-KB/s in 0s
2023-11-03 08:11:58 (35.9 MB/s) - ‘script.deb.sh’ saved [6865/6865]
staf@tstgitlab:~/gitlab$
Review the script.
staf@tstgitlab:~/gitlab$ vi script.deb.sh
Execute it.
staf@tstgitlab:~/gitlab$ sudo bash script.deb.sh
Detected operating system as debian/12.
Checking for curl...
Detected curl...
Checking for gpg...
Detected gpg...
Running apt-get update... done.
Installing debian-archive-keyring which is needed for installing
apt-transport-https on many Debian systems.
Installing apt-transport-https... done.
Installing /etc/apt/sources.list.d/gitlab_gitlab-ce.list...done.
Importing packagecloud gpg key... done.
Running apt-get update... done.
The repository is setup! You can now install packages.
staf@tstgitlab:~/gitlab$
The installation is straightforward, you need to specify the EXTERNAL_URL
environment variable and install the gitlab-ce package.
staf@tstgitlab:~/gitlab$ sudo EXTERNAL_URL="http://tstgitlab" apt install gitlab-ce
<snip>
Notes:
Default admin account has been configured with following details:
Username: root
Password: You didn't opt-in to print initial root password to STDOUT.
Password stored to /etc/gitlab/initial_root_password. This file will be cleaned up in first reconfigure run after 24 hours.
NOTE: Because these credentials might be present in your log files in plain text, it is highly recommended to reset the password following https://docs.gitlab.com/ee/security/reset_use
r_password.html#reset-your-root-password.
gitlab Reconfigured!
*. *.
*** ***
***** *****
.****** *******
******** ********
,,,,,,,,,***********,,,,,,,,,
,,,,,,,,,,,*********,,,,,,,,,,,
.,,,,,,,,,,,*******,,,,,,,,,,,,
,,,,,,,,,*****,,,,,,,,,.
,,,,,,,****,,,,,,
.,,,***,,,,
,*,.
_______ __ __ __
/ ____(_) /_/ / ____ _/ /_
/ / __/ / __/ / / __ `/ __ \
/ /_/ / / /_/ /___/ /_/ / /_/ /
\____/_/\__/_____/\__,_/_.___/
Thank you for installing GitLab!
GitLab should be available at http://tstgitlab
For a comprehensive list of configuration options please see the Omnibus GitLab readme
https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md
Help us improve the installation experience, let us know how we did with a 1 minute survey:
https://gitlab.fra1.qualtrics.com/jfe/form/SV_6kVqZANThUQ1bZb?installation=omnibus&release=16-5
staf@tstgitlab:~/gitlab$
Open your browser to the GitLab URL if everything goes well you see the GitLab login screen.
You can log in as the root user and the initial password that was created during the installation.
This password is stored in the /etc/gitlab/initial_root_password
file.
Please note that the password file will be deleted by GitLab automatically, so copy it and keep it in a safe place.
Display the password.
staf@tstgitlab:~$ ls -l /etc/gitlab/initial_root_password
-rw------- 1 root root 749 Nov 3 08:18 /etc/gitlab/initial_root_password
staf@tstgitlab:~$
staf@tstgitlab:~$ sudo cat /etc/gitlab/initial_root_password
[sudo] password for staf:
# WARNING: This value is valid only in the following conditions
# 1. If provided manually (either via `GITLAB_ROOT_PASSWORD` environment variable or via `gitlab_rails['initial_root_password']` setting in `gitlab.rb`, it was provided before database was seeded for the first time (usually, the first reconfigure run).
# 2. Password hasn't been changed manually, either via UI or via command line.
#
# If the password shown here doesn't work, you must reset the admin password following https://docs.gitlab.com/ee/security/reset_user_password.html#reset-your-root-password.
Password: <snip>
# NOTE: This file will be automatically deleted in the first reconfigure run after 24 hours.
staf@tstgitlab:~$
And login:
GitLab uses NGINX under the hood, enabling https on our installation is reconfiguring NGINX. GitLab also comes with Let’s Encrypt support; see https://docs.gitlab.com/omnibus/settings/ssl/ for more information. When you specify https://
as part of the EXTERNAL_URL
environment variable during the installation the GitLab installer will set up Let’s Encrypt automatically. But I didn’t try this, in this blog post we’ll set up a self-signed certificate.
In a corporate environment, you might want to use an internal certificate authority. But if you want/need to set up an internal CA authority it needs to be secured.
You need to protect your private keys and limit access to the authority, if you don’t have the resources or the time available to set up CA authority in a decent way, it’s better not you use an internal CA authority as this is an attack vector - if an attacker has gained access to the CA authority - or has copied the private key - he/she can generate certificates for every host/domain.
Bottom line; if you don’t have the time to set up CA authority in a secure way it’s better to use Let’s Encrypt or another third-party Authority. Or even just use a self-signing certificate.
I use a HSM - SmartCard-HSM and my CA Authority is offline see https://stafwag.github.io/blog/blog/2020/04/29/setup-an-ca-with-smartcard/
I mainly followed the official documentation at GitLab: https://docs.gitlab.com/omnibus/settings/ssl/index.html#configure-https-manually
Open your favourite editor.
root@gitlab:~# nvi /etc/gitlab/gitlab.rb
root@gitlab:~#
Update the external_url
to use https://
.
## GitLab URL
##! URL on which GitLab will be reachable.
##! For more details on configuring external_url see:
##! https://docs.gitlab.com/omnibus/settings/configuration.html#configuring-the-external-url-for-gitlab
##!
##! Note: During installation/upgrades, the value of the environment variable
##! EXTERNAL_URL will be used to populate/replace this value.
##! On AWS EC2 instances, we also attempt to fetch the public hostname/IP
##! address from AWS. For more details, see:
##! https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html
external_url 'https://gitlab.stafnet.local'
Set letsencrypt['enable'] = false
to disable let’s encrypt support.
################################################################################
# Let's Encrypt integration
################################################################################
letsencrypt['enable'] = false
A certificate is a way to distribute a public key. A certificate is normally signed by a CA authority private key. You trust the CA authority public key distributed in the CA certificate.
A “self-signed certificate” is just a certificate that is signed with the corresponding private key of the certificate instead of the private key of the CA authority.
Create the /etc/gitlab/ssl
directory.
root@gitlab:/etc/gitlab# ls
config_backup gitlab.rb gitlab-secrets.json trusted-certs
root@gitlab:/etc/gitlab# mkdir ssl
root@gitlab:/etc/gitlab#
And set the permissions.
root@gitlab:/etc/gitlab# chmod 755 ssl
root@gitlab:/etc/gitlab#
We’ll create a private key for our self-signed certificate, please note that we don’t encrypt the private key as this would as we would need to type in the password for the private key password each time we start GitLab.
Create a private key.
root@gitlab:/etc/gitlab/ssl# openssl genrsa -out gitlab.stafnet.local.key 4096
root@gitlab:/etc/gitlab/ssl#
Create a certificate request.
root@gitlab:/etc/gitlab/ssl# openssl req -key gitlab.stafnet.local.key -new -out gitlab.stafnet.local.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:BE
State or Province Name (full name) [Some-State]:Antwerp
Locality Name (eg, city) []:Antwerp
Organization Name (eg, company) [Internet Widgits Pty Ltd]:stafnet
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:gitlab.stafnet.local
Email Address []:
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
root@gitlab:/etc/gitlab/ssl#
Sign your certificate with your private key.
root@gitlab:/etc/gitlab/ssl# openssl x509 -signkey gitlab.stafnet.local.key -in gitlab.stafnet.local.csr -req -days 365 -out gitlab.stafnet.local.crt
Certificate request self-signature ok
subject=C = BE, ST = Antwerp, L = Antwerp, O = stafnet, CN = gitlab.stafnet.local
root@gitlab:/etc/gitlab/ssl#
Make sure that your private key is not world-readable.
root@gitlab:/etc/gitlab/ssl# ls -l
total 12
-rw-r--r-- 1 root root 1895 Nov 12 10:22 gitlab.stafnet.local.crt
-rw-r--r-- 1 root root 1691 Nov 12 10:20 gitlab.stafnet.local.csr
-rw------- 1 root root 3272 Nov 8 20:25 gitlab.stafnet.local.key
root@gitlab:/etc/gitlab/ssl#
Open gitlab.rb
in your favourite editor.
root@gitlab:/etc/gitlab/ssl# vi /etc/gitlab/gitlab.rb
root@gitlab:/etc/gitlab/ssl#
Update the external_url
setting.
##! EXTERNAL_URL will be used to populate/replace this value.
##! On AWS EC2 instances, we also attempt to fetch the public hostname/IP
##! address from AWS. For more details, see:
##! https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html
external_url 'https://gitlab.stafnet.local'
Add the following settings to the bottom of the file.
nginx['redirect_http_to_https'] = true
registry_nginx['redirect_http_to_https'] = true
mattermost_nginx['redirect_http_to_https'] = true
Run gitlab-ctl reconfigure
to reconfigure gitlab.
root@gitlab:/etc/gitlab/ssl# gitlab-ctl reconfigure
oot@gitlab:/etc/gitlab/ssl# gitlab-ctl reconfigure
[2023-11-12T11:05:43+01:00] INFO: Started Cinc Zero at chefzero://localhost:1 with repository at /opt/gitlab/embedded (One version per cookbook)
Cinc Client, version 17.10.0
Patents: https://www.chef.io/patents
Infra Phase starting
[2023-11-12T11:05:43+01:00] INFO: *** Cinc Client 17.10.0 ***
[2023-11-12T11:05:43+01:00] INFO: Platform: aarch64-linux
[2023-11-12T11:05:43+01:00] INFO: Cinc-client pid: 52849
[2023-11-12T11:05:46+01:00] INFO: Setting the run_list to ["recipe[gitlab]"] from CLI options
[2023-11-12T11:05:46+01:00] INFO: Run List is [recipe[gitlab]]
[2023-11-12T11:05:46+01:00] INFO: Run List expands to [gitlab]
<snip>
* template[/var/opt/gitlab/postgres-exporter/queries.yaml] action create (up to date)
* consul_service[postgres-exporter] action delete
* file[/var/opt/gitlab/consul/config.d/postgres-exporter-service.json] action delete (up to date)
(up to date)
Recipe: gitlab::database_reindexing_disable
* crond_job[database-reindexing] action delete
* file[/var/opt/gitlab/crond/database-reindexing] action delete (up to date)
(up to date)
[2023-11-12T11:06:20+01:00] INFO: Cinc Client Run complete in 34.625114871 seconds
Running handlers:
[2023-11-12T11:06:20+01:00] INFO: Running report handlers
Running handlers complete
[2023-11-12T11:06:20+01:00] INFO: Report handlers complete
Infra Phase complete, 0/805 resources updated in 37 seconds
gitlab Reconfigured!
If everything goes well you’re able to login to your gitlab instance over https://
.
By default, GitLab will use localhost
as the email server. You either need to reconfigure the local email server to be able to send out emails. Or you can reconfigure GitLab to use an external SMTP server.
You can find more information on how to use an external email server in GitLab at: https://docs.gitlab.com/omnibus/settings/smtp.
The GitLab backup/restore procedure is explained at: https://docs.gitlab.com/ee/administration/backup_restore/
To execute a backup run the gitlab-backup create
command.
The backups are restored in the /var/opt/gitlab/backups
directory.
root@gitlab:/etc/gitlab/ssl# ls -l /var/opt/gitlab/backups
total 3684
-rw------- 1 git git 491520 Oct 7 11:53 1696672275_2023_10_07_16.4.0_gitlab_backup.tar
-rw------- 1 git git 512000 Nov 3 11:36 1699007796_2023_11_03_16.4.1_gitlab_backup.tar
-rw------- 1 git git 1382400 Nov 12 11:49 1699786111_2023_11_12_16.5.1_gitlab_backup.tar
-rw------- 1 git git 1382400 Nov 12 11:53 1699786384_2023_11_12_16.5.1_gitlab_backup.tar
root@gitlab:/etc/gitlab/ssl#
Have fun!
In my previous blog post, I reviewed BastilleBSD. In this post, we go through the required steps to migrate the Jails from ezjail to BastilleBSD.
To test the Jail migration, we’ll first create a test Jail with ezjail. This test Jail will migrate to a BastilleBSD Jail.
We use the ezjail-admin create staftestje001 'vtnet0|<ip>'
command to create the test Jail.
root@pi-rataplan:~ # ezjail-admin create staftestje001 'vtnet0|<ip>'
Warning: Some services already seem to be listening on all IP, (including 192.168.1.51)
This may cause some confusion, here they are:
root nfsd 93987 5 tcp4 *:2049 *:*
root nfsd 93987 6 tcp6 *:2049 *:*
root mountd 92576 6 udp6 *:1014 *:*
root mountd 92576 7 tcp6 *:1014 *:*
root mountd 92576 8 udp4 *:1014 *:*
root mountd 92576 9 tcp4 *:1014 *:*
root ntpd 88967 20 udp6 *:123 *:*
root ntpd 88967 21 udp4 *:123 *:*
root rpc.statd 86127 4 udp6 *:654 *:*
root rpc.statd 86127 5 tcp6 *:654 *:*
root rpc.statd 86127 6 udp4 *:654 *:*
root rpc.statd 86127 7 tcp4 *:654 *:*
root rpcbind 85696 6 udp6 *:111 *:*
root rpcbind 85696 7 udp6 *:702 *:*
root rpcbind 85696 8 tcp6 *:111 *:*
root rpcbind 85696 9 udp4 *:111 *:*
root rpcbind 85696 10 udp4 *:996 *:*
root rpcbind 85696 11 tcp4 *:111 *:*
root@pi-rataplan:~ #
Review the created Jail.
root@pi-rataplan:~ # ezjail-admin list
STA JID IP Hostname Root Directory
--- ---- --------------- ------------------------------ ------------------------
ZS N/A 192.168.1.51 staftestje001 /usr/jails/staftestje001
root@pi-rataplan:~ #
Start the Jail with ezjail-admin start staftst1
root@pi-rataplan:~ # ezjail-admin start staftst1
Starting jails: staftst1.
/etc/rc.d/jail: WARNING: Per-jail configuration via jail_* variables is obsolete. Please consider migrating to /etc/jail.conf.
root@pi-rataplan:~ #
Access the console with ezjail-admin console
root@pi-rataplan:~ # ezjail-admin console staftestje001
FreeBSD 13.2-RELEASE-p2 GENERIC
Welcome to FreeBSD!
Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories: https://www.FreeBSD.org/security/
FreeBSD Handbook: https://www.FreeBSD.org/handbook/
FreeBSD FAQ: https://www.FreeBSD.org/faq/
Questions List: https://www.FreeBSD.org/lists/questions/
FreeBSD Forums: https://forums.FreeBSD.org/
Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with: pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.
Show the version of FreeBSD installed: freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages: man man
FreeBSD directory layout: man hier
To change this login announcement, see motd(5).
root@staftestje001:~ #
Add a user.
root@staftestje001:~ # adduser
Username: staf
Full name: staf
Uid (Leave empty for default):
Login group [staf]:
Login group is staf. Invite staf into other groups? []: wheel
Login class [default]:
Shell (sh csh tcsh nologin) [sh]:
Home directory [/home/staf]:
Home directory permissions (Leave empty for default):
Use password-based authentication? [yes]:
Use an empty password? (yes/no) [no]:
Use a random password? (yes/no) [no]:
Enter password:
Enter password again:
Lock out the account after creation? [no]: no
Username : staf
Password : *****
Full Name : staf
Uid : 1001
Class :
Groups : staf wheel
Home : /home/staf
Home Mode :
Shell : /bin/sh
Locked : no
OK? (yes/no): yes
adduser: INFO: Successfully added (staf) to the user database.
Add another user? (yes/no): no
Goodbye!
Become the user test user and create some files.
root@staftestje001:~ # su - staf
You can use aliases to decrease the amount of typing you need to do to get
commands you commonly use. Examples of fairly popular aliases include (in
Bourne shell style, as in /bin/sh, bash, ksh, and zsh):
alias lf="ls -FA"
alias ll="ls -lA"
alias su="su -m"
In csh or tcsh, these would be
alias lf ls -FA
alias ll ls -lA
alias su su -m
To remove an alias, you can usually use 'unalias aliasname'. To list all
aliases, you can usually type just 'alias'.
staf@staftestje001:~ $
staf@staftestje001:~ $ vi testfile
Execute ezjail-admin stop
to stop the Jail.
root@pi-rataplan:~ # ezjail-admin stop staftestje001
Stopping jails: staftestje001.
root@pi-rataplan:~ #
Use ezjail-admin archive
to create a tar dump of the Jail.
root@pi-rataplan:~ # ezjail-admin archive staftestje001
pax: Access/modification time set failed on: ./var/empty <Operation not permitted>
Warning: Archiving jail staftestje001 was not completely successful.\n Please refer to the output above for problems the archiving tool encountered.\n You may ignore reports concerning setting access and modification times.\n You might want to check and remove /usr/jails/ezjail_archives/staftestje001-202308161229.21.tar.gz.Warning: Archiving jail staftestje001 was not completely successful. For a running jail this is not unusual.
root@pi-rataplan:~ #
The tar file is created at /usr/jails/ezjail_archives
root@pi-rataplan:~ # ls -l /usr/jails/ezjail_archives
total 267233
-rw-r--r-- 1 root wheel 136712524 Aug 16 12:29 staftestje001-202308161229.21.tar.gz
root@pi-rataplan:~ #
It’s possible to import the ezjail archive with bastille import
.
[root@pi-rataplan ~]# bastille import /usr/jails/ezjail_archives/staftestje001-202308161229.21.tar.gz
Importing 'staftestje001' from foreign compressed .tar.gz archive.
Preparing ZFS environment...
Extracting files from 'staftestje001-202308161229.21.tar.gz' archive...
tar: Removing leading '/' from member names
Generating jail.conf...
Updating symlinks...
Container 'staftestje001' imported successfully.
[root@pi-rataplan ~]#
List the Jails.
[root@pi-rataplan ~]# bastille list -a
JID State IP Address Published Ports Hostname Release Path
bastille-tst001 Up 192.168.1.50 - bastille-tst001 13.2-RELEASE-p2 /usr/local/bastille/jails/bastille-tst001/root
staftestje001 Down vtnet0|192.168.1.51 - staftestje001 13.2-RELEASE-p2 /usr/local/bastille/jails/staftestje001/root
[root@pi-rataplan ~]#
Our archived test Jail is imported.
We defined the interface as part of the ezjail-admin create
command. But this ended up in the IP Address
configuration.
Let’s see how this is defined in our Jail configuration.
Go to the Jail dataset.
root@pi-rataplan:~ # cd /usr/local/bastille/jails/staftestje001/
root@pi-rataplan:/usr/local/bastille/jails/staftestje001 #
List the configuration files.
root@pi-rataplan:/usr/local/bastille/jails/staftestje001 # ls
fstab
fstab.ezjail
jail.conf
prop.ezjail-staftestje001-202309032022.27-pi_rataplan-13.2_RELEASE_p2-aarch64
root
root@pi-rataplan:/usr/local/bastille/jails/staftestje001 #
Edit the jail.conf
root@pi-rataplan:/usr/local/bastille/jails/staftestje001 # vi jail.conf
staftestje001 {
devfs_ruleset = 4;
enforce_statfs = 2;
exec.clean;
exec.consolelog = /var/log/bastille/staftestje001_console.log;
exec.start = '/bin/sh /etc/rc';
exec.stop = '/bin/sh /etc/rc.shutdown';
host.hostname = staftestje001;
mount.devfs;
mount.fstab = /usr/local/bastille/jails/staftestje001/fstab;
path = /usr/local/bastille/jails/staftestje001/root;
securelevel = 2;
interface = vtnet0;
ip4.addr = vtnet0|192.168.1.51;
ip6 = disable;
}
The interface is defined in the interface config and the ip4.addr
.
Remove the interface from the ip4.addr
.
ip4.addr = 192.168.1.51;
Execute bastille list -a
to verify.
root@pi-rataplan:/usr/local/bastille/jails/staftestje001 # bastille list -a
JID State IP Address Published Ports Hostname Release Path
bastille-tst001 Down 192.168.1.50 - bastille-tst001 13.2-RELEASE-p2 /usr/local/bastille/jails/bastille-tst001/root
staftestje001 Down 192.168.1.51
Start the Jail with bastille start
root@pi-rataplan:/usr/local/bastille/jails/staftestje001 # bastille start staftestje001
[staftestje001]:
staftestje001: created
root@pi-rataplan:/usr/local/bastille/jails/staftestje001 #
Test that the test user and files are imported correctly.
[staftestje001]:
Last login: Sun Sep 3 18:02:03 on pts/2
FreeBSD 13.2-RELEASE-p2 GENERIC
Welcome to FreeBSD!
Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories: https://www.FreeBSD.org/security/
FreeBSD Handbook: https://www.FreeBSD.org/handbook/
FreeBSD FAQ: https://www.FreeBSD.org/faq/
Questions List: https://www.FreeBSD.org/lists/questions/
FreeBSD Forums: https://forums.FreeBSD.org/
Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with: pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.
Show the version of FreeBSD installed: freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages: man man
FreeBSD directory layout: man hier
To change this login announcement, see motd(5).
root@staftestje001:~ # su - staf
Need to quickly return to your home directory? Type "cd".
-- Dru <genesis@istar.ca>
staf@staftestje001:~ $ ls
testfile
staf@staftestje001:~ $
The last step is to remove the “old” ezjail.
[root@pi-rataplan ~]# ezjail-admin list
STA JID IP Hostname Root Directory
--- ---- --------------- ------------------------------ ------------------------
ZS N/A 192.168.1.51 staftestje001 /usr/jails/staftestje001
ZR 2 192.168.1.49 stafscm /usr/jails/stafscm
ZR 3 192.168.1.45 stafproxy /usr/jails/stafproxy
ZR 4 192.168.1.47 stafmail /usr/jails/stafmail
ZR 5 192.168.1.41 staffs /usr/jails/staffs
ZR 6 192.168.1.85 stafdns /usr/jails/stafdns
[root@pi-rataplan ~]# ezjail-admin delete staftestje001
[root@pi-rataplan ~]#
ezjail delete
only removes the Jail configuration. The storage is still there. Might be useful if you want to restore the Jail. And we still have a backup in /usr/local/jails/archives
if for some reason we need to restore the old ezjail.
[root@pi-rataplan ~]# zfs list | grep -i testje001
zroot/bastille/jails/staftestje001 219M 153G 144K /usr/local/bastille/jails/staftestje001
zroot/bastille/jails/staftestje001/root 219M 153G 219M /usr/local/bastille/jails/staftestje001/root
zroot/usr/jails/staftestje001 219M 153G 219M /usr/jails/staftestje001
[root@pi-rataplan ~]#
As the procedure seems to work, I’ll continue with migration with the ezjail Jails to BastilleBSD :-)
I use FreeBSD on my home network to serve services like email, git, fileserver, etc. For some other services, I use k3s with GNU/Linux application containers.
The FreeBSD services run as Jails. For those who aren’t familiar with FreeBSD Jails. Jails started the whole concept of “containers”.
FreeBSD Jails inspired Sun Microsystems to create Solaris zones.
If you want to know more about the history of FreeBSD Jails, Solaris zones and containers on Un!x systems in general and the challenges to run containers securely I recommend the video;
“Papers We Love: Jails and Solaris Zones by Bryan Cantrill”
Sun took containers to the next level with Solaris zones , allowing a fine-grade CPU and memory allocation.
On GNU/Linux LXC was the most popular container framework. …Till Docker came along.
To the credit of Docker, Docker made the concept of application containers popular.
System containers run the complete operating system and can be used like virtual machines without the overhead.
Application containers run a single application binary inside a container that holds all the dependencies for this application.
FreeBSD Jails can be used for both “application” containers and “system” containers. FreeBSD Jails is a framework that separates the host and the Jails with security in mind.
To make the management of Jails easier we have management tools. I started to use ezjail in 2013 after my OpenSolaris system died and Oracle killed OpenSolaris.
ezjail isn’t developed that actively.
BastilleBSD is the most the popular Jail management tool for FreeBSD Jails. It supports both application and system containers.
I migrated all the services to Raspberry Pi’s to save electricity. But continued to use ezjail, just to stick to something that I knew and was stable. Migrating to BastileBSD was still on my to-do list and I finally found the time to do it.
My blog posts are mostly my installation notes that I publish in the hope that they are useful to somebody else.
In this blog post, you’ll find my journey to explore BastileBSD, in a next blog post, I’ll go through the migration from ezjail to BastilleBSD.
If you are interested in the concepts behind BastilleBSD, I recommend the video:
FreeBSD Fridays: Introduction to BastilleBSD
We’ll execute all actions on a virtual machine running FreeBSD on a Raspberry PI. The Raspberry PI is running Debian GNU/Linux with the KVM/libvirt hypervisor.
The virtual machine is running FreeBSD 13.2.
root@pi-rataplan:~ # freebsd-version
13.2-RELEASE-p2
root@pi-rataplan:~ #
root@pi-rataplan:~ # uname -a
FreeBSD pi-rataplan 13.2-RELEASE-p2 FreeBSD 13.2-RELEASE-p2 GENERIC arm64
root@pi-rataplan:~ #
The first step is to install bastille
.
root@pi-rataplan:~ # pkg install -y bastille
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
The following 1 package(s) will be affected (of 0 checked):
New packages to be INSTALLED:
bastille: 0.9.20220714
Number of packages to be installed: 1
38 KiB to be downloaded.
[1/1] Fetching bastille-0.9.20220714.pkg: 100% 38 KiB 38.8kB/s 00:01
Checking integrity... done (0 conflicting)
[1/1] Installing bastille-0.9.20220714...
[1/1] Extracting bastille-0.9.20220714: 100%
root@pi-rataplan:~ #
BastilleBSD is not a daemon that runs in the background, but starts the containers when the bastille_enable
system configuration rc.conf variable is set to YES
.
FreeBSD has a nice tool to set the system configuration variables; sysrc.
root@pi-rataplan:~ # sysrc bastille_enable=YES
bastille_enable: -> YES
root@pi-rataplan:~ #
It’s possible to specify a list of containers in the bastille_list
, the containers are started in the order of the list.
BastilleBSD can use OpenZFS, but this isn’t enabled by default. When ZFS support is enabled a new ZFS dataset is created when a container is created.
root@pi-rataplan:/usr/local/etc/bastille # cd /usr/local/etc/bastille/
root@pi-rataplan:/usr/local/etc/bastille #
root@pi-rataplan:/usr/local/etc/bastille # vi bastille.conf
To enable OpenZFS support you need to set bastille_zfs_enable
to YES
and specify the zpool to be used.
Please note that YES
need to be in upper case.
## ZFS options
bastille_zfs_enable="YES" ## default: ""
bastille_zfs_zpool="zroot" ## default: ""
bastille_zfs_prefix="bastille" ## default: "${bastille_zfs_zpool}/bast
ille"
bastille_zfs_options="-o compress=lz4 -o atime=off" ## default: "-o compress=lz4 -o atime=o
ff"
In order to start a container you need to bootstrap
a FreeBSD release first.
root@pi-rataplan:/usr/local/share/bastille # freebsd-version
13.2-RELEASE-p2
A FreeBSD release is without the patch level.
The command below will bootstrap the 13.2-RELEASE
on bastilleBSD. The update
option will also include the latest patches.
root@pi-rataplan:/usr/local/share/bastille # bastille bootstrap 13.2-RELEASE update
<snip>
Installing updates...
Restarting sshd after upgrade
Performing sanity check on sshd configuration.
Stopping sshd.
Waiting for PIDS: 93019.
Performing sanity check on sshd configuration.
Starting sshd.
Scanning /usr/local/bastille/releases/13.2-RELEASE/usr/share/certs/blacklisted for certificates...
Scanning /usr/local/bastille/releases/13.2-RELEASE/usr/share/certs/trusted for certificates...
done.
root@pi-rataplan:/usr/local/share/bastille #
You can use the bastille list release
command to list the bootstrapped releases.
root@pi-rataplan:/usr/local/bastille/jails/bastille-tst001 # bastille list release
13.1-RELEASE
13.2-RELEASE
root@pi-rataplan:/usr/local/bastille/jails/bastille-tst001 #
The releases are stored in the bastilleBSD dataset /usr/local/bastille/releases
in our case.
root@pi-rataplan:/usr/local/bastille # cd releases/
root@pi-rataplan:/usr/local/bastille/releases # ls
13.1-RELEASE 13.2-RELEASE
root@pi-rataplan:/usr/local/bastille/releases #
The downloaded tar archive are stored in the cache
ZFS dataset.
root@pi-rataplan:/usr/local/bastille/releases/13.1-RELEASE # zfs list | grep -i bastille | grep -i cache
zroot/bastille/cache 334M 93.6G 104K /usr/local/bastille/cache
zroot/bastille/cache/13.1-RELEASE 165M 93.6G 165M /usr/local/bastille/cache/13.1-RELEASE
zroot/bastille/cache/13.2-RELEASE 169M 93.6G 169M /usr/local/bastille/cache/13.2-RELEASE
root@pi-rataplan:/usr/local/bastille/releases/13.1-RELEASE #
root@pi-rataplan:/usr/local/bastille/releases/13.1-RELEASE # ls -l /usr/local/bastille/cache/13.2-RELEASE
total 345866
-rw-r--r-- 1 root wheel 782 Apr 7 07:01 MANIFEST
-rw-r--r-- 1 root wheel 176939748 Apr 7 07:01 base.txz
root@pi-rataplan:/usr/local/bastille/releases/13.1-RELEASE #
With bastille verify
you can verify a release to be sure that no files are altered.
root@pi-rataplan:/usr/local/bastille/releases # bastille verify 13.2-RELEASE
src component not installed, skipped
Looking up update.FreeBSD.org mirrors... 2 mirrors found.
Fetching metadata signature for 13.2-RELEASE from update2.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
root@pi-rataplan:/usr/local/bastille/releases #
“Thin” Jails are created by default. With a “thin” Jail the operating system is “shared” with the release. This saves a lot of disk space. It’s possible to convert a container to a “thick” Jail after a Jail is created.
BastilleBSD has a lot of network options, it can also create dynamic firewall rules to expose services.
See https://docs.bastillebsd.org/en/latest/chapters/networking.html for more information.
In this example we’ll use a “shared” network interface vtnet0
with the host system.
root@pi-rataplan:/usr/local/bastille/releases # bastille create bastille-tst001 13.2-RELEASE <ip_address> vtnet0
Valid: (<ip_address>).
Valid: (vtnet0).
Creating a thinjail...
[bastille-tst001]:
bastille-tst001: created
[bastille-tst001]:
Applying template: default/thin...
[bastille-tst001]:
Applying template: default/base...
[bastille-tst001]:
[bastille-tst001]: 0
[bastille-tst001]:
syslogd_flags: -s -> -ss
[bastille-tst001]:
sendmail_enable: NO -> NO
[bastille-tst001]:
sendmail_submit_enable: YES -> NO
[bastille-tst001]:
sendmail_outbound_enable: YES -> NO
[bastille-tst001]:
sendmail_msp_queue_enable: YES -> NO
[bastille-tst001]:
cron_flags: -> -J 60
[bastille-tst001]:
/etc/resolv.conf -> /usr/local/bastille/jails/bastille-tst001/root/etc/resolv.conf
Template applied: default/base
Template applied: default/thin
rdr-anchor not found in pf.conf
[bastille-tst001]:
bastille-tst001: removed
[bastille-tst001]:
bastille-tst001: created
root@pi-rataplan:/usr/local/bastille/releases #
A new ZFS dataset is created for the Jail.
root@pi-rataplan:/usr/local/bastille/releases/13.1-RELEASE # zfs list | grep -i tst001
zroot/bastille/jails/bastille-tst001 5.61M 93.6G 116K /usr/local/bastille/jails/bastille-tst001
zroot/bastille/jails/bastille-tst001/root 5.50M 93.6G 5.50M /usr/local/bastille/jails/bastille-tst001/root
root@pi-rataplan:/usr/local/bastille/releases/13.1-RELEASE #
The mounted Jail dataset holds the Jail configuration file and fstab
that is used by the Jail.
root@pi-rataplan:/usr/local/bastille/jails/bastille-tst001 # cd /usr/local/bastille/jails/bastille-tst001
root@pi-rataplan:/usr/local/bastille/jails/bastille-tst001 # ls
fstab jail.conf root
root@pi-rataplan:/usr/local/bastille/jails/bastille-tst001 #
jail.conf:
root@pi-rataplan:/usr/local/bastille/jails/bastille-tst001 # cat jail.conf
bastille-tst001 {
devfs_ruleset = 4;
enforce_statfs = 2;
exec.clean;
exec.consolelog = /var/log/bastille/bastille-tst001_console.log;
exec.start = '/bin/sh /etc/rc';
exec.stop = '/bin/sh /etc/rc.shutdown';
host.hostname = bastille-tst001;
mount.devfs;
mount.fstab = /usr/local/bastille/jails/bastille-tst001/fstab;
path = /usr/local/bastille/jails/bastille-tst001/root;
securelevel = 2;
interface = vtnet0;
ip4.addr = <ip_address>;
ip6 = disable;
}
root@pi-rataplan:/usr/local/bastille/jails/bastille-tst001
fstab:
root@pi-rataplan:/usr/local/bastille/jails/bastille-tst001 # cat /usr/local/bastille/jails/bastille-tst001/fstab
/usr/local/bastille/releases/13.2-RELEASE /usr/local/bastille/jails/bastille-tst001/root/.bastille nullfs ro 0 0
root@pi-rataplan:/usr/local/bastille/jails/bastille-tst001 #
With bastille list
we can list the containers. When no option is given it’ll list the running containers only.
To list all containers (stopped and running) you can use the -a
option.
Note that bastille list
is a wrapper around the jls command and also lists the running ezjail containers.
root@pi-rataplan:/usr/jails/stafproxy/basejail # bastille list
JID IP Address Hostname Path
stafscm <ip> stafscm /usr/jails/stafscm
stafproxy <ip> stafproxy /usr/jails/stafproxy
stafmail <ip> stafmail /usr/jails/stafmail
staffs <ip> staffs /usr/jails/staffs
stafdns <ip> stafdns /usr/jails/stafdns
bastille-tst001 <ip> bastille-tst001 /usr/local/bastille/jails/bastille-tst001/root
root@pi-rataplan:/usr/jails/stafproxy/basejail #
To gain console access you use the bastille console
command.
root@pi-rataplan:/usr/jails/stafproxy/basejail # bastille console bastille-tst001
[bastille-tst001]:
root@bastille-tst001:~ #
Verify the disk space. We’re using a “thin” Jail only, 5.5M
of disk space is used.
root@bastille-tst001:~ # df -h .
Filesystem Size Used Avail Capacity Mounted on
zroot/bastille/jails/bastille-tst001/root 154G 5.5M 154G 0% /
root@bastille-tst001:~ #
We’re using “thin” Jails. The system binaries are read-only.
[root@bastille-tst001 ~]# ls -l /bin
lrwxr-xr-x 1 root wheel 14 Aug 16 10:25 /bin -> /.bastille/bin
[root@bastille-tst001 ~]#
[root@bastille-tst001 ~]# touch /bin/ls
touch: /bin/ls: Read-only file system
[root@bastille-tst001 ~]#
[root@bastille-tst001 ~]# freebsd-version
13.2-RELEASE-p2
[root@bastille-tst001 ~]# uname -a
FreeBSD bastille-tst001 13.2-RELEASE-p2 FreeBSD 13.2-RELEASE-p2 GENERIC arm64
[root@bastille-tst001 ~]#
Bootstrap the pkg
command to install packages.
root@bastille-tst001:~ # pkg
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:13:aarch64/quarterly, please wait...
Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done
[bastille-tst001] Installing pkg-1.19.2...
[bastille-tst001] Extracting pkg-1.19.2: 100%
pkg: not enough arguments
Usage: pkg [-v] [-d] [-l] [-N] [-j <jail name or id>|-c <chroot path>|-r <rootdir>] [-C <configuration file>] [-R <repo config dir>] [-o var=value] [-4|-6] <command> [<args>]
For more information on available commands and options see 'pkg help'.
root@bastille-tst001:~ #
I use ansible to manage my homelab python3
and sudo
are required for this, so these are usually the first packages I install.
root@bastille-tst001:~ # pkg install -y sudo python3 bash
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
The following 9 package(s) will be affected (of 0 checked):
New packages to be INSTALLED:
bash: 5.2.15
gettext-runtime: 0.21.1
indexinfo: 0.3.1
libffi: 3.4.4
mpdecimal: 2.5.1
python3: 3_3
python39: 3.9.17
readline: 8.2.1
sudo: 1.9.14p3
Number of packages to be installed: 9
The process will require 140 MiB more space.
21 MiB to be downloaded.
[bastille-tst001] [1/9] Fetching indexinfo-0.3.1.pkg: 100% 5 KiB 5.5kB/s 00:01
[bastille-tst001] [2/9] Fetching mpdecimal-2.5.1.pkg: 100% 292 KiB 299.4kB/s 00:01
[bastille-tst001] [3/9] Fetching python39-3.9.17.pkg: 100% 17 MiB 2.6MB/s 00:07
[bastille-tst001] [4/9] Fetching libffi-3.4.4.pkg: 100% 36 KiB 36.6kB/s 00:01
[bastille-tst001] [5/9] Fetching readline-8.2.1.pkg: 100% 345 KiB 353.1kB/s 00:01
[bastille-tst001] [6/9] Fetching sudo-1.9.14p3.pkg: 100% 2 MiB 1.6MB/s 00:01
[bastille-tst001] [7/9] Fetching python3-3_3.pkg: 100% 1 KiB 1.1kB/s 00:01
[bastille-tst001] [8/9] Fetching bash-5.2.15.pkg: 100% 2 MiB 1.6MB/s 00:01
[bastille-tst001] [9/9] Fetching gettext-runtime-0.21.1.pkg: 100% 160 KiB 164.0kB/s 00:01
Checking integrity... done (0 conflicting)
[bastille-tst001] [1/9] Installing indexinfo-0.3.1...
[bastille-tst001] [1/9] Extracting indexinfo-0.3.1: 100%
[bastille-tst001] [2/9] Installing mpdecimal-2.5.1...
[bastille-tst001] [2/9] Extracting mpdecimal-2.5.1: 100%
[bastille-tst001] [3/9] Installing libffi-3.4.4...
[bastille-tst001] [3/9] Extracting libffi-3.4.4: 100%
[bastille-tst001] [4/9] Installing readline-8.2.1...
[bastille-tst001] [4/9] Extracting readline-8.2.1: 100%
[bastille-tst001] [5/9] Installing gettext-runtime-0.21.1...
[bastille-tst001] [5/9] Extracting gettext-runtime-0.21.1: 100%
[bastille-tst001] [6/9] Installing python39-3.9.17...
[bastille-tst001] [6/9] Extracting python39-3.9.17: 100%
[bastille-tst001] [7/9] Installing sudo-1.9.14p3...
[bastille-tst001] [7/9] Extracting sudo-1.9.14p3: 100%
[bastille-tst001] [8/9] Installing python3-3_3...
[bastille-tst001] [8/9] Extracting python3-3_3: 100%
[bastille-tst001] [9/9] Installing bash-5.2.15...
[bastille-tst001] [9/9] Extracting bash-5.2.15: 100%
=====
Message from python39-3.9.17:
--
Note that some standard Python modules are provided as separate ports
as they require additional dependencies. They are available as:
py39-gdbm databases/py-gdbm@py39
py39-sqlite3 databases/py-sqlite3@py39
py39-tkinter x11-toolkits/
Let’s install neofetch.
[root@bastille-tst001 ~]# pkg install -y neofetch
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
The following 1 package(s) will be affected (of 0 checked):
New packages to be INSTALLED:
neofetch: 7.1.0_1
Number of packages to be installed: 1
79 KiB to be downloaded.
[bastille-tst001] [1/1] Fetching neofetch-7.1.0_1.pkg: 100% 79 KiB 81.0kB/s 00:01
Checking integrity... done (0 conflicting)
[bastille-tst001] [1/1] Installing neofetch-7.1.0_1...
[bastille-tst001] [1/1] Extracting neofetch-7.1.0_1: 100%
[root@bastille-tst001 ~]#
[root@bastille-tst001 ~]# neofetch
` `.....---.......--.``` -/ --------------------
+o .--` /y:` +. OS: FreeBSD 13.2-RELEASE-p2 aarch64
yo`:. :o `+- Uptime: 3 days, 14 hours, 13 mins
y/ -/` -o/ Packages: 11 (pkg)
.- ::/sy+:. Shell: csh tcsh 6.22.04
/ `-- / Terminal: /dev/pts/1
`: :` CPU: ARM Cortex-A72 r0p3 (4)
`: :` Memory: 2193MiB / 3039MiB
/ /
.- -.
-- -.
`:` `:`
.-- `--.
.---.....----.
[root@bastille-tst001 ~]#
[root@bastille-tst001 ~]#
exit
root@bastille-tst001:~ # logout
List the releases.
root@pi-rataplan:~ # bastille list release
13.1-RELEASE
13.2-RELEASE
root@pi-rataplan:~ #
Update a release to the latest patch level.
root@pi-rataplan:/usr/jails/stafproxy/basejail # bastille update 13.2-RELEASE
src component not installed, skipped
Looking up update.FreeBSD.org mirrors... 2 mirrors found.
Fetching metadata signature for 13.2-RELEASE from update2.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files... done.
No updates needed to update system to 13.2-RELEASE-p2.
No updates are available to install.
You can use bastille cmd
to execute a command inside a container which is a wrapper around the jexec.
The syntax is
bastille cmd <container|ALL> <command string>
root@pi-rataplan:~ # bastille cmd ALL ps aux
[bastille-tst001]:
grep: /usr/local/bastille/jails/ALL/fstab: No such file or directory
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
root 35869 0.0 0.1 12876 2496 - IJ 10:55 0:00.00 cron: running job (cron)
root 46730 0.0 0.1 12704 2696 - SsJ 10:25 0:00.01 /usr/sbin/syslogd -ss
root 55608 0.0 0.1 12876 2496 - IJ 10:55 0:00.00 cron: running job (cron)
root 69222 0.0 0.1 12876 2512 - IsJ 10:25 0:00.01 /usr/sbin/cron -J 60 -s
root 67674 0.0 0.1 13440 2960 1 R+J 10:55 0:00.00 ps aux
[bastille-tst001]: 0
root@pi-rataplan:~ #
FreeBSD has implemented the vulnerability management in the correct way. The vulnerability database is outside of the packages management database as it should be.
Unfortunately the FreeBSD vulnerability database isn’t compatible with the OVAL standard.
This makes auditing the installed FreeBSD packages with the pkg audit command in a way comparable with a security audit with OpenSCAP on a GNU/Linux system.
But this might be a topic for another blog post :-)
root@pi-rataplan:~ # bastille pkg ALL audit -F
[bastille-tst001]:
[bastille-tst001] Fetching vuln.xml.xz: 100% 1 MiB 349.2kB/s 00:03
0 problem(s) in 0 installed package(s) found.
root@pi-rataplan:~ #
Have fun!
And created a few ansible to provision the virtual machines with cloud image with cloud-init and deploy k3s on it.
I updated the roles below to be compatible with the latest Debian release: Debian 12 bookworm.
I created a movie to demonstrate how you can setup a kubernetes homelab in few minutes.
The latest version 1.1.0 is available at: https://github.com/stafwag/ansible-k3s-on-vms
Have fun!
stafwag.cloud_localds 2.1.2 is available at: https://github.com/stafwag/ansible-role-cloud_localds
stafwag.virt_install_vm 1.1.0 is available at: https://github.com/stafwag/ansible-role-virt_install_vm
Debian GNU/Linux 12 templates
stafwag.delegated_vm_install 2.0.0 is available at: https://github.com/stafwag/ansible-role-delegated_vm_install
Debian GNU/Linux 12 template Latest
This time none of the commits are created by me :-)
Thanks to https://github.com/fazlerabbi37 for your contributions!
Have fun!
stafwag.qemu_img 2.2.0 is available at: https://github.com/stafwag/ansible-role-qemu_img
I use the lightweight Kubernetes K3s on a 3-node Raspberry Pi 4 cluster. I wrote a few blog posts on how the Raspberry Pi’s are installed.
I run K3s on virtual machines.
Why virtual machines?
Virtual makes it easier to redeploy or to bring a system down and up if your want to test something.
Another reason is that I also run FreeBSD virtual machines on the Raspberry Pis.
I use Debian GNU/Linux as the Operating system with KVM/libvirt as the hypervisor.
I use Ansible to set up the cluster in an automated way. Got finality the time to clean up the code a bit and release it on Github: https://github.com/stafwag/ansible-k3s-on-vms
The code can also - and will by default - be used on x86 systems.
The playbook is a wrapper around the roles:
To set up the virtual machines.
To install and configure K3s on the virtual machines.
To enable libvirt on the vm_kvm_host
.
The sample inventory will install the virtual machines on localhost. It’s possible to install the virtual machine on multiple lbvirt/KVM hypervisors.
This should enable you setup the virtual machine with k3s in … 5 minutes
(*) if everything goes well the first time :-)
Have fun
]]>The Ansible role stafwag.users is available at: https://github.com/stafwag/ansible-role-users
This release implements a shell parameters to define shell for an user. See the github issue for more details.
shell parameter
Have fun!
An ansible role to manage user and user files - files in the home directory -.
None
The following variables are set by the role.
state: absent | present (default) |
password_lock: no | yes lock password (ansible 2.6+) |
state: absent | present (default) Whether the given key (with the given key_options) should or should not be in the file. |
exclusive: no (default) | yes. Whether to remove all other non-specified keys from the authorized_keys file. |
state: absent | present (default) |
backup: no (default) | yes. create a backup file. |
dir_create: false (default) | true. |
dir_recurse: no (default) | yes create the directory recursively. |
state: absent | present (default) |
backup: no (default) | yes Create a backup |
None
- name: add user & ssh_authorized_key
hosts: testhosts
become: true
vars:
users:
- name: test0001
group: test0001
password: ""
state: "present"
ssh_authorized_keys:
- key: ""
key_options: "no-agent-forwarding"
roles:
- stafwag.users
- name: add user to the sudo group
hosts: testhosts
become: true
vars:
users:
- name: test0001
groups: ""
append: true
roles:
- stafwag.users
- name: setup tyr ssh_config
become: true
hosts: tyr
vars:
users:
- name: staf
user_files:
- name: ssh config
path: .ssh/config
dir_create: true
state: present
- name: ssh config.d/intern_config
path: .ssh/config.d/intern_config
content: ""
dir_create: true
user_lineinfiles:
- name: include intern_config
path: .ssh/config
state: present
regexp: "^include config.d/intern_config"
line: "include config.d/intern_config"
roles:
- stafwag.users
MIT/BSD
Created by Staf Wagemakers, email: staf@wagemakers.be, website: http://www.wagemakers.be.
An ansible role to manage user and user files - files in the home directory -.
None
The following variables are set by the role.
state: absent | present (default) |
password_lock: no | yes lock password (ansible 2.6+) |
state: absent | present (default) Whether the given key (with the given key_options) should or should not be in the file. |
exclusive: no (default) | yes. Whether to remove all other non-specified keys from the authorized_keys file. |
state: absent | present (default) |
backup: no (default) | yes. create a backup file. |
dir_create: false (default) | true. |
dir_recurse: no (default) | yes create the directory recursively. |
state: absent | present (default) |
backup: no (default) | yes Create a backup |
None
- name: add user & ssh_authorized_key
hosts: testhosts
become: true
vars:
users:
- name: test0001
group: test0001
password: ""
state: "present"
ssh_authorized_keys:
- key: ""
key_options: "no-agent-forwarding"
roles:
- stafwag.users
- name: add user to the sudo group
hosts: testhosts
become: true
vars:
users:
- name: test0001
groups: ""
append: true
roles:
- stafwag.users
- name: setup tyr ssh_config
become: true
hosts: tyr
vars:
users:
- name: staf
user_files:
- name: ssh config
path: .ssh/config
dir_create: true
state: present
- name: ssh config.d/intern_config
path: .ssh/config.d/intern_config
content: ""
dir_create: true
user_lineinfiles:
- name: include intern_config
path: .ssh/config
state: present
regexp: "^include config.d/intern_config"
line: "include config.d/intern_config"
roles:
- stafwag.users
MIT/BSD
Created by Staf Wagemakers, email: staf@wagemakers.be, website: http://www.wagemakers.be.
]]>