stafwag Blog

staf wagemakers blog

32 Bits Matters!

"32bits_opnsense.jpg"

pfsense 2.3

My firewall is a pcengines alix.

It was running pfsense and was quite happy about it. Pfsense dropped support for 32 bits in their pfsense 2.4 release.

This would left me with a unsupported firewall which was one of the reasons to use pfsense instead of a closed source commercial router.

I could have moved to a new firewall like the pcengines apu but there is no reason to replace hardware that works fine.

The nice thing about opensource software is that we’ve options to choose from if software doesn’t match your usecase we’ve other options to choose from.

OPNsense

So I decided to give opnsense a try. OPNsense is a fork of pfsense, both are a fork of m0n0wall.

"opnsense_swapspace.png"

swapspace

My firewall only has 256 MB of memory which is a bit low even for a firewall.

The OPNsense developers made it very easy to add swapspace from the GUI. To add swap space go to [ System ] > [ Miscellaneous ] and activate the [ Add a 2 GB swap file to the system ] checkbox.

I’m verify satisfied with the upgrade from pfsense to OPNsense, OPNsense has a new release very month which is nice to get the latest security updates and it’s possible to audit the systems for security updates from the GUI.

"duckdns"

DuckDns

I move my ADSL with a fixed ip address to a VDSL line with a dynamic ip address so I was looking a good free dynamic dns provider and settled with duckdns.

Have fun

How to Start DLM Monitoring on a VDSL Line in Belgium

In Belgium/Flanders we have two main internet line providers;

  • telenet the cablenet network provider.
  • proximus is the telephone network provider.

On telephone network there are alternative internet providers but they use the network of proximus.

I switched my internet connection from ADSL to VDSL and switched to a new provider ( edpnet). The internet speed was below the expectations and my modem reported errors on the line. After fixing the internal phone cabbeling in my appartment I wanted the retrigger the DLM monitoring.

The process is explained in the this post https://userbase.be/forum/viewtopic.php?t=48767 at usebase.be

To start the DLM monitoring in Belgium you need to call 0800 22 424 and type in your line number. If you don’t have a proximus phone number the line number is not the same as your phone number. To get your line number you need to connect an analog phone to our line and call 1924 this will read aloud your line number.

Have fun

High Screen Resolution on a KVM Virtual Machine With QXL

When you create an new virtual KVM virtual system the video ram is limited to 16MB by default to use a higer screen resolution you need to increase the video ram. The available resolution reported by the virtual screen may also not include the resolution that you want to utilize.

You’ll find my journey to enable higher screen resolutions in my KVM (qemu) virtual systems below.

Ubuntu 16.04

There is an issue with Ubuntu 16.04 and the latest HWE kernel https://wiki.ubuntu.com/Kernel/LTSEnablementStack. Even a full HD resultion (1920 x 1080 ) if you have the latest HWE kernel on your system.

To resolve this issue your can uninstall the latest kernel or install the LTS kernel.

Install the LTS Kernel

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
staf@ubuntu:~$ sudo apt-get install linux-generic-lts-xenial
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  linux-generic linux-headers-4.4.0-119 linux-headers-4.4.0-119-generic linux-headers-generic
  linux-image-4.4.0-119-generic linux-image-extra-4.4.0-119-generic linux-image-generic
Suggested packages:
  fdutils linux-doc-4.4.0 | linux-source-4.4.0 linux-tools
The following NEW packages will be installed:
  linux-generic linux-generic-lts-xenial linux-headers-4.4.0-119 linux-headers-4.4.0-119-generic
  linux-headers-generic linux-image-4.4.0-119-generic linux-image-extra-4.4.0-119-generic linux-image-generic
0 upgraded, 8 newly installed, 0 to remove and 0 not upgraded.
Need to get 69,3 MB of archives.
After this operation, 301 MB of additional disk space will be used.
Do you want to continue? [Y/n] 
<snip>
Setting up linux-image-generic (4.4.0.119.125) ...
Setting up linux-headers-4.4.0-119 (4.4.0-119.143) ...
Setting up linux-headers-4.4.0-119-generic (4.4.0-119.143) ...
Setting up linux-headers-generic (4.4.0.119.125) ...
Setting up linux-generic (4.4.0.119.125) ...
Setting up linux-generic-lts-xenial (4.4.0.119.125) ...
staf@ubuntu:~$ 

Remove the HWE kernel

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
staf@ubuntu:~$ sudo apt-get purge linux-image-4.13*
Reading package lists... Done
Building dependency tree       
Reading state information... Done
<snip>
done
The link /vmlinuz.old is a damaged link
Removing symbolic link vmlinuz.old 
 you may need to re-run your boot loader[grub]
The link /initrd.img.old is a damaged link
Removing symbolic link initrd.img.old 
 you may need to re-run your boot loader[grub]
Purging configuration files for linux-image-4.13.0-38-generic (4.13.0-38.43~16.04.1) ...
Examining /etc/kernel/postrm.d .
run-parts: executing /etc/kernel/postrm.d/initramfs-tools 4.13.0-38-generic /boot/vmlinuz-4.13.0-38-generic
run-parts: executing /etc/kernel/postrm.d/zz-update-grub 4.13.0-38-generic /boot/vmlinuz-4.13.0-38-generic

Cleanup

1
2
3
4
5
6
7
8
9
10
11
12
13
14
staf@ubuntu:~$ sudo apt autoremove
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be REMOVED:
  linux-headers-4.13.0-36 linux-headers-4.13.0-36-generic linux-headers-generic-hwe-16.04
0 upgraded, 0 newly installed, 3 to remove and 0 not upgraded.
After this operation, 83,1 MB disk space will be freed.
Do you want to continue? [Y/n] 
(Reading database ... 234149 files and directories currently installed.)
Removing linux-headers-4.13.0-36-generic (4.13.0-36.40~16.04.1) ...
Removing linux-headers-4.13.0-36 (4.13.0-36.40~16.04.1) ...
Removing linux-headers-generic-hwe-16.04 (4.13.0.38.57) ...
staf@ubuntu:~$ 

Reboot

After a reboot higher resolutions are possible on ubuntu 16.04

Increase the video RAM

Required video ram

When you create a new KVM virtual machine it has 16MB of video RAM. Below you’ll the calculation for the required video RAM for a 4k resolution ( 3840 x 2160 ).

3840 x 2160 = 8294400
8294400 x 32 = 265420800
265420800 / 8 = 33177600
33177600 / (1024*1024) = 31.640625 MB

So 32 MB video ram is enough for a 4k resolution, to take some overhead into account we’ll increase the video ram to 64 MB.

list the domains

1
2
3
4
5
6
7
8
9
10
11
12
13
14
[swagemakers@staflaptop ~]$ sudo virsh
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # list --all
 Id    Name                           State
----------------------------------------------------
 -     centos7.0                      shut off
 -     debian                         shut off
 -     fedora27                       shut off

virsh # 

edit the domain settings

1
virsh # edit --domain debian
update the memory settings
1
2
3
4
5
<video>
  <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</video>
<redirdev bus='usb' type='spicevmc'>

to

1
2
3
4
<video>
  <model type='qxl' ram='65536' vram='65536' vgamem='65536' heads='1' primary='yes'/>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</video>

xrandr

Even with the additional RAM higer resolution aren’t possible (yet), the virtual screen doesn’t report the higer screen resolution. It’s possible to add the higher screen resolution with xrandr.

display current settings
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
staf@debian:~$ xrandr 
Screen 0: minimum 320 x 200, current 1920 x 1080, maximum 8192 x 8192
Virtual-0 connected primary 1920x1080+0+0 0mm x 0mm
   1024x768      59.95 +
   1920x1200     59.95  
   1920x1080     60.00* 
   1600x1200     59.95  
   1680x1050     60.00  
   1400x1050     60.00  
   1280x1024     59.95  
   1440x900      59.99  
   1280x960      59.99  
   1280x854      59.95  
   1280x800      59.96  
   1280x720      59.97  
   1152x768      59.95  
   800x600       59.96  
   848x480       59.94  
   720x480       59.94  
   640x480       59.94  
Virtual-1 disconnected
Virtual-2 disconnected
Virtual-3 disconnected
staf@debian:~$ 
get the modeline
1
2
3
4
staf@debian:~$ cvt 2560 1440 
# 2560x1440 59.96 Hz (CVT 3.69M9) hsync: 89.52 kHz; pclk: 312.25 MHz
Modeline "2560x1440_60.00"  312.25  2560 2752 3024 3488  1440 1443 1448 1493 -hsync +vsync
staf@debian:~$ 
# create the new mode line
1
2
staf@debian:~$ xrandr --newmode "2560x1440_60.00"  312.25  2560 2752 3024 3488  1440 1443 1448 1493 -hsync +vsync
staf@debian:~$ 
# add the mode to your screen
1
2
staf@debian:~$ xrandr --addmode Virtual-0 2560x1440_60.00
staf@debian:~$ 
# use the new mode
1
2
staf@debian:~$ xrandr --output Virtual-0 --mode 2560x1440_60.00
staf@debian:~$ 
## 4k

To use a 4k resolution you can use the commands

1
2
3
4
5
6
7
staf@debian:~$  cvt 3840 2160
# 3840x2160 59.98 Hz (CVT 8.29M9) hsync: 134.18 kHz; pclk: 712.75 MHz
Modeline "3840x2160_60.00"  712.75  3840 4160 4576 5312  2160 2163 2168 2237 -hsync +vsync
staf@mydevolo:~$ xrandr --newmode "3840x2160_60.00"  712.75  3840 4160 4576 5312  2160 2163 2168 2237 -hsync +vsync
staf@mydevolo:~$ xrandr --addmode Virtual-0 3840x2160_60.00
staf@mydevolo:~$ xrandr --output Virtual-0 --mode 3840x2160_60.00
staf@mydevolo:~$ 

Add the new screen resolution permanently

Debian & Co

Create a monitor configuration file in /usr/share/X11/xorg.conf.d

1
root@mydevolo:/usr/share/X11/xorg.conf.d# vi 10-monitor.conf

And add the modeline fgor your screen resolution. With the Option “PreferredMode” you can set the prferred resolution.

1
2
3
4
5
6
section "Monitor"
    Identifier "Virtual-0 "
    Modeline "2560x1440_60.00"  312.25  2560 2752 3024 3488  1440 1443 1448 1493 -hsync +vsync
    Modeline "3840x2160_60.00"  712.75  3840 4160 4576 5312  2160 2163 2168 2237 -hsync +vsync
    Option "PreferredMode" "2560x1440_60.00"
EndSection

Other GNU/Linux distros

Most other GNU/Linux distribution use /etc/X11/xorg.conf.d/

Have fun

Links

Postfix Smarthost With Authentication

"postfix"

I used the relay host of my internet provider but this was causing issues since my email was getting mark as SPAM in gmail.
 
It was already on my to-do list to move my outgoing mail to my mail provider also to make it easier to move to another ISP or to implement SPF but was not on the top of my to-do list.
 
My email provider requires authentication, so I needed to reconfigure postfix in my FreeBSD mail jail to use a relay host with authentication.

Install postfix-sasl

To use authentication with postfix the postfix-sasl package is required. If postfix is already installed it’ll be replace by postfix-sasl.

1
root@stafmail:/root # pkg install postfix-sasl

Configuration

Update the relay host

main.cf

1
2
3
4
5
6
7
relayhost = [smtp.mailprovider.domain]:465
smtp_use_tls=yes
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/usr/local/etc/postfix/relay_pass
smtp_sasl_security_options =
smtp_tls_wrappermode = yes
smtp_tls_security_level = encrypt

relay_pass

The credentials are in the relay_pass file the password is in the file as plain-text so we it with the correct file permissions.

1
2
3
root@stafmail:/usr/local/etc/postfix # touch relay_pass
root@stafmail:/usr/local/etc/postfix # chmod 600 relay_pass
root@stafmail:/usr/local/etc/postfix # vi relay_pass
1
[smtp.mailprovider.domain]:465 user:password

Create the hash file.

1
root@stafmail:/usr/local/etc/postfix # postmap relay_pass

Verify the file permissions.

1
2
3
4
root@stafmail:/usr/local/etc/postfix # ls -l relay_pass*
-rw-------  1 root  wheel      60 Feb 23 22:43 relay_pass
-rw-------  1 root  wheel  131072 Feb 23 22:43 relay_pass.db
root@stafmail:/usr/local/etc/postfix # 

Restart

We replaced postfix with postfix-sasl a restart is required.

1
root@stafmail:/usr/local/etc/postfix # /usr/local/etc/rc.d/postfix restart

Have fun

Update Your CPU Microcode on Arch Linux

Meltdown & spectre

With Meldown https://nvd.nist.gov/vuln/detail/CVE-2017-5754, Spectre Variant 1 https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-5753 and Spectre Variant 2 https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-5753 out in the wild there is a lot of confusing going about updating microcode.

There is a “Spectre & Meltdown Checker” available at https://github.com/speed47/spectre-meltdown-checker

Usage is very easy just clone the git repository and run the script.

Microcode

Microcode isn’t uploaded to the CPU but loaded during the boot strap of the CPU. Normally the BIOS upload the microcode to the CPU but this can also be done by the by the bootloader, or the operating system kernel.

Grub

Normally you get an updated bios for your motherboard or computer vendor to get new microcode for your CPU.

But when your vendor hasn’t released a new Bios yet or when you are using old hardware you might not get a new BIOS with updated microcode.

Lucky microcode can also loaded by bootloader this way you can get new microcode without a BIOS update if the new microcode cuase issues you disable it in the bootloader.

The process for Arch Linux is describe at the Arch Wiki https://wiki.archlinux.org/index.php/Microcode

You’ll find journey how to update the microcode on my Arch GNU/Linux system below.

Current microcode

1
2
3
4
[staf@frija ~]$ dmesg | grep -i microcode
[    2.102649] microcode: sig=0x40661, pf=0x20, revision=0xa
[    2.102981] microcode: Microcode Update Driver: v2.01 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
[staf@frija ~]$ 

Install intel-ucode

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
[root@vicky ~]# pacman -Syy intel-ucode
:: Synchronizing package databases...
 core                     126.8 KiB  12.4M/s 00:00 [######################] 100%
 extra                   1629.4 KiB  11.4M/s 00:00 [######################] 100%
 community                  4.1 MiB  11.0M/s 00:00 [######################] 100%
 multilib                 167.2 KiB  8.16M/s 00:00 [######################] 100%
resolving dependencies...
looking for conflicting packages...

Packages (1) intel-ucode-20180108-1

Total Download Size:   1.12 MiB
Total Installed Size:  1.55 MiB

:: Proceed with installation? [Y/n] y
:: Retrieving packages...
 intel-ucode-2018010...  1145.0 KiB   916K/s 00:01 [######################] 100%
(1/1) checking keys in keyring                     [######################] 100%
(1/1) checking package integrity                   [######################] 100%
(1/1) loading package files                        [######################] 100%
(1/1) checking for file conflicts                  [######################] 100%
(1/1) checking available disk space                [######################] 100%
:: Processing package changes...
(1/1) installing intel-ucode                       [######################] 100%
:: Running post-transaction hooks...
(1/1) Arming ConditionNeedsUpdate...
[root@vicky ~]# 

Verify the available microcode for your CPU

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
[staf@frija ~]$ yaourt  iucode-tool
1 aur/iucode-tool 2.2-1 (59) (4.87)
    Tool to manipulate IntelĀ® IA-32/X86-64 microcode bundles
==> Enter nĀ° of packages to be installed (e.g., 1 2 3 or 1-3)
==> ----------------------------------------------------------
==> 1


==> Downloading iucode-tool PKGBUILD from AUR...
x .SRCINFO
x PKGBUILD
oxe commented on 2017-10-01 17:50          
issue with pgp key and have tried various times and not sure what I might be doing wrong but why do you have so many self-signed sigs?

gpg --keyserver hkps.pool.sks-keyservers.net  --recv-keys C467A717507BBAFED3C160920BD9E81139CB4807

uid  Henrique de Moraes Holschuh hmh@debian.org
sig!3        0BD9E81139CB4807 2012-06-26  [self-signature]
uid  Henrique de Moraes Holschuh hmh@hmh.eng.br
sig!3        0BD9E81139CB4807 2012-06-26  [self-signature]
sub  A4B9D9AFC03142CD
sig!         0BD9E81139CB4807 2012-06-26  [self-signature]
sub  981C05C79F47CF26
sig!         0BD9E81139CB4807 2012-06-26  [self-signature]
sub  9137FBD3DE6F0A93
sig!         0BD9E81139CB4807 2014-03-23  [self-signature]
sub  FFDB99C00EABDE2E
sig!         0BD9E81139CB4807 2014-03-23  [self-signature]
sub  FE11BFA68B158E98
sig!         0BD9E81139CB4807 2016-03-26  [self-signature]
sub  A4B1618F7F267286
sig!         0BD9E81139CB4807 2016-03-26  [self-signature]
key 0BD9E81139CB4807:
6 duplicate signatures removed
45 signatures not checked due to missing keys
gpg: key 0BD9E81139CB4807: "Henrique de Moraes Holschuh hmh@hmh.eng.br" not changed
gpg: Total number processed: 1
gpg:              unchanged: 1

please advise

progandy commented on 2017-10-01 18:19             
@oxe: I am not Henrique, so I don't know what he did with his key that it looks this strange, but it doesn't affect the package. The build works, and the signature is properly validated.

Cbhihe commented on 2017-10-10 19:12           
Hi:
During install with '$ makepkg -sric ' I got: a PGP signature error: 

A simplified output follows because I am typing (not copy/pasting) this on a different box than the one (4.13.4.-1-ARCH) where the install took place:

== making package: iucode-tool 2.2-1 (Tue Oct 10...2017)
== Checking runtime dependencies...
== Checking buildtime dependencies...
== Retrieving sources...
downloads ok [...]
== Validating source files with sha256sums...
passed [...]
== Verifying source files with gpg...
iucode-tool_2.2.tar.xz ... FAILED (unknown public key FE11BFA68B158E98)
== ERROR: One or more PGP signatures could not be verified !

Can you explain that unknown PGP public key error ? 
Is it a problem on my side ? 
Please advise. I will be waiting for your response before I actually execute that code. Cheers.

progandy commented on 2017-10-13 15:28             
@Cbhihe: I did not have time and then forgot, sorry. Still, it should be obvious from the previous comments that you need to import the key in your gpg keyring with gpg, as described in the wiki for makepkg [1],[2]

gpg --recv-keys FE11BFA68B158E98
or
gpg --recv-keys C467A717507BBAFED3C160920BD9E81139CB4807
or
gpg --keyserver hkps.pool.sks-keyservers.net --recv-keys C467A717507BBAFED3C160920BD9E81139CB4807

[1]: https://wiki.archlinux.org/index.php/Makepkg#Signature_checking
[2]: https://wiki.archlinux.org/index.php/GnuPG#Use_a_keyserver

Cbhihe commented on 2017-10-14 17:40           
Thank you. Yes it WAS obvious and I had tried 
gpg --recv-keys FE11BFA68B158E98
already, but for some reason I do not get, either the keyring did not register correctly or I screwed up something, or both. 

I have reinstalled the Gnome keyring, re-imported my saved signatures and  
gpg --keyserver hkps.pool.sks-keyservers.net --recv-keys C467A717507BBAFED3C160920BD9E81139CB4807
worked this time. :-)
Cheers.

iucode-tool 2.2-1  (2017-09-13 07:49)
( Unsupported package: Potentially dangerous ! )
==> Edit PKGBUILD ? [Y/n] ("A" to abort)
==> ------------------------------------
==> n

==> iucode-tool dependencies:


==> Continue building iucode-tool ? [Y/n]
==> -------------------------------------
==> 

==> Building and installing package
==> Making package: iucode-tool 2.2-1 (Sun Jan 21 12:48:37 CET 2018)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
  -> Downloading iucode-tool_2.2.tar.xz...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  146k  100  146k    0     0  74948      0  0:00:02  0:00:02 --:--:-- 63193
  -> Downloading iucode-tool_2.2.tar.xz.asc...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   833  100   833    0     0    833      0  0:00:01  0:00:01 --:--:--   478
==> Validating source files with sha256sums...
    iucode-tool_2.2.tar.xz ... Passed
    iucode-tool_2.2.tar.xz.asc ... Skipped
==> Verifying source file signatures with gpg...
    iucode-tool_2.2.tar.xz ... Passed
==> Extracting sources...
  -> Extracting iucode-tool_2.2.tar.xz with bsdtar
==> Starting build()...
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether configure.ac should try to override CFLAGS... no
checking whether configure.ac should try to override LDFLAGS... no
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking minix/config.h usability... no
checking minix/config.h presence... no
checking for minix/config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking whether gcc understands -c and -o together... (cached) yes
checking dependency style of gcc... (cached) gcc3
checking for ANSI C header files... (cached) yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking for stdint.h... (cached) yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking for unistd.h... (cached) yes
checking time.h usability... yes
checking time.h presence... yes
checking for time.h... yes
checking cpuid.h usability... yes
checking cpuid.h presence... yes
checking for cpuid.h... yes
checking whether byte ordering is bigendian... no
checking for inline... inline
checking for int32_t... yes
checking for size_t... yes
checking for ssize_t... yes
checking for uint16_t... yes
checking for uint32_t... yes
checking for uint8_t... yes
checking for stdlib.h... (cached) yes
checking for GNU libc compatible malloc... yes
checking for stdlib.h... (cached) yes
checking for GNU libc compatible realloc... yes
checking whether lstat correctly handles trailing slash... yes
checking whether stat accepts an empty string... no
checking for memset... yes
checking for strcasecmp... yes
checking for strdup... yes
checking for strerror... yes
checking for strrchr... yes
checking for strtoul... yes
checking for timegm... yes
checking for library containing argp_parse... none required
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking for flockfile... yes
checking for fgets_unlocked... yes
configure: project-wide base CPPFLAGS: -D_FORTIFY_SOURCE=2
configure: project-wide base CFLAGS:   -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt
configure: project-wide base LDFLAGS:  -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating iucode_tool.8
config.status: creating iucode_tool_config.h
config.status: executing depfiles commands
make  all-am
make[1]: Entering directory '/home/staf/tmp/yaourt-tmp-staf/aur-iucode-tool/src/iucode-tool-2.2'
gcc -DHAVE_CONFIG_H -I.   -D_FORTIFY_SOURCE=2  -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt -MT intel_microcode.o -MD -MP -MF .deps/intel_microcode.Tpo -c -o intel_microcode.o intel_microcode.c
gcc -DHAVE_CONFIG_H -I.   -D_FORTIFY_SOURCE=2  -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt -MT iucode_tool.o -MD -MP -MF .deps/iucode_tool.Tpo -c -o iucode_tool.o iucode_tool.c
mv -f .deps/intel_microcode.Tpo .deps/intel_microcode.Po
mv -f .deps/iucode_tool.Tpo .deps/iucode_tool.Po
gcc  -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt  -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -o iucode_tool intel_microcode.o iucode_tool.o  
make[1]: Leaving directory '/home/staf/tmp/yaourt-tmp-staf/aur-iucode-tool/src/iucode-tool-2.2'
==> Entering fakeroot environment...
==> Starting package()...
make[1]: Entering directory '/home/staf/tmp/yaourt-tmp-staf/aur-iucode-tool/src/iucode-tool-2.2'
 /usr/bin/mkdir -p '/home/staf/tmp/yaourt-tmp-staf/aur-iucode-tool/pkg/iucode-tool//usr/bin'
 /usr/bin/mkdir -p '/home/staf/tmp/yaourt-tmp-staf/aur-iucode-tool/pkg/iucode-tool//usr/share/man/man8'
  /usr/bin/install -c iucode_tool '/home/staf/tmp/yaourt-tmp-staf/aur-iucode-tool/pkg/iucode-tool//usr/bin'
 /usr/bin/install -c -m 644 iucode_tool.8 '/home/staf/tmp/yaourt-tmp-staf/aur-iucode-tool/pkg/iucode-tool//usr/share/man/man8'
make[1]: Leaving directory '/home/staf/tmp/yaourt-tmp-staf/aur-iucode-tool/src/iucode-tool-2.2'
==> Tidying install...
  -> Removing libtool files...
  -> Purging unwanted files...
  -> Removing static library files...
  -> Stripping unneeded symbols from binaries and libraries...
  -> Compressing man and info pages...
==> Checking for packaging issue...
==> Creating package "iucode-tool"...
  -> Generating .PKGINFO file...
  -> Generating .BUILDINFO file...
  -> Generating .MTREE file...
  -> Compressing package...
==> Leaving fakeroot environment.
==> Finished making: iucode-tool 2.2-1 (Sun Jan 21 12:48:44 CET 2018)
==> Cleaning up...

==> Continue installing iucode-tool ? [Y/n]
==> [v]iew package contents [c]heck package with namcap
==> ---------------------------------------------------
==> y

loading packages...
resolving dependencies...
looking for conflicting packages...

Packages (1) iucode-tool-2.2-1

Total Installed Size:  0.06 MiB

:: Proceed with installation? [Y/n] y
(1/1) checking keys in keyring                                   [####################################] 100%
(1/1) checking package integrity                                 [####################################] 100%
(1/1) loading package files                                      [####################################] 100%
(1/1) checking for file conflicts                                [####################################] 100%
(1/1) checking available disk space                              [####################################] 100%
:: Processing package changes...
(1/1) installing iucode-tool                                     [####################################] 100%
ldconfig: File /usr/lib/libmlt.so.6.4.0 is empty, not checked.
ldconfig: File /usr/lib/libmlt++.so.6.4.0 is empty, not checked.
ldconfig: File /usr/lib32/libmng.so.2 is empty, not checked.
ldconfig: File /usr/lib32/libmng.so is empty, not checked.
ldconfig: File /usr/lib32/libmng.so.2.0.2 is empty, not checked.
:: Running post-transaction hooks...
(1/1) Arming ConditionNeedsUpdate...
[staf@frija ~]$ 
1
2
3
4
5
6
[root@frija ~]# bsdtar -Oxf /boot/intel-ucode.img | iucode_tool -tb -lS - 
iucode_tool: system has processor(s) with signature 0x00040661
microcode bundle 1: (stdin)
selected microcodes:
  001/143: sig 0x00040661, pf_mask 0x32, 2017-11-20, rev 0x0018, size 25600
[root@frija ~]# 

Recreate grub.cfg

grub-mkconfig will detect the microcode and add it the grub configuration.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
[root@vicky ~]# grub-mkconfig -o /boot/grub/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-linux-lts
Found initrd image(s) in /boot: intel-ucode.img initramfs-linux-lts.img
Found fallback initrd image(s) in /boot: intel-ucode.img initramfs-linux-lts-fallback.img
Found linux image: /boot/vmlinuz-linux-hardened
Found initrd image(s) in /boot: intel-ucode.img initramfs-linux-hardened.img
Found fallback initrd image(s) in /boot: intel-ucode.img initramfs-linux-hardened-fallback.img
Found linux image: /boot/vmlinuz-linux-ck
Found initrd image(s) in /boot: intel-ucode.img initramfs-linux-ck.img
Found fallback initrd image(s) in /boot: intel-ucode.img initramfs-linux-ck-fallback.img
Found linux image: /boot/vmlinuz-linux
Found initrd image(s) in /boot: intel-ucode.img initramfs-linux.img
Found fallback initrd image(s) in /boot: intel-ucode.img initramfs-linux-fallback.img
done
[root@vicky ~]# 

When take a look at the newly created grub.cfg you see that microcode image is added to the initrd image. If you new micro code cause issue you can just remove the entry in grub configuration

1
2
3
4
5
6
7
8
9
10
[root@vicky ~]# cat /boot/grub/grub.cfg | grep initrd
  initrd  /__active/rootvol/boot/intel-ucode.img /__active/rootvol/boot/initramfs-linux-lts.img
  initrd  /__active/rootvol/boot/intel-ucode.img /__active/rootvol/boot/initramfs-linux-lts-fallback.img
  initrd  /__active/rootvol/boot/intel-ucode.img /__active/rootvol/boot/initramfs-linux-hardened.img
  initrd  /__active/rootvol/boot/intel-ucode.img /__active/rootvol/boot/initramfs-linux-hardened-fallback.img
  initrd  /__active/rootvol/boot/intel-ucode.img /__active/rootvol/boot/initramfs-linux-ck.img
  initrd  /__active/rootvol/boot/intel-ucode.img /__active/rootvol/boot/initramfs-linux-ck-fallback.img
  initrd  /__active/rootvol/boot/intel-ucode.img /__active/rootvol/boot/initramfs-linux.img
  initrd  /__active/rootvol/boot/intel-ucode.img /__active/rootvol/boot/initramfs-linux-fallback.img
[root@vicky ~]# 

Reboot your system and verify

1
2
3
4
5
[staf@frija ~]$ dmesg | grep -i microcode
[    0.000000] microcode: microcode updated early to revision 0x18, date = 2017-11-20
[    1.852726] microcode: sig=0x40661, pf=0x20, revision=0x18
[    1.853029] microcode: Microcode Update Driver: v2.2.
[staf@frija ~]$ 

Have fun

Bacula on FreeBSD (Part 3 Storage Setup)

"bacula setup"

I finally got the time to continue with my bacula backup setup. See my previous posts about the start of my bacula setup.

Storage setup

I created a new zfs pool “bigpool” with some old harddisks I probably need to replace them with bigger harddisk in the further.

zfs filesystem

First we create a zfs filesystem for our bacula storage.

1
2
root@rataplan:~ # zfs create bigpool/bacula
root@rataplan:~ # 

delegate to jail

jailed

We want to use the zfs dataset in the bacula jail so we need to delegate the control to the dataset into the bacula jail.

1
2
root@rataplan:~ # zfs set jailed=on bigpool/bacula
root@rataplan:~ # zfs jail stafbacula bigpool/bacula

verify

When we logon to the jail we see that the zfs dateset is available.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
root@rataplan:~ # ezjail-admin console stafbacula
Last login: Wed Dec 27 10:52:27 on pts/2
FreeBSD 11.1-RELEASE-p4 (GENERIC) #0: Tue Nov 14 06:12:40 UTC 2017

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories:   https://www.FreeBSD.org/security/
FreeBSD Handbook:      https://www.FreeBSD.org/handbook/
FreeBSD FAQ:           https://www.FreeBSD.org/faq/
Questions List: https://lists.FreeBSD.org/mailman/listinfo/freebsd-questions/
FreeBSD Forums:        https://forums.FreeBSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with:  pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.

Show the version of FreeBSD installed:  freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages:  man man
FreeBSD directory layout:      man hier

Edit /etc/motd to change this login announcement.
You have new mail.
root@stafbacula:~ # zfs list
NAME             USED  AVAIL  REFER  MOUNTPOINT
bigpool         1.14M   433G    23K  /bigpool
bigpool/bacula    23K   433G    23K  /bigpool/bacula
root@stafbacula:~ # 

When we restart the jail we see that the dataset isn’t available anymore in the jail

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
root@stafbacula:~ # logout
root@rataplan:~ # /usr/local/etc/rc.d/ezjail restart stafbacula
Stopping jails: stafbacula.
Starting jails: stafbacula.
/etc/rc.d/jail: WARNING: Per-jail configuration via jail_* variables  is obsolete.  Please consider migrating to /etc/jail.conf.
root@rataplan:~ # ezjail-admin console stafbacula
Last login: Wed Dec 27 10:58:33 on pts/2
FreeBSD 11.1-RELEASE-p4 (GENERIC) #0: Tue Nov 14 06:12:40 UTC 2017

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories:   https://www.FreeBSD.org/security/
FreeBSD Handbook:      https://www.FreeBSD.org/handbook/
FreeBSD FAQ:           https://www.FreeBSD.org/faq/
Questions List: https://lists.FreeBSD.org/mailman/listinfo/freebsd-questions/
FreeBSD Forums:        https://forums.FreeBSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with:  pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.

Show the version of FreeBSD installed:  freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages:  man man
FreeBSD directory layout:      man hier

Edit /etc/motd to change this login announcement.
You have new mail.
root@stafbacula:~ # zfs list
no datasets available
root@stafbacula:~ # 

make persistent

enable zfs in the jail

When the jail is booted it need to bring the zfs filesystem online. We need to add zfs_enable=YES to the jail rc.conf

1
root@rataplan:~ # vi /usr/jails/stafbacula/etc/rc.conf
1
2
3
bacula_dir="start"
bacula_dir_enable="yes"
zfs_enable="YES"

update the ezjail configuration

ezjail needs to jail the zfs dataset to the jail when it’s start the jail.

1
root@rataplan:~ # vi /usr/local/etc/ezjail/stafbacula

We need to add the dataset to the jail’s zfs_datasets. By default a jail isn’t allowed to mount the zfs dataset so need to update jail’s parmeters.

1
2
export jail_stafbacula_zfs_datasets="bigpool/bacula"
export jail_stafbacula_parameters="enforce_statfs=0 allow.mount=1 allow.mount.zfs=1 allow.mount.procfs=1 allow.mount.devfs=1"

Restart the jail and verify that the zfs filesystem is available inside the jail

1
2
3
4
5
root@rataplan:~ # ezjail-admin restart stafbacula
Stopping jails: stafbacula.
Starting jails: stafbacula.
/etc/rc.d/jail: WARNING: Per-jail configuration via jail_* variables  is obsolete.  Please consider migrating to /etc/jail.conf.
root@rataplan:~ # 

Verify that dataset is mounted

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
root@rataplan:~ # ezjail-admin console stafbacula
Last login: Wed Dec 27 11:36:04 on pts/2
FreeBSD 11.1-RELEASE-p4 (GENERIC) #0: Tue Nov 14 06:12:40 UTC 2017

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories:   https://www.FreeBSD.org/security/
FreeBSD Handbook:      https://www.FreeBSD.org/handbook/
FreeBSD FAQ:           https://www.FreeBSD.org/faq/
Questions List: https://lists.FreeBSD.org/mailman/listinfo/freebsd-questions/
FreeBSD Forums:        https://forums.FreeBSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with:  pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.

Show the version of FreeBSD installed:  freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages:  man man
FreeBSD directory layout:      man hier

Edit /etc/motd to change this login announcement.
You have new mail.
root@stafbacula:~ # zfs list
NAME             USED  AVAIL  REFER  MOUNTPOINT
bigpool         1.14M   433G    23K  /bigpool
bigpool/bacula    23K   433G    23K  /bigpool/bacula
root@stafbacula:~ # df -h /bigpool/bacula
Filesystem                    Size    Used   Avail Capacity  Mounted on
zroot/usr/jails/stafbacula    2.6T    618M    2.6T     0%    /usr/jails/stafbacula
root@stafbacula:~ # 

Links

Model-m Tux Update...

"modelm_tux_only.jpg"

I own a Unicomp model-m keyboard. The keyboard has a nice key feel but it has windows super key(s).


I don’t use super key(s), and would prefer to have a keyboard without it. But when it has super keys I’d rather have it without the windows logo on it so it was time to replace them with the tux version

Pictures

modelm_tux_package.jpg modelm_all_keys.jpg modelm_tux_only.jpg modelm_with_tux_keys.jpg

Jenkins Build With 20 Cores

I finally got the time to try out my jenkins build on my new 20 Core Dual Processor Jenkins Build Workstation

I’m able to run all test on multiple operation systems now. I still need to review this setup and perhaps move some tests to docker instead of the virtual machines to save some memory. …but this jenkins setup was configured before docker was a thing.

Have fun

20 Core Dual Processor Jenkins Build Workstation

"Xeon CPU Side"


My jenkins builds are taking too long mainly due the lack of memory. I mainly use jenkins to verify that my software work on different operation systems (GNU/Linux distributions / *BSD / Solaris).

Looking for a solution that is still affordable I ended up with building a dual Xeon workstation. CPU and memory comes from www.ebay.be


 

Part list:

  • CPU: 2 * Intel Xeon E5-2660v2 This CPU has 10 cores and 20 thread, so I get 40 threads.
  • Motherboard: Asrock EP2C602-4L/D16 I choose this motherboard because it has a lot of DIMM slots so I can upgrade the memory in the further. Downside is that layout is SSI EEB that limits the case choose.
  • Memory: 4 * SAMSUNG M393B2G70BH0-CK0 16GB which gives me 64 GB ECC memory
  • CPU Cooler 2 * Thermaltake Water 3.0 Performer C For the first I used watercooling mainly because I wanted to make sure that the cooling will not block the access to the DIMM slots.
  • PSU: Seasonic FOCUS Plus 750 Gold I needed a power supply with 2 * 8 pins CPU connectors.
  • Case: Phanteks Enthoo Pro This case supports SSI EEB and is not too expensive.

Pictures

Xeon CPU side Xeon CPU side Xeon CPU side

Still need to verify if jenkins works on this system :-)

Bacula on FreeBSD (Part 2 Bacula Catalog Over SSL )

"PostgreSSL"

In my previous post, I setup on my PostgresSQL FreeBSD jail, In this post we continue with the bacaula server.

In this post we will continue with the database connection (Catalog) we’ll go the extra mile 1,609344 km and encrypt the catalog connection with ssl. Why? We encrypt.. because we can!

Bacula Components

  • Bacula Director
    The Bacula Director is daemon that runs in the backgroud that control all backup operations.

  • Bacula Console
    The Bacula console is an administrator program that allows an system administrator to control the Bacula director.

  • Bacula File
    The Bacula File is a backup client install on the backup client.

  • Bacula Storage
    The backup media.

  • Catalog
    The Catalog is the index of the backups. Bacula supports three types of index databases mySQL ( mariaDB), PostgreSQL and SQLite

  • Bacula monitor
    A Bacula monitor service is a program that allows the system administrator to cerify the status of the bacula Directors, Bacula File Daemons and Bacula Storage Daemons.

Bacula Server

Jail

Create the Bacula Server Jail

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
root@rataplan:~ # ezjail-admin create stafbacula "em0|192.168.1.52"
Warning: Some services already seem to be listening on all IP, (including 192.168.1.52)
  This may cause some confusion, here they are:
root     ntpd       754   20 udp6   *:123                 *:*
root     ntpd       754   21 udp4   *:123                 *:*
root     rpc.statd  717   4  udp6   *:846                 *:*
root     rpc.statd  717   5  tcp6   *:846                 *:*
root     rpc.statd  717   6  udp4   *:846                 *:*
root     rpc.statd  717   7  tcp4   *:846                 *:*
root     nfsd       713   5  tcp4   *:2049                *:*
root     nfsd       713   6  tcp6   *:2049                *:*
root     mountd     707   5  udp6   *:823                 *:*
root     mountd     707   6  tcp6   *:823                 *:*
root     mountd     707   7  udp4   *:823                 *:*
root     mountd     707   8  tcp4   *:823                 *:*
root     rpcbind    676   6  udp6   *:111                 *:*
root     rpcbind    676   7  udp6   *:779                 *:*
root     rpcbind    676   8  tcp6   *:111                 *:*
root     rpcbind    676   9  udp4   *:111                 *:*
root     rpcbind    676   10 udp4   *:768                 *:*
root     rpcbind    676   11 tcp4   *:111                 *:*
root     syslogd    656   6  udp6   *:514                 *:*
root     syslogd    656   7  udp4   *:514                 *:*
root@rataplan:~ # 

Start the jail

1
2
3
4
root@rataplan:~ # ezjail-admin start stafbacula
Starting jails: stafbacula.
/etc/rc.d/jail: WARNING: Per-jail configuration via jail_* variables  is obsolete.  Please consider migrating to /etc/jail.conf.
root@rataplan:~ # 

Open the console

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
root@rataplan:~ # ezjail-admin console stafbacula
FreeBSD 11.1-RELEASE-p1 (GENERIC) #0: Wed Sep  9 11:55:48 UTC 2017

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories:   https://www.FreeBSD.org/security/
FreeBSD Handbook:      https://www.FreeBSD.org/handbook/
FreeBSD FAQ:           https://www.FreeBSD.org/faq/
Questions List: https://lists.FreeBSD.org/mailman/listinfo/freebsd-questions/
FreeBSD Forums:        https://forums.FreeBSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with:  pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.

Show the version of FreeBSD installed:  freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages:  man man
FreeBSD directory layout:      man hier

Edit /etc/motd to change this login announcement.
root@stafbacula:~ # 

Bacula installation

Install pkg

Set up dns

1
root@stafbacula:~ # vi /etc/resolv.conf
1
nameserver 192.168.1.1

Bootstrap pkg

1
2
3
4
5
6
7
8
9
10
11
12
root@stafbacula:~ # pkg
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:11:amd64/quarterly, please wait...
Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done
[stafbacula] Installing pkg-1.10.1...
[stafbacula] Extracting pkg-1.10.1: 100%
pkg: not enough arguments
Usage: pkg [-v] [-d] [-l] [-N] [-j <jail name or id>|-c <chroot path>|-r <rootdir>] [-C <configuration file>] [-R <repo config dir>] [-o var=value] [-4|-6] <command> [<args>]

For more information on available commands and options see 'pkg help'.
root@stafbacula:~ # 

Install the bacula server package

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
root@stafbacula:~ # pkg install bacula-server
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
Updating database digests format: 100%
The following 8 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        bacula-server: 7.4.7_1
        bacula-client: 7.4.7_1
        readline: 7.0.3
        indexinfo: 0.2.6
        gettext-runtime: 0.19.8.1_1
        lzo2: 2.10_1
        postgresql95-client: 9.5.7_1
        perl5: 5.24.1_1

Number of packages to be installed: 8

The process will require 69 MiB more space.
17 MiB to be downloaded.

Proceed with this action? [y/N]: y
[stafbacula] [1/8] Fetching bacula-server-7.4.7_1.txz: 100%  678 KiB 694.6kB/s    00:01    
[stafbacula] [2/8] Fetching bacula-client-7.4.7_1.txz: 100%  286 KiB 292.8kB/s    00:01    
[stafbacula] [3/8] Fetching readline-7.0.3.txz: 100%  334 KiB 342.4kB/s    00:01    
[stafbacula] [4/8] Fetching indexinfo-0.2.6.txz: 100%    5 KiB   5.3kB/s    00:01    
[stafbacula] [5/8] Fetching gettext-runtime-0.19.8.1_1.txz: 100%  148 KiB 151.1kB/s    00:01    
[stafbacula] [6/8] Fetching lzo2-2.10_1.txz: 100%  113 KiB 115.4kB/s    00:01    
[stafbacula] [7/8] Fetching postgresql95-client-9.5.7_1.txz: 100%    2 MiB 772.9kB/s    00:03    
[stafbacula] [8/8] Fetching perl5-5.24.1_1.txz: 100%   13 MiB 874.0kB/s    00:16    
Checking integrity... done (0 conflicting)
[stafbacula] [1/8] Installing indexinfo-0.2.6...
[stafbacula] [1/8] Extracting indexinfo-0.2.6: 100%
[stafbacula] [2/8] Installing readline-7.0.3...
[stafbacula] [2/8] Extracting readline-7.0.3: 100%
[stafbacula] [3/8] Installing gettext-runtime-0.19.8.1_1...
[stafbacula] [3/8] Extracting gettext-runtime-0.19.8.1_1: 100%
[stafbacula] [4/8] Installing lzo2-2.10_1...
[stafbacula] [4/8] Extracting lzo2-2.10_1: 100%
[stafbacula] [5/8] Installing perl5-5.24.1_1...
[stafbacula] [5/8] Extracting perl5-5.24.1_1: 100%
[stafbacula] [6/8] Installing bacula-client-7.4.7_1...
===> Creating groups.
Creating group 'bacula' with gid '910'.
===> Creating users
Creating user 'bacula' with uid '910'.
[stafbacula] [6/8] Extracting bacula-client-7.4.7_1: 100%
[stafbacula] [7/8] Installing postgresql95-client-9.5.7_1...
[stafbacula] [7/8] Extracting postgresql95-client-9.5.7_1: 100%
[stafbacula] [8/8] Installing bacula-server-7.4.7_1...
===> Creating groups.
Using existing group 'bacula'.
===> Creating users
Using existing user 'bacula'.
[stafbacula] Extracting bacula-server-7.4.7_1: 100%
Message from perl5-5.24.1_1:
The /usr/bin/perl symlink has been removed starting with Perl 5.20.
For shebangs, you should either use:

#!/usr/local/bin/perl

or

#!/usr/bin/env perl

The first one will only work if you have a /usr/local/bin/perl,
the second will work as long as perl is in PATH.
Message from bacula-client-7.4.7_1:
################################################################################

NOTE:
Sample files are installed in /usr/local/etc/bacula:

  bconsole.conf.sample, bacula-fd.conf.sample

################################################################################
Message from postgresql95-client-9.5.7_1:
The PostgreSQL port has a collection of "side orders":

postgresql-docs
  For all of the html documentation

p5-Pg
  A perl5 API for client access to PostgreSQL databases.

postgresql-tcltk
  If you want tcl/tk client support.

postgresql-jdbc
  For Java JDBC support.

postgresql-odbc
  For client access from unix applications using ODBC as access
  method. Not needed to access unix PostgreSQL servers from Win32
  using ODBC. See below.

ruby-postgres, py-PyGreSQL
  For client access to PostgreSQL databases using the ruby & python
  languages.

postgresql-plperl, postgresql-pltcl & postgresql-plruby
  For using perl5, tcl & ruby as procedural languages.

postgresql-contrib
  Lots of contributed utilities, postgresql functions and
  datatypes. There you find pg_standby, pgcrypto and many other cool
  things.

etc...
Message from bacula-server-7.4.7_1:
###############################################################################

bacula server was installed

An auto-changer manipulation script based on FreeBSDs
chio command is included and installed at

  /usr/local/sbin/chio-bacula

Please have a look at it if you want to use an
autochanger. You have to configure the usage in

  /usr/local/etc/bacula/bacula-dir.conf

Take care of correct permissions for changer and
tape device (e.g. /dev/ch0 and /dev/n[r]sa0) i.e.
they must be accessible by user bacula.

Due to lack of some features in the FreeBSD tape driver
implementation you MUST add some OS dependent options to
the bacula-sd.conf file:

  Hardware End of Medium = no;
  Backward Space Record  = no;
  Backward Space File    = no;

With 2 filemarks at EOT (see man mt):
  Fast Forward Space File = no;
  BSF at EOM = yes;
  TWO EOF    = yes;

With 1 filemarks at EOT (see man mt):
  Fast Forward Space File = yes;
  BSF at EOM = no;
  TWO EOF   = no;

NOTE: YOU CAN SWITCH EOT model ONLY when starting
      from scratch with EMPTY tapes.

It is also important that all the scripts accessed
by RunBeforeJob and RunAfterJob will be executed by
the user bacula.  Check your permissions.

For USB support read the bacula manual. It could be necessary
to configure/compile a new kernel.

Look at /usr/local/share/bacula/update_bacula_tables for
database update procedure. Details can be found in the
ReleaseNotes

If you are using sqlite you need to run the make_sqlite_tables script as
the bacula user. Do this using 'sudo su -m bacula'.

################################################################################
root@stafbacula:~ # 

Initialize the bacula catalog

We’ll have a postgreSQL server running in a FreeBSD jail as our catalog (see http://stafwag.github.io/blog/blog/2017/08/06/bacula-on-freebsd:w_part1/ howto install PostgreSQL into a FreeBSD jail).

PostgreSQL setup

The setup below describes howto configure the PostgreSQL catalog with certificate and username/password authentication. This might be overkill the bacula server runs on the same physical host so no data is going out on the network. But I wanted to setup the database conneection as secure as possible and will reuse this setup for my other database connection. We’ll setup a “self signed” root ca for now, but I replace this with my own CA in further.

PostgreSQL authentication methods

PostgreSQL support a lot of authentication methods you’ll find a description of the supported of the support authentication methods below (without too much details):

  • Trust Authentication
    With trust authentication the postgreSQL trust the connection from the remote host, this is the default for localhost host connection and “socket” connections.
     
  • Password Authentication
    Authentication with login/password
     
  • GSSAPI Authentication
    Authentication with the Generic Security Services Application Program Interface
     
  • SSPI Authentication
    Authentication with the Security Support Provider Interface - SSPI is a proprietary variant of GSSAPI with extensions and very Windows-specific data types -
     
  • Kerberos Authentication
    Authentication using the Kerberos protocol
     
  • Ident Authentication
    Authentication using the ident protocol
     
  • Peer Authentication
    Authentication using the getpeereid() kernel function, only supported for local connection on BSD, MacOS and GNU/Linux.
     
  • LDAP Authentication
    LDAP authentication.
     
  • RADIUS Authentication
    Radius authentication.
     
  • Certificate Authentication
    Authentication with a PKI certificate.
     
  • PAM Authentication
    PAM based authentication
     

     
    I wanted to use password authentication over ssl with a client certificate. The bacula documents isn’t very clear on howto configure it. After a quick lot at the bacula source code it should be supported, so let’s give it a try…

Configure the PostgreSQL jail

Allow network connections

Logon the postgreSQL server jail move to the postgreSQL data directory and edit postgresql.conf to allow TCP/IP connections.

1
2
3
root@stafdb:/var/db/postgres/data96 # pwd
/var/db/postgres/data96
root@stafdb:/var/db/postgres/data96 # vim postgresql.conf
1
2
3
4
5
6
7
8
# - Connection Settings -

listen_addresses = '192.168.1.51'               # what IP address(es) to listen on;
# listen_addresses = 'localhost'                # what IP address(es) to listen on;
                                        # comma-separated list of addresses;
                                        # defaults to 'localhost'; use '*' for all
                                        # (change requires restart)
#port = 5432                            # (change requires restart)

SSL encryption

It’s always a good idea to encrypt your connections.

SSL Server setup
Umask

set the umask to prevent somebody can read you private key.

1
2
3
root@stafdb:/var/db/postgres/data96 # su - postgres
$ umask 077 
$ 
Create a private key

Create a private key without encrypting it.

1
2
3
4
5
6
openssl genrsa -out server.key 4096
Generating RSA private key, 4096 bit long modulus
........................................................................................................................................................................................................................++
......++
e is 65537 (0x10001)
$ 
Create a self-signed certificate
1
2
3
4
5
$ openssl req -new -key server.key -days 3650 -out server.crt -x509 -subj '/C=BE/ST=Flanders/L=Antwerp/O=stafnet/CN=stafdb'
$ ls -ltr                                                                                               total 23
-rw-------   1 postgres  postgres  3247 Sep  9 11:47 server.key
-rw-------   1 postgres  postgres  1964 Sep  9 11:52 server.crt
$ 
Root ca

We created a self signed certificate so the server certificate is our trusted ca root.

1
2
3
4
5
6
7
$ ln -s server.crt root.crt
$ ls -ltr
total 24
-rw-------   1 postgres  postgres  3247 Sep  9 11:47 server.key
-rw-------   1 postgres  postgres  1964 Sep  9 11:52 server.crt
lrwx------   1 postgres  postgres    10 Sep  9 11:53 root.crt -> server.crt
$ 
Enable ssl

Edit postgresql.conf and update the ssl setting

1
$ vi postgresql.conf

By default ssl_ca_file is not set but this directive is required so don’t forget to set it. We disable the 3DES ciphers they’re obsolete… We don’t speficy a crl for now.

1
2
3
4
5
6
7
8
9
10
11
#authentication_timeout = 1min          # 1s-600s
ssl = on                                # (change requires restart)
ssl_ciphers = 'HIGH:MEDIUM:!3DES:!aNULL' # allowed SSL ciphers
                                        # (change requires restart)
ssl_prefer_server_ciphers = on          # (change requires restart)
#ssl_ecdh_curve = 'prime256v1'          # (change requires restart)
ssl_cert_file = 'server.crt'            # (change requires restart)
ssl_key_file = 'server.key'             # (change requires restart)
ssl_ca_file = 'root.crt'                        # (change requires restart)
#ssl_crl_file = ''                      # (change requires restart)
#password_encryption = on
SSL Client setup
Become bacula

Logon to the bacula jail and become the bacula user. We use “su -m …” to logon to the locked daemon account, this will take over the root environment.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
root@rataplan:~ # ezjail-admin console stafbacula
Last login: Wed Sep  9 09:33:16 on pts/0
FreeBSD 11.1-RELEASE-p1 (GENERIC) #0: Wed Sep  9 11:55:48 UTC 2017

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories:   https://www.FreeBSD.org/security/
FreeBSD Handbook:      https://www.FreeBSD.org/handbook/
FreeBSD FAQ:           https://www.FreeBSD.org/faq/
Questions List: https://lists.FreeBSD.org/mailman/listinfo/freebsd-questions/
FreeBSD Forums:        https://forums.FreeBSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with:  pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.

Show the version of FreeBSD installed:  freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages:  man man
FreeBSD directory layout:      man hier

Edit /etc/motd to change this login announcement.
You have new mail.
root@stafbacula:~ # su -m bacula -c "/bin/sh"
$ id
uid=910(bacula) gid=910(bacula) groups=910(bacula)
$ 
umask

Set the umask to prevent somebody can read you private key.

1
2
$ umask 077
$ 
move to the bacula home directory
1
2
3
$ cat /etc/passwd | grep bacula
bacula:*:910:910:Bacula Daemon:/var/db/bacula:/usr/sbin/nologin
$ cd /var/db/bacula
create the .postges directory
1
2
3
4
5
6
7
8
$ mkdir .postgres
$ ls -la
total 12
drwxrwx---   3 bacula  bacula   3 Sep  9 09:41 .
drwxr-xr-x  14 root    wheel   18 Sep  9 14:41 ..
drwx------   2 bacula  bacula   2 Sep  9 09:41 .postgres
$ cd .postgres/
$ 
Create a private key

We took over the root evironment therefor we need to set the RANDFILE variable to randfile in the bacula home directory.

1
2
3
4
$ pwd
/var/db/bacula/.postgres
$ export RANDFILE=/var/db/bacula/.rnd
$ 

Create the private key.

1
2
3
4
5
6
$ openssl genrsa -out `hostname`.key 4096
Generating RSA private key, 4096 bit long modulus
..........++
.......................................................++
e is 65537 (0x10001)
$ 
Create the client csr
1
2
$ openssl req -new -key stafbacula.key -out stafbacula.csr -subj '/C=BE/ST=Flanders/L=Antwerp/O=stafnet/CN=stafbacula'
$ 
Create the client certifocate

Logon to the postgreSQL jail as postgres and sign the client csr.

1
2
3
4
5
[postgres@stafdb ~/data96]$ openssl x509 -req -in stafbacula.csr -CAcreateserial -CA root.crt -CAkey server.key -out stafbacula.crt 
Signature ok
subject=/C=BE/ST=Flanders/L=Antwerp/O=stafnet/CN=stafbacula
Getting CA Private Key
[postgres@stafdb ~/data96]$ 
Copy the client certificate and the trusted root certificate to bacula jail
1
2
3
4
5
6
7
8
9
10
11
$ uname -a
FreeBSD stafbacula 11.1-RELEASE-p1 FreeBSD 11.1-RELEASE-p1 #0: Wed Sep  9 11:55:48 UTC 2017     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64
$ pwd
/var/db/bacula/.postgres
$ ls -ltr
total 24
-rw-------  1 bacula  bacula  3243 Sep  9 09:47 stafbacula.key
-rw-------  1 bacula  bacula  1679 Sep  9 09:54 stafbacula.csr
-rw-------  1 bacula  bacula  1964 Sep  9 10:04 root.crt
-rw-------  1 bacula  bacula  1850 Sep  9 10:06 stafbacula.crt
$ 

Host file on the bacula jail

The hostname of the posgresql jail has to match with the CN of the server certificate. So we’ll add the hostname to /etc/hosts

1
root@stafbacula:~ # vi /etc/hosts
1
192.168.1.51    stafdb

Setup the bacula database

Create the bacula database user

1
2
3
4
5
6
7
8
9
[postgres@stafdb ~/data96]$ id
uid=770(postgres) gid=770(postgres) groups=770(postgres)
[postgres@stafdb ~/data96]$ psql postgres
psql (9.6.3)
Type "help" for help.

postgres=# create user bacula WITH PASSWORD 'xxxxxx';
CREATE ROLE
postgres=# 

To update the user password;

1
2
3
postgres=# alter user bacula PASSWORD 'yyyyyyy';
ALTER ROLE
postgres=# 

You can view the new permissions in the pg_user table or by execute the \du (describe user shortcut), by default the user has minimal permissions.

1
2
3
4
5
6
7
postgres=# select * from pg_user where usename = 'bacula';
 usename | usesysid | usecreatedb | usesuper | userepl | usebypassrls |  passwd  | valuntil | useconfig 
---------+----------+-------------+----------+---------+--------------+----------+----------+-----------
 bacula  |    16386 | f           | f        | f       | f            | ******** |          | 
(1 row)

postgres=# 
1
2
3
4
5
6
7
8
postgres=# \du
                                   List of roles
 Role name |                         Attributes                         | Member of 
-----------+------------------------------------------------------------+-----------
 bacula    |                                                            | {}
 postgres  | Superuser, Create role, Create DB, Replication, Bypass RLS | {}

postgres=# 

Allow the bacula user to create databases

The bacula database script will try to create the bacula catalog database. We’ll allow the bacula user to create databases,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
postgres=# alter user bacula CREATEDB;
ALTER ROLE
postgres=# select * from pg_user where usename = 'bacula';
 usename | usesysid | usecreatedb | usesuper | userepl | usebypassrls |  passwd  | valuntil | useconfig 
---------+----------+-------------+----------+---------+--------------+----------+----------+-----------
 bacula  |    16386 | t           | f        | f       | f            | ******** |          | 
(1 row)

postgres=# \du
                                   List of roles
 Role name |                         Attributes                         | Member of 
-----------+------------------------------------------------------------+-----------
 bacula    | Create DB                                                  | {}
 postgres  | Superuser, Create role, Create DB, Replication, Bypass RLS | {}

postgres=# 

Create the bacula database

We’ll create a bacula database so we can verify the database connection from the bacula user to the bacula database.

Create a new bacula database

1
2
3
postgres=# create database bacula;
CREATE DATABASE
postgres=# 

Grant all permissions to the bacula user

1
2
3
postgres=# grant ALL on DATABASE bacula to bacula;
GRANT
postgres=# 

Update pg_hba

The pg_hba.conf configuration controls the Host Based Access to your postgreSQL database(s).

1
2
3
4
5
[postgres@stafdb ~/data96]$ pwd
/var/db/postgres/data96
[postgres@stafdb ~/data96]$ id
uid=770(postgres) gid=770(postgres) groups=770(postgres)
[postgres@stafdb ~/data96]$ vi pg_hba.conf 

And add the next lines;

1
2
3
4
5
# TYPE  DATABASE        USER            ADDRESS                 METHOD
hostssl bacula          bacula          192.168.1.52/32         md5 clientcert=1
hostssl template0       bacula          192.168.1.52/32         md5 clientcert=1
hostssl template1       bacula          192.168.1.52/32         md5 clientcert=1
hostssl postgres        bacula          192.168.1.52/32         md5 clientcert=1

Our bacula jail 192.168.1.52 only needs to have to the bacula database with the bacula user over ssl hostssl passwords will be send as a md5 hash and a client certificate is required clientcert=1.

We could also used the cert method and map the client certificate to postgresql user so we could authenticate with the client certificate only…

We allow access to the template* and the postgres database because it’s required for the bacula database xcreate script. We can remove them ( only allow access to the bacula database ) after the catalog database is created.

Restart postgresql

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
root@stafdb:/var/db/postgres/data96 # service postgresql restart
DEBUG:  postgres: PostmasterMain: initial environment dump:
DEBUG:  -----------------------------------------
DEBUG:          LC_TIME=C
DEBUG:          LC_NUMERIC=C
DEBUG:          LC_MONETARY=C
DEBUG:          LC_MESSAGES=C
DEBUG:          LC_CTYPE=C
DEBUG:          LC_COLLATE=C
DEBUG:          MAIL=/var/mail/postgres
DEBUG:          PGLOCALEDIR=/usr/local/share/locale
DEBUG:          PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/var/db/postgres/bin
DEBUG:          PGDATA=/var/db/postgres/data96
DEBUG:          PWD=/var/db/postgres
DEBUG:          PGSYSCONFDIR=/usr/local/etc/postgresql
DEBUG:          HOME=/var/db/postgres
DEBUG:          USER=postgres
DEBUG:          SHELL=/bin/sh
DEBUG:          PG_GRANDPARENT_PID=79045
DEBUG:          BLOCKSIZE=K
DEBUG:  -----------------------------------------
LOG:  could not create IPv6 socket: Protocol not supported
LOG:  could not bind IPv4 socket: Address already in use
HINT:  Is another postmaster already running on port 5432? If not, wait a few seconds and retry.
WARNING:  could not create listen socket for "192.168.1.51"
DEBUG:  invoking IpcMemoryCreate(size=148480000)
DEBUG:  SlruScanDirectory invoking callback on pg_notify/0000
DEBUG:  removing file "pg_notify/0000"
DEBUG:  dynamic shared memory system will support 288 segments
DEBUG:  created dynamic shared memory control segment 773439544 (2316 bytes)
DEBUG:  max_safe_fds = 984, usable_fds = 1000, already_open = 6
LOG:  ending log output to stderr
HINT:  Future log output will go to log destination "syslog".
DEBUG:  CommitTransaction
DEBUG:  name: unnamed; blockState:       STARTED; state: INPROGR, xid/subid/cid: 0/1/0, nestlvl: 1, children: 
root@stafdb:/var/db/postgres/data96 #

Test the database connection

Verify

Verify the database connection for the bacula jail. See https://www.postgresql.org/docs/9.6/static/libpq-connect.html

1
2
3
4
5
6
7
8
9
10
[bacula@stafbacula /var/db/bacula/.postgres]$ psql "sslmode=verify-full host=stafdb dbname=bacula sslcert=`pwd`/postgresql.crt sslkey=`pwd`/postgresql.key sslrootcert=`pwd`/root.crt"
Password:
DEBUG:  CommitTransaction
DEBUG:  name: unnamed; blockState:       STARTED; state: INPROGR, xid/subid/cid: 0/1/0, nestlvl: 1, children:
psql (9.5.7, server 9.6.3)
WARNING: psql major version 9.5, server major version 9.6.
         Some psql features might not work.
SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off)
Type "help" for help.
bacula=>

Create environment script

Bacula comes with a few scripts to popilate the catalog we will create an “environment” script to setup the required environment variabeles to connect to the database. https://www.postgresql.org/docs/9.6/static/libpq-envars.html gives an overview of PostgreSQL environment variabeles.

1
[bacula@stafbacula /var/db/bacula]$ vi psql_env.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
PGHOST=stafdb
PGUSER=bacula
PGSSLMODE=verify-full
PGSSLCERT=/var/db/bacula/.postgres/postgresql.crt
PGSSLKEY=/var/db/bacula/.postgres/postgresql.key
PGSSLROOTCERT=/var/db/bacula/.postgres/root.crt

export PGHOST
export PGUSER
export PGSSLMODE
export PGSSLCERT
export PGSSLKEY
export PGSSLROOTCERT

Test the environment script

1
2
3
4
5
6
7
8
9
10
11
12
13
root@stafbacula:~ # su -m bacula -c /bin/sh
$ . /var/db/bacula/psql_env.sh
$ psql bacula
Password: 
DEBUG:  CommitTransaction
DEBUG:  name: unnamed; blockState:       STARTED; state: INPROGR, xid/subid/cid: 0/1/0, nestlvl: 1, children: 
psql (9.5.8, server 9.6.4)
WARNING: psql major version 9.5, server major version 9.6.
         Some psql features might not work.
SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off)
Type "help" for help.

bacula=> 

Configure the bacula catalog

Configuration directives

I found the bacula documention not very clear howto setup the catalog connection with certificate authentication - or I looked at the wrong place - so I downloaded the bacula source code ( version 7.4.7 )to verify the required directives. ./src/dird/d/dird_conf.c

1
[staf@vicky bacula-7.4.7]$ vim ./src/dird/dird_conf.c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
/*
   Bacula(R) - The Network Backup Solution

   Copyright (C) 2000-2016 Kern Sibbald

   The original author of Bacula is Kern Sibbald, with contributions
   from many others, a complete list can be found in the file AUTHORS.

   You may use this file and others of this release according to the
   license defined in the LICENSE file, which includes the Affero General
   Public License, v3.0 ("AGPLv3") and some additional permissions and
   terms pursuant to its AGPLv3 Section 7.

   This notice must be preserved when any source code is
   conveyed and/or propagated.

   Bacula(R) is a registered trademark of Kern Sibbald.
*/

<snip>

/*
 *    Catalog Resource Directives
 *
 *   name          handler     value                 code flags    default_value
 */
static RES_ITEM cat_items[] = {
   {"Name",     store_name,     ITEM(res_cat.hdr.name),    0, ITEM_REQUIRED, 0},
   {"Description", store_str,   ITEM(res_cat.hdr.desc),    0, 0, 0},
   {"dbaddress", store_str,     ITEM(res_cat.db_address),  0, 0, 0},
   {"Address",  store_str,      ITEM(res_cat.db_address),  0, 0, 0},
   {"DbPort",   store_pint32,   ITEM(res_cat.db_port),      0, 0, 0},
   /* keep this password as store_str for the moment */
   {"dbpassword", store_str,    ITEM(res_cat.db_password), 0, 0, 0},
   {"Password", store_str,      ITEM(res_cat.db_password), 0, 0, 0},
   {"dbuser",   store_str,      ITEM(res_cat.db_user),     0, 0, 0},
   {"User",     store_str,      ITEM(res_cat.db_user),     0, 0, 0},
   {"DbName",   store_str,      ITEM(res_cat.db_name),     0, ITEM_REQUIRED, 0},
   {"dbdriver", store_str,      ITEM(res_cat.db_driver),   0, 0, 0},
   {"DbSocket", store_str,      ITEM(res_cat.db_socket),   0, 0, 0},
   {"dbsslkey", store_str,      ITEM(res_cat.db_ssl_key),  0, 0, 0},
   {"dbsslcert", store_str,     ITEM(res_cat.db_ssl_cert),  0, 0, 0},
   {"dbsslca", store_str,       ITEM(res_cat.db_ssl_ca),  0, 0, 0},
   {"dbsslcapath", store_str,   ITEM(res_cat.db_ssl_capath),  0, 0, 0},
   {"dbsslcipher", store_str,   ITEM(res_cat.db_ssl_cipher),  0, 0, 0},
   /* Turned off for the moment */
   {"MultipleConnections", store_bit, ITEM(res_cat.mult_db_connections), 0, 0, 0},
   {"DisableBatchInsert", store_bool, ITEM(res_cat.disable_batch_insert), 0, ITEM_DEFAULT, false},
   {NULL, NULL, {0}, 0, 0, 0}
};

The ssl directives didn’t seem to work with postgresql :-( If we feed the postgresql environment variables with the correct ssl settings to the bacula director it seems to work.

Initialize the database

Drop the existing bacula database

The bacacla create script will try to create a new bacaula database so we’ll to drop or test database on our database server.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
root@stafdb:~ # su - postgres
$ psql
psql (9.6.4)
Type "help" for help.

postgres=# drop database bacula ;
DROP DATABASE
postgres=# \l
                             List of databases
   Name    |  Owner   | Encoding | Collate | Ctype |   Access privileges   
-----------+----------+----------+---------+-------+-----------------------
 postgres  | postgres | UTF8     | C       | C     | 
 template0 | postgres | UTF8     | C       | C     | =c/postgres          +
           |          |          |         |       | postgres=CTc/postgres
 template1 | postgres | UTF8     | C       | C     | =c/postgres          +
           |          |          |         |       | postgres=CTc/postgres
(3 rows)

postgres=# 

Create the database

Logon the bacula jail and create the bacula database.

1
2
3
4
5
6
7
8
9
10
11
$ . /var/db/bacula/psql_env.sh
$ ./create_bacula_database
Creating postgresql database
Password: 
Password: 
CREATE DATABASE
ALTER DATABASE
Creation of bacula database succeeded.
Password: 
Password: 
Database encoding OK

Populate the bacula tables

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
$ ./make_bacula_tables 
Making postgresql tables
Password: 
CREATE TABLE
ALTER TABLE
CREATE INDEX
CREATE TABLE
ALTER TABLE
CREATE INDEX
CREATE TABLE
CREATE INDEX
CREATE INDEX
CREATE TABLE
CREATE INDEX
CREATE TABLE
CREATE INDEX
CREATE TABLE
CREATE INDEX
CREATE TABLE
CREATE TABLE
CREATE INDEX
CREATE TABLE
CREATE INDEX
CREATE TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE INDEX
CREATE TABLE
CREATE INDEX
CREATE TABLE
CREATE INDEX
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE INDEX
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE INDEX
CREATE TABLE
CREATE INDEX
CREATE TABLE
CREATE TABLE
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
CREATE TABLE
CREATE INDEX
INSERT 0 1
Creation of Bacula PostgreSQL tables succeeded.
$ 

Verify

Logon the bacula database and verify that the database populated.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
$ psql
Password: 
psql (9.5.8, server 9.6.4)
WARNING: psql major version 9.5, server major version 9.6.
         Some psql features might not work.
SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off)
Type "help" for help.

bacula=> \d
                       List of relations
 Schema |               Name                |   Type   | Owner  
--------+-----------------------------------+----------+--------
 public | basefiles                         | table    | bacula
 public | basefiles_baseid_seq              | sequence | bacula
 public | cdimages                          | table    | bacula
 public | client                            | table    | bacula
 public | client_clientid_seq               | sequence | bacula
 public | counters                          | table    | bacula
 public | device                            | table    | bacula
 public | device_deviceid_seq               | sequence | bacula
 public | file                              | table    | bacula
 public | file_fileid_seq                   | sequence | bacula
 public | filename                          | table    | bacula
 public | filename_filenameid_seq           | sequence | bacula
 public | fileset                           | table    | bacula
 public | fileset_filesetid_seq             | sequence | bacula
 public | job                               | table    | bacula
 public | job_jobid_seq                     | sequence | bacula
 public | jobhisto                          | table    | bacula
 public | jobmedia                          | table    | bacula
 public | jobmedia_jobmediaid_seq           | sequence | bacula
 public | location                          | table    | bacula
 public | location_locationid_seq           | sequence | bacula
 public | locationlog                       | table    | bacula
 public | locationlog_loclogid_seq          | sequence | bacula
 public | log                               | table    | bacula
 public | log_logid_seq                     | sequence | bacula
 public | media                             | table    | bacula
 public | media_mediaid_seq                 | sequence | bacula
 public | mediatype                         | table    | bacula
 public | mediatype_mediatypeid_seq         | sequence | bacula
 public | path                              | table    | bacula
 public | path_pathid_seq                   | sequence | bacula
 public | pathhierarchy                     | table    | bacula
 public | pathvisibility                    | table    | bacula
 public | pool                              | table    | bacula
 public | pool_poolid_seq                   | sequence | bacula
 public | restoreobject                     | table    | bacula
 public | restoreobject_restoreobjectid_seq | sequence | bacula
 public | snapshot                          | table    | bacula
 public | snapshot_snapshotid_seq           | sequence | bacula
--More--(byte 2667)

Cleanup

Disable the access to template? and postgres databases.

1
root@stafdb:/var/db/postgres/data96 # vi pg_hba.conf
1
2
3
4
5
host    all             all             ::1/128                 trust
hostssl bacula          bacula          192.168.1.52/32         md5 clientcert=1
# hostssl       template0       bacula          192.168.1.52/32         md5 clientcert=1
# hostssl       template1       bacula          192.168.1.52/32         md5 clientcert=1
# hostssl       postgres        bacula          192.168.1.52/32         md5 clientcert=1

Reload

1
2
root@stafdb:/var/db/postgres/data96 # service postgresql reload
root@stafdb:/var/db/postgres/data96 # 

Test it. Verify that access to the postgres database is denied from the bacula host.

1
2
3
$ psql postgres
psql: FATAL:  no pg_hba.conf entry for host "192.168.1.52", user "bacula", database "postgres", SSL on
$ 

Bacula catalog configuration

Update the bacula director configuration

1
root@stafbacula:/usr/local/etc/bacula # vi bacula-dir.conf
1
2
3
4
5
6
7
8
# Generic catalog service
Catalog { 
  Name = MyCatalog
  dbname = "bacula"; dbuser = "bacula"; dbpassword = "********" ; dbsslkey = "/var/db/bacula/.postgres
/postgresql.key"; dbsslcert = "/var/db/bacula/.postgres/postgresql.crt"; dbsslca= "/var/db/bacula/.postgres
/root.crt"

}

Test the catalog connection

bacula include a program to verify the bacula catalog “dbcheck”, the -c switch select the bacula director configuration file the -B switch print out the configuration.

1
2
3
4
5
6
7
8
9
10
11
12
bacula@stafbacula /usr/local]$ dbcheck -c /usr/local/etc/bacula/bacula-dir.conf -B -v
catalog=MyCatalog
db_name=bacula
db_driver=
db_user=bacula
db_password=*******
db_address=stafdb
db_port=0
db_socket=
db_type=PostgreSQL
working_dir=/var/db/bacula
[bacula@stafbacula /usr/local]$ 

For some reason the ssl directives aren’t include and the connection fails

1
2
3
4
5
6
7
8
[bacula@stafbacula /usr/local/etc/rc.d]$ dbcheck -c /usr/local/etc/bacula/bacula-dir.conf -v
dbcheck: Fatal Error at dbcheck.c:303 because:
postgresql.c:271 Unable to connect to PostgreSQL server. Database=bacula User=bacula
Possible causes: SQL server not running; password incorrect; max_connections exceeded.
09-Sep 14:22 dbcheck: Fatal Error at dbcheck.c:303 because:
postgresql.c:271 Unable to connect to PostgreSQL server. Database=bacula User=bacula
Possible causes: SQL server not running; password incorrect; max_connections exceeded.
[bacula@stafbacula /usr/local/etc/rc.d]$ 

On our postgres host we get the error message that the bacula host tries to connect without SSL.

1
2
3
4
5
6
7
8
9
10
11
oot@stafdb:/var/db/postgres/data96 # tail -f /var/log/messages
Sep  9 14:22:10 stafdb postgres[14183]: [10-1] FATAL:  connection requires a valid client certificate
Sep  9 14:22:10 stafdb postgres[14184]: [10-1] FATAL:  no pg_hba.conf entry for host "192.168.1.52", user "bacula", database "bacula", SSL off
Sep  9 14:22:15 stafdb postgres[14185]: [10-1] FATAL:  connection requires a valid client certificate
Sep  9 14:22:15 stafdb postgres[14186]: [10-1] FATAL:  no pg_hba.conf entry for host "192.168.1.52", user "bacula", database "bacula", SSL off
Sep  9 14:22:20 stafdb postgres[14187]: [10-1] FATAL:  connection requires a valid client certificate
Sep  9 14:22:20 stafdb postgres[14188]: [10-1] FATAL:  no pg_hba.conf entry for host "192.168.1.52", user "bacula", database "bacula", SSL off
Sep  9 14:22:25 stafdb postgres[14190]: [10-1] FATAL:  connection requires a valid client certificate
Sep  9 14:22:25 stafdb postgres[14191]: [10-1] FATAL:  no pg_hba.conf entry for host "192.168.1.52", user "bacula", database "bacula", SSL off
Sep  9 14:22:30 stafdb postgres[14193]: [10-1] FATAL:  connection requires a valid client certificate
Sep  9 14:22:30 stafdb postgres[14194]: [10-1] FATAL:  no pg_hba.conf entry for host "192.168.1.52", user "bacula", database "bacula", SSL off

When set the postgresql varialables with the correct ssl settings the connnection works fine.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
[bacula@stafbacula /usr/local/etc/rc.d]$ dbcheck -c /usr/local/etc/bacula/bacula-dir.conf -v
Hello, this is the database check/correct program.
Modify database is off. Verbose is on.
Please select the function you want to perform.

     1) Toggle modify database flag
     2) Toggle verbose flag
     3) Check for bad Filename records
     4) Check for bad Path records
     5) Check for duplicate Filename records
     6) Check for duplicate Path records
     7) Check for orphaned Jobmedia records
     8) Check for orphaned File records
     9) Check for orphaned Path records
    10) Check for orphaned Filename records
    11) Check for orphaned FileSet records
    12) Check for orphaned Client records
    13) Check for orphaned Job records
    14) Check for all Admin records
    15) Check for all Restore records
    16) All (3-15)
    17) Quit
Select function number: 

Bacula director

Enable the bacula director

1
2
3
root@stafbacula:/usr/local/etc/rc.d # sysrc bacula_dir_enable=yes
bacula_dir_enable:  -> yes
root@stafbacula:/usr/local/etc/rc.d # 

Create the bacula.log

1
2
root@stafbacula:/var/log # touch /var/log/bacula.log
root@stafbacula:/var/log # chown bacula:bacula /var/log/bacula.log

Include the postgreSQL ssl settings in the bacula director startup script

Update the bacula-dir startup sript to include the ssl settings.

1
2
3
4
5
6
7
8
9
10
# Add the following lines to /etc/rc.conf.local or /etc/rc.conf
# to enable this service:
#
# bacula_dir_enable  (bool):   Set to NO by default.
#                Set it to YES to enable bacula_dir.
# bacula_dir_flags (params):   Set params used to start bacula_dir.
#

. /etc/rc.subr
. /var/db/bacula/psql_env.sh

bconsole access

To test that the catalog works correctly with the director we need to setup bconsole access. Open the bacula director configuration file.

1
[root@stafbacula /usr/local/etc/bacula]# vim bacula-dir.conf

And defined and Password

1
2
3
4
5
6
7
8
9
10
Director {                            # define myself
  Name = MyBaculaDirector
  DIRport = 9101                # where we listen for UA connections
  QueryFile = "/usr/local/share/bacula/query.sql"
  WorkingDirectory = "/var/db/bacula"
  PidDirectory = "/var/run"
  Maximum Concurrent Jobs = 20
  Password = "*******"         # Console password
  Messages = Daemon
}

Open the bconsole configuration file

1
[root@stafbacula /usr/local/etc/bacula]# vi bconsole.conf

and setup the same password

1
2
3
4
5
6
7
8
9
10
11
12
# Bacula User Agent (or Console) Configuration File
#
# Copyright (C) 2000-2015 Kern Sibbald
# License: BSD 2-Clause; see file LICENSE-FOSS
#

Director {
  Name = MyBaculaDirector
  DIRport = 9101
  address = localhost
  Password = "*****"
}

Start the director & test

Start the bacula-dir service

1
2
3
4
5
6
7
root@stafbacula /usr/local/etc/bacula]# service bacula-dir start
Starting bacula_dir.
[root@stafbacula /usr/local/etc/bacula]# ps aux | grep -i bacula 
bacula 14416  0.0  0.1 51424 6588  -  SsJ  14:40   0:00.12 /usr/local/sbin/bacula-dir -u bacula -g bacula 
root   14420  0.0  0.0 14796 1968  0  R+J  14:40   0:00.00 grep -i bacula
root   13530  0.0  0.0  8300 1596  2  I+J  13:47   0:00.00 tail -f /var/log/bacula.log
[root@stafbacula /usr/local/etc/bacula]#

And test the console access

1
2
3
4
5
6
7
8
bacula@stafbacula:/usr/local/etc/bacula % bconsole
Connecting to Director localhost:9101
1000 OK: 102 MyBaculaDirector Version: 7.4.7 (16 March 2017)
Enter a period to cancel a command.
*version
MyBaculaDirector Version: 7.4.7 (16 March 2017) amd64-portbld-freebsd11.0 freebsd 11.0-RELEASE-p12 
You have messages.
*

In a next blog post we’ll continue with the bacula configuration.

Have fun!

Links