Bacula on FreeBSD (part 1 PostgresSQL in a jail)
I do take backups; my current solution are couple of shell script wrapper around dump/zfs send/btrfs send/rsync which is a mess. So decided give bacula a try
I use ezjail to manage my FreeBSD jails. PostgresSQL is my favorite database and will use this database as the backend for bacula and will use this database as the backend for bacula. I want to move all my databases to 1 FreeBSD jail this should make the easier to create reliable database backup in the further. For this reason we’ll setup 2 FreeBSD jails 1 for the database and 1 for bacula.
You’ll find my journey of installing PostgreSQL on a FreeBSD jail. In another blog post we will continue with the installation of bacula.
PostgreSQL
Jail
Create the PostgreSQL Jail
root@rataplan:~ # ezjail-admin create stafdb "em0|192.168.1.51"
Warning: Some services already seem to be listening on all IP, (including 192.168.1.51)
This may cause some confusion, here they are:
root ntpd 754 20 udp6 *:123 *:*
root ntpd 754 21 udp4 *:123 *:*
root rpc.statd 717 4 udp6 *:640 *:*
root rpc.statd 717 5 tcp6 *:640 *:*
root rpc.statd 717 6 udp4 *:640 *:*
root rpc.statd 717 7 tcp4 *:640 *:*
root nfsd 713 5 tcp4 *:2049 *:*
root nfsd 713 6 tcp6 *:2049 *:*
root mountd 707 5 udp6 *:753 *:*
root mountd 707 6 tcp6 *:753 *:*
root mountd 707 7 udp4 *:753 *:*
root mountd 707 8 tcp4 *:753 *:*
root rpcbind 676 6 udp6 *:111 *:*
root rpcbind 676 7 udp6 *:847 *:*
root rpcbind 676 8 tcp6 *:111 *:*
root rpcbind 676 9 udp4 *:111 *:*
root rpcbind 676 10 udp4 *:766 *:*
root rpcbind 676 11 tcp4 *:111 *:*
root syslogd 657 6 udp6 *:514 *:*
root syslogd 657 7 udp4 *:514 *:*
root@rataplan:~ #
PostgreSQL requires shared memory
PostgreSQL uses shared memory it’s required to set “allow.sysvipc=1” for the jail. I don’t want to enable this globaly since this might be a security risk. Shared memory has permissions set based on the uid enabling sysvipc on a jail might cause the jail to read shared memory from the host system or another jail.
To enable “allow.sysvipc=1” a jail we can update the ezjail configuration. Ezjail keep the jail configuration in /usr/local/etc/ezjail
root@rataplan:~ # cd /usr/local/etc/ezjail
root@rataplan:/usr/local/etc/ezjail # ls
stafansible staffs stafproxy staftestbuild
stafdb stafmail stafpuppet
root@rataplan:/usr/local/etc/ezjail #
Open the database jail file and update the configuration.
root@rataplan:/usr/local/etc/ezjail # vi stafdb
#
# Required to run PostgeSQL in the jail
#
export jail_stafdb_parameters="allow.sysvipc=1"
Start the database jail
root@rataplan:~ # ezjail-admin start stafdb
Starting jails: stafdb.
/etc/rc.d/jail: WARNING: Per-jail configuration via jail_* variables is obsolete. Please consider migrating to /etc/jail.conf.
root@rataplan:~ #
Console access
root@rataplan:~ # ezjail-admin console stafdb
FreeBSD 11.0-RELEASE-p9 (GENERIC) #0: Tue Apr 11 08:48:40 UTC 2017
Welcome to FreeBSD!
Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories: https://www.FreeBSD.org/security/
FreeBSD Handbook: https://www.FreeBSD.org/handbook/
FreeBSD FAQ: https://www.FreeBSD.org/faq/
Questions List: https://lists.FreeBSD.org/mailman/listinfo/freebsd-questions/
FreeBSD Forums: https://forums.FreeBSD.org/
Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with: pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.
Show the version of FreeBSD installed: freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages: man man
FreeBSD directory layout: man hier
Edit /etc/motd to change this login announcement.
root@stafdb:~ #
root@stafdb:~ #
ProgreSQL installation
Install pkg
Set up dns
root@stafdb:~ # vi /etc/resolv.conf
nameserver 192.168.1.1
Bootstrap pkg
root@stafdb:~ # pkg
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:11:amd64/quarterly, please wait...
Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done
[stafdb] Installing pkg-1.10.1...
[stafdb] Extracting pkg-1.10.1: 100%
pkg: not enough arguments
Usage: pkg [-v] [-d] [-l] [-N] [-j <jail name or id>|-c <chroot path>|-r <rootdir>] [-C <configuration file>] [-R <repo config dir>] [-o var=value] [-4|-6] <command> [<args>]
For more information on available commands and options see 'pkg help'.
root@stafdb:~ #
Install PostgreSQL
Search for the latest PostgreSQL server version.
root@stafdb:~ # pkg search postgresql | grep server
pgtcl-postgresql92-2.0.0_1 TCL extension for accessing a PostgreSQL server (PGTCL-NG)
pgtcl-postgresql93-2.0.0_1 TCL extension for accessing a PostgreSQL server (PGTCL-NG)
pgtcl-postgresql94-2.0.0_1 TCL extension for accessing a PostgreSQL server (PGTCL-NG)
pgtcl-postgresql95-2.0.0_1 TCL extension for accessing a PostgreSQL server (PGTCL-NG)
pgtcl-postgresql96-2.0.0_1 TCL extension for accessing a PostgreSQL server (PGTCL-NG)
postgresql92-server-9.2.21_1 PostgreSQL is the most advanced open-source database available anywhere
postgresql93-server-9.3.17_1 PostgreSQL is the most advanced open-source database available anywhere
postgresql94-server-9.4.12_1 PostgreSQL is the most advanced open-source database available anywhere
postgresql95-server-9.5.7_1 PostgreSQL is the most advanced open-source database available anywhere
postgresql96-server-9.6.3_1 PostgreSQL is the most advanced open-source database available anywhere
root@stafdb:~ #
Install the PostgreSQL package.
root@stafdb:~ # pkg install postgresql96-server
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
The following 8 package(s) will be affected (of 0 checked):
New packages to be INSTALLED:
postgresql96-server: 9.6.3_1
libxml2: 2.9.4
icu: 58.2_2,1
gettext-runtime: 0.19.8.1_1
indexinfo: 0.2.6
postgresql96-client: 9.6.3_2
perl5: 5.24.1_1
readline: 7.0.3
Number of packages to be installed: 8
The process will require 131 MiB more space.
30 MiB to be downloaded.
Proceed with this action? [y/N]: y
[stafdb] [1/8] Fetching postgresql96-server-9.6.3_1.txz: 100% 4 MiB 357.1kB/s 00:11
[stafdb] [2/8] Fetching libxml2-2.9.4.txz: 100% 802 KiB 410.4kB/s 00:02
[stafdb] [3/8] Fetching icu-58.2_2,1.txz: 100% 9 MiB 313.3kB/s 00:30
[stafdb] [4/8] Fetching gettext-runtime-0.19.8.1_1.txz: 100% 147 KiB 151.0kB/s 00:01
[stafdb] [5/8] Fetching indexinfo-0.2.6.txz: 100% 5 KiB 5.3kB/s 00:01
[stafdb] [6/8] Fetching postgresql96-client-9.6.3_2.txz: 100% 2 MiB 300.0kB/s 00:08
[stafdb] [7/8] Fetching perl5-5.24.1_1.txz: 100% 13 MiB 341.5kB/s 00:41
[stafdb] [8/8] Fetching readline-7.0.3.txz: 100% 334 KiB 342.3kB/s 00:01
Checking integrity... done (0 conflicting)
[stafdb] [1/8] Installing indexinfo-0.2.6...
[stafdb] [1/8] Extracting indexinfo-0.2.6: 100%
[stafdb] [2/8] Installing gettext-runtime-0.19.8.1_1...
[stafdb] [2/8] Extracting gettext-runtime-0.19.8.1_1: 100%
[stafdb] [3/8] Installing perl5-5.24.1_1...
[stafdb] [3/8] Extracting perl5-5.24.1_1: 100%
[stafdb] [4/8] Installing readline-7.0.3...
[stafdb] [4/8] Extracting readline-7.0.3: 100%
[stafdb] [5/8] Installing libxml2-2.9.4...
[stafdb] [5/8] Extracting libxml2-2.9.4: 100%
[stafdb] [6/8] Installing icu-58.2_2,1...
[stafdb] [6/8] Extracting icu-58.2_2,1: 100%
[stafdb] [7/8] Installing postgresql96-client-9.6.3_2...
[stafdb] [7/8] Extracting postgresql96-client-9.6.3_2: 100%
[stafdb] [8/8] Installing postgresql96-server-9.6.3_1...
===> Creating groups.
Creating group 'postgres' with gid '770'.
===> Creating users
Creating user 'postgres' with uid '770'.
=========== BACKUP YOUR DATA! =============
As always, backup your data before
upgrading. If the upgrade leads to a higher
minor revision (e.g. 8.3.x -> 8.4), a dump
and restore of all databases is
required. This is *NOT* done by the port!
===========================================
[stafdb] Extracting postgresql96-server-9.6.3_1: 100%
Message from perl5-5.24.1_1:
The /usr/bin/perl symlink has been removed starting with Perl 5.20.
For shebangs, you should either use:
#!/usr/local/bin/perl
or
#!/usr/bin/env perl
The first one will only work if you have a /usr/local/bin/perl,
the second will work as long as perl is in PATH.
Message from postgresql96-client-9.6.3_2:
The PostgreSQL port has a collection of "side orders":
postgresql-docs
For all of the html documentation
p5-Pg
A perl5 API for client access to PostgreSQL databases.
postgresql-tcltk
If you want tcl/tk client support.
postgresql-jdbc
For Java JDBC support.
postgresql-odbc
For client access from unix applications using ODBC as access
method. Not needed to access unix PostgreSQL servers from Win32
using ODBC. See below.
ruby-postgres, py-PyGreSQL
For client access to PostgreSQL databases using the ruby & python
languages.
postgresql-plperl, postgresql-pltcl & postgresql-plruby
For using perl5, tcl & ruby as procedural languages.
postgresql-contrib
Lots of contributed utilities, postgresql functions and
datatypes. There you find pg_standby, pgcrypto and many other cool
things.
etc...
Message from postgresql96-server-9.6.3_1:
For procedural languages and postgresql functions, please note that
you might have to update them when updating the server.
If you have many tables and many clients running, consider raising
kern.maxfiles using sysctl(8), or reconfigure your kernel
appropriately.
The port is set up to use autovacuum for new databases, but you might
also want to vacuum and perhaps backup your database regularly. There
is a periodic script, /usr/local/etc/periodic/daily/502.pgsql, that
you may find useful. You can use it to backup and perform vacuum on all
databases nightly. Per default, it performs `vacuum analyze'. See the
script for instructions. For autovacuum settings, please review
~pgsql/data/postgresql.conf.
If you plan to access your PostgreSQL server using ODBC, please
consider running the SQL script /usr/local/share/postgresql/odbc.sql
to get the functions required for ODBC compliance.
Please note that if you use the rc script,
/usr/local/etc/rc.d/postgresql, to initialize the database, unicode
(UTF-8) will be used to store character data by default. Set
postgresql_initdb_flags or use login.conf settings described below to
alter this behaviour. See the start rc script for more info.
To set limits, environment stuff like locale and collation and other
things, you can set up a class in /etc/login.conf before initializing
the database. Add something similar to this to /etc/login.conf:
---
postgres:\
:lang=en_US.UTF-8:\
:setenv=LC_COLLATE=C:\
:tc=default:
---
and run `cap_mkdb /etc/login.conf'.
Then add 'postgresql_class="postgres"' to /etc/rc.conf.
======================================================================
To initialize the database, run
/usr/local/etc/rc.d/postgresql initdb
You can then start PostgreSQL by running:
/usr/local/etc/rc.d/postgresql start
For postmaster settings, see ~pgsql/data/postgresql.conf
NB. FreeBSD's PostgreSQL port logs to syslog by default
See ~pgsql/data/postgresql.conf for more info
======================================================================
To run PostgreSQL at startup, add
'postgresql_enable="YES"' to /etc/rc.conf
Enable the postgresql daemon at the jail startup
root@stafdb:~ # sysrc postgresql_enable="YES"
postgresql_enable: -> YES
root@stafdb:~ # grep postgresql_enable /etc/rc.conf
postgresql_enable="YES"
root@stafdb:~ #
Initialize the database
root@stafdb:~ # service postgresql initdb
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /var/db/postgres/data96 ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/usr/local/bin/pg_ctl -D /var/db/postgres/data96 -l logfile start
root@stafdb:~ #
Start the database
Droot@stafdb:~ # service postgresql start
LOG: could not create IPv6 socket: Protocol not supported
LOG: ending log output to stderr
HINT: Future log output will go to log destination "syslog".
root@stafdb:~ #
Verify the database
root@stafdb:~ # su - postgres
$ psql -l
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
$ ^Droot@stafdb:~ # service postgresql start
LOG: could not create IPv6 socket: Protocol not supported
LOG: ending log output to stderr
HINT: Future log output will go to log destination "syslog".
root@stafdb:~ # su - postgres
$ psql -l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+---------+-------+-----------------------
postgres | postgres | UTF8 | C | C |
template0 | postgres | UTF8 | C | C | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | C | C | =c/postgres +
| | | | | postgres=CTc/postgres
(3 rows)
$
Have fun!
Links
- https://dan.langille.org/2015/01/10/bacula-on-freebsd-with-zfs/
- https://cwharton.com/blog/2016/10/postgresql-and-freebsd-quick-start/
- https://dan.langille.org/2013/07/09/fatal-could-not-create-shared-memory-segment-function-not-implemented/
Leave a comment