stafwag Blog

staf wagemakers blog

Setting Up OpenStack-Ansible All-In-One on a Centos 7 System

"openstack-logo"

Openstack is nice platform to deploy an Infrastructure as a service and is a collection of projects but it can be a bit difficult to setup. The documentation is really great if you want to setup openstack by hand and there are a few openstack distributions that makes it easier to install it.

Ansible is a very nice tool for system automatisation and is one that’s easier to learn.

"ansible-logo-red"

Wouldn’t be nice if we could make the openstack installation easier with ansible? That’s exactly what Openstack-Ansible does.

In this blog post we’ll setup “an all-in-one” openstack installation on Centos 7. The installer will install openstack into lxc containers and it’s nice way to learn how openstack works and how to operate it.

Preparation

System requirements

I use a Centos 7 virtual system running as a KVM instance with nested KVM virtualasation enabled. The system requiremensts The minimun requiremenst are:

  • 8 CPU cores
  • 50 GB of free diskspace
  • 8GB RAM

update ….

Make sure that your system is up-to-update

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
[staf@openstack ~]$ sudo yum update -y

We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:

    #1) Respect the privacy of others.
    #2) Think before you type.
    #3) With great power comes great responsibility.

[sudo] password for staf: 
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: distrib-coffee.ipsl.jussieu.fr
 * extras: mirror.in2p3.fr
 * updates: centos.mirror.fr.planethoster.net
base                                                                                                                                    | 3.6 kB  00:00:00     
extras                                                                                                                                  | 3.4 kB  00:00:00     
updates                                                                                                                                 | 3.4 kB  00:00:00     
No packages marked for update
[staf@openstack ~]$ 

Install git

We’ll need git to install the ansible playbooks and the Openstack-Ansible installation scripts.

1
2
3
4
5
6
7
8
9
10
11
12
[staf@openstack ~]$ yum install git
Loaded plugins: fastestmirror
You need to be root to perform this command.
[staf@openstack ~]$ sudo yum install git
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.in2p3.fr
 * extras: mirror.in2p3.fr
 * updates: centos.mirror.fr.planethoster.net
Package git-1.8.3.1-20.el7.x86_64 already installed and latest version
Nothing to do
[staf@openstack ~]$ 

Ansible….

This is a bit of a pitfail… The Openstack-Ansible bootstrap script will download and install his own version of ansible and create a link to /usr/local/bin. So /usr/local/bin must be in your $PATH. Ansible shouldn’t be installed on your system or if it is installed it shouln’t be executed instead of the ansible version that is builded with Openstack-Ansible.

On most GNU/Linux distributions have /usr/local/bin and /usr/local/sbin is in the $PATH but not on centos, so we’ll need to add it.

Make sure that ansible insn’t installed

1
2
3
[staf@openstack ~]$ sudo rpm -qa | grep -i ansible
[sudo] password for staf: 
[staf@openstack ~]$ 

Update your $PATH

1
[root@openstack ~]# export PATH=/usr/local/bin:$PATH

If you want to have /usr/local/bin in your $PATH update /etc/profile or $HOME/.profile

ssh password authentication

The ansibe playbooks will disable PasswordAuthentication, make sure that you login with a ssh key. - Password authentication is obsolete anyway -

firewalld

Firewall is enabled on Centos by default, the default iptables rules prevent communication between the openstack containers.

stop and disable firewalld

1
2
3
4
[root@openstack ~]# systemctl stop firewalld
[root@openstack ~]# systemctl disable firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

verify

1
2
3
4
5
6
7
8
9
10
root@openstack ~]# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
[root@openstack ~]# 

Openstack installation

The installation will take some time therefor it’s recommended to use an session manager like tmux or GNU screen

Bootstrap

git clone

clone the openstack-ansible git repo

1
2
3
4
5
6
7
8
[root@openstack ~]# git clone https://git.openstack.org/openstack/openstack-ansible /opt/openstack-ansible
Cloning into '/opt/openstack-ansible'...
remote: Counting objects: 67055, done.
remote: Compressing objects: 100% (32165/32165), done.
remote: Total 67055 (delta 45474), reused 52564 (delta 32073)
Receiving objects: 100% (67055/67055), 14.60 MiB | 720.00 KiB/s, done.
Resolving deltas: 100% (45474/45474), done.
[root@openstack ~]# 
1
2
[root@openstack ~]# cd /opt/openstack-ansible
[root@openstack openstack-ansible]# 

choose you Openstack releases

Openstack has release shedule about every 6 months the current stable release is Rocky. Every Openstack release has his own branch in the git repo. Each Openstack-Ansible release is tagged in the git repo. So either you’ll need checkout Openstack-Ansible release tag or the bracnh. We’ll checkout the Rocky branch.

get the list of branches

1
2
3
4
5
6
7
8
9
[root@openstack openstack-ansible]# git branch -a
* master
  remotes/origin/HEAD -> origin/master
  remotes/origin/master
  remotes/origin/stable/ocata
  remotes/origin/stable/pike
  remotes/origin/stable/queens
  remotes/origin/stable/rocky
[root@openstack openstack-ansible]# 
checkout the branch
1
2
3
4
[root@openstack openstack-ansible]# git checkout stable/rocky
Branch stable/rocky set up to track remote branch stable/rocky from origin.
Switched to a new branch 'stable/rocky'
[root@openstack openstack-ansible]# 

Bootstrap ansible

Execute scripts/bootstrap-ansible.sh this will install the required packages and ansible playbooks.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
[root@openstack openstack-ansible]# scripts/bootstrap-ansible.sh
+ export HTTP_PROXY=
+ HTTP_PROXY=
+ export HTTPS_PROXY=
+ HTTPS_PROXY=
+ export ANSIBLE_PACKAGE=ansible==2.5.14
+ ANSIBLE_PACKAGE=ansible==2.5.14
+ export ANSIBLE_ROLE_FILE=ansible-role-requirements.yml
+ ANSIBLE_ROLE_FILE=ansible-role-requirements.yml
+ export SSH_DIR=/root/.ssh
+ SSH_DIR=/root/.ssh
+ export DEBIAN_FRONTEND=noninteractive
+ DEBIAN_FRONTEND=noninteractive
<SNIP>
+ unset ANSIBLE_LIBRARY
+ unset ANSIBLE_LOOKUP_PLUGINS
+ unset ANSIBLE_FILTER_PLUGINS
+ unset ANSIBLE_ACTION_PLUGINS
+ unset ANSIBLE_CALLBACK_PLUGINS
+ unset ANSIBLE_CALLBACK_WHITELIST
+ unset ANSIBLE_TEST_PLUGINS
+ unset ANSIBLE_VARS_PLUGINS
+ unset ANSIBLE_STRATEGY_PLUGINS
+ unset ANSIBLE_CONFIG
+ '[' false == true ']'
+ echo 'System is bootstrapped and ready for use.'
System is bootstrapped and ready for use.
[root@openstack openstack-ansible]# 

Verify

scripts/bootstrap-ansible created /opt/ansible-runtime and create amd updated //usr/local/bin with a few links.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
[root@openstack openstack-ansible]# ls -ld /opt/*
drwxr-xr-x.  5 root root   56 Jan 12 11:42 /opt/ansible-runtime
drwxr-xr-x. 14 root root 4096 Jan 12 11:43 /opt/openstack-ansible
[root@openstack openstack-ansible]# ls -ltr /usr/local/bin/
total 8
lrwxrwxrwx. 1 root root   32 Jan 12 11:43 ansible -> /usr/local/bin/openstack-ansible
lrwxrwxrwx. 1 root root   39 Jan 12 11:43 ansible-config -> /opt/ansible-runtime/bin/ansible-config
lrwxrwxrwx. 1 root root   43 Jan 12 11:43 ansible-connection -> /opt/ansible-runtime/bin/ansible-connection
lrwxrwxrwx. 1 root root   40 Jan 12 11:43 ansible-console -> /opt/ansible-runtime/bin/ansible-console
lrwxrwxrwx. 1 root root   39 Jan 12 11:43 ansible-galaxy -> /opt/ansible-runtime/bin/ansible-galaxy
lrwxrwxrwx. 1 root root   36 Jan 12 11:43 ansible-doc -> /opt/ansible-runtime/bin/ansible-doc
lrwxrwxrwx. 1 root root   42 Jan 12 11:43 ansible-inventory -> /opt/ansible-runtime/bin/ansible-inventory
lrwxrwxrwx. 1 root root   32 Jan 12 11:43 ansible-playbook -> /usr/local/bin/openstack-ansible
lrwxrwxrwx. 1 root root   37 Jan 12 11:43 ansible-pull -> /opt/ansible-runtime/bin/ansible-pull
lrwxrwxrwx. 1 root root   38 Jan 12 11:43 ansible-vault -> /opt/ansible-runtime/bin/ansible-vault
-rw-r--r--. 1 root root 3169 Jan 12 11:43 openstack-ansible.rc
-rwxr-xr-x. 1 root root 2638 Jan 12 11:43 openstack-ansible

Verify that ansible command is one that’s installed bu the Openstack-Ansible bootstrap script.

1
2
[root@openstack openstack-ansible]# which ansible
/usr/local/bin/ansible

Bootstrap AIO

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
[root@openstack openstack-ansible]# scripts/bootstrap-aio.sh
+ export BOOTSTRAP_OPTS=
+ BOOTSTRAP_OPTS=
+++ dirname scripts/bootstrap-aio.sh
++ readlink -f scripts/..
+ export OSA_CLONE_DIR=/opt/openstack-ansible
TASK [Gathering Facts] *****************************************************************************************************
ok: [localhost]

TASK [sshd : Set OS dependent variables] ***********************************************************************************
ok: [localhost] => (item=/etc/ansible/roles/sshd/vars/RedHat_7.yml)

TASK [sshd : OS is supported] **********************************************************************************************
ok: [localhost] => {
    "changed": false, 
    "msg": "All assertions passed"
}

TASK [sshd : Install ssh packages] 
<SNIP>
EXIT NOTICE [Playbook execution success] **************************************
===============================================================================
+ popd
/opt/openstack-ansible
+ unset ANSIBLE_INVENTORY
+ unset ANSIBLE_VARS_PLUGINS
+ unset HOST_VARS_PATH
+ unset GROUP_VARS_PATH
[root@openstack openstack-ansible]# 

Run the playbooks

We’ll to run a few playbooks to setup the containers and our Openstack environment.

Move to the openstack-ansible playbook directory.

1
2
3
4
[root@aio1 ~]# cd /opt/openstack-ansible/playbooks/
[root@aio1 playbooks]# pwd
/opt/openstack-ansible/playbooks
[root@aio1 playbooks]# 

and exexcute the playbooks.

1
2
3
[root@openstack playbooks]# openstack-ansible setup-hosts.yml
[root@openstack playbooks]# openstack-ansible setup-infrastructure.yml
[root@aio1 playbooks]# openstack-ansible setup-openstack.yml

If all goes well your openstack installation is completed.

You can verify the openstack containers with lxc-ls

1
2
3
4
5
6
7
8
9
10
11
12
13
14
[root@aio1 playbooks]# lxc-ls --fancy
NAME                                   STATE   AUTOSTART GROUPS            IPV4                                           IPV6 
aio1_cinder_api_container-c211b759     RUNNING 1         onboot, openstack 10.255.255.43, 172.29.237.244, 172.29.244.190  -    
aio1_galera_container-9a90cbd9         RUNNING 1         onboot, openstack 10.255.255.50, 172.29.239.126                  -    
aio1_glance_container-c05aab79         RUNNING 1         onboot, openstack 10.255.255.218, 172.29.236.160, 172.29.247.238 -    
aio1_horizon_container-81943ba2        RUNNING 1         onboot, openstack 10.255.255.160, 172.29.237.37                  -    
aio1_keystone_container-a5859104       RUNNING 1         onboot, openstack 10.255.255.40, 172.29.236.95                   -    
aio1_memcached_container-ab998d0e      RUNNING 1         onboot, openstack 10.255.255.175, 172.29.239.49                  -    
aio1_neutron_server_container-439aeb90 RUNNING 1         onboot, openstack 10.255.255.137, 172.29.239.13                  -    
aio1_nova_api_container-c83e5ef0       RUNNING 1         onboot, openstack 10.255.255.216, 172.29.236.52                  -    
aio1_rabbit_mq_container-4fd792fb      RUNNING 1         onboot, openstack 10.255.255.2, 172.29.239.62                    -    
aio1_repo_container-b39d88a1           RUNNING 1         onboot, openstack 10.255.255.227, 172.29.237.146                 -    
aio1_utility_container-fff0b6df        RUNNING 1         onboot, openstack 10.255.255.117, 172.29.237.82                  -    
[root@aio1 playbooks]# 

Find the correct ip address

You should see horizon running with netstat

1
2
3
4
5
6
[root@aio1 ~]# netstat -pan | grep -i 443
tcp        0      0 172.29.236.100:443      0.0.0.0:*               LISTEN      12908/haproxy       
tcp        0      0 192.168.122.23:443      0.0.0.0:*               LISTEN      12908/haproxy       
unix  3      [ ]         STREAM     CONNECTED     73443    31134/tmux           
unix  2      [ ]         DGRAM                    1244303  23435/rsyslogd       
[root@aio1 ~]# 

Logon to the openstack GUI (Horizon)

Password…

1
[root@aio1 ~]# grep keystone_auth_admin_password /etc/openstack_deploy/user_secrets.yml

"openstack-ansible-aio-login.png"

Have fun

Links

Best Wishes 2019

2019

How to Configure DNS-over-TLS on OPNsense

DNS-over-TLS

In my previous blog posts we configured Stubby on GNU/Linux and FreeBSD.

"Logo_OPNsense.jpg"

In this blog article we’ll configure DNS-over-TLS with Unbound on OPNsense. Both Stubby and Unbound are written by NLnet.

DNS resolvers

Stubby is a small dns resolver to encrypt your dns traffic, which makes it perfect to increase end-user privacy. Stubby can be integrated into existing dns setups.

DNSmasq is small dns resolver that can cache dns queries and forward dns traffic to other dns servers.

Unbound is fast validating, caching DNS resolver that supports DNS-over-TLS. Unbound or dnsmaq are not full feature dns servers like BIND.

The main difference beteen Unbound and DNSmasq is that Unbound can talk the the root servers directly while dnsmasq always needs to forward your dns queries to another dns server - your ISP dns server or a public dns servicve like (Quad9, cloudfare, google, …) -

Unbound has build-in support for DNS-over-TLS. DNSmasq needs an external DNS-over-TLS resolver like Stubby.

Which one to use?

It depends - as always -, Stubby can integrating easily in existing dns setups like dnsmasq. Unbound is one package that does it all and is more feature rich compared to DNSmasq.

OPNsense

I use OPNsense as my firewall. Unbound is the default dns resolver on OPNsense so it makes (OPN)sense to use Unbound.

Choose your upstream DNS service

There’re a few public DNS providers that supports DNS-over-tls the best known are Quad9, cloudfare. Quad9 will block malicious domains on the default dns servers 9.9.9.9/149.112.112.10 while 9.9.9.10 has no security blocklist.

In this article we’ll use Quad9 but you could also with cloudfare or another dns provider that you trust and has support for DNS-over-tls.

Enable DNS-over-TLS

opnsense_enable_dns_tls.png

You need to configure your firewall to use your upstream dns provider. You also want to make sure your isp dns servers aren’t used.

Sniffing

If you snif the DNS traffic on your firewall tcpdump -i wan_interface udp port 53 you’ll see that the DNS traffic is unencrypted.

Configuration

To enable DNS-over-TLS we’ll need to reconfigure unbound.

Go to [ Services ] -> [Unbound DNS ] -> [General] And copy/paste the setting below

1
2
3
4
5
6
server:
forward-zone:
name: "."
forward-ssl-upstream: yes
forward-addr: 9.9.9.9@853
forward-addr: 149.112.112.112@853

to Custom options these settings will reconfigure Unbound to forward the dns for the upstream dns servers Quad9 over ssl.

Verify

If you snif the udp traffic on you firewall with tcpdump -i wan_interface udp port 53 you’ll not see any unencrypted traffic anymore - unless not all your clients are configured to use your firewall as the dns server -.

If your snif TCP PORT 853 tcpdump -i vr1 tcp port 853 we’ll see your encrypted dns-over-tls traffic.

General DNS settings

You also want to make sure that your firewall isn’t configure to use an unecrypted DNS server.

opnsense_set_dns.png

Configuration

Go to [ system ] -> [ settings ] -> [ general ] and set the dns servers also make sure that [ ] Allow DNS server list to be overridden by DHCP/PPP on WAN is unchecked.

Verify

You can verify the configuration by logging on to your firewall over ssh and reviewing the contents of /etc/resolv.conf.

Have fun!

Links

DNS Privacy With Stubby (Part 2 FreeBSD)

FreeBSD

In my previous blog article we install on GNU/Linux which is my main desktop operation system. My NAS and the services that are required to be always running are on FreeBSD.

In this arcticle we will setup Stubby - the DNS Privacy Daemon - on FreeBSD.

Jails

FreeBSD jails are verify nice to keep services separated in a secure way.

ezjail

ezjail is a very nice tool for managing FreeBSD jails.

1
root@rataplan:~ # pkg install ezjail

To loopback or not to loopback….

The loopback ip address is mapped to the jail ip address on FreeBSD by default. There are two options

  • use the jail ip address, make sure that you setup a firewall rule if you want disable traffic from external.
  • use a cloned loopback interface; keep in mind that with a cloned interface this interface is shared between your jails.

We’ll use the jail ip address.

create the jail

We create a new jail and we assign the cloned loop interface with a loopback ip address - this loopback ip address must be unique for each for each jail - and outside interface and ip address.

1
root@rataplan:~ # ezjail-admin create stafdns 're0|192.168.1.53'

start the jail

1
2
3
4
root@rataplan:/usr/local/etc/ezjail # ezjail-admin start stafdns
Starting jails: stafdns.
/etc/rc.d/jail: WARNING: Per-jail configuration via jail_* variables  is obsolete.  Please consider migrating to /etc/jail.conf.
root@rataplan:/usr/local/etc/ezjail #

console login and install pkg

Logon to the jail and install pkg we might need to configure a dns server or use a proxy sever to install pkg.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
root@rataplan:/usr/local/etc/ezjail # ezjail-admin console stafdns
root@stafdns:~ # echo "nameserver 9.9.9.9" > /etc/resolv.conf
root@stafdns:~ # pkg
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:11:amd64/quarterly, please wait...
Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done
[stafdns] Installing pkg-1.10.5_1...
[stafdns] Extracting pkg-1.10.5_1: 100%
pkg: not enough arguments
Usage: pkg [-v] [-d] [-l] [-N] [-j <jail name or id>|-c <chroot path>|-r <rootdir>] [-C <configuration file>] [-R <repo config dir>] [-o var=value] [-4|-6] <command> [<args>]

For more information on available commands and options see 'pkg help'.
root@stafdns:~ #

Install stubby

Stubby available in the FreeBSD Ports in the getdns package, …but it isn’t installed when you install the binary package. To install stubby we need to it from source.

dig

To debug dns issues dig a handy tool to have….

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
root@rataplan:/usr/ports/dns/getdns # pkg install bind-tools
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
Updating database digests format: 100%
The following 4 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        bind-tools: 9.12.2P1
        idnkit: 1.0_7
        py27-ply: 3.11
        json-c: 0.13

Number of packages to be installed: 4

The process will require 42 MiB more space.
4 MiB to be downloaded.

Proceed with this action? [y/N]: y

Update your ports tree

Physical system

On a physical FreeBSD system execute portsnap fetch and portsnap extract

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
root@rataplan:~ # portsnap fetch
Looking up portsnap.FreeBSD.org mirrors... 6 mirrors found.
Fetching snapshot tag from ec2-eu-west-1.portsnap.freebsd.org... done.
Fetching snapshot metadata... done.
Updating from Sat Sep  8 09:31:35 CEST 2018 to Sun Sep  9 09:51:49 CEST 2018.
Fetching 4 metadata patches... done.
Applying metadata patches... done.
Fetching 0 metadata files... done.
Fetching 44 patches. 
(44/44) 100.00%  done.                                
done.
Applying patches... 
done.
Fetching 2 new ports or files... done.
root@rataplan:~ # 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
root@rataplan:~ # portsnap extract
/usr/ports/.arcconfig
/usr/ports/.gitattributes
/usr/ports/.gitauthors
/usr/ports/.gitignore
/usr/ports/.gitmessage
/usr/ports/CHANGES
/usr/ports/CONTRIBUTING.md
/usr/ports/COPYRIGHT
/usr/ports/GIDs
/usr/ports/Keywords/desktop-file-utils.ucl
/usr/ports/Keywords/fc.ucl
/usr/ports/Keywords/fcfontsdir.ucl

<snip>

/usr/ports/x11/xzoom/
/usr/ports/x11/yad/
/usr/ports/x11/yakuake-kde4/
/usr/ports/x11/yakuake/
/usr/ports/x11/yalias/
/usr/ports/x11/yeahconsole/
/usr/ports/x11/yelp/
/usr/ports/x11/zenity/
Building new INDEX files... done.

Jail

I use ezjail to manage my FreeBSD jails. Execute the ezjail-admin update -P to update the ports tree inside your jails.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
root@rataplan:~ # ezjail-admin update -P
Looking up portsnap.FreeBSD.org mirrors... 6 mirrors found.
Fetching snapshot tag from ec2-eu-west-1.portsnap.freebsd.org... done.
Ports tree hasn't changed since last snapshot.
No updates needed.
Removing old files and directories... done.
Extracting new files:
/usr/jails/basejail/usr/ports/archivers/py-lz4/
/usr/jails/basejail/usr/ports/astro/wmsolar/
/usr/jails/basejail/usr/ports/audio/musicpd/
/usr/jails/basejail/usr/ports/biology/seaview/
/usr/jails/basejail/usr/ports/deskutils/gsimplecal/
/usr/jails/basejail/usr/ports/deskutils/xfce4-tumbler/
/usr/jails/basejail/usr/ports/devel/eric6/
/usr/jails/basejail/usr/ports/devel/es-eric6/
/usr/jails/basejail/usr/ports/devel/ioncube/
/usr/jails/basejail/usr/ports/devel/liblouis/
/usr/jails/basejail/usr/ports/devel/monodevelop/
/usr/jails/basejail/usr/ports/devel/rudeconfig/
/usr/jails/basejail/usr/ports/emulators/ppsspp-qt5/
/usr/jails/basejail/usr/ports/emulators/ppsspp/
/usr/jails/basejail/usr/ports/german/eric6/
/usr/jails/basejail/usr/ports/java/linux-oracle-jdk10/
/usr/jails/basejail/usr/ports/java/linux-oracle-jre10/
/usr/jails/basejail/usr/ports/java/openjdk8/
/usr/jails/basejail/usr/ports/lang/gcc6-devel/
/usr/jails/basejail/usr/ports/lang/gcc7-devel/
/usr/jails/basejail/usr/ports/lang/gcc8-devel/
/usr/jails/basejail/usr/ports/lang/gcc9-devel/
/usr/jails/basejail/usr/ports/misc/ree/
/usr/jails/basejail/usr/ports/net-im/psi/
/usr/jails/basejail/usr/ports/net-mgmt/p5-Net-SNMP/
/usr/jails/basejail/usr/ports/net/Makefile
/usr/jails/basejail/usr/ports/net/charm/
/usr/jails/basejail/usr/ports/net/linknx/
/usr/jails/basejail/usr/ports/net/py-maxminddb/
/usr/jails/basejail/usr/ports/net/py-shodan/
/usr/jails/basejail/usr/ports/net/tcpreen/
/usr/jails/basejail/usr/ports/ports-mgmt/pkg-devel/
/usr/jails/basejail/usr/ports/print/ghostscript9-agpl-base/
/usr/jails/basejail/usr/ports/russian/eric6/
/usr/jails/basejail/usr/ports/science/Makefile
/usr/jails/basejail/usr/ports/science/metaphysicl/
/usr/jails/basejail/usr/ports/science/namd/
/usr/jails/basejail/usr/ports/security/sancp/
/usr/jails/basejail/usr/ports/security/testssl.sh/
/usr/jails/basejail/usr/ports/textproc/scim-bridge/
/usr/jails/basejail/usr/ports/www/orangehrm/
/usr/jails/basejail/usr/ports/www/smarty3/
/usr/jails/basejail/usr/ports/www/tinytinyhttpd/
/usr/jails/basejail/usr/ports/x11-wm/spectrwm/
/usr/jails/basejail/usr/ports/x11/plasma5-plasma-workspace/
/usr/jails/basejail/usr/ports/x11/sddm/
Building new INDEX files... done.
root@rataplan:~ # 

Install stubby

Go to the getdns ports directory

1
2
root@stafproxy:/root # cd /usr/ports/dns/getdns/
root@stafproxy:/usr/ports/dns/getdns # make config

and run make config select [ ] STUBBY Build with Stubby DNS/TLS resolver

1
2
3
4
5
6
7
8
9
10
11
┌─────────────────────────────── getdns-1.4.2 ─────────────────────────────────┐
│ ┌──────────────────────────────────────────────────────────────────────────┐ │
│ │+[x] DOCS      Build and/or install documentation                         │ │
│ │+[ ] LIBEV     Build with libev extension                                 │ │
│ │+[ ] LIBEVENT  Build with libevent extension                              │ │
│ │+[ ] LIBUV     Build with libuv extension                                 │ │
│ │+[x] STUBBY    Build with Stubby DNS/TLS resolver                         │ │
│ └──────────────────────────────────────────────────────────────────────────┘ │
├──────────────────────────────────────────────────────────────────────────────┤
│                       <  OK  >            <Cancel>                           │
└──────────────────────────────────────────────────────────────────────────────┘

run make and accept the defaults.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
root@stafproxy:/usr/ports/dns/getdns # make
===>  License BSD3CLAUSE accepted by the user
===>   getdns-1.4.2 depends on file: /usr/local/sbin/pkg - found
=> getdns-1.4.2.tar.gz doesn't seem to exist in /var/ports/distfiles/.
=> Attempting to fetch https://getdnsapi.net/dist/getdns-1.4.2.tar.gz
getdns-1.4.2.tar.gz                           100% of 1034 kB 1092 kBps 00m01s
===> Fetching all distfiles required by getdns-1.4.2 for building
===>  Extracting for getdns-1.4.2
=> SHA256 Checksum OK for getdns-1.4.2.tar.gz.

<snip>

/usr/bin/install -c -m 644 getdns_service_sync.3 /var/ports/basejail/usr/ports/dns/getdns/work/stage/usr/local/man/man3
/usr/bin/install -c -m 644 getdns_validate_dnssec.3 /var/ports/basejail/usr/ports/dns/getdns/work/stage/usr/local/man/man3
/usr/bin/strip /var/ports/basejail/usr/ports/dns/getdns/work/stage/usr/local/lib/libgetdns*.so.*
/usr/bin/strip /var/ports/basejail/usr/ports/dns/getdns/work/stage/usr/local/bin/getdns_*
/usr/bin/strip /var/ports/basejail/usr/ports/dns/getdns/work/stage/usr/local/bin/stubby
/bin/mv /var/ports/basejail/usr/ports/dns/getdns/work/stage/usr/local/etc/stubby/stubby.yml  /var/ports/basejail/usr/ports/dns/getdns/work/stage/usr/local/etc/stubby/stubby.yml.sample
====> Compressing man pages (compress-man)
===> Staging rc.d startup script(s)

make install

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
root@stafproxy:/usr/ports/dns/getdns # make install
===>  Installing for getdns-1.4.2
===>  Checking if getdns already installed
===>   Registering installation for getdns-1.4.2
[stafproxy] Installing getdns-1.4.2...
***
***  !!! IMPORTANT !!!!  libgetdns needs a DNSSEC trust anchor!
***
***  For the library to be able to perform DNSSEC, the root
***  trust anchor needs to be present in presentation format
***  in the file:
***     /usr/local/etc/unbound/root.key
***
***  We recomend using unbound-anchor to retrieve and install
***  the root trust anchor like this:
***     su -m unbound -c /usr/local/sbin/unbound-anchor
***

===> SECURITY REPORT: 
      This port has installed the following files which may act as network
      servers and may therefore pose a remote security risk to the system.
/usr/local/lib/libgetdns.a(stub.o)
/usr/local/lib/libgetdns.so.10.0.2
/usr/local/lib/libgetdns.a(server.o)

      This port has installed the following startup scripts which may cause
      these network services to be started at boot time.
/usr/local/etc/rc.d/stubby

      If there are vulnerabilities in these programs there may be a security
      risk to the system. FreeBSD makes no guarantee about the security of
      ports included in the Ports Collection. Please type 'make deinstall'
      to deinstall the port if this is a concern.

      For more information, and contact details about the security
      status of this software, see the following webpage: 
https://getdnsapi.net/
root@stafproxy:/usr/ports/dns/getdns # 

Lock the package to avoid that the package gets replaced by a getdns package without stubby.

1
2
3
4
root@stafproxy:/usr/ports/dns/getdns # pkg lock getdns
getdns-1.4.2: lock this package? [y/N]: y
Locking getdns-1.4.2
root@stafproxy:/usr/ports/dns/getdns # 

Configure stubby

Enable the stubby service

Use sysrc to enable the stubby service…

1
2
3
4
5
6
7
8
9
10
11
root@stafproxy:/usr/local/etc # service stubby start
Cannot 'start' stubby. Set stubby_enable to YES in /etc/rc.conf or use 'onestart' instead of 'start'.
root@stafproxy:/usr/local/etc # service stubby rcvar
# stubby
#
stubby_enable="NO"
#   (default: "")

root@stafproxy:/usr/local/etc # sysrc stubby_enable="YES"
stubby_enable:  -> YES
root@stafproxy:/usr/local/etc # 

choose your upstream dns provider

Edit the stubby.yml file and uncomment the upstream dns server that you want the use. Stubby will loadbalance the dns traffic to all configured upstream dns servers by default. This is configured with the round_robin_upstreams directive, if set to 1 the traffic is loadbalanced, if set 0 stubby will use the first configured dns server.

1
root@rataplan:/usr/local/etc # vi stubby/stubby.yml

Change the port

We’ll setup dnsmasq to cache our dns requests modify the listen_addresses directive and set the port 53000

1
2
3
listen_addresses:
  - 127.0.0.1@53000
  - 0::1@53000

Start it

1
2
3
4
root@stafproxy:/usr/local/etc # service stubby start
Starting stubby.
[07:51:37.865826] STUBBY: Read config from file /usr/local/etc/stubby/stubby.yml
root@stafproxy:/usr/local/etc # 

test it

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
root@stafproxy:/root # dig @<ip_of_the_jail> -p 53000 www.wagemakers.be

; <<>> DiG 9.8.3-P4 <<>> @127.0.0.53 -p 53000 www.wagemakers.be
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56970
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.wagemakers.be.             IN      A

;; ANSWER SECTION:
www.wagemakers.be.      85181   IN      CNAME   wagemakers.be.
wagemakers.be.          85181   IN      A       95.215.185.144

;; Query time: 110 msec
;; SERVER: 127.0.0.53#53000(127.0.0.53)
;; WHEN: Sat Sep 22 13:16:11 2018
;; MSG SIZE  rcvd: 119

dnsmasq

Install dnsmasq.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
root@stafproxy:/root # pkg install dnsmasq
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        dnsmasq: 2.79,1

Number of packages to be installed: 1

329 KiB to be downloaded.

Proceed with this action? [y/N]: y
[stafproxy] [1/1] Fetching dnsmasq-2.79,1.txz: 100%  329 KiB 336.4kB/s    00:01
Checking integrity... done (0 conflicting)
[stafproxy] [1/1] Installing dnsmasq-2.79,1...
[stafproxy] [1/1] Extracting dnsmasq-2.79,1: 100%
Message from dnsmasq-2.79,1:

*** To enable dnsmasq, edit /usr/local/etc/dnsmasq.conf and
*** set dnsmasq_enable="YES" in /etc/rc.conf[.local]
***
*** Further options and actions are documented inside
*** /usr/local/etc/rc.d/dnsmasq
root@stafproxy:/root #

Enable dnsmasq.

Usae sysrc to enable the dnsmasq service.

1
2
3
root@stafproxy:/root # sysrc dnsmasq_enable="YES"
dnsmasq_enable:  -> YES
root@stafproxy:/root #

Configure dnsmasq

1
2
root@stafproxy:/usr/local/etc # mv dnsmasq.conf dnsmasq.conf_org
root@stafproxy:/usr/local/etc # vi dnsmasq.conf
1
2
3
4
server=<ip_address_of_the_jail>#53000
listen-address=<ip_address_of_the_jail>
interface=<netork_interface_of_the_jail>
bind-interfaces

start dnsmasq

1
2
3
root@stafproxy:/usr/local/etc # service dnsmasq start
Starting dnsmasq.
root@stafproxy:/usr/local/etc #

test it

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
root@stafproxy:/usr/local/etc # dig @192.168.1.45 www.wagemakers.be

; <<>> DiG 9.8.3-P4 <<>> @192.168.1.45 www.wagemakers.be
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 32987
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.wagemakers.be.             IN      A

;; ANSWER SECTION:
www.wagemakers.be.      86000   IN      CNAME   wagemakers.be.
wagemakers.be.          86000   IN      A       95.215.185.144

;; Query time: 308 msec
;; SERVER: 192.168.1.45#53(192.168.1.45)
;; WHEN: Sun Oct  7 09:16:51 2018
;; MSG SIZE  rcvd: 119

root@stafproxy:/usr/local/etc #

Update /etc/resolv.conf

Update your /etc/resolv.conf

1
root@stafproxy:/usr/local/etc # vi /etc/resolv.conf
1
nameserver <ip_address_of_the_jail>

and test it;

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
root@stafproxy:/usr/local/etc # dig www.wagemakers.be

; <<>> DiG 9.8.3-P4 <<>> www.wagemakers.be
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27629
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;www.wagemakers.be.             IN      A

;; ANSWER SECTION:
www.wagemakers.be.      85702   IN      CNAME   wagemakers.be.
wagemakers.be.          85702   IN      A       95.215.185.144

;; Query time: 1 msec
;; SERVER: 192.168.1.45#53(192.168.1.45)
;; WHEN: Sun Oct  7 09:21:49 2018
;; MSG SIZE  rcvd: 78

root@stafproxy:/usr/local/etc #

Have fun!

Links

DNS Privacy With Stubby (Part 1 GNU/Linux)

Installing and configuring an encrypted dns server is straightforward, there is no reason to use an unencrypted dns service.

DNS is not secure or private

DNS traffic is insecure and runs over UDP port 53 (TCP for zone transfers ) unecrypted by default.

This make your unencrypted DNS traffic a privacy risk and a security risk:

  • anyone that is able to sniff your network traffic can collect a lot information from your leaking DNS traffic.
  • with a DNS spoofing attack an attacker can trick you let go to malicious website or try to intercept your email traffic.

Encrypt your dns traffic

Encrypting your network traffic is always a good idea for privacy and security reasons - we encrypt, because we can! - . More information about dns privacy can be found at https://dnsprivacy.org/

On this site you’ll find also the DNS Privacy Daemon - Stubby that let’s you send your DNS request over TLS to an alternative DNS provider. You should use a DNS provider that you trust and has a no logging policy. quad9, cloudflare and google dns are well-known alternative dns providers. At https://dnsprivacy.org/wiki/display/DP/DNS+Privacy+Test+Servers you can find a few other options.

You’ll find my journey to setup Stubby on a few operation systems I use (or I’m force to use) below …

GNU/Linux

Arch Linux

I use Arch Linux on my main workstation. Stubby is already in the Arch repositories this make installation straightforward.

Install stubby

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
[root@vicky ~]# pacman -S stubby
resolving dependencies...
looking for conflicting packages...

Packages (5) fstrm-0.4.0-1  getdns-1.4.2-1  protobuf-c-1.3.0-3  unbound-1.7.3-4
             stubby-0.2.3-1

Total Download Size:   1.09 MiB
Total Installed Size:  5.68 MiB

:: Proceed with installation? [Y/n] 
:: Retrieving packages...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 88476  100 88476    0     0   403k      0 --:--:-- --:--:-- --:--:--  403k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 62480  100 62480    0     0  1271k      0 --:--:-- --:--:-- --:--:-- 1271k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  632k  100  632k    0     0   750k      0 --:--:-- --:--:-- --:--:--  749k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  302k  100  302k    0     0  1615k      0 --:--:-- --:--:-- --:--:-- 1606k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 34052  100 34052    0     0   831k      0 --:--:-- --:--:-- --:--:--  831k
(5/5) checking keys in keyring                       [###########################] 100%
(5/5) checking package integrity                     [###########################] 100%
(5/5) loading package files                          [###########################] 100%
(5/5) checking for file conflicts                    [###########################] 100%
(5/5) checking available disk space                  [###########################] 100%
:: Processing package changes...
(1/5) installing fstrm                               [###########################] 100%
(2/5) installing protobuf-c                          [###########################] 100%
(3/5) installing unbound                             [###########################] 100%
Optional dependencies for unbound
    expat: unbound-anchor [installed]
(4/5) installing getdns                              [###########################] 100%
(5/5) installing stubby                              [###########################] 100%
:: Running post-transaction hooks...
(1/4) Reloading system manager configuration...
(2/4) Creating system user accounts...
(3/4) Creating temporary files...
(4/4) Arming ConditionNeedsUpdate...
[root@vicky ~]# 

choose your upstream dns provider

Edit the stubby.yml file and uncomment the upstream dns server that you want the use. Stubby will loadbalance the dns traffic to all configured upstream dns servers by default. This is configured with the round_robin_upstreams directive, if set to 1 the traffic is loadbalanced, if set 0 stubby will use the first configured dns server.

1
[staf@vicky ~]$ sudo vi /etc/stubby/stubby.yml

enable and start stubby

1
2
3
4
[root@vicky ~]# systemctl enable stubby
Created symlink /etc/systemd/system/multi-user.target.wants/stubby.service -> /usr/lib/systemd/system/stubby.service.
[root@vicky ~]# systemctl start stubby
[root@vicky ~]# 

test

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
[root@vicky ~]# dig @127.0.0.1 www.wagemakers.be

; <<>> DiG 9.13.2 <<>> @127.0.0.1 www.wagemakers.be
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18226
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: fe9d3618b821614f174436385b7acb64a4f4cc6657e14626 (good)
;; QUESTION SECTION:
;www.wagemakers.be.             IN      A

;; ANSWER SECTION:
www.wagemakers.be.      86000   IN      CNAME   wagemakers.be.
wagemakers.be.          86000   IN      A       95.215.185.144

;; Query time: 128 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Mon Aug 20 16:08:36 CEST 2018
;; MSG SIZE  rcvd: 147

[root@vicky ~]# 

Local dns cache with dnsmasq

Change the stubby port.

Edit /etc/stubby/stubby.yml

1
[root@vicky ~]# vi /etc/stubby/stubby.yml

And change the port by modifing the listen_addresses directive

1
2
3
listen_addresses:
  - 127.0.0.1@53000
  - 0::1@53000

restart stubby

1
[root@vicky ~]# systemctl restart stubby.service

and verify that the dns on 127.0.0.1:53 doesn’t work anymore.

1
2
3
4
5
6
7
[root@vicky ~]# dig @127.0.0.1 www.wagemakers.be

; <<>> DiG 9.13.2 <<>> @127.0.0.1 www.wagemakers.be
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached
[root@vicky ~]# 

ensure that stubby does work on port 53000

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
[root@frija etc]# dig @127.0.0.1 -p 53000 www.wagemakers.be

; <<>> DiG 9.13.2 <<>> @127.0.0.1 -p 53000 www.wagemakers.be
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27173
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65535
;; QUESTION SECTION:
;www.wagemakers.be.             IN      A

;; ANSWER SECTION:
www.wagemakers.be.      43200   IN      CNAME   wagemakers.be.
wagemakers.be.          43200   IN      A       95.215.185.144

;; Query time: 250 msec
;; SERVER: 127.0.0.1#53000(127.0.0.1)
;; WHEN: Tue Aug 21 13:26:37 CEST 2018
;; MSG SIZE  rcvd: 119

[root@frija etc]# 

Install dnsmasq

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
[root@vicky ~]# pacman -S dnsmasq
warning: dnsmasq-2.79-1 is up to date -- reinstalling
resolving dependencies...
looking for conflicting packages...

Packages (1) dnsmasq-2.79-1

Total Installed Size:  0.70 MiB
Net Upgrade Size:      0.00 MiB

:: Proceed with installation? [Y/n] y
(1/1) checking keys in keyring                       [###########################] 100%
(1/1) checking package integrity                     [###########################] 100%
(1/1) loading package files                          [###########################] 100%
(1/1) checking for file conflicts                    [###########################] 100%
(1/1) checking available disk space                  [###########################] 100%
:: Processing package changes...
(1/1) reinstalling dnsmasq                           [###########################] 100%
:: Running post-transaction hooks...
(1/3) Reloading system manager configuration...
(2/3) Creating system user accounts...
(3/3) Arming ConditionNeedsUpdate...
[root@vicky ~]# 

Configure dnsmasq

1
2
3
[root@vicky etc]# cd /etc
[root@vicky etc]# mv /etc/dnsmasq.conf /etc/dnsmasq.conf_org
[root@vicky etc]# vi dnsmasq.conf

It is import to configure stubby to listen the localhost interface only. If you use Linux KVM you probably have a dns serivce running on your bridge interfaces for your virtual machines.

1
2
3
4
server=127.0.0.1#53000
listen-address=127.0.0.1
interface=lo
bind-interfaces

Start and enable dnsmasq

1
2
3
4
[root@vicky ~]# systemctl start dnsmasq
[root@vicky ~]# systemctl enable dnsmasq
Created symlink /etc/systemd/system/multi-user.target.wants/dnsmasq.service -> /usr/lib/systemd/system/dnsmasq.service.
[root@vicky ~]# 

Reconfigure your system

reconfigure your system to use dnsmasq as the dns service.

I use netctl on my system. You can update the network configuration with netctl

1
2
[root@vicky netctl]# netctl edit <network_name>
[root@vicky netctl]# netctl restart  <network_name>

If you networkmanager you can use nmcli, nmtui or the GUI network configuration in your desktop environment.

GNU/Linux is GNU/Linux

The configuration on other GNU/Linux distributions is the same as on Arch apart from the installation process. The same method can be use if your (favorite) Linux distribution doesn’t have a stubby package, the installation method of the required package will be different of course.

Debian

Current testing release Debian “buster”

1
$ sudo apt install stubby dnsmasq

Current stable Debian 9 “strech”

Stubby in the getdns-utils in Debian stretch, it’s an older version. Therefor I ended up with building stubby from the source code.

Install the required packages

Install the required packages to build stubby.

1
staf@stretch:~/github$ sudo apt install build-essential git libtool autoconf libssl-dev libyaml-dev
git clone

The getdns git repo;

1
2
3
4
5
6
7
staf@stretch:~/github$ git clone https://github.com/getdnsapi/getdns.git
Cloning into 'getdns'...
remote: Counting objects: 16154, done.
remote: Total 16154 (delta 0), reused 0 (delta 0), pack-reused 16154
Receiving objects: 100% (16154/16154), 9.72 MiB | 1.13 MiB/s, done.
Resolving deltas: 100% (12413/12413), done.
staf@stretch:~/github$ 
checkout the latest stable release

Verify the lastest release tag. The current stable release 1.4.2

1
2
3
4
5
6
7
8
9
10
11
12
13
14
staf@stretch:~/github/getdns$ git tag
TNW2015
list
v0.1.0
v0.1.1
v0.1.2
<snip>
v1.4.0
v1.4.0-rc1
v1.4.1
v1.4.1-rc1
v1.4.2
v1.4.2-rc1
staf@stretch:~/github/getdns$ 

checkout the latest stable release.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
staf@stretch:~/github/getdns$ git checkout v1.4.2
Note: checking out 'v1.4.2'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at e481273... Last minute update
staf@stretch:~/github/getdns$ 
build it…
1
2
3
4
5
6
7
staf@stretch:~/github/getdns$ git submodule update --init
staf@stretch:~/github/getdns$ libtoolize -ci
staf@stretch:~/github/getdns$ autoreconf -fi
staf@stretch:~/github/getdns$ mkdir build
staf@stretch:~/github/getdns$ cd build/
staf@stretch:~/github/getdns/build$ ../configure --prefix=/usr/local --without-libidn --without-libidn2 --enable-stub-only --with-stubby
staf@stretch:~/github/getdns/build$ make
make install
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
staf@stretch:~/github/getdns/build$ sudo make install
[sudo] password for staf: 
cd src && make install
make[1]: Entering directory '/home/staf/github/getdns/build/src'
<snip>
make[1]: Leaving directory '/home/staf/github/getdns/build/doc'
***
***  !!! IMPORTANT !!!!
***
***  From release 1.2.0, getdns comes with built-in DNSSEC
***  trust anchor management.  External trust anchor management,
***  for example with unbound-anchor, is no longer necessary
***  and no longer recommended.
***
***  Previously installed trust anchors, in the default location -
***
***        /usr/local/etc/unbound/getdns-root.key
***
***  - will be preferred and used for DNSSEC validation, however
***  getdns will fallback to trust-anchors obtained via built-in
***  trust anchor management when the anchors from the default
***  location fail to validate the root DNSKEY rrset.
***
***  To prevent expired DNSSEC trust anchors to be used for
***  validation, we strongly recommend removing the trust anchors
***  on the default location when there is no active external
***  trust anchor management keeping it up-to-date.
***
staf@stretch:~/github/getdns/build$ sudo make install
systemd service

Stubby comes with a systemd service definition. Copy it to the correct location.

1
2
3
staf@stretch:~/github/getdns/build$ cd ..
staf@stretch:~/github/getdns$ cd stubby/systemd/
staf@stretch:~/github/getdns/stubby/systemd$ sudo cp stubby.service /lib/systemd/system/

Update the path to /usr/local

1
staf@stretch:~/github/getdns/stubby/systemd$ sudo vi /lib/systemd/system/stubby.service
1
2
3
4
5
6
7
8
9
10
11
12
13
14
[Unit]
Description=stubby DNS resolver

[Service]
User=stubby
DynamicUser=yes
CacheDirectory=stubby
WorkingDirectory=/var/cache/stubby
ExecStart=/usr/local/bin/stubby
AmbientCapabilities=CAP_NET_BIND_SERVICE
CapabilityBoundingSet=CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

And create the stubby working directory

1
root@stretch:~# mkdir /var/cache/stubby

ldconfig

update your library cache

1
staf@stretch:~/github/getdns/stubby/systemd$ sudo ldconfig -v

Update the configuration

Edit the stubby.yml configuration file.

1
staf@stretch:~/github/getdns/stubby/systemd$ sudo nvi /usr/local/etc/stubby/stubby.yml

Update the port where stubby will listen to and select the upstream dns service you want to use.

1
2
3
listen_addresses:
  - 127.0.0.1@53000
  - 0::1@53000

start and test

Start stubby….

1
2
3
4
5
6
staf@stretch:~/github/getdns/stubby/systemd$ sudo systemctl list-unit-files | grep -i stubby
stubby.service                              disabled
staf@stretch:~/github/getdns/stubby/systemd$ sudo systemctl enable stubby
Created symlink /etc/systemd/system/multi-user.target.wants/stubby.service /lib/systemd/system/stubby.service.
staf@stretch:~/github/getdns/stubby/systemd$ sudo systemctl start stubby
staf@stretch:~/github/getdns/stubby/systemd$ 

and test it

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
root@stretch:~# dig @127.0.0.1 -p 53000 www.wagemakers.be

; <<>> DiG 9.10.3-P4-Debian <<>> @127.0.0.1 -p 53000 www.wagemakers.be
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17510
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.wagemakers.be.             IN      A

;; ANSWER SECTION:
www.wagemakers.be.      49704   IN      CNAME   wagemakers.be.
wagemakers.be.          81815   IN      A       95.215.185.144

;; Query time: 72 msec
;; SERVER: 127.0.0.1#53000(127.0.0.1)
;; WHEN: Sun Sep 02 10:33:53 CEST 2018
;; MSG SIZE  rcvd: 119

root@stretch:~# 

dnsmasq

Install dnsmasq

1
root@stretch:/etc# apt-get install dnsmasq

Configure dnsmasq

1
2
root@stretch:/etc# mv dnsmasq.conf dnsmasq.conf_org
root@stretch:/etc# vi dnsmasq.conf
1
2
3
4
server=127.0.0.1#53000
listen-address=127.0.0.1
interface=lo
bind-interfaces

Enable and start it…

1
2
3
4
root@stretch:/etc# systemctl enable dnsmasq
Synchronizing state of dnsmasq.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable dnsmasq
root@stretch:/etc# systemctl restart dnsmasq

Verify

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
root@stretch:/etc# dig @127.0.0.1 www.wagemakers.be

; <<>> DiG 9.10.3-P4-Debian <<>> @127.0.0.1 www.wagemakers.be
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57295
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.wagemakers.be.             IN      A

;; ANSWER SECTION:
www.wagemakers.be.      48645   IN      CNAME   wagemakers.be.
wagemakers.be.          80756   IN      A       95.215.185.144

;; Query time: 72 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sun Sep 02 10:51:32 CEST 2018
;; MSG SIZE  rcvd: 119

root@stretch:/etc# 

reconfigure you system to use dnsmasq….

1
root@stretch:/etc# nvi resolv.conf
1
nameserver 127.0.0.1

Have fun!

Links

Migrate a Windows Vmware Virtual Machine to Linux KVM

Linux KVM is getting more and more useable for desktop virtualization thanks to the the virtio and QXL/SPICE drivers.

Most Linux distributes have the virtio & QXL drivers you might need to install the spice-vdagent.

On Windows you can download and install the virtio and QXL drivers.

Using the virtio drivers will improve your guest system performance and your virtualization experience.

Convert the disk image

merge the vmware disk images…

If you use split disk images on vmware ( or vmware player ) migrate them to a single disk images with the vmware-vdiskmanager command.

1
2
3
4
5
$ vmware-vdiskmanager -r mywin.vmdk -t 0 /tmp/mywin._combined.vmdk
Creating disk '/var/lib/libvirt/images/tmp/mywin._combined.vmdk'
  Convert: 100% done.
Virtual disk conversion successful.
$

convert the vmdk image to qcow2

Convert the VMDK disk image to qcow2

1
[staf@vicky vboxes]$ qemu-img convert -f vmdk -O qcow2 mywin._combined.vmdk mywin.qcow2

mv

1
2
[staf@vicky vboxes]$ sudo mv mywin_combined.qcow2 /var/lib/libvirt/images/
[sudo] password for staf: 

Import the disk image to KVM

We’ll inport the disk image with virt-install it’s also posible to import the images with virt-manager if you prefer a graphical interface or or just being lazy :-)

Available os options

To list the supported operation system you can use the osinfo-query os command

1
2
3
4
5
6
7
8
9
10
11
[staf@vicky ~]$ osinfo-query os | head
 Short ID             | Name                                               | Version  | ID                                      
----------------------+----------------------------------------------------+----------+-----------------------------------------
 alpinelinux3.5       | Alpine Linux 3.5                                   | 3.5      | http://alpinelinux.org/alpinelinux/3.5  
 alpinelinux3.6       | Alpine Linux 3.6                                   | 3.6      | http://alpinelinux.org/alpinelinux/3.6  
 alpinelinux3.7       | Alpine Linux 3.7                                   | 3.7      | http://alpinelinux.org/alpinelinux/3.7  
 altlinux1.0          | Mandrake RE Spring 2001                            | 1.0      | http://altlinux.org/altlinux/1.0        
 altlinux2.0          | ALT Linux 2.0                                      | 2.0      | http://altlinux.org/altlinux/2.0        
 altlinux2.2          | ALT Linux 2.2                                      | 2.2      | http://altlinux.org/altlinux/2.2        
 altlinux2.4          | ALT Linux 2.4                                      | 2.4      | http://altlinux.org/altlinux/2.4        
 altlinux3.0          | ALT Linux 3.0                                      | 3.0      | http://altlinux.org/altlinux/3.0        
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
[staf@vicky ~]$ osinfo-query os |  grep -i windows
 win1.0               | Microsoft Windows 1.0                              | 1.0      | http://microsoft.com/win/1.0            
 win10                | Microsoft Windows 10                               | 10.0     | http://microsoft.com/win/10             
 win2.0               | Microsoft Windows 2.0                              | 2.0      | http://microsoft.com/win/2.0            
 win2.1               | Microsoft Windows 2.1                              | 2.1      | http://microsoft.com/win/2.1            
 win2k                | Microsoft Windows 2000                             | 5.0      | http://microsoft.com/win/2k             
 win2k12              | Microsoft Windows Server 2012                      | 6.3      | http://microsoft.com/win/2k12           
 win2k12r2            | Microsoft Windows Server 2012 R2                   | 6.3      | http://microsoft.com/win/2k12r2         
 win2k16              | Microsoft Windows Server 2016                      | 10.0     | http://microsoft.com/win/2k16           
 win2k3               | Microsoft Windows Server 2003                      | 5.2      | http://microsoft.com/win/2k3            
 win2k3r2             | Microsoft Windows Server 2003 R2                   | 5.2      | http://microsoft.com/win/2k3r2          
 win2k8               | Microsoft Windows Server 2008                      | 6.0      | http://microsoft.com/win/2k8            
 win2k8r2             | Microsoft Windows Server 2008 R2                   | 6.1      | http://microsoft.com/win/2k8r2          
 win3.1               | Microsoft Windows 3.1                              | 3.1      | http://microsoft.com/win/3.1            
 win7                 | Microsoft Windows 7                                | 6.1      | http://microsoft.com/win/7              
 win8                 | Microsoft Windows 8                                | 6.2      | http://microsoft.com/win/8              
 win8.1               | Microsoft Windows 8.1                              | 6.3      | http://microsoft.com/win/8.1            
 win95                | Microsoft Windows 95                               | 4.0      | http://microsoft.com/win/95             
 win98                | Microsoft Windows 98                               | 4.1      | http://microsoft.com/win/98             
 winme                | Microsoft Windows Millennium Edition               | 4.9      | http://microsoft.com/win/me             
 winnt3.1             | Microsoft Windows NT Server 3.1                    | 3.1      | http://microsoft.com/winnt/3.1          
 winnt3.5             | Microsoft Windows NT Server 3.5                    | 3.5      | http://microsoft.com/winnt/3.5          
 winnt3.51            | Microsoft Windows NT Server 3.51                   | 3.51     | http://microsoft.com/winnt/3.51         
 winnt4.0             | Microsoft Windows NT Server 4.0                    | 4.0      | http://microsoft.com/winnt/4.0          
 winvista             | Microsoft Windows Vista                            | 6.0      | http://microsoft.com/win/vista          
 winxp                | Microsoft Windows XP                               | 5.1      | http://microsoft.com/win/xp             
[staf@vicky ~]$ 

import

We need to import the disk image as IDE device since we don’t have the virtio driver in our windows disk image (yet).

1
2
3
4
5
[root@vicky ~]# virt-install --name "mywin" --ram 8192 --cpu host --os-variant win10 --vcpu 8 --disk /var/lib/libvirt/images/mywin_combined.qcow2,bus=ide --network bridge=virbr0 --import

Starting install...

(virt-viewer:3361): GSpice-WARNING **: 16:49:26.546: Warning no automount-inhibiting implementation available

Install the virtio drivers and QXL graphics drivers

Get them…

Type of virtio drivers

The following virtio windows drivers are available.

  • block (disk drivers)
  • network
  • baloon ((dynamic memory management)

The fedoraproject provides pre compiled iso images containig all the virtio drivers and installation images for windows XP.

ISO contents

  • NetKVM/ - Virtio network driver
  • viostor/ - Virtio block driver
  • vioscsi/ - Virtio Small Computer System Interface (SCSI) driver
  • viorng/ - Virtio RNG driver
  • vioser/ - Virtio serial driver
  • Balloon/ - Virtio memory balloon driver
  • qxl/ - QXL graphics driver for Windows 7 and earlier. (build virtio-win-0.1.103-1 and later)
  • qxldod/ - QXL graphics driver for Windows 8 and later. (build virtio-win-0.1.103-2 and later)
  • pvpanic/ - QEMU pvpanic device driver (build virtio-win-0.1.103-2 and later)
  • guest-agent/ - QEMU Guest Agent 32bit and 64bit MSI installers
  • qemupciserial/ - QEMU PCI serial device driver
  • *.vfd VFD floppy images for using during install of Windows XP

Download

The virtio windows driver images are available from https://docs.fedoraproject.org/quick-docs/en-US/creating-windows-virtual-machines-using-virtio-drivers.html

I use arch linux and download virtio-win AUR package with pacaur. You can download the images directly or use the installation packages for your Linux distribution.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
[staf@vicky ~]$ pacaur -S virtio-win
:: Package virtio-win not found in repositories, trying AUR...
:: resolving dependencies...
:: looking for inter-conflicts...

AUR Packages  (1) virtio-win-0.1.149.2-1  

:: Proceed with installation? [Y/n] 
<snip>
  -> Compressing package...
==> Leaving fakeroot environment.
==> Finished making: virtio-win 0.1.149.2-1 (Sat Jun 16 20:00:22 2018)
==> Cleaning up...
:: Installing virtio-win package(s)...
loading packages...
resolving dependencies...
looking for conflicting packages...

Packages (1) virtio-win-0.1.149.2-1

Total Installed Size:  314.84 MiB

:: Proceed with installation? [Y/n] 
(1/1) checking keys in keyring                                         [#######################################] 100%
(1/1) checking package integrity                                       [#######################################] 100%
(1/1) loading package files                                            [#######################################] 100%
(1/1) checking for file conflicts                                      [#######################################] 100%
(1/1) checking available disk space                                    [#######################################] 100%
:: Processing package changes...
(1/1) installing virtio-win                                            [#######################################] 100%
Optional dependencies for virtio-win
    qemu [installed]
:: Running post-transaction hooks...
(1/1) Arming ConditionNeedsUpdate...
[staf@vicky ~]$ ls -l /var/li

This install virtio images to /usr/share/virtio/

1
2
3
4
5
6
[staf@vicky ~]$  ls -l /usr/share/virtio/
total 321308
-rw-r--r-- 1 root root 324233216 Jun 16 19:58 virtio-win.iso
-rw-r--r-- 1 root root   2949120 Jun 16 19:58 virtio-win_x86_32.vfd
-rw-r--r-- 1 root root   2949120 Jun 16 19:58 virtio-win_x86_64.vfd
[staf@vicky ~]$ 

virtio-win.iso is the ISO cdrom image containing all the drivers.

Installation

mount the iso image

"mount_cdrom_000.png"

Make sure that the cdrom is mounted in windows.

"mount_cdrom_000.png"

Install

Open Device Manager

Open device Manager in the control panel or type devmgmt.msc on the command prompt.

"mount_cdrom_000.png"

Update the drivers

  • balloon, the balloon driver affects the PCI device
  • vioserial, affects the PCI simple communication controler
  • NetKVM, the network driver affects the Network adapters.
  • viostor, the block driver affects the Disk drives.
Update the PCI drivers

In windows 10 the PCI device and the PCI Simple Communications Controller have the missing driver icon. Right click on the PCI device and select update driver -> click on Browse my computer for driver software Specify the cdrom as the search location and click Next, this will install the Balloon driver.

Do the same for the PCI Simple Communications Controller this will install the “VirtIO Serial Driver”

"update_pci_000.png" "update_pci_001.png" "update_pci_002.png" "update_pci_003.png"

install the VioStor driver

Add a temporary disk to the virtual machine and use VirtIO as the Bus Type In the Device Manager you’ll get a new device SCSI Controller right click it and update the driver. This will install the Red Hat VirtIO SCSI controller

"install_viostor_000.png" "install_viostor_001.png" "install_viostor_002.png"

Go to the device settings of your virtual machine and change the Disk bus to VirtIO and shutdown you virtual machine.

"install_viostor_003.png"

You can remove the temporary disk now or leave it if you can find some use for it…

Make sure that you disk is selected as the bootable device.

"install_viostor_004.png"

Start the virtual machine and make sure that the system is bootable.

install the netKVM driver

Update the Device model to virtio.

"use_virtio_net_000.png"

Start devmgmt.msc and update the driver as we did before….

"install_netkvm_000.png" "install_netkvm_001.png"

And verify that you network card works correctly.

"install_netkvm_002.png"

install the QXL graphical driver

Update the Microsoft Basic Display Adapter

"install_qxl_000.png" "install_qxl_001.png" "install_qxl_002.png"

After the installation you can change the the display resolution.

"install_qxl_003.png"

If you want to use higher screen resolutions you need to increase the video ram

Have fun!

Links

Nested Virtualization in KVM

KVM

Kernel-based Virtual Machine (KVM) has become the defacto hypervisor on GNU/Linux systems it works with great performance as it utilizes the CPU virtualization extensions Inetl VT-x or AMD-V). KVM doesn’t emulate hardware but uses QEMU for this.

Nested Virtual guest

It’s possible to use nested virtualization this make it possible to run a hypervisor inside a KVM virtual machine.

Enabling nested virtualization in KVM

Verify

To verify if nested virtualization is enabled on your system can check /sys/module/kvm_intel/parameters/nested on Intal systems or /sys/module/kvm_amd/parameters/nested

1
2
3
[staf@frija ~]$ cat /sys/module/kvm_intel/parameters/nested
N
[staf@frija ~]$ 

Enable

Shutdown all virtual machines

Make sure that there no virtual machines running.

1
2
3
4
5
6
7
8
9
10
11
[root@frija ~]# virsh 
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # list
 Id    Name                           State
----------------------------------------------------

virsh # 

Unload KVM

Unload the KVM kernel module.

1
2
[root@frija ~]# modprobe -r kvm_intel
[root@frija ~]# 

Load KVM and activate nested

Reload the KVM with the nested feature enabled.

1
2
[root@frija ~]# modprobe kvm_intel nested=1
[root@frija ~]# 

Verify

1
2
3
[root@frija ~]# cat /sys/module/kvm_intel/parameters/nested
Y
[root@frija ~]# 

To enable the nested feature permanently create /etc/modprobe.d/kvm_intel.conf

1
[root@frija ~]# vi /etc/modprobe.d/kvm_intel.conf

and enable the nested option.

1
options kvm_intel nested=1

Enabling nested virtialization in the virtual machine

When you logon to a virtual machine and verify the virtualization extensions on the cpu the flags aren’t available.

1
2
[staf@centos7 ~]$ cat /proc/cpuinfo | grep  -i -E "vmx|svm"
[staf@centos7 ~]$ 

To enable nested virtualization in a vritual machine you can

  • start virsh and and edit the the virtual machine and change the CPU line to <cpu mode='host-model' check='partial'/>
  • Open virt-manager and select Copy host CPU configuration on the CPU configuration
1
2
3
4
5
6
7
8
9
10
11
12
root@frija ~]# virsh 
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # list
 Id    Name                           State
----------------------------------------------------
 1     centos7.0                      running

virsh # edit centos7.0 

Change the cpu settings

1
2
3
4
5
6
7
8
  <features>
    <acpi/>
    <apic/>
    <vmport state='off'/>
  </features>
  <cpu mode='host-model' check='partial'>
    <model fallback='allow'/>
  </cpu>

Shutdown the virtual machine

1
2
3
4
virsh # reboot centos7.0 
Domain centos7.0 is being rebooted

virsh # 

Start the virtual machine

1
2
virsh # start centos7.0  
Domain centos7.0 started

Verify that the feature policies on the cpu are updated.

1
virsh # dumpxml centos7.0 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
 <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>Haswell-noTSX-IBRS</model>
    <vendor>Intel</vendor>
    <feature policy='require' name='vme'/>
    <feature policy='require' name='ss'/>
    <feature policy='require' name='f16c'/>
    <feature policy='require' name='rdrand'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='require' name='arat'/>
    <feature policy='require' name='tsc_adjust'/>
    <feature policy='require' name='xsaveopt'/>
    <feature policy='require' name='pdpe1gb'/>
    <feature policy='require' name='abm'/>
    <feature policy='require' name='ibpb'/>
 </cpu>

Logon to the virtual machine and verify the cpu flags;

1
2
3
4
5
6
7
[staf@centos7 ~]$ cat /proc/cpuinfo | grep -i vmx
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt ibpb ibrs arat spec_ctrl
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt ibpb ibrs arat spec_ctrl
[staf@centos7 ~]$ cat /proc/cpuinfo | grep  -i "vmx|svm"
[staf@centos7 ~]$ cat /proc/cpuinfo | grep  -i -E "vmx|svm"
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt ibpb ibrs arat spec_ctrl
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt ibpb ibrs arat spec_ctrl

Execute the virt-host-validate

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
[staf@centos7 ~]$ virt-host-validate
  QEMU: Checking for hardware virtualization                                 : PASS
  QEMU: Checking if device /dev/kvm exists                                   : PASS
  QEMU: Checking if device /dev/kvm is accessible                            : PASS
  QEMU: Checking if device /dev/vhost-net exists                             : PASS
  QEMU: Checking if device /dev/net/tun exists                               : PASS
  QEMU: Checking for cgroup 'memory' controller support                      : PASS
  QEMU: Checking for cgroup 'memory' controller mount-point                  : PASS
  QEMU: Checking for cgroup 'cpu' controller support                         : PASS
  QEMU: Checking for cgroup 'cpu' controller mount-point                     : PASS
  QEMU: Checking for cgroup 'cpuacct' controller support                     : PASS
  QEMU: Checking for cgroup 'cpuacct' controller mount-point                 : PASS
  QEMU: Checking for cgroup 'cpuset' controller support                      : PASS
  QEMU: Checking for cgroup 'cpuset' controller mount-point                  : PASS
  QEMU: Checking for cgroup 'devices' controller support                     : PASS
  QEMU: Checking for cgroup 'devices' controller mount-point                 : PASS
  QEMU: Checking for cgroup 'blkio' controller support                       : PASS
  QEMU: Checking for cgroup 'blkio' controller mount-point                   : PASS
  QEMU: Checking for device assignment IOMMU support                         : WARN (No ACPI DMAR table found, IOMMU either disabled in BIOS or not supported by this hardware platform)
   LXC: Checking for Linux >= 2.6.26                                         : PASS
   LXC: Checking for namespace ipc                                           : PASS
   LXC: Checking for namespace mnt                                           : PASS
   LXC: Checking for namespace pid                                           : PASS
   LXC: Checking for namespace uts                                           : PASS
   LXC: Checking for namespace net                                           : PASS
   LXC: Checking for namespace user                                          : PASS
   LXC: Checking for cgroup 'memory' controller support                      : PASS
   LXC: Checking for cgroup 'memory' controller mount-point                  : PASS
   LXC: Checking for cgroup 'cpu' controller support                         : PASS
   LXC: Checking for cgroup 'cpu' controller mount-point                     : PASS
   LXC: Checking for cgroup 'cpuacct' controller support                     : PASS
   LXC: Checking for cgroup 'cpuacct' controller mount-point                 : PASS
   LXC: Checking for cgroup 'cpuset' controller support                      : PASS
   LXC: Checking for cgroup 'cpuset' controller mount-point                  : PASS
   LXC: Checking for cgroup 'devices' controller support                     : PASS
   LXC: Checking for cgroup 'devices' controller mount-point                 : PASS
   LXC: Checking for cgroup 'blkio' controller support                       : PASS
   LXC: Checking for cgroup 'blkio' controller mount-point                   : PASS
   LXC: Checking if device /sys/fs/fuse/connections exists                   : FAIL (Load the 'fuse' module to enable /proc/ overrides)
[staf@centos7 ~]$ 

Have fun

Links

32 Bits Matters!

"32bits_opnsense.jpg"

pfsense 2.3

My firewall is a pcengines alix.

It was running pfsense and was quite happy about it. Pfsense dropped support for 32 bits in their pfsense 2.4 release.

This would left me with a unsupported firewall which was one of the reasons to use pfsense instead of a closed source commercial router.

I could have moved to a new firewall like the pcengines apu but there is no reason to replace hardware that works fine.

The nice thing about opensource software is that we’ve options to choose from if software doesn’t match your usecase we’ve other options to choose from.

OPNsense

So I decided to give opnsense a try. OPNsense is a fork of pfsense, both are a fork of m0n0wall.

"opnsense_swapspace.png"

swapspace

My firewall only has 256 MB of memory which is a bit low even for a firewall.

The OPNsense developers made it very easy to add swapspace from the GUI. To add swap space go to [ System ] > [ Miscellaneous ] and activate the [ Add a 2 GB swap file to the system ] checkbox.

I’m verify satisfied with the upgrade from pfsense to OPNsense, OPNsense has a new release very month which is nice to get the latest security updates and it’s possible to audit the systems for security updates from the GUI.

"duckdns"

DuckDns

I move my ADSL with a fixed ip address to a VDSL line with a dynamic ip address so I was looking a good free dynamic dns provider and settled with duckdns.

Have fun

How to Start DLM Monitoring on a VDSL Line in Belgium

In Belgium/Flanders we have two main internet line providers;

  • telenet the cablenet network provider.
  • proximus is the telephone network provider.

On telephone network there are alternative internet providers but they use the network of proximus.

I switched my internet connection from ADSL to VDSL and switched to a new provider ( edpnet). The internet speed was below the expectations and my modem reported errors on the line. After fixing the internal phone cabbeling in my appartment I wanted the retrigger the DLM monitoring.

The process is explained in the this post https://userbase.be/forum/viewtopic.php?t=48767 at usebase.be

To start the DLM monitoring in Belgium you need to call 0800 22 424 and type in your line number. If you don’t have a proximus phone number the line number is not the same as your phone number. To get your line number you need to connect an analog phone to our line and call 1924 this will read aloud your line number.

Have fun

High Screen Resolution on a KVM Virtual Machine With QXL

When you create an new virtual KVM virtual system the video ram is limited to 16MB by default to use a higer screen resolution you need to increase the video ram. The available resolution reported by the virtual screen may also not include the resolution that you want to utilize.

You’ll find my journey to enable higher screen resolutions in my KVM (qemu) virtual systems below.

Ubuntu 16.04

There is an issue with Ubuntu 16.04 and the latest HWE kernel https://wiki.ubuntu.com/Kernel/LTSEnablementStack. Even a full HD resultion (1920 x 1080 ) if you have the latest HWE kernel on your system.

To resolve this issue your can uninstall the latest kernel or install the LTS kernel.

Install the LTS Kernel

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
staf@ubuntu:~$ sudo apt-get install linux-generic-lts-xenial
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  linux-generic linux-headers-4.4.0-119 linux-headers-4.4.0-119-generic linux-headers-generic
  linux-image-4.4.0-119-generic linux-image-extra-4.4.0-119-generic linux-image-generic
Suggested packages:
  fdutils linux-doc-4.4.0 | linux-source-4.4.0 linux-tools
The following NEW packages will be installed:
  linux-generic linux-generic-lts-xenial linux-headers-4.4.0-119 linux-headers-4.4.0-119-generic
  linux-headers-generic linux-image-4.4.0-119-generic linux-image-extra-4.4.0-119-generic linux-image-generic
0 upgraded, 8 newly installed, 0 to remove and 0 not upgraded.
Need to get 69,3 MB of archives.
After this operation, 301 MB of additional disk space will be used.
Do you want to continue? [Y/n] 
<snip>
Setting up linux-image-generic (4.4.0.119.125) ...
Setting up linux-headers-4.4.0-119 (4.4.0-119.143) ...
Setting up linux-headers-4.4.0-119-generic (4.4.0-119.143) ...
Setting up linux-headers-generic (4.4.0.119.125) ...
Setting up linux-generic (4.4.0.119.125) ...
Setting up linux-generic-lts-xenial (4.4.0.119.125) ...
staf@ubuntu:~$ 

Remove the HWE kernel

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
staf@ubuntu:~$ sudo apt-get purge linux-image-4.13*
Reading package lists... Done
Building dependency tree       
Reading state information... Done
<snip>
done
The link /vmlinuz.old is a damaged link
Removing symbolic link vmlinuz.old 
 you may need to re-run your boot loader[grub]
The link /initrd.img.old is a damaged link
Removing symbolic link initrd.img.old 
 you may need to re-run your boot loader[grub]
Purging configuration files for linux-image-4.13.0-38-generic (4.13.0-38.43~16.04.1) ...
Examining /etc/kernel/postrm.d .
run-parts: executing /etc/kernel/postrm.d/initramfs-tools 4.13.0-38-generic /boot/vmlinuz-4.13.0-38-generic
run-parts: executing /etc/kernel/postrm.d/zz-update-grub 4.13.0-38-generic /boot/vmlinuz-4.13.0-38-generic

Cleanup

1
2
3
4
5
6
7
8
9
10
11
12
13
14
staf@ubuntu:~$ sudo apt autoremove
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be REMOVED:
  linux-headers-4.13.0-36 linux-headers-4.13.0-36-generic linux-headers-generic-hwe-16.04
0 upgraded, 0 newly installed, 3 to remove and 0 not upgraded.
After this operation, 83,1 MB disk space will be freed.
Do you want to continue? [Y/n] 
(Reading database ... 234149 files and directories currently installed.)
Removing linux-headers-4.13.0-36-generic (4.13.0-36.40~16.04.1) ...
Removing linux-headers-4.13.0-36 (4.13.0-36.40~16.04.1) ...
Removing linux-headers-generic-hwe-16.04 (4.13.0.38.57) ...
staf@ubuntu:~$ 

Reboot

After a reboot higher resolutions are possible on ubuntu 16.04

Increase the video RAM

Required video ram

When you create a new KVM virtual machine it has 16MB of video RAM. Below you’ll the calculation for the required video RAM for a 4k resolution ( 3840 x 2160 ).

3840 x 2160 = 8294400
8294400 x 32 = 265420800
265420800 / 8 = 33177600
33177600 / (1024*1024) = 31.640625 MB

So 32 MB video ram is enough for a 4k resolution, to take some overhead into account we’ll increase the video ram to 64 MB.

list the domains

1
2
3
4
5
6
7
8
9
10
11
12
13
14
[swagemakers@staflaptop ~]$ sudo virsh
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # list --all
 Id    Name                           State
----------------------------------------------------
 -     centos7.0                      shut off
 -     debian                         shut off
 -     fedora27                       shut off

virsh # 

edit the domain settings

1
virsh # edit --domain debian
update the memory settings
1
2
3
4
5
<video>
  <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</video>
<redirdev bus='usb' type='spicevmc'>

to

1
2
3
4
<video>
  <model type='qxl' ram='65536' vram='65536' vgamem='65536' heads='1' primary='yes'/>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</video>

xrandr

Even with the additional RAM higer resolution aren’t possible (yet), the virtual screen doesn’t report the higer screen resolution. It’s possible to add the higher screen resolution with xrandr.

display current settings
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
staf@debian:~$ xrandr 
Screen 0: minimum 320 x 200, current 1920 x 1080, maximum 8192 x 8192
Virtual-0 connected primary 1920x1080+0+0 0mm x 0mm
   1024x768      59.95 +
   1920x1200     59.95  
   1920x1080     60.00* 
   1600x1200     59.95  
   1680x1050     60.00  
   1400x1050     60.00  
   1280x1024     59.95  
   1440x900      59.99  
   1280x960      59.99  
   1280x854      59.95  
   1280x800      59.96  
   1280x720      59.97  
   1152x768      59.95  
   800x600       59.96  
   848x480       59.94  
   720x480       59.94  
   640x480       59.94  
Virtual-1 disconnected
Virtual-2 disconnected
Virtual-3 disconnected
staf@debian:~$ 
get the modeline
1
2
3
4
staf@debian:~$ cvt 2560 1440 
# 2560x1440 59.96 Hz (CVT 3.69M9) hsync: 89.52 kHz; pclk: 312.25 MHz
Modeline "2560x1440_60.00"  312.25  2560 2752 3024 3488  1440 1443 1448 1493 -hsync +vsync
staf@debian:~$ 
# create the new mode line
1
2
staf@debian:~$ xrandr --newmode "2560x1440_60.00"  312.25  2560 2752 3024 3488  1440 1443 1448 1493 -hsync +vsync
staf@debian:~$ 
# add the mode to your screen
1
2
staf@debian:~$ xrandr --addmode Virtual-0 2560x1440_60.00
staf@debian:~$ 
# use the new mode
1
2
staf@debian:~$ xrandr --output Virtual-0 --mode 2560x1440_60.00
staf@debian:~$ 
## 4k

To use a 4k resolution you can use the commands

1
2
3
4
5
6
7
staf@debian:~$  cvt 3840 2160
# 3840x2160 59.98 Hz (CVT 8.29M9) hsync: 134.18 kHz; pclk: 712.75 MHz
Modeline "3840x2160_60.00"  712.75  3840 4160 4576 5312  2160 2163 2168 2237 -hsync +vsync
staf@mydevolo:~$ xrandr --newmode "3840x2160_60.00"  712.75  3840 4160 4576 5312  2160 2163 2168 2237 -hsync +vsync
staf@mydevolo:~$ xrandr --addmode Virtual-0 3840x2160_60.00
staf@mydevolo:~$ xrandr --output Virtual-0 --mode 3840x2160_60.00
staf@mydevolo:~$ 

Add the new screen resolution permanently

Debian & Co

Create a monitor configuration file in /usr/share/X11/xorg.conf.d

1
root@mydevolo:/usr/share/X11/xorg.conf.d# vi 10-monitor.conf

And add the modeline fgor your screen resolution. With the Option “PreferredMode” you can set the prferred resolution.

1
2
3
4
5
6
section "Monitor"
    Identifier "Virtual-0 "
    Modeline "2560x1440_60.00"  312.25  2560 2752 3024 3488  1440 1443 1448 1493 -hsync +vsync
    Modeline "3840x2160_60.00"  712.75  3840 4160 4576 5312  2160 2163 2168 2237 -hsync +vsync
    Option "PreferredMode" "2560x1440_60.00"
EndSection

Other GNU/Linux distros

Most other GNU/Linux distribution use /etc/X11/xorg.conf.d/

Have fun

Links