blog.vyos.net

LLDP bug

James Brown reported on phabricator that LLDP is not working in 1.1.8. Quite a mess up on our side: the reason it's not working is that it's built against the old OpenSSL 0.9.8 which is no longer in VyOS, but since the debian/control was missing dependency on libssl, it wasn't detected as dependent on OpenSSL and thus wasn't rebuilt.

If you want LLDP back right now, you can install the helium3 package from our repository (http://dev.packages.vyos.net/legacy/repos/vyos/pool/main/l/lldpd/).

OSPF route-map

One feature is missing from the changelog because it was committed "stealthily", without a task number, and thus I missed it. It's the command for setting up route-map for installing routes imported from OSPF.

set protocols ospf route-map MyMap

This route-map can prevent routes from getting installed into the kernel routing table, but make no mistake, it is not incoming route filtering (which would be very bad for OSPF). The routes will stay in OSPFd and in the RIB (they will be displayed as inactive).

The future of 1.1.x

So far it looks like we are going to make an 1.1.9 release to fix the LLDP bug. Perhaps we should also cherry-pick something safe from 1.2.0 too, if you have any specific bugfix or a tiny feature is mind, let us know.

VyOS 1.1.8 on AWS

The official 1.1.8 AMI passed the automated tests and it's on its way to the marketplace, the manual review by the marketplace team will take perhaps a week or so.

If you want to make you own, you can already use the AMI build scripts and point it to the (signed) release image URL.

And while we are at it: IPv6 in VyOS 1.2.0

I've re-enabled the old patch for IPv6 VRRP in the current branch and it will be in today's nightly build. In 1.1.x, we had to disable it because in the older keepalived version IPv4 and IPv6 VRRP were mutually exclusive, however, in the current branch, it seems to work. If you feel adventurous, please test the nightly build (on lab VMs!) and tell us it it works for you.

Also, on the forum, it was reported that the current branch image doesn't build. I've resolved that problem today so if you want to build an image, it should work.

1.1.8, the major minor release, is available for download from https://downloads.vyos.io/?dir=release/1.1.8 (mirrors are syncing up).

It breaks the semantic versioning convention, while the version number implies a bugfix-only release, it actually includes a number of new features. This is because 1.2.0 number is already assigned to the Jessie-based release that is still in beta, but not including those features that have been in the codebase for a while and a few of them have already been in production for some users would feel quite wrong, especially considering the long delay between the releases. Overall it's pretty close in scope to the original 1.2.0 release plan before Debian Squeeze was EOLd and we had to switch the effort to getting rid of the legacy that was keeping us from moving to a newer base distro.

You can find the full changelog here.

The release is available for both 64-bit and 32-bit machines. The i586-virt flavour, however, was discontinued since a) according to web server logs and user comments, there is no demand for it, unlike a release for 32-bit physical machines b) hypervisors capable of running on 32-bit hardware went extinct years ago. The current 32-bit image is built with paravirtual drivers for KVM/Xen, VMware, and Hyper-V, but without PAE, so you shouldn't have any problem running it on small x86 boards and testing it on virtual machines.

We've also made a 64-bit OVA that works with VMware and VirtualBox.

Security

Multiple vulnerabilities in OpenSSL, dnsmasq, and hostapd were patched, including the recently found remote code execution in dnsmasq.

Bugfixes

Some notable bugs that were fixed include:

• Protocol negation in NAT not working correctly (it had exactly opposite effect and made the rule match the negated protocol instead)
• Inability to reconfigure L2TPv3 interface tunnel and session ID after interface creation
• GRUB not getting installed on RAID1 members
• Lack of USB autosuspend causing excessive CPU load in KVM guests
• VTI interfaces not coming back after tunnel reset
• Cluster failing to start on boot if network links take too long to get up

New features

User/password authentication for OpenVPN client mode

A number of VPN providers (and some corporate VPNs) require that you use user/password authentication and do not support x.509-only authentication. Now this is supported by VyOS:

set interfaces openvpn vtun0 authentication username jrandomhacker
set interfaces openvpn vtun0 authentication password qwerty
set interfaces openvpn vtun0 tls ca-cert-file /config/auth/ca.crt
set interfaces openvpn vtun0 mode client
set interfaces openvpn vtun0 remote-host 192.0.2.1

Bridged OpenVPN servers no longer require subnet settings

New OpenVPN options exposed in the CLI

A few OpenVPN options that formerly would have to be configured through openvpn-option are now available in the CLI:

set interfaces openvpn vtun0 use-lzo-compression
set interfaces openvpn vtun0 keepalive interval 10
set interfaces openvpn vtun0 keepalive failure-count 5

Point to point VXLAN tunnels are now supported

set interfaces vxlan vxlan0 address 10.1.1.1/24
set interfaces vxlan vxlan0 remote 203.0.113.50
set interfaces vxlan vxlan0 vni 10

AS-override option for BGP

The as-override option that is often used as an alternative to allow-as-in is now available in the CLI:

set protocols bgp 64512 neighbor 192.0.2.10 as-override

as-path-exclude option for route-maps

set policy route-map Foo rule 10 action permit
set policy route-map Foo rule 10 set as-path-exclude 64600

Buffer size option for NetFlow/sFlow

set system flow-accounting buffer-size 512 # megabytes

set system flow-accounting netflow source-ip 192.0.2.55
set system flow-accounting netflow max-flows 2097152

VLAN QoS mapping options

set interfaces ethernet eth0 vif 42 egress-qos 1:6
set interfaces ethernet eth0 vif 42 ingress-qos 1:6

Ability to set custom sysctl options

set system sysctl custom $key value $value

Custom client ID for DHCPv6

set interfaces ethernet eth0 dhcpv6-options duid foobar

Ethernet offload options

Under "set interfaces ethernet ethX offload-options" you can find a number of options that control NIC offload.

Syslog level "all"

Now you can specify options for the *.* syslog pattern, for example:

set system syslog global facility all level notice

Unresolved or partially resolved issues

Acknowledgements

I would like to say thanks to everyone who contributed and made this release possible, namely: Kim Hagen, Alex Harpin, Yuya Kusakabe, Yuriy Andamasov, Ray Soucy, Nikolay Krasnoyarski, Jason Hendry, Kevin Blackham, kouak, upa, Logan Attwood, Panagiotis Moustafellos, Thomas Courbon, and Ildar Ibragimov (hope I didn't forget anyone).

A note on the downloads server

The original packages.vyos.net server is still having IO performance problems and won't handle a traffic spike associated with release well. We've setup the https://downloads.vyos.io server on our new host specially for release images and will later migrate the rest of the old server including package repositories and the rsync setup.

A new image is available for testing: http://dev.packages.vyos.net/iso/testing/vyos-1.1.8-rc2-amd64.iso 

If you are using VLAN interfaces, avoid the rc1 one and install the rc2 right away. Thanks to Samir we have identified a bug in VLAN interface configuration scripts (https://phabricator.vyos.net/T435) that prevented them from loading correctly. It was caused by an overlooked syntax error in the new options for ingress/egress VLAN QoS options.

Oddly, the current branch had a correct version of that patch, so 1.2.0 builds are not affected.

The long overdue 1.1.8 release candidate is available for download from http://dev.packages.vyos.net/iso/testing/vyos-1.1.8-rc1-amd64.iso

While a number of people have already been running 1.2.0 nightly builds in production, we do acknowledge there are people who are not in position to install updates that are not completely stable, and recently discovered vulnerabilities in dnsmasq that potentially allow remote code execution are impossible to ignore (unlike many older vulnerabilities that are only locally or aren't practical to exploint).

It's stable for all practical purposes, but since it includes pretty big updates and a few new features, I suppose it's better to go through the release candidate phase. If in say a week no one finds any issues.

The release is only available for 64-bit machines at the moment. We can provide it for 32-bit, but we are wondering if anyone still wants it, when even small boards have 64-bit CPUs.

You can read the full changelog here: https://wiki.vyos.net/wiki/1.1.8_changelog_proposal 

Among package updates, there are openssl 1.0.2l and dnsmasq 2.72. Since squeeze is long EOL, the OpenSSL update required re-compiling everything that depends on OpenSSL ourselves, which took longer than we hoped.

Among VyOS fixes and features, there are user/password authentication for OpenVPN, as-override option for BGP neighbors, as-path-exclude option for route-map rules, tweakable pipe (buffer) size for netflow/sflow (too small hardcoded value could cause pmacct crash on high traffic routers), peer-to-peer VXLAN interfaces, and multiple fixes for bugs of varying severity, such as overly high CPU load on KVM guests or protocol negation in NAT rules not working.

A lot of features from 1.2.0 are not backportable due to big code changes and dependencies on way newer software versions than 1.1.x could provide, so features for cherry-picking had to be carefully chosen and even that needed quite a bit of merge conflict resolution. Quite a few of those were meant for the ill-fated "lithium" release that was supposed to be named 1.2.0 and be the last squeeze-based release, but then squeeze EOL'd, then serious life circumstances forced Alex Harpin to put all his VyOS work on hold thus leaving the maintainers team even more understaffed, and then the company we started to fund VyOS development through commercial support and services had a hard time when it almost reached the point of bankruptcy and dissolution (and, since it's self-funded, its founders almost reached the point of personal bankruptcy along with it), so by the time we could get things back on track a feature release based on squeeze wouldn't be feasible, especially considering how much we had to change to make the old codebase run on jessie. In a sense, it's a lithium that could have been, at least partially, rather than a straight maintenance release with nothing but bugfixes.
But, many of those features spent so much time in the limbo without making it into a release called stable that we felt compelled to include at least some of them.

I would like to say thanks to everyone who contributed and made this release possible, namely: Kim Hagen, Alex Harpin, Yuya Kusakabe, Yuriy Andamasov, Ray Soucy, Nikolay Krasnoyarski, Jason Hendry, Kevin Blackham, kouak, upa, Logan Attwood, Panagiotis Moustafellos, Thomas Courbon, and Ildar Ibragimov (hope I didn't forget anyone).

Someone on Facebook rightfully noted that lately there's been more work on the infrastructure than development. This is true, but that work on infrastructure was long overdue and we just had to do it some time. There is even more work on the infrastructure waiting to be done, though it's more directly related to development, like restructuring the package repos.

Anyway, it doesn't mean all development has stopped while we've been working on infrastructure. Today we released a Python library for managing VyOS routers remotely.

Before I get to the details, have a quick example of what using it is like:

import vymgmt

vyos = vymgmt.Router('192.0.2.1', 'vyos', password='vyos', port=22)

vyos.login()
vyos.configure()

vyos.set("protocols static route 203.0.113.0/25 next-hop 192.0.2.20")
vyos.delete("system options reboot-on-panic")
vyos.commit()

vyos.save()
vyos.exit()
vyos.logout()

If you want to give it a try, you can install it from PyPI ("pip install vymgmt"), it's compatible with both Python 2.7 and Python 3. You can read the API reference at http://vymgmt.readthedocs.io/en/latest/ or get the source code at https://github.com/vyos/python-vyos-mgmt .

Now to the details. This is not a true remote API, the library connects to VyOS over SSH and sends commands as if it was a user session. Surprisingly, one of the tricky parts was to find an SSH/expect library that can cope with VyOS shell environment well, and is compatible with both 2.7 and 3. All credit for this goes to our contributor who goes by Hochikong, who tried a whole bunch of them, settled with pexpect and wrote a prototype.

How the library is better than using pexpect directly, if it's a rather thin wrapper for it? First, it's definitely more convenient to just call set() or delete() or commit() than to format command strings yourself and take care of the sending and receiving lines.

Second, common error conditions are detected (through simple regex matching) and raise appropriate exceptions such as ConfigError (for set/delete failures) or CommitError for commit errors. There's also a special ConfigLocked exception (a subclass of CommitError) that is raised when commit fails due to another commit in progress, so you can recover from it by sleep() and retry. This may seem uncommon, but people who use VRRP transition scripts and the like on VyOS already reported that they ran into it.

Third, the library is aware of the state machine of VyOS sessions, and will not let you accidentally do wrong things such as trying to enter set/delete commands before entering the conf mode. By default it also doesn't let you exit configure sessions if there are uncommited or unsaved changes, though you can override it. If a timeout occursm an exception will be raised too (while pexpect returns False in this case).

Right now it only supports set, delete, and commit, of all high level methods. This should be enough for the start, but if you want something else, there are generic methods for running op and conf mode commands (run_op_mode_command() and run_conf_mode_command() respectively). We are not sure what people want most, so what we implement depends on your requests ans suggestions (and pull requests of course!). Other things that are planned but that aren't there yet are SSH public key auth and top level words other than set and delete (rename, copy etc.). We are not sure if commit-confirm is really friendly to programmatic access, but if you have any ideas how to handle it, share with us.

On an unrelated note, syncer and his graphics designer friend made a design for VyOS t-shirts. If anyone buys that stuff, the funds will be used for the project needs. The base cost is around 20 eur, but you can get them with 15% discount by using VYOSMGTLIB promo code: https://teespring.com/stores/vyos?source=blog&pr=VYOSMGTLIB