We are doing some maintenance on ci.vyos.net and the build hosts. They may be inaccessible for an hour or two. We will notify you if it takes longer than expected.
We have restored VyOS builds. Nightly build should work as expected today, and you can build it by hand as well if you want. This is not exactly the end of the story for us since we need to finish some reconfiguration of Jenkins to accomodate the new setup, but nothing dramatic should happen to the ISO builds any soon, or so we hope.
HTTPS, however, is another story. It still doesn't work for me, and I'm not sure if it's APT itself to blame or anything in our build setup. Since this is not a pressing issue, I'm not going to put much effort into it right now, but if you have a build setup, please check if it works for you. If it doesn't work for anyone, then we can write it off as an APT issue.
We've made HTTP on the dev.packages.vyos.net host optional, and restored the real directory index (provided by the Apache HTTP's mod_autoindex) instead of using the DirectoryLister that was proven a bit problematic with APT.
Since we had to change the default repository URL anyway, I also took a chance to finally make it configurable rather than hardcoded (T519). Now you can specify a custom URL with the --vyos-mirror="$URL" option. It defaults to the plain HTTP URL right now for the reason stated below.
I have also found a way to make live-build include apt-transport-https packages at the bootstrap stage and enable it to use HTTPS servers for building images. However, for some reason it doesn't work for me, apt says it cannot fetch the package index, while fetching that file with curl works just fine from the same host. I'm not sure what the issue may be. If you verify that it works for you or doesn't, or you know how to make it work, please comment upon T422.
Actual builds "still" don't work but for a completely unrelated reasons: mdns-repeater package needed for the recently merged mDNS repeater feature is not yet in the repository. We will fix it shortly, now that the builds otherwise work.
The original host where the packages.vyos.net and dev.packages.vyos.net web servers used to live has kept having serious I/O performance issues ever since a RAID failure event, and the situation hasn't improved much. After we've started having very noticeable latency even with bash completion (not speaking of very slow download speed for our users and mirror maintainers), we decided that it's time to move it elsewhere.
We do not blame the hoster, to the contrary, we are grateful for hosting it for us at no cost for so long despite its bandwidth consumption, and for offering it to us at the point in the VyOS project life when it has exactly zero budget for anything. But, it seems now it's time to move on.
The dev.packages.vyos.net host is used for the package repositories of the current branch and for nightly builds, as well as experimental images we make available for testing. It is already migrated to the new host and operational, even though some things are not working as well as they should, including temporary (we hope) loss of IPv6 access to it, and somewhat untidy directory links generated by the (otherwise very nice) DirectoryLister script.
We are sorry for the migration taking longer than it should have and for not communicating it to the community. The dev.packages.vyos.net stayed inaccessible for a whole day because of miscommunication: we have migrated the data, but I assumed that Yuriy (syncer) will setup the web server, while he assumed I will. Our maintenance procedures definitely could use some improvement.
Note that apart from physical data migration, we have also done some directory restructuring. We hope it's now more logical and that it will not have any impact on anyone, but if it does, we can setup redirects to make old direct links work again.
The packages.vyos.net server will stay on the old machine for a bit as we also need to migrate the rsync setup that the mirrors are using. We are still not sure whether to use downloads.vyos.io as a central server for mirrors too.
As of building VyOS images, we'll make adjustments to the build scripts to make them use the new paths by default and images will be buildable again.
We'll be migrating the CI server (https://ci.vyos.net) to a different host this weekend. It means over this weekend there will be no automatic package or image builds for a time.
We hope this will not take too long, but in the worst case we will have it working by Monday.
Nothing else will be affected by this maintenance so the overall impact on the end users should be very low.
In preparation for the new 1.2.0 (jessie-based) beta release, we are re-populating the package repositories. The old repositories are now archived, you still can find then in the /legacy/repos directory on dev.packages.vyos.net
The purpose of this is two-fold. First, the old repo got quite messy, and Debian people (rightfully!) keep reminding us about it, but it would be difficult to do a gradual cleanup. Second, since the CI server has moved, and so did the build hosts, we need to test how well the new procedures are working. And, additionally, it should tell us if we are prepared to restore VyOS from its source should anything happen to the packages.vyos.net server or its contents.
For perhaps a couple of days, there will be no new nightly builds, and you will not be able to build ISOs yourself, unless you change the repo path in ./configure options by hand. Stay tuned.
I know you are tired of this already, but... yes, you guessed it right, we are migrating the phabricator again!
This time we are moving it to the host that currently houses the wiki, a VM at OpenITC (thanks, Sean!). This should be the last migration for a long while. We plan to consolidate all web resources on that host: while this is not so good for redundancy, it's easier to manage. Since the blog is not hosted there, and we also have a twitter, we still have out of band channels to notify people about outages and resolution timeframe estimates, should that server ever failed.
Today we'll shutdown the phabricator to migrate all the data on the new host and re-deploy it there (it would be nice to put it in read only mode instead, but it currently doesn't have it, it's planned for future versions). We also need to reconfigure the web server for the new setup, so there may be short periods of downtime for the wiki too.
We'll notify you when migration is complete.
We are working on the servers now, moving some things around again, and the phabricator is temporarily inaccessible. We'll let you know when it's resolved.
Hi everyone,
I bet you are all tired of this stuff already, but then again, so we are. We ourselves hope it's the last big maintenance we get to do at that site and then everything will stay stable for a while.
We are moving all VMs to new servers with more optimal hardware configuration and optimizing some configs. Among the VMs that will go down, likely for a few hours, are phabricator and jessie build host.
Things should get back to normal once we are done, we'll make another post when it's complete.
The maintainer of the Ukrainian VyOS mirror (0.ua.mirrors.vyos.net) is re-organizing his network, and this required to change the IPv4 address of the mirror host. We have updated its DNS records, but it may take some time to update, if it's still inaccessible for you, wait a bit or reset the DNS cache by hand.